You are here

Article Text

Article menu

Download PDFPDF

XML
Paediatrics
Research
The design and user-testing of a question prompt list for attention-deficit/hyperactivity disorder
  1. Rana Ahmed1,
  2. David K Raynor2,
  3. Kirsten J McCaffery3,
  4. Parisa Aslani1
  1. 1Faculty of Pharmacy, University of Sydney, Sydney, New South Wales, Australia
  2. 2School of Healthcare, University of Leeds, Leeds, UK
  3. 3School of Public Health, University of Sydney, Sydney, New South Wales, Australia
  1. Correspondence to Rana Ahmed; rana.ahmed{at}sydney.edu.au

Abstract

Objectives This study involved the development of a question prompt list (QPL) booklet designed to facilitate communication and shared decision-making between parents/carers of children diagnosed with attention-deficit/hyperactivity disorder (ADHD) and their clinicians; and user-testing of the QPL to assess its usability.

Design Best practice in information writing and design was used to format the QPL content into a 16-page booklet. We then applied user-testing, which uses mixed methods to assess document performance with small cohorts of participants and then improves it in an iterative process. Individual interviews assessed the ability of QPL users to locate and understand key points of information, followed by a semistructured questionnaire, to ascertain their general views about the booklet.

Setting and participants Testing was undertaken with two cohorts of 10 parents/carers of children with ADHD (n=20); matched on age, gender and educational attainment.

Tested documents In round 1, we tested 15 key points of information related to the QPL. Participant responses and feedback from round 1 informed a revised version of the booklet, tested in a subsequent round.

Primary outcome measure The target was for 8/10 of the participants to be able to find and demonstrate an understanding of all key information points, in accordance with European guidelines for medicine leaflet testing.

Results After round 1, problems related to 4/15 information points were identified (booklet purpose; preparing for appointments; asking about a second medical opinion; selecting which questions to ask). Participants also made suggestions regarding the booklet's layout and design. After round 2, all information points were located and understood by at least 8/10 participants.

Conclusions This is the first study to have developed a usable ADHD-specific QPL for use by parents/carers of children with ADHD during clinical consultations, and the first demonstration of the utility of user-testing methods in ensuring QPL usability.

  • attention deficit hyperactivity disorder
  • shared decision making
  • communication
  • question prompt list
  • user testing

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2014-006585

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This is the first study to have developed a tailored resource intended to facilitate communication and shared decision-making between parents/carers of children with attention-deficit/hyperactivity disorder and their clinicians.

    • The study represents the first demonstration of the utility of user-testing as a method in assessing the performance of this type of resource.

    • The user-testing method does not test the documents’ influence on treatment decision-making or long-term outcomes such as adherence to therapy, which require assessment in future work.

    Introduction

    Attention-deficit/hyperactivity disorder (ADHD) is a chronic and impairing neurodevelopmental disorder of childhood.1 It is characterised by symptoms of inattention, hyperactivity and impulsivity.2 ,3 The target of first-line treatment with stimulant agents (eg, methylphenidate) is to enhance the action of norepinephrine and dopamine, thereby alleviating ADHD symptoms.4–6 Despite an understanding of the neurobiological origins of ADHD and the demonstrated efficacy of these medicines, there remains a significant amount of controversy surrounding ADHD and a strong sense of unease within the public sphere about using stimulant medicines as first-line therapy.7–9

    These polemic discussions have only been strengthened by the recognition that the prevalence of ADHD continues to rise,10 a fact that many advocate is the result of lax diagnostic and prescribing practices, and widening of the diagnostic criteria used to define the disorder.11–14

    Therefore, although the use of pharmacotherapy is regarded as standard clinical practice for the management of ADHD symptoms throughout international treatment guidelines, parents and carers (henceforth referred to as parents for ease of reference) of children who have received an ADHD diagnosis often have difficulty making decisions about treatment.15 ,16 Parents have expressed frustration and confusion with sources of ADHD-related information and a desire to access relevant, reliable resources to assist in their treatment decision-making.17

    Non-adherence to prescribed treatments for ADHD may be as high as 87% in some instances,18 and has been associated with poorer outcomes for the child and overall increased healthcare burden.19–21 While this may be attributed to a number of factors, lack of adequate information provision about the disorder and its treatments appears to repeatedly underscore poor adherence.18 ,22 ,23

    Information from healthcare professionals and shared decision-making

    Healthcare professionals (HCPs) are an important source of reliable information for parents.17 ,24 However, some parents have reported difficulties communicating with HCPs during clinical consultations, raising concerns such as: general difficulty obtaining information, receiving insufficient information, and receiving excessive information that is irrelevant to their specific concerns and difficult to absorb during the limited consultation time.17 ,25 ,26 These communication difficulties can lead to an inability to express treatment preferences and poor adherence to prescribed regimens.26

    This is why the practice of shared decision-making (SDM), a collaborative approach used between clinicians and patients to arrive at agreed treatment decisions, has become a focus of great interest in the literature.27 ,28 Recognised by many as the gold-standard in the delivery of healthcare services,29 SDM requires clinicians to engage with their patients during clinical consultations, facilitating an exchange of information and values to assist in reaching a point of shared agreement about treatment.29 This process decreases the asymmetry of information and authority that can often be present during clinical consultations and empowers patients to take control over their treatment decisions.28 In the paediatric care setting, involving parents in treatment decision-making has been demonstrated to improve treatment adherence and overall health outcomes for the child.30

    With regard to ADHD and its management, the importance of SDM has been emphasised throughout international treatment guidelines.31–33 However, greater efforts are required to facilitate SDM during clinical consultations.34 Tools such as question prompt lists (QPLs), which assist patients in asking questions during clinical consultations, may prove to be a useful approach in addressing this.

    QPL for ADHD

    QPLs contain structured lists of disease and treatment-specific questions intended for use by patients as a prompt for question-asking during clinical consultations. QPLs are designed to facilitate communication between patients and their clinicians and, in turn, encourage SDM. They have been demonstrated to be effective facilitators for communication during clinical consultations in oncology and palliative care settings.35 ,36

    Development of a QPL for ADHD may help address a number of issues, such as: (1) concerns raised by parents of children with ADHD about the availability of relevant and reliable information sources; (2) difficulties experienced communicating with HCPs during clinical consultations and (3) need for greater efforts to promote SDM. Such a QPL would have the additional benefit of addressing parents’ desire to use written resources as a prompt for communication with a HCP and the inability of some parents to ask the right questions during consultations.17 ,25 ,26

    In light of this, we developed and validated the content of an ADHD-specific QPL. The questions were derived through a systematic analysis of existing ADHD and QPL-related resources, and validated by clinicians, researchers, parents and consumer advocates in a three-round web-based Delphi study (submitted for publication). The QPL consists of 88 questions addressing a range of ADHD-related issues including: (1) diagnosis; (2) understanding ADHD; (3) treatment: (i) medicines, (ii) psychological and alternative; (4) healthcare team; (5) monitoring ADHD; (6) managing ADHD; (7) future expectations: (i) approaching adolescence, (ii) health and medicines, (iii) academic progress, (iv) social progress; and (8) support and information.

    The QPL does not include any information about ADHD or ADHD-related issues, rather, it consists of a list of questions pertaining to the above eight topic areas, which parents can choose to ask their child's clinicians. By encouraging question-asking during clinical consultations, it is anticipated that the QPL will help increase parents’ knowledge about ADHD and its treatments and consequently enhance the potential for SDM between parents and clinicians about treatment options.

    Prior to assessing these outcomes, it is essential to first ensure that the QPL is presented in a user-friendly format and that its content is easy to understand. User-testing was deemed to be a suitable and thorough approach to evaluating these aspects of the QPL. This study aimed to: (1) format the 88 questions derived from our previous work into a booklet using principles of good information writing and design and (2) test the performance of this booklet using established user-testing methods. To the best of our knowledge, this is the first application of user-testing methods to evaluating any QPL.

    In utilising this approach we asked two research questions, first, whether parents of children with ADHD could locate and understand key questions and pieces of information in the QPL and second, if the iterative application of user-testing could inform the development of a revised and improved version of the QPL.

    Methods

    There were two key phases involved in this study: (1) formatting the QPL into a booklet; and (2) applying user-testing methods to evaluate its performance.

    Formatting QPL into booklet form

    The 88 questions formed the main text of the QPL and were incorporated into a booklet format using a similar approach to that adopted by Langbecker et al.37 The booklet lists the questions according to their respective topics and includes instructions for parents, outlining who the booklet is for and how it should be used.

    The instructions emphasise that the booklet may not provide exhaustive coverage of the questions parents may wish to ask and encourage them to add their own questions. Parents are also advised against asking all the questions during one consultation and asked, rather, to identify those questions that are relevant to their child's needs at that specific point in time.

    Key writing and design principles for producing easy-to-understand healthcare materials38 were followed and included use of a large, clear font; inclusion of white space around the text; use of subheadings, bulleted points and bold text to highlight information; inclusion of culturally diverse images achieved by applying an artistic cross-hatch effect over the images so faces were not readily identifiable; and inclusion of a cover designed to be attractive to parents. A colour-coded contents page was included to further enhance the usability of the booklet. A blank, lined page was provided at the end of each topic for inclusion of additional questions or notes.

    The first draft of the QPL was a 16-page, slightly smaller than A5 sized, wire spiral-bound booklet titled “Asking Questions about ADHD: Questions to ask your child's healthcare provider about ADHD and its treatment.”

    User-testing

    User-testing is an established method that involves the performance-based evaluation of written patient materials, specifically, their ease of use and clarity.39 ,40 It has been primarily used to evaluate medicine information leaflets developed by pharmaceutical manufacturers, medicine information booklets and participant information sheets for clinical trials,41–43 but has also been applied to decision aids44 and medicine label wording.45 Unlike readability formulae, which rely purely on word and sentence length,46 ,47 user-testing assesses how a document performs with its intended users.

    The process involves individual interviews with cohorts of 10 participants, where they are provided with a copy of the document, and presented with a series of approximately 15 questions to determine their ability to locate and understand key points of information within it.39 ,40 ,48 The questionnaire is followed by a brief semistructured interview to ascertain participants’ views about the format, design and layout of the document.48 After the first round of interviews is completed, the document is revised to address any problems identified from participant feedback, using good practise in writing and information design.49 The revised document is tested with a second cohort and this iterative process continues until all issues with the document are resolved. According to the standards set by the European Union (EU), this is indicated by 8 of the 10 users being able to find and understand responses to all questions in the structured questionnaire.49

    Participants

    Twenty parents of children (aged between 3 and 18 years) with a clinical diagnosis of ADHD (the intended users of the QPL) were recruited by a market research company or through an Australian ADHD support group Facebook page.

    In each cohort of 10 participants, there were no more than 3 participants who had completed tertiary education and at least 1 belonged to the following age categories 30–39, 40–49 and 50–59 years. Similar participant profiles in terms of likely influences on testing (gender, age and educational level) were maintained in the two rounds of testing. To increase the rigour of the testing process, participants could not take part if they regularly used written information documents as part of their occupation or if they were HCPs.

    Tested materials

    The materials tested were: (1) the first draft of our ADHD-specific QPL, comprising 16 pages and (2) a revised version of the QPL, with changes made to the wording, layout and format based on the responses to the user-testing questionnaire and parent feedback from round 1, and by applying good practice in information writing and design.

    Outcomes

    The main outcome measure was participants’ ability to locate and demonstrate an understanding of 15 key points of information and questions in the QPL (table 1). These key items were selected by RA to test the usability and clarity of the information in the QPL, and checked for relevance by PA and DKR, after which some modifications were made. Any further differences were discussed between RA and PA until consensus about the questions was achieved. The questions were categorised into three themes (facts, actions and explanations) and each was presented to the participants in an order different to that of the natural order of the information in the QPL. Participant responses were used to score whether the information was found (‘yes’ or ‘no’) and, if found, whether it was understood (‘yes’ or ‘no’). The time taken to read the booklet and to complete the questionnaire was also measured. The interviewer also made field notes to document how the booklet was being used and listed any comments made by the participants during the testing process.

    View this table:
    Table 1

    User-testing questions relating to the 15 key information points in the QPL and participant responses

    Procedure

    Round 1: testing the original QPL booklet

    Participants were given a copy of the booklet and instructed to read it at their own pace, without the interviewer present. After reading the booklet, they were asked to use it to locate the answer to each of the 15 structured questions and to explain what they had understood, where applicable. Participants were next asked a few open-ended questions about the QPL booklet, namely, their general impressions, appearance and booklet size, font style and size, images and graphics, and organisation of information, to gather qualitative data about the booklet. All semi-structured interviews were audio recorded and transcribed verbatim with participant permission. Thematic analysis50 was used to identify the key themes in the qualitative data.

    Round 2: rewording, redesign and reassessment of the QPL booklet

    Following round 1, the QPL booklet was edited based on participant responses. Changes made were either content or aesthetics-based. Content changes were those that were anticipated to assist participants in locating and understanding items in the structured questionnaire, while aesthetic changes were those related to participant feedback during the semistructured interview. The revised QPL booklet was tested using the same procedure outlined previously.

    Results

    Testing of original QPL booklet (round 1)

    Quantitative data

    The original QPL booklet was tested by 10 parents of children diagnosed with ADHD. Of these, seven were women and three were men, aged between 33 and 50 years. Only three had obtained a tertiary level of education.

    Participants took an average of 8 min (range 6–12) to read the booklet. The structured questionnaire was completed in an average of 22 min (range 8–48). Table 1 outlines the number of participants who were able to locate and understand the questionnaire items in each round of testing. On the basis of these results, participants could not locate the appropriate section in the booklet (rather than not being able to understand the information) for the following 4 (of the 15) points (table 1):

    1. The main purpose of the booklet (Question 1);

    2. Using the booklet to prepare for an upcoming appointment (Question 3);

    3. Selecting which questions to ask the clinician (Question 6);

    4. Asking about obtaining a second medical opinion (Question 10).

    Qualitative data

    The thematic analysis of the semistructured interviews identified four themes: (1) concept of a QPL booklet, (2) appearance and graphics, (3) content and language and (4) organisation of information and user-friendliness. Similarities and differences in the participants’ views regarding these themes were noted and illustrated by verbatim quotes from the participants.

    Concept of a QPL booklet

    The QPL booklet was extremely well-received by participants in round 1, with all indicating that they would use this resource if made available to them: “I actually have got more information from here [QPL] than what I've had in years…The key about learning about this disease is to constantly ask questions.” [P6]; “It's fantastic, it's the best [resource] I've seen for ADHD…this is brilliant” [P2].

    They felt that the QPL would address some of the difficulties they experienced during clinical consultations: “Most parents are still in this grey area [regarding] what to ask and do feel frazzled when they go to the doctors” [P1].

    The parents also provided insight into their views on the potential applications and benefits of the resource: “I didn't really think…how is that [puberty] going to affect him [son] until I read this booklet” [P1]; “When you get a bombardment of information, you don't always remember. So it gives you the chance to write down the answers that the health care professional has given you…” [P2].

    The QPL was viewed by some parents as a resource they could share with their friends and children: “I'd actually encourage him [son] to read this because it may help him understand a bit more…what the condition is” [P3].

    The only reservation parents had about the QPL was the anticipated need for increased HCP awareness and education about the resource.

    Appearance and graphics

    All participants agreed that the booklet itself was an appropriate size: “Small enough…to put in a work bag or handbag” [P3] as well as the font size of the content.

    There was a general sentiment that the QPL was ‘very well put together’ [P2], of ‘brilliant quality’ [P2], “The colours are nice and vibrant so it grabs your attention” [P3] and the colours used created a ‘positive vibe’ [P6]. One parent, however, made the remark that “You might want to think of having a more durable cover” [P4].

    There were mixed views regarding the images, specifically the artistic cross-hatch effect to blur and de-identify the subjects. The majority responded positively to these images and provided interesting comments about the merit of the approach used, aside from imparting anonymity to the subjects: “It's very hard to represent the full diversity of cultures and backgrounds in photos. So I think it's clever…otherwise it could be misinterpreted as being exclusive” [P4]; “That…effect on the photo reflects what you feel about your child…and maybe what your child is feeling like as well” [P9].

    Three participants expressed a preference for ‘normal’ [P10] clear images primarily noting the sentiment: “It's more personalized when you can see the faces” [P3]. However, as the majority preferred the effect used, this was maintained in the revised version of the QPL.

    Content and language

    Overall, the parents expressed that the content of the QPL was appropriate, and affirmed the relevance of the included instructions and questions. “You've divided it into easy to digest paragraphs which makes it easier to read” [P7].

    The different topics in the booklet were described as being ‘really clearly defined’ [P4] and “It's good that it's [the questions] all in point [bullet] form” [P6]. All agreed that the language used throughout the QPL was easy to understand: “It's clearly and plainly written which I think will help a variety of people with a variety of literacy levels” [P4].

    Requests to improve the content of the QPL related to the inclusion of:

    • Information about disorders related to ADHD;

    • A list of the various medications available for ADHD and their effects;

    • A list of the types of HCPs that should be involved in ADHD management;

    • Information about ‘common misconceptions’ [P10] surrounding ADHD;

    • Contact details for ADHD support groups and websites.

    Some parents also requested the inclusion of positive affirmations and parenting tips, and a section about the long-term outcomes of children with ADHD.

    The authors chose not to include these items in the revised version of the QPL as they were viewed to potentially alter the purpose of the booklet from one which encourages parents to ask questions and obtain tailored responses to one which provides general information that may be misinterpreted by parents or irrelevant to their particular needs. The questions included in multiple sections of the QPL provide opportunities for parents to discuss these topics with their clinicians and to obtain the best advice for their child's particular situation.

    However, we included a question on the impact of diet on ADHD in the revised version.

    Order of information and user-friendliness

    The order of the information in the booklet was felt to be appropriate by all participants. Positive comments were also provided about the user-friendliness of the booklet, particularly the colour-coding, paper quality and the use of ring-binding to hold the booklet together.

    Parents suggested four key improvements to enhance the booklet's usability: (1) inclusion of a cover page for each topic; (2) inclusion of tabbed topic dividers; (3) addition of greater writing space and (4) change in the paper type to one with a more matte finish, “Not every pen would work on this paper” [P7].

    Revisions of original booklet

    Revisions were made to the booklet to address the four key points of information parents had difficulty locating as well as the suggestions provided in the qualitative data. The revised booklet was A5 in size (slightly larger than the original version) and 50 pages in length (vs 16 in the original)—selected pages of the original and revised versions of the booklet are presented in figures 1 and 2.

    Figure 1
    Figure 1

    Introductory page from the original version of the question prompt list (on left) and revised version (on right) following first round of testing. Key change displayed here is the addition of subheadings to break up the text and to aid navigation. ADHD, attention-deficit/hyperactivity disorder.

    Figure 2
    Figure 2

    New sections added to the revised version of the question prompt list following the first round of testing. The page on the left provides room for parents to personalise the booklet by including their child's name and a contact number. The page on the right is the ‘My Contacts’ section, which provides space for parents to include the contact details of the healthcare professionals involved in their child's care. ADHD, attention-deficit/hyperactivity disorder.

    Revisions of QPL content

    The overall structure of the booklet remained largely unchanged, however, some adjustments were made to the headings in the introductory section to address the trouble experienced by parents in locating information points in the first round of user-testing (specifically, points A, B and C, above). These changes are outlined in box 1.

    Box 1

    Revisions made to the question prompt list (QPL) content after round 1 of testing.

    • A. The main purpose of the booklet (Question 1).

      • We modified the heading “Why should I use this booklet?” to “How will this booklet help me?”

    • B. Using the booklet to prepare for an upcoming appointment (Question 3).

      • The section, “Using this booklet with your child's doctor” was divided with the following subheadings to help navigation:

        •   1. “Before your appointment”

        •   2. “During your appointment”

        •   3. “After your appointment”

    • C. Selecting which questions to ask the clinician (Question 6).

      • We modified the heading, “How should I use this booklet?” to “Which questions should I ask?”

    Other content changes

    • The font used for the subheadings in the treatment and future expectations topics was made bold to help distinguish the separate sections.

    • The booklet was made more personal by including a section at the beginning titled “This booklet belongs to…” where parents could write their name alongside their child's and include a contact number in case of loss of the booklet.

    • An additional section titled ‘My Contacts’ was added to the back of the booklet to allow parents to write down the contact details of their child's school and the various healthcare professionals involved in their child's care.

    • The addition of a question regarding the impact of diet on attention-deficit/hyperactivity disorder (ADHD) as per the participants’ requests. The question was “How does diet affect ADHD?” and was included under Topic 2, ‘Understanding ADHD’.

    Aesthetic modifications

    Aesthetic changes were made to enhance the user-friendliness of the booklet, help better differentiate the sections, and allow parents to navigate the booklet with greater ease (and to locate the response to point D above, (asking about obtaining a second medical opinion (Question 10))). These changes are outlined in box 2.

    Box 2

    Aesthetic revisions made to the question prompt list (QPL) after round 1 of testing.

    1. Section dividers

      • Overhanging tabbed section dividers were created for each of the QPL topics, also serving as a cover page for each topic.

      • The dividers were coloured in keeping with the colour- coding used in the initial booklet.

    2. Greater writing space

      • Two double-sided additional lined pages were provided at the end of each topic for the inclusion of further questions or notes by parents.

    3. Weight and finish of paper

      • Heavier weight paper was used for the cover of the booklet to enhance its durability.

      • Matte-based paper was used for the content pages of the booklet to allow for the use of different pens.

    Testing of revised information (round 2)

    Quantitative data

    The revised booklet was tested by a further 10 parents: 6 women and 4 men, aged between 31 and 53 years, with only 3 having obtained a tertiary level of education.

    Participants took an average of 7 min (range 3–14, median 5.5 min) to read the booklet, which was similar to round 1. The structured questionnaire was completed in an average of 21 min (range 15–30, median 20 min), again, similar to round 1. These results suggest that despite the increase in the overall thickness of the booklet during the second round, parents were able to navigate the booklet within the same timeframe.

    Table 1 shows that responses to all 15 of the structured user-testing questions were located and understood by at least 8 of the 10 participants. As this is the target set by the EU in medicine leaflet testing,49 we concluded the user-testing process at this stage (although further small changes were made based on participant feedback).

    Qualitative data

    Concept of a QPL booklet

    As in round 1, all parents expressed that they would use the booklet. Again, the QPL was met with very positive responses from participants who reiterated the importance of such tailored information resources being made available to them: “Sometimes you walk into the doctor's surgery, you're overwhelmed, you forget [things to ask], you walk out thinking…I didn't ask what I was supposed to” [P15]. “I would call it [the QPL] a confidence book…A question book is better [than a book of information] because it makes the parent think about things rather than being told how to do it, it allows the parent to use their own interpretations and their own initiative” [P13].

    The relevance of the QPL and its potential applications and benefits were also addressed by the parents: “There's a lot of questions in here that…I wouldn't have thought of…so it gives you that extra edge” [P12]. The QPL was seen as a resource that could also prove useful to family and friends: “If the parents and the child sit down and read it together…when they go to see the doctor, the child can ask the doctor some questions” [P13].

    Appearance and graphics

    The size of the QPL was viewed to be appropriate by all except two participants, who felt the QPL could be slightly smaller. However, all agreed that the font size used was appropriate.

    The colour scheme used and the booklet's aesthetic appearance received equally positive praise: “I love it because…it's not identifiable as [a resource for] ADHD. It looks like a diary, you know I want it to be discrete, and you've done that” [P15].

    The images used and the artistic effect previously described were well received by all except two: “I like the vaguery of the imagery. It's implying that the condition is still a bit unknown but it's not beyond help” [P13]; “It's the recognition that this could be anybody's child, boy or girl, all ages—it's wonderful” [P11].

    Content and language

    The appropriateness and relevance of the QPL content in addition to the newly added ‘This booklet belongs to’ page and the ‘My Contacts’ section was confirmed by all participants: “I like how I can put his [son's] name here, it becomes personal…I love this part [‘My Contacts’], I would be writing all of my contacts here” [P15].

    The language used throughout the QPL was, again, viewed to be straightforward and easy to understand. For the same reasons outlined following round 1, we decided not to include substantive information about ADHD, despite some requests for this, as the purpose of the booklet is to encourage question asking rather than providing general information that may not be appropriate or relevant to all users.

    Order of information and user-friendliness

    The results from the interviews revealed that all of the parents were happy with the order of the information, with one stating: “I like that you've gone through the process…really, from the beginning through to the future expectations as they [children] have gone through the years” [P11].

    All parents agreed the space provided for the addition of notes or further questions was excellent. The ring binding was described as being ‘sturdy and strong’ [P11] and enhanced the functionality of the booklet.

    Despite the increase in the thickness of the booklet compared with its initial tested format, the parents found the revised version to have great user-friendliness and the inclusion of the tabbed section dividers was particularly well-received.

    However, there was a request for greater contrast in the colours used to distinguish the different sections/topics. As this was only an aesthetic change to the revised version of the booklet and given that the EU targets for document testing were achieved in round 2, the authors deemed that a subsequent round of testing would not be required.

    Discussion

    Guidelines for producing written health materials and principles of good information design were adopted to inform the rigorous development of an ADHD-specific QPL (in booklet form) intended for use by parents and carers of children with ADHD. The QPL is intended to empower parents to ask questions during clinical consultations, thereby increasing their knowledge about ADHD and its treatments and enhancing the potential for SDM with clinicians. For the first time, user-testing methods were applied to evaluate the performance of the QPL with its intended users. In doing so, we were able to confirm: (1) that parents were able to locate and understand key questions as well as pieces of information within the booklet and (2) that the iterative process of user-testing led to the identification of weaknesses in the document and, consequently, to the development of an improved version of the QPL addressing these issues.

    To the best of our knowledge, this is the first demonstration of the utility of user-testing methods in assessing the performance and usability of any QPL. In a previous study involving development of a QPL for palliative care, the authors noted that a number of HCPs and an expert in consumer materials reviewed the QPL prior to its preliminary testing in a clinical environment.36 Although little detail was provided, the review process did not involve feedback from the intended users of the QPL and also appeared to be more focused on the relevance and appropriateness of the QPL content, rather than usability of the QPL. This was also the case in the study by Langbecker et al,37 which involved development of a QPL for patients with primary brain tumours. Their approach involved an iterative review process whereby the QPL was mailed to intended users and a telephone interview conducted a week later to ascertain areas of improvement. Based on the findings of the current study, we propose that user-testing may provide a more structured approach to not only ensuring the relevance of the QPL content, but also that the intended users of the document can actually be observed when locating and understanding the information they need. The mixed-methods approach afforded by user-testing also allows for greater insight into how the document performs by providing opportunities for qualitative feedback regarding its formatting, layout and usability.

    User-testing has been traditionally applied to evaluate the performance of written medicine information leaflets and booklets, and also to other forms of patient information.41–44 In the latter, more than 1 round of revisions to the document and subsequent testing were needed to reach the targets set by the EU for testing. Perhaps the key difference between these and the present study is that the first version of the QPL was designed by the research team using best practice principles of information design in the first instance, whereas previous studies have involved the testing of already published medicine information leaflets and booklets that may not have necessarily adhered to these guidelines. This reinforces the potential benefits associated with the revision of any drafted patient information in line with these guidelines, prior to testing. Only minimal changes were made to the layout, structure and formatting of the QPL as a result of the testing, further reinforcing the importance of these principles and guidelines for the production of written healthcare materials for consumers. It is also important to note that the actual content of the QPL, particularly the included questions, remained largely unaltered throughout the user-testing process. This is a testament to the rigorous process used in the generation of the questions and their validation by parents, consumer advocates, clinicians and researchers in our Delphi study (submitted for publication).

    The success of the user-testing process was demonstrated by the improvement in the ability of parents to locate and understand key information points following revisions to the original booklet. Perhaps most importantly, however, were the positive responses to the concept of the QPL as a resource, particularly that it would give parents confidence to play an active role during their child's clinical consultations. This positive response asserts the importance of previous work conducted by the research team in elucidating the information needs of parents of children with ADHD and reinforces the appropriateness of the QPL as a resource to assist them in meeting these needs.17

    To the best of our knowledge, this is the first ADHD-specific QPL to be developed and the first intervention targeting communication between parents of children with ADHD and their child's clinicians with the potential to enhance their capacity for SDM. This is particularly important in light of the ongoing controversies surrounding ADHD and parents’ consequent desire for clear and tailored information to assist with their treatment decision-making. Furthermore, given the recent interest in the development of programmes and interventions to afford patients greater opportunities for active involvement in treatment decisions, we believe this QPL is a well-timed and well-placed resource. This is especially relevant for ADHD, an area where parents as well as clinicians have been shown to view SDM favourably but, seemingly, no work has yet been conducted to assist in the realisation of this outcome.34 Therefore, the development and ultimate use of this ADHD-specific QPL in clinical environments may prove to be one of the first steps taken towards specifically addressing this void in the literature. As the QPL is anticipated to improve parents’ understanding about ADHD and its treatments, it may also serve to improve adherence to medications or other treatments agreed on with clinicians.

    The findings of this study should be considered in light of some limitations. We did not specifically enquire about or record the treatment histories of the participants’ children nor did we assess their level of ADHD-related knowledge or directly assess their health literacy levels, although the parents’ level of educational attainment was used as an indicator of their literacy. It is possible that parents’ familiarity with certain treatments and their ADHD-knowledge more generally may have influenced their ability to locate and understand certain pieces of information or questions. Furthermore, we chose to recruit parents or carers of children with a clinical diagnosis of ADHD to participate in this study, rather than parents without any experience related to the disorder. This decision was made to ensure that the booklet was being evaluated by parents with a lived experience related to ADHD and, in this way, that appropriate feedback could be obtained about the QPL.

    User-testing specifically looks at whether people can find and understand information within a document, and although it has benefits including its mixed-methods nature and small participant burden, it is limited by its outcomes. The method does not test the documents’ influence on treatment decision-making or long-term outcomes such as adherence to therapy, which require assessment in future work. It is important that the usability, acceptability and impact of this QPL are evaluated during clinical consultations between parents and their child's clinicians. We are currently evaluating the use of the QPL in such settings. Pending the outcomes of this study, we anticipate that there may be potential for broader roll-out of the resource and its integration as part of routine clinical care for these families. While we expect the QPL to be of benefit to families regardless of what stage they are at with their child's ADHD, it is likely that it will be of particular use to those families who are seeking medical advice regarding a potential ADHD diagnosis. To ensure that these families are able to access the QPL as early as possible, copies of the resource will be distributed to primary care physicians (eg, general practitioners) for provision to families being referred to specialists (eg, developmental paediatricians). Copies will also be distributed to specialists’ clinics to facilitate access to the resource for families who have already received an ADHD diagnosis. The timing of QPL provision in these clinics would ultimately be at the clinicians’ discretion but may be useful in situations where families are likely to have new concerns and issues to discuss, for example, before starting a trial of pharmacotherapy or as the child is approaching adolescence. To increase the practicality for roll-out, the QPL will also be uploaded to the Internet in a printer-friendly format that can be downloaded by interested parents and clinicians.

    Conclusions

    Guidelines for producing written healthcare materials were used to inform the design of an ADHD-specific QPL booklet intended for use by parents of children with ADHD. This, coupled with the novel application of user-testing methods to determine the performance of the QPL, ultimately resulted in the development of a highly relevant, easy to understand and user-friendly resource. User-testing may provide a more structured and rigorous approach to testing the performance of future QPLs or written healthcare materials other than written medicine information. The QPL itself is the first intervention targeted at addressing parents’ unmet information needs about ADHD and its treatments. This resource has the potential to empower parents’ treatment decisions and to enhance the potential for SDM during clinical consultations.

    References

    1. ↵
      1. Warikoo N,
      2. Faraone SV
      . Background, clinical features and treatment of attention deficit hyperactivity disorder in children. Expert Opin Pharmacother 2013;14:1885–906. doi:10.1517/14656566.2013.818977
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Arnsten AF
      . Fundamentals of attention-deficit/hyperactivity disorder: circuits and pathways. J Clin Psychiatry 2006;67(Suppl 8):7–12. doi:10.4088/JCP.v67n0102
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Biederman J,
      2. Spencer T
      . Attention-deficit/hyperactivity disorder (ADHD) as a noradrenergic disorder. Biol Psychiatry 1999;46:1234–42. doi:10.1016/S0006-3223(99)00192-4
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Wilens TE,
      2. Biederman J,
      3. Spencer TJ
      . Attention deficit/hyperactivity disorder across the lifespan. Annu Rev Med 2002;53:113–31. doi:10.1146/annurev.med.53.082901.103945
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Wilens TE
      . Effects of methylphenidate on the catecholaminergic system in attention-deficit/hyperactivity disorder. J Clin Psychopharmacol 2008;28:S46–53. doi:10.1097/JCP.0b013e318173312f
      OpenUrlCrossRefPubMed
    6. ↵
      1. del Campo N,
      2. Chamberlain SR,
      3. Sahakian BJ, et al
      . The roles of dopamine and noradrenaline in the pathophysiology and treatment of attention-deficit/hyperactivity disorder. Biol Psychiatry 2011;69:e145–57. doi:10.1016/j.biopsych.2011.02.036
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Pescosolido BA,
      2. Jensen PS,
      3. Martin JK, et al
      . Public knowledge and assessment of child mental health problems: findings from the national stigma study-children. J Am Acad Child Adolesc Psychiatry 2008;47:339–49. doi:10.1097/CHI.0b013e318160e3a0
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Mayes R,
      2. Bagwell C,
      3. Erkulwater J
      . ADHD and the rise in stimulant use among children. Harv Rev Psychiatry 2008;16:151–66. doi:10.1080/10673220802167782
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Radomsky M
      . Kids on speed? [television broadcast]. Sydney: Australian Broadcasting Corporation, 2014.
    10. ↵
      1. Boyle CA,
      2. Boulet S,
      3. Schieve LA, et al
      . Trends in the prevalence of developmental disabilities in US children, 1997–2008. Pediatrics 2011;127:1034–42. doi:10.1542/peds.2010-2989
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Moynihan R,
      2. Doust J,
      3. Henry D
      . Preventing overdiagnosis: how to stop harming the healthy. BMJ 2012;344:e3502. doi:10.1136/bmj.e3502
      OpenUrlFREE Full Text
    12. ↵
      1. Frances A
      . The first draft of DSM-V. BMJ 2010;340:c1168. doi:10.1136/bmj.c1168
      OpenUrlFREE Full Text
    13. ↵
      1. Ghanizadeh A
      . Agreement between Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, and the proposed DSM-V attention deficit hyperactivity disorder diagnostic criteria: an exploratory study. Compr Psychiatry 2013;54:7–10. doi:10.1016/j.comppsych.2012.06.001
      OpenUrl
    14. ↵
      1. Batstra L,
      2. Frances A
      . DSM-5 further inflates attention deficit hyperactivity disorder. J Nerv Ment Dis 2012;200:486–8. doi:10.1097/NMD.0b013e318257c4b6
      OpenUrlPubMed
    15. ↵
      1. Brinkman WB,
      2. Sherman SN,
      3. Zmitrovich AR, et al
      . Parental angst making and revisiting decisions about treatment of attention-deficit/hyperactivity disorder. Pediatrics 2009;124:580–9. doi:10.1542/peds.2008-2569
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Hansen DL,
      2. Hansen EH
      . Caught in a balancing act: Parents’ dilemmas regarding their ADHD child's treatment with stimulant medication. Qual Health Res 2006;16:1267–85. doi:10.1177/1049732306292543
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Ahmed R,
      2. Borst JM,
      3. Yong CW, et al
      . Do parents of children with attention-deficit/hyperactivity disorder (ADHD) receive adequate information about the disorder and its treatments? A qualitative investigation. Patient Prefer Adherence 2014;8:661–70. doi:10.2147/PPA.S60164
      OpenUrl
    18. ↵
      1. Ahmed R,
      2. Aslani P
      . Attention-deficit/hyperactivity disorder: an update on medication adherence and persistence in children, adolescents and adults. Expert Rev Pharmacoecon Outcomes Res 2013;13:791–815. doi:10.1586/14737167.2013.841544
      OpenUrl
    19. ↵
      1. Narayan S,
      2. Hay J
      . Cost effectiveness of methylphenidate versus AMP/DEX mixed salts for the first-line treatment of ADHD. Expert Rev Pharmacoecon Outcomes Res 2004;4:625–34. doi:10.1586/14737167.4.6.625
      OpenUrlPubMed
    20. ↵
      1. Chacko A,
      2. Newcorn JH,
      3. Feirsen N, et al
      . Improving medication adherence in chronic pediatric health conditions: a focus on ADHD in youth. Curr Pharm Des 2010;16:2416–23. doi:10.2174/138161210791959908
      OpenUrlPubMed
    21. ↵
      1. Clay D,
      2. Farris K,
      3. McCarthy AM, et al
      . Family perceptions of medication administration at school: errors, risk factors, and consequences. J Sch Nurs 2008;24:95–102. doi:10.1177/10598405080240020801
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Ahmed R,
      2. Borst J,
      3. Wei YC, et al
      . Parents’ perspectives about factors influencing adherence to pharmacotherapy for ADHD. J Atten Disord 2013. doi:10.1177/1087054713499231.
    23. ↵
      1. Ahmed R,
      2. McCaffery KJ,
      3. Aslani P
      . Factors influencing parental decision making about stimulant treatment for attention-deficit/hyperactivity disorder. J Child Adolesc Psychopharmacol 2013;23:163–78. doi:10.1089/cap.2012.0087
      OpenUrl
    24. ↵
      1. Sciberras E,
      2. Iyer S,
      3. Efron D, et al
      . Information needs of parents of children with attention-deficit/hyperactivity disorder. Clin Pediatr 2010;49:150–7. doi:10.1177/0009922809346730
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Hummelinck A,
      2. Pollock K
      . Parents’ information needs about the treatment of their chronically ill child: a qualitative study. Patient Educ Couns 2006;62:228–34. doi:10.1016/j.pec.2005.07.006
      OpenUrlCrossRefPubMed
    26. ↵
      1. Coulter A,
      2. Entwistle V,
      3. Gilbert D
      . Sharing decisions with patients: is the information good enough? BMJ 1999;318:318–22. doi:10.1136/bmj.318.7179.318
      OpenUrlFREE Full Text
    27. ↵
      1. Charles C,
      2. Gafni A,
      3. Whelan T
      . Decision-making in the physician–patient encounter: revisiting the shared treatment decision-making model. Soc Sci Med 1999;49:651–61. doi:10.1016/S0277-9536(99)00145-8
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Joosten EAG,
      2. DeFuentes-Merillas L,
      3. de Weert GH, et al
      . Systematic review of the effects of shared decision-making on patient satisfaction, treatment adherence and health status. Psychother Psychosom 2008;77:219–26. doi:10.1159/000126073
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Charles C,
      2. Gafni A,
      3. Whelan T
      . Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med 1997;44:681–92. doi:10.1016/S0277-9536(96)00221-3
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Drotar DC,
      2. Crawford P,
      3. Bonner M
      . Collaborative decision-making and promoting treatment adherence in pediatric chronic illness. Patient Intell 2010;2:1–7. doi:10.2147/PI.S8820
      OpenUrl
    31. ↵
      1. Wolraich M,
      2. Brown L,
      3. Brown RT, et al
      . ADHD: clinical practice guideline for the diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Pediatrics 2011;128:1007–22. doi:10.1542/peds.2011-2654
      OpenUrlAbstract/FREE Full Text
    32. ↵
      (CADDRA) CADHDRA. Canadian ADHD practice guidelines. 3rd edn. Toronto: CADDRA, 2011.
    33. ↵
      National Health and Medical Research Council. Clinical practice points on the diagnosis, assessment and management of attention deficit hyperactivity disorder in children and adolescents. Canberra, Australia: Australian Government, 2012.
    34. ↵
      1. Fiks AG,
      2. Hughes CC,
      3. Gafen A, et al
      . Contrasting parents’ and pediatricians’ perspectives on shared decision-making in ADHD. Pediatrics 2011;127:e188–96. doi:10.1542/peds.2010-1510
      OpenUrlAbstract/FREE Full Text
    35. ↵
      1. Clayton JM,
      2. Butow PN,
      3. Tattersall MH, et al
      . Randomized controlled trial of a prompt list to help advanced cancer patients and their caregivers to ask questions about prognosis and end-of-life care. J Clin Oncol 2007;25:715–23. doi:10.1200/JCO.2006.06.7827
      OpenUrlAbstract/FREE Full Text
    36. ↵
      1. Clayton J,
      2. Butow P,
      3. Tattersall M, et al
      . Asking questions can help: development and preliminary evaluation of a question prompt list for palliative care patients. Br J Cancer 2003;89:2069–77. doi:10.1038/sj.bjc.6601380
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Langbecker D,
      2. Janda M,
      3. Yates P
      . Development and piloting of a brain tumour-specific question prompt list. Eur J Cancer Care 2012;21:517–26. doi:10.1111/j.1365-2354.2012.01328.x
      OpenUrl
    38. ↵
      Centers for Disease Control and Prevention. Simply put: a guide for creating easy-to-understand materials. Atlanta, Georgia: Centers for Disease Control and Prevention, 2009.
    39. ↵
      1. Jay E,
      2. Aslani P,
      3. Raynor D
      . User testing of consumer medicine information in Australia. Health Educ J 2011;70:420–7. doi:10.1177/0017896910376131
      OpenUrlAbstract/FREE Full Text
    40. ↵
      1. Sless D,
      2. Shrensky R
      . Writing about medicines for people: usability guidelines for consumer product information: Australian Self-Medication Industry, Incorporated. 2007.
    41. ↵
      1. Knapp P,
      2. Raynor DK,
      3. Silcock J, et al
      . Performance-based readability testing of participant materials for a phase I trial: TGN1412. J Med Ethics 2009;35:573–8. doi:10.1136/jme.2008.026708
      OpenUrlAbstract/FREE Full Text
    42. ↵
      1. Knapp P,
      2. Raynor DK,
      3. Silcock J, et al
      . Can user testing of a clinical trial patient information sheet make it fit-for-purpose? A randomized controlled trial. BMC Med 2011;9:89. doi:10.1186/1741-7015-9-89
      OpenUrlCrossRefPubMed
    43. ↵
      1. Knapp P,
      2. Raynor DK,
      3. Silcock J, et al
      . Performance-based readability testing of participant information for a Phase 3 IVF trial. Trials 2009;10:79. doi:10.1186/1745-6215-10-79
      OpenUrlCrossRefPubMed
    44. ↵
      1. Knapp P,
      2. Wanklyn P,
      3. Raynor DK, et al
      . Developing and testing a patient information booklet for thrombolysis used in acute stroke. Int J Pharm Pract 2010;18:362–9. doi:10.1111/j.2042-7174.2010.00060.x
      OpenUrlPubMed
    45. ↵
      1. Harris E,
      2. Enright D
      . New words for cautionary and advisory labels make them easily understood. Pharm J 2011;286:278–9.
      OpenUrl
    46. ↵
      1. Dale E,
      2. Chall JS
      . A formula for predicting readability. Educ Res Bull 1948;27:11–28.
      OpenUrl
    47. ↵
      1. Laughlin GHM
      . SMOG grading—a new readability formula. J Read 1969;12:639–46.
      OpenUrlWeb of Science
    48. ↵
      1. Raynor DK
      . User testing in developing patient medication information in Europe. Res Social Adm Pharm 2013;9:640–5. doi:10.1016/j.sapharm.2013.02.007
      OpenUrl
    49. ↵
      1. Raynor DK,
      2. Knapp P,
      3. Silcock J, et al
      . “User-testing” as a method for testing the fitness-for-purpose of written medicine information. Patient Educ Couns 2011;83:404–10. doi:10.1016/j.pec.2011.03.016
      OpenUrlCrossRefPubMed
    50. ↵
      1. Kelly M
      . The role of theory in qualitative health research. Fam Pract 2010;27:285–90. doi:10.1093/fampra/cmp077
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Contributors All authors contributed to the study design. RA conducted all user-testing interviews and, with the assistance of PA, analysed the data. RA wrote the manuscript, which was critically reviewed by DKR, KJM and PA.

    • Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests DKR is the co-founder and academic advisor for Luto Research Ltd, a company that provides performance-based health information testing services.

    • Ethics approval The Human Research Ethics Committee at The University of Sydney.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.