You are here

Article Text

Article menu

Download PDFPDF

XML
Health informatics
Protocol
STANDING Collaboration: a study protocol for developing clinical standards
  1. Louise K Wiles1,2,
  2. Peter D Hibbert1,2,3,4,5,
  3. Jacqueline H Stephens1,
  4. Enrico Coiera5,
  5. Johanna Westbrook3,
  6. Jeffrey Braithwaite2,
  7. Ric O Day6,
  8. Ken M Hillman7,8,
  9. William B Runciman1,4
  1. 1 Patient Safety and Healthcare Human Factors, Centre for Population Health Research, School of Health Sciences, Sansom Institute for Health Research, University of South Australia, Adelaide, South Australia, Australia
  2. 2 Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, New South Wales, Australia
  3. 3 Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, New South Wales, Australia
  4. 4 Australian Patient Safety Foundation, Adelaide, South Australia, Australia
  5. 5 Centre for Health Informatics, Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, New South Wales, Australia
  6. 6 St Vincent’s Clinical School, University of New South Wales, Kensington, New South Wales, Australia
  7. 7 The Simpson Centre for Health Services Research, South Western Sydney Clinical School, The University of New South Wales, Kensington, New South Wales, Australia
  8. 8 Intensive Care Unit, Liverpool Hospital, Liverpool, New South Wales, Australia
  1. Correspondence to Professor William B Runciman; william.runciman{at}unisa.edu.au

Abstract

Introduction Despite widespread availability of clinical practice guidelines (CPGs), considerable gaps continue between the care that is recommended (‘appropriate care’) and the care provided. Problems with current CPGs are commonly cited as barriers to providing ’appropriate care'.

Our study aims to develop and test an alternative method to keep CPGs accessible and up to date. This method aims to mitigate existing problems by using a single process to develop clinical standards (embodied in clinical indicators) collaboratively with researchers, healthcare professionals, patients and consumers. A transparent and inclusive online curated (purpose-designed, custom-built, wiki-type) system will use an ongoing and iterative documentation process to facilitate synthesis of up-to-date information and make available its provenance. All participants are required to declare conflicts of interest. This protocol describes three phases: engagement of relevant stakeholders; design of a process to develop clinical standards (embodied in indicators) for ‘appropriate care’ for common medical conditions; and evaluation of our processes, products and feasibility.

Methods and analysis A modified e-Delphi process will be used to gain consensus on ‘appropriate care’ for a range of common medical conditions. Clinical standards and indicators will be developed through searches of national and international guidelines, and formulated with explicit criteria for inclusion, exclusion, time frame and setting. Healthcare professionals and consumers will review the indicators via the wiki-based modified e-Delphi process. Reviewers will declare conflicts of interest which will be recorded and managed according to an established protocol. The provenance of all indicators and suggestions included or excluded will be logged from indicator inception to finalisation. A mixed-methods formative evaluation of our research methodology will be undertaken.

Ethics and dissemination Human Research Ethics Committee approval has been received from the University of South Australia. We will submit the results of the study to relevant journals and offer national and international presentations.

  • quality indicators
  • healthcare
  • practice guidelines as topic
  • consensus development conferences as topic
  • quality assurance

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2016-014048

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • We will develop and evaluate a method for generating and ratifying clinical standards and indicators of ‘appropriate care’ for common health conditions which has been designed to overcome deficiencies in current methods.

    • This study will obtain expert consensus on ‘appropriate care,’ underpinned by evidence, for a range of common medical conditions.

    • The recruitment of healthcare professionals, patients and consumers to review clinical indicators may introduce selection biases.

    • The use of English language clinical practice guidelines may not be representative of all available evidence, and limits the generalisability of study findings.

    Introduction

    In Australia, ‘appropriate care’ (care in line with evidence or consensus-based guidelines) is provided to adults, on average, only 57% of the time, with large variations across common medical conditions and providers.1 Problems with clinical practice guidelines (CPGs), standards and indicators (see box 1 for definitions) are commonly cited as one of the barriers to providing appropriate care2; these include large numbers of repositories and guidelines; duplication and overlap among guidelines; differing recommendations for care practices; lack of currency; inconsistent structure and content; voluminous documents which are not easy to assimilate or use3–5; and recommendations which are often vague and difficult to measure.2 6 7 In addition, most CPGs lack detail of how evidence was interpreted and weighted to formulate recommendations, offer little opportunity for end-users to provide formal feedback8 and have been developed by people with (often undisclosed) professional or commercial conflicts of interest (COIs).2 6 9

    Box 1

    Definitions for clinical practice guideline, standard, indicator and tool.

    A clinical practice guideline:

    ‘Statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.10 11

    A clinical standard2:

    • is an agreed process that should be undertaken or an outcome that should be achieved for a particular circumstance, symptom, sign or diagnosis (or a defined combination of these)

    • should be evidence based, specific, feasible to apply, easy and unambiguous to measure, and produce a clinical benefit and/or improve the safety and/or quality of care, at least at the population level.

    If a standard cannot or should not be complied with, the reason/s should be briefly stated.

     A clinical indicator2:

    • describes a measurable component of the standard, with explicit criteria for inclusion, exclusion, time frame and setting.

    A clinical tool2 6 12–14:

    • should implicitly or explicitly incorporate a standard or a component of a standard

    • should constitute a guide to care that facilitates compliance with the standard

    • should be easy to audit, preferably electronically, to provide feedback

    • should be able to be incorporated into workflows and medical records.

    Implementability of CPGs is a key factor affecting their perceived utility and uptake11 15; in response, international efforts are being directed at developing clinical standards and indicators to identify evidence and service delivery gaps and areas for improvement, and understand and measure the quality of care provided.16 17 Emerging schools of thought suggest that ‘appropriate care’ may be enhanced through greater patient (health consumer) engagement.12–14 18 This could be facilitated by involving patients and interested laypeople as well as healthcare professionals (HCPs) in CPG development,6 19 20 and using online technologies to enhance transparency, accessibility and currency of both content and development processes.6 21

    The strategies employed in this protocol aim to mitigate problems with existing CPG development processes (table 1) by adopting a single approach to avoid duplication, using an ongoing and iterative documentation process to facilitate transparent synthesis of up-to-date information and make its provenance accessible, and requiring all participants to declare their COIs.2 Indicators will be developed using selection criteria to reflect ‘essential’ clinical practice and be expressed one concept at a time in plain English to help create standards that are clear, concise, measurable and easy to use.

    View this table:
    Table 1

    Key aspects of the STANDING Collaboration methodology, designed to mitigate issues with current CPG development

    The aims of the STANDING Collaboration study are to (1) provide proof of concept for an alternative method for creating sets of nationally-agreed evidence-based standards and clinical indicators, and (2) obtain consensus on ‘appropriate care’ for a range of common medical conditions. To do so, we will use a three-phase approach to engage relevant stakeholders, develop clinical indicators representative of ‘appropriate care’ (which constitute the standard) for a range of common conditions, and evaluate our processes, products and feasibility. We plan to develop an inclusive, transparent, collaborative process, which allows HCPs and patients or consumers to develop and keep up-to-date clinical standards comprising indicators with defined attributes, using an online curated wiki-based platform to facilitate ongoing review and updating of the standard, or individual indicators, as soon as new evidence emerges. In this study, the term ‘wiki’ refers to an interactive information management system which will allow users (eg, HCPs and patients) to collaborate directly in formulating and refining indicators that are relevant to their clinical practice and lived experience.21 22 The source and provenance of each indicator, including all suggestions, will be posted online and updated as necessary.

    Methods and analysis

    Our three-phase approach (figure 1) comprises:

    Figure 1
    Figure 1

    Overview of STANDING Collaboration research methodology and definitions for participant groups. CPG, clinical practice guideline.

    1. stakeholder analysis;

    2. development and test of a process for creating clinical indicators representative of ‘appropriate care’ for a range of common conditions;

    3. evaluation of processes, products and feasibility.

    Phase 1

    In order to gain an understanding of potential barriers, facilitators and the overall feasibility of the STANDING Collaboration methodology, stakeholder perspectives will be captured through a series of semistructured qualitative telephone interviews. HCPs and consumers will be invited to participate. Relevant medical colleges, professional and consumer associations and networks will be contacted, using publicly available information, to request assistance with the recruitment of interview participants. Invitations will comprise email notifications to members and media releases and articles within newsletters, asking potential participants to contact the Research Team members. The telephone interviews will be conducted at a time convenient to the research team member and the participant. Based on the sample sizes reported in similar stakeholder analyses,23 24 we anticipate conducting approximately 18–25 interviews in total (9–13 interview participants per stakeholder type), or until saturation is reached.

    Using guidelines for stakeholder analyses25 and the schedules from previous qualitative research as a guide,19 24 a range of topics will be explored in the interviews (table 2).

    View this table:
    Table 2

    Topics (and their rationale) for exploration in the qualitative stakeholder interviews

    Interviews will be recorded with consent and transcribed by a professional transcription company. Transcripts and summaries will be returned to participants for the purpose of making comments or corrections and providing feedback on the findings.26 Content analyses will be used to derive common themes. We will use open coding and inductive reasoning with two coders to group similar responses into categories and assign labels capturing specific themes. Any discrepancies will be discussed among Research Team members. After agreement is reached on the composition and label for each category, we will assign (axial coding) categories to the central phenomena of interest (table 2).27 28

    Phase 2

    Clinical indicators will be developed for individual conditions using a four-stage process:

    1. source, select and search relevant CPGs;

    2. extract all concepts from each CPG together with the relevant text in which they appear (original recommendation), and tabulate common concepts to select, draft and format the proposed clinical indicators based on identified concepts;

    3. review the indicators internally;

    4. review the indicators externally.

    Stage I: source, select and search relevant CPGs

    Interview data from the stakeholder analysis (phase 1) will be used in conjunction with national health priority areas, burden of disease and prevalence data to identify candidate conditions for clinical standard and indicator development.1 7 Clinical indicators will be drawn initially from the latest CPGs. A systematic search will be undertaken of national-level Australian CPGs endorsed by the National Health and Medical Research Council, and international-level guidelines from the National Institute for Health and Care Excellence in England, Scottish Intercollegiate Guidelines Network, and the Agency for Healthcare Research and Quality’s National Guideline Clearinghouse in the USA, and the Guidelines International Network. In the absence of Australian national or international CPGs, relevant professional medical college and association CPGs may also be searched, as well as those published at state or professional level and in international journals. Details of the search strategy are provided in appendix A, online supplementary file 1.29 30 In order to describe the quality of the evidence sources from which our clinical indicators will be derived, two members of the Research Team will independently appraise CPGs using the AGREE-II (Appraisal of Guidelines for Research and Evaluation II) tool.31–34

    supplementary appendix A

    Stage II: select, draft and format proposed indicators

    Recommendations (and their key underlying concepts) from each CPG will be collated and used to inform the content of the proposed clinical indicators. Not all recommendations published in CPGs will become indicators. Recommendations will be flagged for potential exclusion based on the following criteria:

    • strength of the wording of the recommendation (ie, ’may' and ’could' statements would be excluded; ’should' and ’must' statements would be included)

    • vague guiding or aspirational statements and those without recommended actions

    • conflicting recommendations from less recent CPGs and those with lower AGREE-II scores.

    All clinical indicators will be written in plain English, one concept at a time, using a structured and standardised format (eg, commencing with the inclusion criteria followed by the compliance action)7 (table 3). For each condition, indicators will be arranged according to phases of care (ie, screening, diagnosis, assessment, acute care, ongoing care) so that together, they constitute a clinical standard amendable to inclusion as a clinical tool over the patient journey.

    View this table:
    Table 3

    Examples of clinical indicators from CareTrack Australia 1 that were written in plain English, one concept at a time using a structured and standardised format

    Stages III and IV: iterative review using a modified e-Delphi approach

    Stages III and IV will involve online wiki-based reviews of proposed clinical indicators; initially by way of an internal review between the Research Team and Curator Group (stage III), followed by an external review by the Curator Group and HCPs and consumers who choose to register to the site (Wiki Registrants) and provide feedback on the proposed clinical indicators.7 This approach has been chosen to facilitate the number and spread of individuals’ perspectives obtained and avoid domination of the consensus process by one or a few participants, and to optimise face validity of the final set of clinical standards and indicators.7 35 All STANDING Collaboration participants (Research Team, Clinical Champion, Curator Group members, and Wiki Registrants) will be required to complete a COI declaration, which will be taken into consideration when accepting or rejecting suggestions, and logged with the provenance of each indicator for transparency.9 36

    Stage III: internal review processes

    Internal reviews will first be conducted within our Research Team, and subsequently by our Clinical Champion and Curator Group members who will comprise a mix of at least two members of the following: clinicians (eg, general practitioners, medical specialists, allied health professionals, nurses), researchers, policymakers or public health specialists or healthcare quality improvement experts, and consumers. Depending on their self-reported scope of practice, expertise and interest, Curator Group members (including consumers) may be able to participate in review panels for more than one condition. The total number of invitations to potential Curator Group members will depend on the skill mix of those invited and the overall pattern of recruitment. This selection strategy is supported within the Delphi process literature with studies using similar criteria to choose potential participants (eg, renown, member of an organisation, recommendation, years of experience, willingness to participate, availability, interest).7 35

    The internal review will consist of a maximum of three rounds to allow sufficient testing of the subsequent phase of the process (ie, external review). In the first round, drafts of proposed clinical indicators, and the recommendations on which they are based, will be sent via email to the Curator Group members. The review criteria to be used are based on the methods from previous studies for developing and measuring indicators of appropriate care.1 7 16 37–39 Curator Group members will be asked to: recommend indicators for inclusion (with or without amendments) or exclusion, provide comments in relation to three key criteria: evidence, feasibility and importance (appendix B, online supplementary file 2) and make additional suggestions (with supporting material). In addition, Research Team members will pose specific questions to the Curator Group members about individual indicators to highlight inconclusive or conflicting CPG recommendations, or to clarify definitions for inclusion criteria and compliance actions. In particular, consumer members of the Curator Group will be asked to vet the plain English wording of clinical indicators and a linked glossary of terms to ensure that content is appropriately targeted to the consumer audience. In this round, Curator Group members will complete their assignments independently to minimise ‘group-think.’40 41 Research Team members (WBR, PDH, LKW, JHS) will collate the feedback and revise the content, structure and format of each indicator. The refined set of indicators (including the original indicators and any feedback and suggestions) will be sent to the same Curator Group members for a second round of scoring. The same approach will be used in the second round, with a request for further refinement and identification of indicators to be included or excluded. If necessary, Curator Group members will discuss the proposed set of indicators via a third round teleconference, with a view to achieving consensus and approving the indicators for the external online wiki-based review process.

    supplementary appendix B

    Stage IV: external review processes

    External reviews will be conducted by HCPs and consumers who have registered to this wiki as reviewers (Wiki Registrants). Relevant medical colleges, professional and consumer associations and networks will be contacted to request assistance with the identification of potential clinical indicator reviewers. Invitations will be by email, media releases and articles within newsletters. HCPs and consumers will self-nominate as reviewers for one or more of the STANDING Collaboration conditions based on their interests, scope of practice and experience.35 42 Wiki Registrants for this process will be required to declare their COIs, which will be taken into account by the Clinical Champion and Curator Group when considering reviewers’ feedback on the indicators.9 36

    The external review will involve an interactive wiki-based process where indicators for each condition from round 3 of the internal review will be posted on an online wiki site. A software development company will be engaged to purpose design and custom build the wiki for this project. The wiki ‘live’ time for each version will depend on the recruitment rate of reviewers and the progress of their reviews, but is anticipated to be no longer than 3 months per round. Reviewers will provide comments on indicators in relation to the three key criteria: evidence, feasibility and importance (appendix B, online supplementary file 2), make recommendations (ie, inclusion, inclusion with amendments, exclusion, hold) and be able to suggest edits in real time. The Clinical Champion and Curator Group for each condition will follow-up and manage external reviewers’ responses, and make final recommendations for that version regarding the inclusion, content, structure and format of indicators. The Clinical Champion and Curator Group will use supporting references when considering and responding to each suggestion related to whether and why they have been included or rejected. In addition, all external reviewers’ comments and recommendations will be logged, classified and presented in subsequent rounds according to whether and why they have or have not been incorporated into the next iteration. This will allow tracking of the evolution of the standards and indicators from the original recommendations on which they were based to their final iteration, as well as the nature and influence of review feedback in shaping the standard. Once the indicators are ‘stable’ with no further significant changes being suggested, that version of the standard will be published as comprising a set of clinical indicators that represents ’appropriate care' for Australians with the candidate conditions at that time. Endorsement will then be sought by relevant professional bodies and consumer organisations. For each medical condition that has undergone indicator development via the STANDING Collaboration process, it will be possible for evidence to be monitored by the Curator Group (or a subgroup comprised of key members of the Curator Group) in order to update standards and indicators as necessary. For each condition, our initial monitoring plan involves using information from automated database searches and feedback from the wiki to initially update indicators every three months and, once stable, at a minimum of every six months.

    Phase 3

    A multimethods evaluation of the process and products of phases 1 and 2 of our research methodology will be undertaken (table 4). Three data sources will be used to inform the evaluation: (A) engagement and utilisation statistics sourced from the wiki logs—these will include demographics of users and rates, times and nature of use; (B) the nature and content of Wiki Registrant, Curator Group and Clinical Champion comments (eg, the format and rationale of proposed changes to indicators, level of agreement between reviewers and resulting changes to the indicators); and (C) all users and stakeholders’ perspectives on the process, usability and appropriateness of the vehicle for developing the standards (ie, the wiki) as well as the acceptance and utility of the final sets of standards and indicators themselves.

    View this table:
    Table 4

    Description of the phase 3 evaluation of phase 1 and phase 2

    Participants will be invited to provide feedback regarding their experiences and perspectives via one of three data collection methods: (1) online user perspectives survey (for external reviewers); (2) semistructured interviews (for Review Panel members and the Clinical Champion); and (3) interviews and focus groups (for phase 1 stakeholders, following publication of the sets of indicators) (table 4). Both quantitative and qualitative analyses of these three data sources will be undertaken including: descriptive statistics for the characteristics of wiki users, their engagement and patterns of use, frequency counts and content analyses of ratings and free-text responses from the online user perspectives survey, semistructured interviews and focus groups (table 4). Using these data, recommendations will be developed regarding the overall feasibility of the wiki process for future indicator development. The next phase of the research is to study the implementation of the standards as a translational tool for clinical practice and ongoing audit.

    Ethics and dissemination

    Ethical approval

    Human Research Ethics Committee approval has been granted from the University of South Australia (protocol number 0000035183). All STANDING Collaboration participants and reviewers will be required to give informed consent (for their chosen study phase(s)) and complete a COI declaration prior to participation, which will be recorded and managed according to an established protocol (appendix c, online supplementary file 3).43–46

    supplementary appendix C

    Dissemination

    We will submit the results of the study to relevant national and international journals with the intention of publishing the results widely. As well, we will make national and international presentations to stakeholder groups including those involving patients, researchers, clinicians, managers and policymakers.

    Discussion

    Notwithstanding the large number of CPGs currently available, delivery of ‘appropriate’ healthcare in Australia and internationally is highly variable and leaves considerable room for improvement. A number of major difficulties have been identified with current CPGs and their development. Our alternative approach for keeping evidence accessible and up to date has been designed to mitigate problems with existing processes. Here, we describe a protocol for developing and testing a process for creating clinical standards, embodied in clinical indicators.

    This process has been designed to systematically address many of the problems identified with current CPGs and their development. The approach is characterised by being inclusive (HCPs, researchers and consumers), transparent (all reviewers’ suggestions are logged, with their provenance, as accepted or rejected by the Clinical Champion and Curator Group), up to date (revisions will be ongoing after a version is published), easy to use (one concept per indicator), written in plain English and able to be integrated into the sequence of work flow in managing a condition.2 29 30

    Findings from this study will be used to inform the design of future studies using Delphi processes to establish consensus on recommended healthcare, and will be relevant for national and international researchers, policymakers, healthcare practitioners and patients. Specifically, there is potential for standards and indicators developed using this methodology to be assembled to comprise the content of electronic tools for the basic care of common conditions (ie, reflects ‘essential’ Australian clinical practice; appendix B, online supplementary file 2). It is envisaged that the clinical tool (box 1) will2:

    1. implicitly or explicitly enunciate the clinical standard for the basic care of the condition in question

    2. inform HCPs, patients and carers about that condition

    3. guide care

    4. document what care has been offered and what has not (and why)

    5. be amenable to audit (preferably automated) so that feedback can be provided—at clinical indicator, patient, provider, facility and eventually population levels.

    We recognise several limitations to our study. The inclusion of only English language CPGs, for pragmatic purposes and contextual consistency, may not be representative of all available evidence and limits the generalisability of our findings. Reviewers will be invited to participate and self-nominate for conditions that are within their scope of practice, interest or experience (ie, healthcare providers and consumers) which introduces a selection bias. Clinical indicators will be developed from recommendations in existing CPGs, which means there is potential for problems with current CPG development processes to contaminate our final sets of indicators and standards. The aim of this study is to provide proof of concept and test a new methodology. Our intention is that the key approaches and characteristics of the STANDING Collaboration clinical standard and indicator development process (ie, governance structure, transparency, HCP and consumer engagement and codesign, access to provenance of both accepted and rejected suggestions, and use of online technologies to facilitate keeping the indicators up to date) will be universally applicable, and be able to be tailored to other healthcare settings and structures.30 47 48 We aim to ameliorate these limitations by adopting a collaborative user-centred approach where feedback from both consumer and HCP groups is sought, communicated transparently and incorporated into guidance for others who wish to develop clinical standards and indicators.6

    References

    1. 1.↵
      2. Runciman WB ,
      3. Hunt TD ,
      4. Hannaford NA , et al
      . CareTrack: assessing the appropriateness of health care delivery in Australia. Med J Aust 2012;197:100–5.doi:10.5694/mja12.10510
      OpenUrlCrossRefPubMedWeb of Science
    2. 2.↵
      2. Runciman WB ,
      3. Coiera EW ,
      4. Day RO , et al
      . Towards the delivery of appropriate health care in Australia. Med J Aust 2012;197:78–81.doi:10.5694/mja12.10799
      OpenUrlPubMed
    3. 3.↵
      2. Francke AL ,
      3. Smit MC ,
      4. de Veer AJ ,
      5. dVA J , et al
      . Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak 2008;8:38.doi:10.1186/1472-6947-8-38
      OpenUrlCrossRefPubMed
    4. 4.↵
      2. Cochrane LJ ,
      3. Olson CA ,
      4. Murray S , et al
      . Gaps between knowing and doing: understanding and assessing the barriers to optimal health care. J Contin Educ Health Prof 2007;27:94–102.doi:10.1002/chp.106
      OpenUrlCrossRefPubMedWeb of Science
    5. 5.↵
      2. Baiardini I ,
      3. Braido F ,
      4. Bonini M , et al
      . Why do doctors and patients not follow guidelines? Curr Opin Allergy Clin Immunol 2009;9:228–33.doi:10.1097/ACI.0b013e32832b4651
      OpenUrlCrossRefPubMed
    6. 6.↵
      2. Elwyn G ,
      3. Wieringa S ,
      4. Greenhalgh T
      . Clinical encounters in the post-guidelines era. BMJ 2016;353:i3200.doi:10.1136/bmj.i3200
      OpenUrlFREE Full Text
    7. 7.↵
      2. Wiles LK ,
      3. Hooper TD ,
      4. Hibbert PD , et al
      . CareTrack Kids-part 1. assessing the appropriateness of healthcare delivered to australian children: study protocol for clinical Indicator development. BMJ Open 2015;5:e007748.doi:10.1136/bmjopen-2015-007748
    8. 8.↵
      2. Scott IA ,
      3. Guyatt GH
      . Clinical practice guidelines: the need for greater transparency in formulating recommendations. Med J Aust 2011;195:29–33.
      OpenUrlPubMed
    9. 9.↵
      2. Williams MJ ,
      3. Kevat DA ,
      4. Loff B
      . Conflict of interest guidelines for clinical guidelines. Med J Aust 2011;195:442–5.doi:10.5694/mja10.11130
      OpenUrlCrossRefPubMedWeb of Science
    10. 10.↵
      2. Graham R ,
      3. Mancher M ,
      4. Wolman DM , et al
      ; Clinical practice guidelines we can trust: national Academies Press, 2011.
    11. 11.↵
      2. Qaseem A ,
      3. Forland F ,
      4. Macbeth F , et al
      . Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med 2012;156:525–31.doi:10.7326/0003-4819-156-7-201204030-00009
      OpenUrlCrossRefPubMedWeb of Science
    12. 12.↵
      The King’s Fund. Experience-based co-design toolkit 2013 http://www.kingsfund.org.uk/projects/ebcd (accessed 13 July 2016).
    13. 13.↵
      2. Epstein RM ,
      3. Fiscella K ,
      4. Lesser CS , et al
      . Why the nation needs a policy push on patient-centered health care. Health Aff 2010;29:1489–95.doi:10.1377/hlthaff.2009.0888
      OpenUrlAbstract/FREE Full Text
    14. 14.↵
      2. Epstein RM ,
      3. Street RL
      . The values and value of patient-centered care. Ann Fam Med 2011;9:100–3.doi:10.1370/afm.1239
      OpenUrlFREE Full Text
    15. 15.↵
      2. Mickan S ,
      3. Burls A ,
      4. Glasziou P
      . Patterns of ’leakage' in the utilisation of clinical guidelines: a systematic review. Postgrad Med J 2011;87:670–9.doi:10.1136/pgmj.2010.116012
      OpenUrlAbstract/FREE Full Text
    16. 16.↵
      National Institute for Health and Care Excellence. Health and Social Directorate Indicators process Guide. 2014 https://www.nice.org.uk/media/default/Get-involved/Meetings-In-Public/indicator-advisory-committee/ioc-process-guide.pdf.
    17. 17.↵
      2. Rosenberg A ,
      3. Agiro A ,
      4. Gottlieb M , et al
      . Early Trends among seven recommendations from the Choosing wisely Campaign. JAMA Intern Med 2015;175:1913–20.doi:10.1001/jamainternmed.2015.5441
      OpenUrlCrossRefPubMed
    18. 18.↵
      Australian Commission on Safety and Quality in Health Care. Patient and Consumer Centred Care 2015 http://www.safetyandquality.gov.au/our-work/patient-and-consumer-centred-care/ (accessed 13 July 2016).
    19. 19.↵
      2. Boivin A ,
      3. Currie K ,
      4. Fervers B , et al
      . Patient and public involvement in clinical guidelines: international experiences and future perspectives. Qual Saf Health Care 2010;19:e22–4.doi:10.1136/qshc.2009.034835
      OpenUrlAbstract/FREE Full Text
    20. 20.↵
      2. Yudkin JS ,
      3. Kavanagh J ,
      4. McCormack JP
      . Guidelines for treating risk factors should include tools for shared decision making. BMJ 2016;353:i3147.doi:10.1136/bmj.i3147
      OpenUrlFREE Full Text
    21. 21.↵
      2. den Breejen EM ,
      3. Nelen WL ,
      4. Knijnenburg JM , et al
      . Feasibility of a wiki as a participatory tool for patients in clinical guideline development. J Med Internet Res 2012;14:e138.doi:10.2196/jmir.2080
      OpenUrlCrossRefPubMed
    22. 22.↵
      2. Brulet A ,
      3. Llorca G ,
      4. Letrilliart L et al
      . Medical wikis dedicated to clinical practice: a systematic review. J Med Internet Res 2015;17:e48.doi:/jmir.3574
      OpenUrl
    23. 23.↵
      2. van der Weijden T ,
      3. Légaré F ,
      4. Boivin A , et al
      . How to integrate individual patient values and preferences in clinical practice guidelines? A research protocol. Implement Sci 2010;5:10.doi:10.1186/1748-5908-5-10
      OpenUrlCrossRefPubMed
    24. 24.↵
      2. Baribeau D ,
      3. Wong J ,
      4. Monga S , et al
      . Selecting quality indicators in child and adolescent mental health care: a “stakeholder-driven” approach. Journal of Participatory Medicine 2016;8.
    25. 25.↵
      2. Nykänen P ,
      3. Brender J ,
      4. Talmon J , et al
      . Guideline for good evaluation practice in health informatics (GEP-HI). Int J Med Inform 2011;80:815–27.doi:10.1016/j.ijmedinf.2011.08.004
      OpenUrlCrossRefPubMed
    26. 26.↵
      2. Tong A ,
      3. Sainsbury P ,
      4. Craig J
      . Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349–57.doi:10.1093/intqhc/mzm042
      OpenUrlCrossRefPubMedWeb of Science
    27. 27.↵
      2. O’Brien BC ,
      3. Harris IB ,
      4. Beckman TJ , et al
      . Standards for reporting qualitative research: a synthesis of recommendations. Acad Med 2014;89:1245–51.doi:10.1097/ACM.0000000000000388
      OpenUrlCrossRefPubMed
    28. 28.↵
      2. Vaismoradi M ,
      3. Turunen H ,
      4. Bondas T
      . Content analysis and thematic analysis: implications for conducting a qualitative descriptive study. Nurs Health Sci 2013;15:398–405.doi:10.1111/nhs.12048
      OpenUrlCrossRefPubMedWeb of Science
    29. 29.↵
      2. Vlayen J ,
      3. Aertgeerts B ,
      4. Hannes K , et al
      . A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. Int J Qual Health Care 2005;17:235–42.doi:10.1093/intqhc/mzi027
      OpenUrlCrossRefPubMedWeb of Science
    30. 30.↵
      2. Alonso-Coello P ,
      3. Irfan A ,
      4. Solà I , et al
      . The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care 2010;19:e58.doi:10.1136/qshc.2010.042077
      OpenUrlAbstract/FREE Full Text
    31. 31.↵
      2. Brouwers MC ,
      3. Kho ME ,
      4. Browman GP , et al
      . Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ 2010;182:E472–E478.doi:10.1503/cmaj.091716
      OpenUrlAbstract/FREE Full Text
    32. 32.↵
      2. Brouwers MC ,
      3. Kho ME ,
      4. Browman GP , et al
      . Development of the AGREE II, part 1: performance, usefulness and Areas for improvement. CMAJ 2010;182:1045–52.doi:10.1503/cmaj.091714
      OpenUrlAbstract/FREE Full Text
    33. 33.↵
      2. Brouwers MC ,
      3. Kho ME ,
      4. Browman GP , et al
      . AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 2010;182:E839–E842.doi:10.1503/cmaj.090449
      OpenUrlFREE Full Text
    34. 34.↵
      2. Enterprise A
      . Appraisal of guidelines for Research and evaluation II (AGREE II) Instrument. 2013 http://www.agreetrust.org/wp-content/uploads/2013/10/AGREE-II-Users-Manual-and-23-item-Instrument_2009_UPDATE_2013.pdf.
    35. 35.↵
      2. Boulkedid R ,
      3. Abdoul H ,
      4. Loustau M , et al
      . Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011;6:e20476.doi:10.1371/journal.pone.0020476
    36. 36.↵
      2. Dunn AG ,
      3. Coiera E ,
      4. Mandl KD , et al
      . Conflict of interest disclosure in biomedical research: a review of current practices, biases, and the role of public registries in improving transparency. Res Integr Peer Rev 2016;1:1(1):1.doi:10.1186/s41073-016-0006-7
      OpenUrl
    37. 37.↵
      2. Mangione-Smith R ,
      3. DeCristofaro AH ,
      4. Setodji CM , et al
      . The quality of ambulatory care delivered to children in the United States. N Engl J Med 2007;357:1515–23.doi:10.1056/NEJMsa064637
      OpenUrlCrossRefPubMedWeb of Science
    38. 38.↵
      2. McGlynn EA ,
      3. Asch SM ,
      4. Adams J , et al
      . The quality of health care delivered to adults in the United States. N Engl J Med 2003;348:2635–45.doi:10.1056/NEJMsa022615
      OpenUrlCrossRefPubMedWeb of Science
    39. 39.↵
      2. Campbell SM ,
      3. Kontopantelis E ,
      4. Hannon K , et al
      . Framework and Indicator testing protocol for developing and piloting quality indicators for the UK quality and outcomes framework. BMC Fam Pract 2011;12:85.doi:10.1186/1471-2296-12-85
      OpenUrlCrossRefPubMed
    40. 40.↵
      2. Franklin KK ,
      3. Hart JK
      . Idea generation and exploration: benefits and Limitations of the policy Delphi Research Method. Innovative Higher Education 2006;31:237–46.doi:10.1007/s10755-006-9022-8
      OpenUrl
    41. 41.↵
      2. Raine R ,
      3. Sanderson C ,
      4. Black N
      . Developing clinical guidelines: a challenge to current methods. BMJ 2005;331:631–3.doi:10.1136/bmj.331.7517.631
      OpenUrlFREE Full Text
    42. 42.↵
      2. Hasson F ,
      3. Keeney S
      . Enhancing rigour in the Delphi technique research. Technol Forecast Soc Change 2011;78:1695–704.doi:10.1016/j.techfore.2011.04.005
      OpenUrlCrossRef
    43. 43.↵
      National Health and Medical Research Council. NHMRC Guideline development and conflict of interest: identifying and managing conflicts of interest of prospective member and members of NHMRC committees and working groups developing guidelines. 2012 http://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/developers/nh155_coi_policy_120710.pdf (accessed 13 July 2016).
    44. 44.↵
      National Institute for Health and Clinical Excellence. A code of practice for declaring and dealing with conflicts of interest. 2007 http://www.nice.org.uk/niceMedia/pdf/NICECodeofPracticeConflictsofInterestApril07.pdf.
    45. 45.↵
      NHMRC Guideline development and conflict of interest: identifying and managing conflicts of interest of prospective member and members of NHMRC committees and working groups developing guidelines 2012, 2012.
    46. 46.↵
      University of South Australia. Conflicts of interest in Research. 2014 http://w3.unisa.edu.au/res/ethics/integrity/frameworksection7.asp.
    47. 47.↵
      2. Fervers B ,
      3. Burgers JS ,
      4. Haugh MC , et al
      . Adaptation of clinical guidelines: literature review and proposition for a framework and procedure. Int J Qual Health Care 2006;18:167–76.doi:10.1093/intqhc/mzi108
      OpenUrlCrossRefPubMedWeb of Science
    48. 48.↵
      2. Kredo T ,
      3. Bernhardsson S ,
      4. Machingaidze S , et al
      . Guide to clinical practice guidelines: the current state of play. Int J Qual Health Care 2016;28:122–8.doi:10.1093/intqhc/mzv115
      OpenUrlCrossRefPubMed
    49. 49.
      2. Eccles MP ,
      3. Grimshaw JM ,
      4. Shekelle P , et al
      . Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implementation Science 2012;7 1.doi:10.1186/1748-5908-7-60
      OpenUrlCrossRef
    50. 50.
      National Asthma Council Australia. Asthma Management Handbook 2006: national Asthma Council, 2006.
    51. 51.
      National Heart Foundation of Australia Guide to management of hypertension 2008 National Heart Foundation of Australia 2010. Updated December 2010.

    Footnotes

    • Contributors WBR is the chief investigator and together with PDH, EC, JW, JB, ROD and KMH initiated the project and led the NHMRC grant proposal. LKW is a Research Team member and developed the first drafting of the protocol manuscript. LKW, PDH, JHS and WBR iteratively developed the methodology. All authors (LKW, PDH, JHS, EC, JW, JB, ROD, KMH, WBR) contributed to the writing of this manuscript.

    • Funding This work is supported by NHMRC Program Grant APP1054146. It is led by the University of South Australia and the Australian Institute of Health Innovation, Macquarie University.

    • Competing interests None declared.

    • Ethics approval University of South Australia Human Research Ethics Committee.

    • Provenance and peer review Not commissioned; externally peer reviewed.