You are here

Article Text

Article menu
Health policy
Original research
Patient and public involvement in the development of clinical practice guidelines: a scoping review
  1. Elizabeth Ann Bryant1,
  2. Anna Mae Scott2,
  3. Hannah Greenwood2,
  4. Rae Thomas2
  1. 1Human Resources, 14 University Drive, Bond University, Robina, Queensland, Australia
  2. 2Institute for Evidence Based Healthcare, 14 University Drive, Bond University, Robina, Queensland, Australia
  1. Correspondence to Elizabeth Ann Bryant; abryant{at}bond.edu.au

Abstract

Objectives Organisations that develop clinical practice guidelines (CPGs) encourage involvement of patients and the publics in their development, however, there are no standard methodologies for doing so. To examine how CPGs report patient and public involvement (PPI), we conducted a scoping review of the evidence addressing the following four questions: (1) who are the patients and publics involved in developing the CPG?; (2) from where and how are the patients and publics recruited?; (3) at what stage in the CPG development process are the patients and publics involved? and (4) how do the patients and publics contribute their views? We also extracted data on the use of PPI reporting checklists by the included studies.

Design We used the methodology developed by Arksey and O’Malley and refined by the Joanna Briggs Institute. We searched PubMed, Embase, CINAHL and PsycINFO, websites of national guideline bodies from the UK, Canada, Australia and the USA, and conducted a forward citation search. No language, date or participant demographics restrictions were applied. Data were synthesised narratively.

Results We included 47 studies addressing 1 or more of the 4 questions. All included studies reported who the patient and publics involved (PPI members) were, and several studies reported PPI members from different groups. Patients were reported in 43/47 studies, advocates were reported in 22/47 studies, patients and advocates reported in 17/47 studies, and general public reported in 2/47 studies. Thirty-four studies reported from where the patients and publics were recruited, with patient groups being the most common (20/34). Stage of involvement was reported by 42/47 studies, most commonly at question identification (26/42) and draft review (18/42) stages. Forty-two studies reported how the patients contributed, most commonly via group meetings (18/42) or individual interviews. Ten studies cited or used a reporting checklist to report findings.

Conclusions Our scoping review has revealed knowledge gaps to inform future research in several ways: replication, terminology and inclusion. First, no standard approach to PPI in CPG development could be inferred from the research. Second, inconsistent terminology to describe patients and publics reduces clarity around which patients and publics have been involved in developing CPGs. Finally, the under-representation of research describing PPI in the development of screening, as opposed to treatment, CPGs warrants further attention.

  • protocols & guidelines
  • epidemiology
  • health policy

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2021-055428

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    STRENGTHS AND LIMITATIONS OF THIS STUDY

    • We adhered to a robust scoping review methodology that deliberately included a very broad, comprehensive search strategy, which resulted in a sizeable scoping review that included 47 studies.

    • The search strategy had no restrictions on language, demographics of patients and publics, study design or date of publication.

    • Although our search strategy was broad, inconsistent patient and public involvement terminology may have limited our ability to identify all relevant studies.

    • We deviated from our published protocol in our search strategy. Forward and backward citation searches were proposed but due to the large number of identified studies, we chose to conduct forward citation searches of included studies only.

    Introduction

    Clinical practice guidelines (CPGs) are recommendations on how to diagnose and treat a medical condition and are intended to facilitate informed decision making and optimise patient care.1 They should be based on the best available research evidence and practice experience and be responsive to patient preferences and needs.2 3 Patient and public involvement (PPI) in healthcare, whether in research or to inform policy decisions, is recommended based on ethical principles and the expectation that it will improve the relevance of the outcomes and quality of the decisions.2 While medical practice is grounded in clinical science, patient management decisions are often influenced by the individual patient’s circumstances.4 CPGs developed in the absence of meaningful involvement of healthcare consumers in the guideline development group therefore cannot meet the needs of the population.5 The word ‘involvement’ is used intentionally instead of participation, engagement or contribution because PPI can be considered as decision making with or by patients and publics rather than for them.6

    International guideline standards include PPI as a core principle for developing high-quality evidence-based CPGs7 but PPI has not been widely adopted in Australian8 or the US guidelines.9 The number of CPGs has increased over several decades10 and guideline development organisations need effective and efficient methods of involving patients and public in the development process.11

    A synthesis of research on how best to identify, incorporate and report patient preferences and needs published before12 and after 2010,13 highlighted the paucity of substantial information about the methods employed and/or recommended. Reviewing guidance documents and methodological handbooks on incorporating patients and their views when developing CPGs confirmed that most institutions recommend the practice but provided little detail on the process.1 The AGREE (Appraisal of Guidelines, Research and Evaluation) Instrument, first published in 200314 and then refined in 2010,15 was developed to assess the methodological rigour of CPG development and act as a guide for development. In 2016, the AGREE Reporting Checklist was published16 as a tool to improve the completeness and transparency of reporting in practice guidelines. In 2017, the RIGHT (Reporting Items for practice Guidelines in HealThcare) Reporting Checklist was developed17 and built on the items included in the 2016 AGREE Reporting Checklist. Both checklists include items for reporting the guideline development group members’ name, role16 17 and the strategy used to capture the patients’ and publics’ views and preferences16 but neither offer guidance or address standardised reporting of PPI in CPG development.

    A previously published study which considered some of our research questions, did not explain their methods for study selection and had not examined studies post-2015.18 The authors suggested that despite governments, funding bodies and guideline developers world-wide seeking to involve patients and the broader public in development of CPGs, there are no standardised methodologies to achieve meaningful PPI in guideline development. It is not surprising then, that 5 years after the IOM3 released standards for development of healthcare guidelines, only 8% of guideline developers in the USA required PPI in guideline development groups and only 20% of guideline developers in the USA created patient-targeted guideline versions.9 This is despite research literature suggesting that PPI has a positive impact on guideline development through augmenting clinical care recommendations with patient-focused issues, thus helping to realise the aim of the guidelines: to optimise patient care and outcomes.12 19 For example, involving infertile couples when developing a multidisciplinary guideline on infertility broadened the scope of the guideline by including patient-identified clinical issues.12

    In the absence of consistent methodology for PPI in the development of CPGs, we aimed to focus on the specifics of the process and synthesise the available research to answer four questions: (1) who are the patients and publics involved in developing the CPG; (2) from where, and how are the patients and publics recruited; (3) at what stage in the CPG development process are the patients and publics involved, and (4) how do the patients and publics contribute their views.

    Methods

    Approach

    We conducted a systematic scoping review to identify when and how publications report the involvement of patients and public in developing CPGs. The scoping review was conducted based on the methodological framework developed20 and refined by the Joanna Briggs Institute.21 Results are reported using Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines.22 The protocol for this study has been published.23

    Search strategy

    The reference list of a previously published study that considered some of our research questions9 yielded seven published studies which were used as a validation set for our search strategy. A search strategy was created in collaboration with a research information specialist and by using Word Frequency Analyser24 to identify key words and phrases from the validation set. Search terms were deliberately broad so as not to unduly limit articles.25 The search strategy constructed for PubMed is shown in online supplemental appendix 1.

    The search was conducted in PubMed, Embase, CINAHL and PsycINFO on 1 October 2019. Targeted internet searches were also conducted to identify published national standards for developing CPGs from countries with a similar social and economic environment to Australia, that is, UK, Canada and the USA, and were searched for additional primary studies. Finally, on 17 August 2020, we conducted forward (citing) citation searches for the included studies.

    Inclusion criteria

    We included published primary studies that report the involvement of PPI in CPG development specifically addressing one or more of our four research questions:

    1. Who are patients and publics involved in developing the CPG.

    2. From where, and how, are the patients and publics recruited.

    3. At what stage in the CPG development process are the patients and publics involved.

    4. How do the patients and publics contribute their views.

    We did not include evaluation of PPI models as a research question in this review because (1) our aim was to identify specific characteristics of PPI in CPG development, and (2) we were aware of a registered systematic review protocol specifically focused on this research question.26

    No language, date or participant demographics (eg, gender, age, health history) restrictions were applied. We excluded letters, opinion pieces, commentaries, reports and studies focused on PPI in health technology assessments.

    Screening

    Two reviewers (AB and HG) independently screened titles and abstracts in multiples of 100 against the inclusion/exclusion criteria until we achieved 90% consistency (ie, agreement for inclusion was achieved 9/10 times). This occurred after screening titles and abstracts of 400 search results. Discrepancies were discussed and dispute resolution occurred via consensus or a third reviewer if required. Full text screening was conducted by the same two reviewers except in multiples of 10 not 100. Full text were also screened to 90% consistency, which occurred after 26 search results.

    Data extraction

    Data extraction templates were designed and piloted independently by two authors (AB and HG) on 10 randomly selected studies. Data extraction forms were amended for suitability and consistency. We continued extracting independently until data extraction consistency between the two authors reached 90% (ie, agreement for extracted data was achieved 9/10 times). All subsequent studies were extracted by AB. Extracted data included study details such as, author and date of publication, study title, study location, population and type of clinical guideline. Extracted study outcomes were focused on our four research questions.

    Data analysis

    Q1: Who are the patients and publics involved in developing the CPG?

    We modified the framework suggested by Degeling et al27 and classified which publics were involved in the CPG development process as: general public—citizens/community who are unfamiliar or only broadly familiar with the health condition; affected public from a screening population (ie, those eligible to be screened from a screening CPG) but without any experience of the health condition; an affected public from a treatment population/patients (ie, those with experience of being treated for the health condition); and advocates—who are representatives of groups interest or engaged in the health condition and/or political organisations.

    Q2: From where, and how, are the patients and publics recruited?

    Data were extracted on three dimensions (sampling frame, source and approach). Sampling frame refers to how patients and publics were recruited (ie, convenience, purposive, random). Where the patients and publics were recruited from was extracted as the recruitment source (ie, patient groups, patient records from healthcare providers, individual patients, contacts of researcher or guideline developer). Finally, we also extracted how these patients and publics were approached to be involved in the development of the CPG development (ie, in person, email, telephone, letter, website, newsletter, clinic notices, social media).

    Q3: At what stage in the CPG development process are the patients and publics involved?

    The classification terminology described in the National Institute for Health and Care Excellence (NICE) manual28 was common to many included studies. Therefore, we extracted data for this question using these categories (ie, topic selection, scoping, identifying the questions, identifying/reviewing the evidence, reviewing the draft CPG, revising the draft CPG and assisting with the patient version). We expanded the classification to include ‘throughout the process’ to capture how some studies reported the data.

    Q4: How do the patients and publics contribute their views?

    Data were extracted across three different styles of contribution (1) in-person—individual interviews or group settings; (2) online—surveys or Delphi process and (3) multimodal—combination of in-person and online contributions.

    An emerging theme which was not anticipated in the study protocol was the inconsistent reporting of PPI in CPG development and we considered it important to identify whether reporting checklists were used in our included studies. To identify whether a study used a reporting checklist (eg, AGREE, AGREE II), we examined each study’s list of references.

    Results

    Search results

    The database searches identified 2015 studies. A further 797 studies were identified in the published National standards from Australia, UK, Canada and USA. A forward citation search identified 15 studies. Of the 2367 unique items, 2258 were excluded based on title and abstract screening. Among the 109 full-text articles that were screened, 62 were excluded because they were not primary studies (11/62), did not relate to developing a CPG (30/62), did not include PPI (7/62), were abstracts only (11/62) or were a duplicate publication (3/62). Forty-seven studies were included in references 29–75 (see figure 1).

    Figure 1
    Figure 1

    PRISMA flow diagram. CPG, clinical practice guideline; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

    Included study characteristics

    A full list of included studies and extracted data is reported in online supplemental table 1 Studies were published between 2000 and 2020 and were conducted in Canada (10/47), The Netherlands (8/47), Australia (6/47), the USA (6/47), Germany (4/47), UK (3/47), Italy (3/47), Spain (2/47) and 1 study each in Belgium, Colombia, Ghana, Madagascar and Turkey. In 36/47 studies the reported aim was to investigate PPI in the development of a CPG. In the other 11/47 studies the reported aim was to develop a CPG and PPI in the process was incidentally noted. Clinical topics varied and included arthritis and joint disease (8/47), mental health (8/47), pain management (7/47), cancer (5/47), infertility (5/47), kidney disease (4/47), multiple sclerosis (2/47), vision impairment (2/47) and 1 each for fibromyalgia, leg ulcers, resumption of work, sickle cell disease, systemic lupus erythematosus and ulcerative colitis. Forty studies described the development of a treatment guideline, three studies described development of a screening guideline and development of a diagnostic guideline was described in one study. Development of a screening and treatment/management guideline was described in three studies.

    Q1: Who are the patients and publics involved in developing the CPG?

    All included studies reported this outcome(online supplemental table 1) Most of the studies (43/47) included PPI members who had been diagnosed with the CPG condition and were therefore defined as members of a treated population, that is, patients. Included in those 43/47 studies were the three screening CPGs. Fearns et al31 included members of the treated population in developing a patient version of an existing glaucoma screening CPG, while the other two29 30 involved the treated population in the use of amyloid positron emission tomography imaging in patients with or at risk of dementia.

    Three studies (3/47) provided details of the severity of symptoms experienced by the patients and publics at the time of involvement in developing the CPG. All three studies noted that the specificity of the patient and publics profile may have limited the applicability of the CPG. Rankin et al54 involved women who had a breast cancer diagnosis within 2 years and were physically and mentally able to complete the survey. Serrano-Aguilar et al56 involved people with a diagnosis of inherited retinal diseases but with enough sight to read and respond to the survey. Posada-Borrero et al53 involved major lower limb amputees in Colombia that were able to attend all meetings in person.

    Advocates were recruited as PPI members in 22/47 studies. Three of these (3/22), included only advocates, another 17/22 studies included both advocates and patients, 1/22 studies included advocates and general public and 1/22 studies included advocates, patients and general public as PPI members. Wilson et al64 and Wolfe et al74 involved the parents of young children with acute respiratory infections and receiving life-altering information respectively but the children were not involved.

    Nine studies (9/47) specified that to be involved, patients and public were older than 18 years of age. The least frequently involved PPI members (2/47) were those from general public and always, as mentioned, in combination with another group.

    Q2: From where, and how, are the patients and publics recruited?

    Where PPI members are recruited from was reported in 35/47 studies(online supplemental table 1) with three of those studies recruiting from more than one source that is, recruitment group. Two studies reported recruiting from patient groups and researcher/developer contacts while one study reported recruiting from patient groups and patient records from healthcare providers. Of the 32 studies reporting only one recruitment group, patient groups were the most common (n=20) source of PPI members, followed by patient records from healthcare providers (n=7) and, direct contact with patients (n=5). Advocates were mostly identified through patient groups (77%), patients most often through patient groups (58%) and patient records (21%) and the general public through patient groups (100%).

    Eighteen studies reported how PPI members are recruited. Multiple recruitment approaches (such as a combination of newsletters, website posts, emails, clinic notices, telephone contact, in-person and letters) were reported in nine studies. A singular approach to recruitment of patients and public was reported in nine studies, newsletter (n=1), in-person (n=2), letter (n=2) and email (n=5).

    The sampling method used to identify PPI members was reported in 14 studies. Purposive sampling was the most frequently reported sampling frame (n=10), followed by convenience sampling (n=3). Random sampling was only reported in one study. Reasons for purposively sampling PPI members included to ensure stakeholder representation (n=1), capture expertise (n=1) and a particular phase of a health condition trajectory (n=2), for demographic purposes only (n=3) and demographics in combination with clinical characteristics (n=3).

    Q3: At what stage in the CPG development process are the patients and publics involved?

    Forty-two studies reported this outcome (online supplemental table 1) . No involvement was reported at either topic selection or scoping the literature stages of the CPG. PPI was reported in the other five stages: identifying the questions (n=26), identify/review evidence (n=4), review draft of the CPG (n=18), revise the draft (n=2) and assist with the patient version of the CPG (n=9). Three studies reported patients and publics involved throughout the process of developing the CPG. Hatemi et al68 and Serrano-Aguilar et al55 reported an advocate as present throughout the CPG development process. Solari et al73 reported a patient participating in all aspects of the CPG development.

    Q4: How do the patients and publics contribute their views?

    The majority of included studies (42/47) described the format in which the patients and publics contributed their views (online supplemental table 1) . An in-person format, either by individual semistructured interview (n=8) or in a group setting (n=18) was most reported. All group settings were facilitated by professionals or researchers except those reported by Tong et al59 which were peer-facilitated. Contributing in an online format was reported in seven studies either by a Delphi process (n=3) or survey (n=4). A multimodal approach to capturing the patient and public view (such as via surveys and in-person groups) was reported in a further seven studies.

    Two studies reported alternative formats for patients and publics within the same study. Westby et al63 reported the option of a semistructured interview for those patients and publics unable to attend the facilitated focus group. In developing a CPG for spinal metastases,67 a patient advocate was included in the working group, but patients contributed separately by responding to a survey and, if invited, via a semistructured interview.

    Reporting guideline use by the included studies

    Very few studies reported using either a reporting checklist or a developmental tool such as the AGREE14 or AGREE II Instrument15 in their methods (online supplemental table 1) . Of the 43 studies published post-2003 (the year that the AGREE Instrument was first published) the AGREE or AGREE II Instrument was referenced but not reported in the methods by five studies and reported as used by only three studies. Two studies, van der Ham et al61 and Pittens et al52 developed their own monitoring framework for PPI in CPG.

    Discussion

    What did we find

    To determine who, how and when patients and publics are involved in published CPG development studies, we searched research literature and guideline developer standards and identified 47 research studies that addressed one or more of our 4 questions: question 1 (n=47), question 2 (n=34), question 3 (n=42) and question 4 (n=42). Most studies included in this review (n=36) stated they were specifically designed to investigate PPI in CPG development. It is encouraging to note the research investigating PPI but whether this translates to routine involvement in CPG development remains to be seen. Unfortunately, we searched CPG development guideline documents from Australia, Canada, the USA and the UK in the expectation that we would identify studies that address who to involve, how to recruit them, and when and how to involve them, but we found none. This concurs with Selva et al1 who reviewed guidance documents from 56 institutions worldwide to find little detail on how to incorporate patients’ views in guideline development and Armstrong and Bloom76 who reported that 5 years after the 2011 release of the IOM standards for development of healthcare guidelines, only 8% of guideline developers in the USA, required PPI in guideline development groups.

    We identified that patients were the most recruited group (91.4%) involved in developing CPGs, which is consistent with Légaré et al12 who reported that patients were recruited in 45 of their 71 (63.3%) included studies. However, because of inconsistencies in terminology between our study and Légaré et al12 we cannot directly compare the recruitment rates of other participant categories. For example, Légaré et al12 classified family and caregivers as ‘patient representatives’ whereas our taxonomy included family and caregivers in the treatment population, that is, affected public with treatment experience. Only 2 of the 47 included studies addressing this question (question 1) involved the general public—a screening guideline and a treatment guideline. However, both studies also recruited either patients and/or advocates. Six of our included studies involved screening CPGs. Screening guidelines might benefit from the inclusion of an ‘affected public’27 (ie, those people who have the potential to be ‘consumers’ or ‘patients’ because they are within the demographics of the screening recommendations but have not been diagnosed with the health condition) rather than advocates or patients. It is this group of people who are the most impacted by a screening CPG.

    How the patients and publics were recruited for the guideline groups also varied. The sampling frame for recruiting patients and publics (eg, purposive, convenience or random) was not reported in more than half the included studies and the method for recruiting was not described with enough detail to allow the sampling frame to be inferred. This was surprising given that diversity is important regarding recruiting individuals with different disease perspectives, ethnicities and roles.30 77 Khodyakov et al78 recommended recruiting demographically and geographically diverse patients and publics and those in different stages of disease progression. Using social media for recruitment into clinical studies is thought to facilitate contact with a broader pool of potential patients and publics79 but we only identified one study that reported using social media as a recruitment approach.

    To categorise our results as to when in the guideline development process patients and publics were involved, we used the development stages reported in the NICE guidelines.28 Six of our included studies that were identified as focusing on PPI in developing CPGs did not report where in the guideline development process, they involved publics and patients. Most studies only included PPI in one stage of the CPG process with the most reported stages being identifying the question and reviewing the draft guideline. Only three studies55 68 73 involved patients and advocates throughout the CPG process. We consider this a wasted opportunity for PPI.

    Most of the included studies involved patients and publics using a single method (eg, group or individual contributions, surveys or Delphis). A few studies enabled more inclusion by conducting multimodal methods of contributions. There was limited use of online methods of capturing the contributions of the patients and publics involved in developing the CPGs. This was surprising given the recency of the studies identified. Online surveys and the Delphi process offer opportunities to include patients and publics physically unable to attend face-to-face sessions potentially reducing participant attrition59 and increasing participant diversity.29 80

    Where does this fit

    This study contributes to the investigation of the involvement of patients and publics in the development of CPGs but from the unique perspective of four specific questions. The questions were designed to gather details of the processes employed and identify any consistencies in methodology to infer best practice. Consistent with our results, previous reviews of primary research12 13 and guidance documents for developing CPGs1 reported little detail on how to identify, incorporate and report patient preferences in clinical guidelines and limited evidence of PPI in CPG development stages.

    We identified inconsistencies in terminology that confuse discussion of which patients and publics are being involved. For example, Armstrong et al30 use the term ‘patient representatives’ to describe patient, caregiver and advocates and Aziato and Adejumo35 use the term ‘patients’ to describe patients and patients’ relatives. Using the same term to describe different participant groups hampers attempts to elucidate best practice methods from the research literature.

    Strengths and limitations

    We adhered to a robust scoping review methodology21 that deliberately included a very broad, comprehensive search strategy resulting in a sizeable scoping review that included 47 studies. The included studies reported on PPI in screening, treatment, and diagnostic CPG development, were from Europe, South America, North America, Africa, the UK and Australia, with CPG from a breadth of health conditions (eg, treatment for physical and mental health conditions, screening for dementia). We used NICE stages of CPG development criteria28 to structure our data extraction for when PPI occurred in the CPG process.

    Limitations to this study must also be acknowledged. We did not include evaluation of PPI models as a research question in this review because (1) our aim was to identify specific characteristics of PPI in CPG development and (2) we were aware of a registered systematic review protocol specifically focused on this research question.26

    Although our search strategy was broad, inconsistent PPI terminology may have limited our ability to identify all relevant studies. We deviated from our published protocol23 in our search strategy. Forward and backward citation searches were proposed but due to the large number of identified studies, we chose to conduct forward citation searches of included studies only. The forward citation searching expanded the data available for our four research questions by locating follow-up studies and identifying new findings and developments.

    Implications

    Our scoping review has revealed knowledge gaps to inform future research in a number of ways: replication, terminology and inclusion.

    First, no standard approach to PPI in CPG development could be inferred from the research literature because the level of detail regarding recruitment groups and approaches or when and how to gather the views of patients and publics was insufficient. In most cases the detail provided would not allow for the study to be replicated even in studies with PPI in development of CPGs as the focus. There are valid and reliable tools available that can be applied to CPGs to assess developmental rigour and standardise reporting, for example, the AGREE Instrument,14 AGREE Reporting Checklist16 and the RIGHT Instrument.17 We recommend investigating the limited use of any of those tools in our included studies. A recently validated tool, PANELVIEW, was developed for guideline developers to involve clinicians, patients and other participants in evaluating their guideline processes.81 Assessing guideline panel members’ perception of the appropriateness of, and satisfaction with, the process, methods and outcome of the development of a health guideline will inform quality improvement of existing or new guideline programmes. Importantly though, none of the above-mentioned tools address standardised reporting of PPI in CPG development. Inconsistent and inadequate reporting hinders the synthesis of PPI not only in CPG development but in all areas of PPI and restricts elucidation of best-practice models.

    Second, the lack of consistent terminology to classify who constitute patients and publics confuses perceptions of which patients and publics have been and should be involved in developing CPGs. Reviewing the evidence and establishing a model of best practice are hampered by this inconsistency. Consensus regarding standard classification terminology is required. We recommend development of standard classification terminology to facilitate clear translation of research or perhaps the adoption (with modifications as required) of the terminology developed for the health technology assessment space. Inconsistent terminology is indeed hindering development of best practice in PPI. However, consistent terminology without consistent and thorough reporting of PPI, in this case, in CPG development won't facilitate reviewing the evidence and developing best practice for PPI.

    Finally, the under-representation of research describing PPI in the development of screening, as opposed to treatment, CPGs warrants further attention. There may be differences in recommended recruitment groups, recruitment approaches and when and how to involve patients and the publics. The target population of any CPG are the patients and publics most impacted. As early as 1990 the IOM3 made a strong recommendation ‘that the process of developing guidelines include representatives of key affected groups and disciplines.’ People who have the potential to be ‘consumers’ or ‘patients’ because they are within the demographics of the screening recommendations but have not been diagnosed with the health condition will be more impacted by a screening CPG than patients and/or advocates. Nevertheless, the growing number of studies in this area suggests that the issue of PPI in CPG is gaining traction with both researchers and policymakers, which may result in a closer alignment of CPGs and patient preferences and needs.

    Patient and public involvement

    Neither patients nor the public have been involved in the design or the conduct of the current review. As the study is a scoping review, there are no participants. We anticipate that the findings of this study will advance the synthesis of information for PPI in CPG development.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as online supplemental information.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Ethics approval is not required as the study involves information that is in the public domain and freely available. The results will be disseminated through a peer-reviewed publication and presentation at conferences targeting an audience involved in clinical guideline development and patient and public involvement in healthcare.

    References

    1. ↵
      1. Selva A,
      2. Sanabria AJ,
      3. Pequeño S, et al
      . Incorporating patients' views in Guideline development: a systematic review of guidance documents. J Clin Epidemiol 2017;88:102–12.doi:10.1016/j.jclinepi.2017.05.018pmid:http://www.ncbi.nlm.nih.gov/pubmed/28579379
      OpenUrlPubMed
    2. ↵
      1. Boivin A,
      2. Richards T,
      3. Forsythe L, et al
      . Evaluating patient and public involvement in research. BMJ 2018;363:k5147.doi:10.1136/bmj.k5147pmid:http://www.ncbi.nlm.nih.gov/pubmed/30522999
      OpenUrlFREE Full Text
    3. ↵
      1. Institute of Medicine
      . Clinical practice guidelines: directions for a new program. Washington, DC: The National Academies Press, 1990: 168.
    4. ↵
      1. Mercuri M,
      2. Sherbino J,
      3. Sedran RJ, et al
      . When guidelines don't guide: the effect of patient context on management decisions based on clinical practice guidelines. Acad Med 2015;90:191–6.doi:10.1097/ACM.0000000000000542pmid:http://www.ncbi.nlm.nih.gov/pubmed/25354075
      OpenUrlCrossRefPubMed
    5. ↵
      1. National Health and Medical Research Council (NHMRC)
      . 2016 NHMRC standards for guidelines, 2016. Available: https://www.nhmrc.gov.au/guidelinesforguidelines/standards
    6. ↵
      1. Ocloo J,
      2. Matthews R
      . From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf 2016;25:626–32.doi:10.1136/bmjqs-2015-004839pmid:http://www.ncbi.nlm.nih.gov/pubmed/26993640
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Armstrong MJ,
      2. Rueda J-D,
      3. Gronseth GS, et al
      . Framework for enhancing clinical practice guidelines through continuous patient engagement. Health Expect 2017;20:3–10.doi:10.1111/hex.12467pmid:http://www.ncbi.nlm.nih.gov/pubmed/27115476
      OpenUrlCrossRefPubMed
    8. ↵
      1. National Health and Medical Research Council (NHMRC)
      . 2014 annual report on Australian clinical practice guidelines Canberra, 2014. Available: https://www.nhmrc.gov.au/file/5041/download?token=oiAWNTrB
    9. ↵
      1. Armstrong MJ,
      2. Mullins CD,
      3. Gronseth GS, et al
      . Recommendations for patient engagement in Guideline development panels: a qualitative focus group study of guideline-naïve patients. PLoS One 2017;12:e0174329.doi:10.1371/journal.pone.0174329pmid:http://www.ncbi.nlm.nih.gov/pubmed/28319201
      OpenUrlPubMed
    10. ↵
      1. Grant S,
      2. Hazlewood GS,
      3. Peay HL, et al
      . Practical considerations for using online methods to engage patients in Guideline development. Patient 2018;11:155–66.doi:10.1007/s40271-017-0280-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/29030831
      OpenUrlPubMed
    11. ↵
      1. Bennett WL,
      2. Robbins CW,
      3. Bayliss EA, et al
      . Engaging stakeholders to inform clinical practice guidelines that address multiple chronic conditions. J Gen Intern Med 2017;32:883–90.doi:10.1007/s11606-017-4039-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/28349409
      OpenUrlPubMed
    12. ↵
      1. Légaré F,
      2. Boivin A,
      3. van der Weijden T, et al
      . Patient and public involvement in clinical practice guidelines: a knowledge synthesis of existing programs. Med Decis Making 2011;31:E45–74.doi:10.1177/0272989X11424401pmid:http://www.ncbi.nlm.nih.gov/pubmed/21959267
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Kim C,
      2. Armstrong MJ,
      3. Berta WB, et al
      . How to identify, incorporate and report patient preferences in clinical guidelines: a scoping review. Health Expect 2020;23:1028–36.doi:10.1111/hex.13099pmid:http://www.ncbi.nlm.nih.gov/pubmed/32656807
      OpenUrlPubMed
    14. ↵
      1. AGREE Collaboration
      . Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the agree project. Qual Saf Health Care 2003;12:18–23.doi:10.1136/qhc.12.1.18pmid:http://www.ncbi.nlm.nih.gov/pubmed/12571340
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Brouwers MC,
      2. Kho ME,
      3. Browman GP, et al
      . Agree II: advancing Guideline development, reporting and evaluation in health care. CMAJ 2010;182:E839–42.doi:10.1503/cmaj.090449pmid:http://www.ncbi.nlm.nih.gov/pubmed/20603348
      OpenUrlFREE Full Text
    16. ↵
      1. Brouwers MC,
      2. Kerkvliet K,
      3. Spithoff K, et al
      . The agree reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ 2016;352:i1152.doi:10.1136/bmj.i1152pmid:http://www.ncbi.nlm.nih.gov/pubmed/26957104
      OpenUrlFREE Full Text
    17. ↵
      1. Chen Y,
      2. Yang K,
      3. Marušic A, et al
      . A reporting tool for practice guidelines in health care: the right statement. Ann Intern Med 2017;166:128–32.doi:10.7326/M16-1565pmid:http://www.ncbi.nlm.nih.gov/pubmed/27893062
      OpenUrlCrossRefPubMed
    18. ↵
      1. Rashid A,
      2. Thomas V,
      3. Shaw T, et al
      . Patient and public involvement in the development of healthcare guidance: an overview of current methods and future challenges. Patient 2017;10:277–82.doi:10.1007/s40271-016-0206-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/27830457
      OpenUrlCrossRefPubMed
    19. ↵
      1. Gregory J,
      2. Hartz-Karp J,
      3. Watson R
      . Using deliberative techniques to engage the community in policy development. Aust New Zealand Health Policy 2008;5:16.doi:10.1186/1743-8462-5-16pmid:http://www.ncbi.nlm.nih.gov/pubmed/18631378
      OpenUrlCrossRefPubMed
    20. ↵
      1. Arksey H,
      2. O'Malley L
      . Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005;8:19–32.doi:10.1080/1364557032000119616
      OpenUrlCrossRef
    21. ↵
      1. Peters M,
      2. Godfrey C,
      3. Khalil H
      . Guidance for the conduct of JBI scoping reviews. In: Aromataris E, Munn Z, eds. Joanna Briggs Institute reviewer's manual. The Joanna Briggs Institute, 2017.
    22. ↵
      1. Tricco AC,
      2. Lillie E,
      3. Zarin W, et al
      . PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018;169:467–73.doi:10.7326/M18-0850pmid:http://www.ncbi.nlm.nih.gov/pubmed/30178033
      OpenUrlCrossRefPubMed
    23. ↵
      1. Bryant EA,
      2. Scott AM,
      3. Thomas R
      . Patient and public involvement in the development of clinical practice guidelines: a scoping review protocol. BMJ Open 2020;10:e037327.doi:10.1136/bmjopen-2020-037327pmid:http://www.ncbi.nlm.nih.gov/pubmed/33020092
      OpenUrlPubMed
    24. ↵
      1. Institute for Evidence Based Health, Bond University
      . Word Frequency Analyser, Research Tools: Systematic Review Accelerator [Internet]. Available: http://iebhbondeduau/education-services/research-tools [Accessed 16 Apr 2022].
    25. ↵
      1. Conklin A,
      2. Morris Z,
      3. Nolte E
      . What is the evidence base for public involvement in health-care policy?: results of a systematic scoping review. Health Expect 2015;18:153–65.doi:10.1111/hex.12038pmid:http://www.ncbi.nlm.nih.gov/pubmed/23252574
      OpenUrlCrossRefPubMed
    26. ↵
      1. Josefine Bjorkqvist SM,
      2. MacLennan S,
      3. Yuan C
      . An evaluation of existing models of patient and public involvement in the development of clinical practice guidelines: a systematic review. PROSPERO 2018 CRD42018116712 2018.
    27. ↵
      1. Degeling C,
      2. Carter SM,
      3. Rychetnik L
      . Which public and why deliberate?--A scoping review of public deliberation in public health and health policy research. Soc Sci Med 2015;131:114–21.doi:10.1016/j.socscimed.2015.03.009pmid:http://www.ncbi.nlm.nih.gov/pubmed/25770463
      OpenUrlCrossRefPubMed
    28. ↵
      1. National Institute for H, Care E
      . Nice process and methods guides. The guidelines manual. London: National Institute for Health and Care Excellence (NICE) Copyright © 2012 National Institute for Health and Clinical Excellence, unless otherwise stated. All rights reserved, 2012.
    29. ↵
      1. Armstrong MJ,
      2. Gronseth GS,
      3. Gagliardi AR, et al
      . Participation and consultation engagement strategies have complementary roles: a case study of patient and public involvement in clinical practice Guideline development. Health Expect 2020;23:423–32.doi:10.1111/hex.13018pmid:http://www.ncbi.nlm.nih.gov/pubmed/31884708
      OpenUrlPubMed
    30. ↵
      1. Armstrong MJ,
      2. Mullins CD,
      3. Gronseth GS, et al
      . Impact of patient involvement on clinical practice Guideline development: a parallel group study. Implement Sci 2018;13:55.doi:10.1186/s13012-018-0745-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/29661195
      OpenUrlPubMed
    31. ↵
      1. Fearns N,
      2. Graham K,
      3. Johnston G, et al
      . Improving the user experience of patient versions of clinical guidelines: user testing of a Scottish Intercollegiate guideline network (sign) patient version. BMC Health Serv Res 2016;16:37.doi:10.1186/s12913-016-1287-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/26837683
      OpenUrlPubMed
    32. ↵
      1. Harding E,
      2. Brown D,
      3. Hayward M, et al
      . Service user perceptions of involvement in developing NICE mental health guidelines: a grounded theory study. J Ment Health 2010;19:249–57.doi:10.3109/09638230903469202pmid:http://www.ncbi.nlm.nih.gov/pubmed/20441489
      OpenUrlPubMed
    33. ↵
      1. Miller HM,
      2. Tong A,
      3. Tunnicliffe DJ, et al
      . Identifying and integrating patient and caregiver perspectives for clinical practice guidelines on the screening and management of infectious microorganisms in hemodialysis units. Hemodial Int 2017;21:213–23.doi:10.1111/hdi.12457pmid:http://www.ncbi.nlm.nih.gov/pubmed/27389043
      OpenUrlPubMed
    34. ↵
      1. Ughi N,
      2. Prevete I,
      3. Ramonda R, et al
      . The Italian Society of rheumatology clinical practice guidelines for the diagnosis and management of gout. Reumatismo 2019;71:50–79.doi:10.4081/reumatismo.2019.1176pmid:http://www.ncbi.nlm.nih.gov/pubmed/31948193
      OpenUrlPubMed
    35. ↵
      1. Aziato L,
      2. Adejumo O
      . Developing a context appropriate clinical guideline for post-operative pain management in Ghana: a participatory approach. Int J Afr Nurs Sci 2015;2:26–33.doi:10.1016/j.ijans.2015.03.001
      OpenUrl
    36. ↵
      1. Berk L,
      2. Jorm AF,
      3. Kelly CM, et al
      . Development of guidelines for caregivers of people with bipolar disorder: a Delphi expert consensus study. Bipolar Disord 2011;13:556–70.doi:10.1111/j.1399-5618.2011.00942.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/22017224
      OpenUrlCrossRefPubMed
    37. ↵
      1. Boelens PG,
      2. Taylor C,
      3. Henning G, et al
      . Involving patients in a multidisciplinary European consensus process and in the development of a 'patient summary of the consensus document for colon and rectal cancer care'. Patient 2014;7:261–70.doi:10.1007/s40271-014-0061-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/24862378
      OpenUrlPubMed
    38. ↵
      1. Brouwers MC,
      2. Vukmirovic M,
      3. Spithoff K, et al
      . Engaging cancer patients in clinical practice Guideline development: a pilot study. Curr Oncol 2018;25:250–6.doi:10.3747/co.25.3943pmid:http://www.ncbi.nlm.nih.gov/pubmed/30111965
      OpenUrlPubMed
    39. ↵
      1. Cronin RM,
      2. Mayo-Gamble TL,
      3. Stimpson S-J, et al
      . Adapting medical guidelines to be patient-centered using a patient-driven process for individuals with sickle cell disease and their caregivers. BMC Hematol 2018;18:12.doi:10.1186/s12878-018-0106-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/29977566
      OpenUrlPubMed
    40. ↵
      1. den Breejen EME,
      2. Hermens RPMG,
      3. Galama WH, et al
      . Added value of involving patients in the first step of multidisciplinary Guideline development: a qualitative interview study among infertile patients. Int J Qual Health Care 2016;28:299–305.doi:10.1093/intqhc/mzw020pmid:http://www.ncbi.nlm.nih.gov/pubmed/26968684
      OpenUrlCrossRefPubMed
    41. ↵
      1. Den Breejen EME,
      2. Hilbink MAHW,
      3. Nelen WLDM, et al
      . A patient-centered network approach to multidisciplinary-guideline development: a process evaluation. Implement Sci 2014;9:68.doi:10.1186/1748-5908-9-68pmid:http://www.ncbi.nlm.nih.gov/pubmed/24898160
      OpenUrlPubMed
    42. ↵
      1. den Breejen EME,
      2. Nelen WLDM,
      3. Knijnenburg JML, et al
      . Feasibility of a wiki as a participatory tool for patients in clinical Guideline development. J Med Internet Res 2012;14:e138.doi:10.2196/jmir.2080pmid:http://www.ncbi.nlm.nih.gov/pubmed/23103790
      OpenUrlCrossRefPubMed
    43. ↵
      1. Devlin JW,
      2. Skrobik Y,
      3. Rochwerg B, et al
      . Methodologic innovation in creating clinical practice guidelines: insights from the 2018 Society of critical care medicine pain, Agitation/Sedation, delirium, immobility, and sleep disruption guideline effort. Crit Care Med 2018;46:1457–63.doi:10.1097/CCM.0000000000003298pmid:http://www.ncbi.nlm.nih.gov/pubmed/29985807
      OpenUrlPubMed
    44. ↵
      1. Fonseka TM,
      2. Pong JT,
      3. Kcomt A, et al
      . Collaborating with individuals with lived experience to adapt CANMAT clinical depression guidelines into a patient treatment guide: the CHOICE-D co-design process. J Eval Clin Pract 2020;26:1259–69.doi:10.1111/jep.13308pmid:http://www.ncbi.nlm.nih.gov/pubmed/31729117
      OpenUrlPubMed
    45. ↵
      1. Fraenkel L,
      2. Miller AS,
      3. Clayton K, et al
      . When patients write the guidelines: patient panel recommendations for the treatment of rheumatoid arthritis. Arthritis Care Res 2016;68:26–35.doi:10.1002/acr.22758pmid:http://www.ncbi.nlm.nih.gov/pubmed/26545701
      OpenUrlPubMed
    46. ↵
      1. Goodman SM,
      2. Miller AS,
      3. Turgunbaev M, et al
      . Clinical practice guidelines: incorporating input from a patient panel. Arthritis Care Res 2017;69:1125–30.doi:10.1002/acr.23275pmid:http://www.ncbi.nlm.nih.gov/pubmed/28620968
      OpenUrlPubMed
    47. ↵
      1. Häuser W,
      2. Petzke F,
      3. Kopp I, et al
      . [Impact of conflicts of interest on guideline recommendations : Empirical study within the second update of the German interdisciplinary S3 guidelines on fibromyalgia syndrome]. Schmerz 2017;31:308–18.doi:10.1007/s00482-017-0218-xpmid:http://www.ncbi.nlm.nih.gov/pubmed/28455823
      OpenUrlPubMed
    48. ↵
      1. Kiltz U,
      2. van der Heijde D,
      3. Mielants H, et al
      . ASAS/EULAR recommendations for the management of ankylosing spondylitis: the patient version. Ann Rheum Dis 2009;68:1381–6.doi:10.1136/ard.2008.096073pmid:http://www.ncbi.nlm.nih.gov/pubmed/18930993
      OpenUrlAbstract/FREE Full Text
    49. ↵
      1. Köpke S,
      2. Giordano A,
      3. Veronese S, et al
      . Patient and caregiver involvement in the formulation of guideline questions: findings from the European Academy of Neurology guideline on palliative care of people with severe multiple sclerosis. Eur J Neurol 2019;26:41–50.doi:10.1111/ene.13760pmid:http://www.ncbi.nlm.nih.gov/pubmed/30035845
      OpenUrlCrossRefPubMed
    50. ↵
      1. Lindsay GM,
      2. Mior SA,
      3. Côté P, et al
      . Patients' experiences with vehicle collision to inform the development of clinical practice guidelines: a narrative inquiry. J Manipulative Physiol Ther 2016;39:218–28.doi:10.1016/j.jmpt.2016.01.005pmid:http://www.ncbi.nlm.nih.gov/pubmed/26926886
      OpenUrlPubMed
    51. ↵
      1. Morin SN,
      2. Djekic-Ivankovic M,
      3. Funnell L, et al
      . Patient engagement in clinical guidelines development: input from > 1000 members of the Canadian Osteoporosis Patient Network. Osteoporos Int 2020;31:867–74.doi:10.1007/s00198-019-05248-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/31838552
      OpenUrlPubMed
    52. ↵
      1. Pittens CACM,
      2. Vonk Noordegraaf A,
      3. van Veen SC, et al
      . The involvement of gynaecological patients in the development of a clinical guideline for resumption of (work) activities in the Netherlands. Health Expect 2015;18:1397–412.doi:10.1111/hex.12121pmid:http://www.ncbi.nlm.nih.gov/pubmed/23992108
      OpenUrlPubMed
    53. ↵
      1. Posada-Borrero AM,
      2. Plata-Contreras JA,
      3. Lugo-Agudelo LH
      . Participation of patients with lower limb amputation in the development of a clinical practice guide in Colombia. Iatreia 2016;29:S-96–S-109.
      OpenUrl
    54. ↵
      1. Rankin N,
      2. Newell S,
      3. Sanson-Fisher R, et al
      . Consumer participation in the development of psychosocial clinical practice guidelines: opinions of women with breast cancer. Eur J Cancer Care 2000;9:97–104.doi:10.1046/j.1365-2354.2000.00198.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/11261017
      OpenUrlPubMed
    55. ↵
      1. Serrano-Aguilar P,
      2. Trujillo-Martin MdelM,
      3. Pérez de la Rosa A, et al
      . Patient participation in a clinical Guideline development for systemic lupus erythematosus. Patient Educ Couns 2015;98:1156–63.doi:10.1016/j.pec.2015.05.022pmid:http://www.ncbi.nlm.nih.gov/pubmed/26095343
      OpenUrlPubMed
    56. ↵
      1. Serrano-Aguilar P,
      2. Trujillo-Martin MM,
      3. del Pino-Sedeño T, et al
      . Patient participation in the development of a clinical guideline for inherited retinal dystrophies. Expert Opin Orphan Drugs 2016;4:691–7.doi:10.1080/21678707.2016.1182907
      OpenUrl
    57. ↵
      1. Solomon P,
      2. O'Brien KK,
      3. Baxter L, et al
      . Community involvement in development of evidence-informed recommendations for rehabilitation for older adults living with HIV. Prog Community Health Partnersh 2016;10:83–8.doi:10.1353/cpr.2016.0008pmid:http://www.ncbi.nlm.nih.gov/pubmed/27018357
      OpenUrlPubMed
    58. ↵
      1. Steinhart AH,
      2. Fernandes A
      . Clinical practice guidelines for the medical management of nonhospitalized ulcerative colitis: the patient perspective. Can J Gastroenterol Hepatol 2015;29:294–6.doi:10.1155/2015/214937pmid:http://www.ncbi.nlm.nih.gov/pubmed/26000730
      OpenUrlPubMed
    59. ↵
      1. Tong A,
      2. Lopez-Vargas P,
      3. Howell M, et al
      . Consumer involvement in topic and outcome selection in the development of clinical practice guidelines. Health Expect 2012;15:410–23.doi:10.1111/j.1369-7625.2011.00676.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/23134217
      OpenUrlCrossRefPubMed
    60. ↵
      1. Tong A,
      2. Tunnicliffe DJ,
      3. Lopez-Vargas P, et al
      . Identifying and integrating consumer perspectives in clinical practice guidelines on autosomal-dominant polycystic kidney disease. Nephrology 2016;21:122–32.doi:10.1111/nep.12579pmid:http://www.ncbi.nlm.nih.gov/pubmed/26235729
      OpenUrlPubMed
    61. ↵
      1. van der Ham AJ,
      2. van Erp N,
      3. Broerse JEW
      . Monitoring and evaluation of patient involvement in clinical practice Guideline development: lessons from the multidisciplinary guideline for employment and severe mental illness, the Netherlands. Health Expect 2016;19:471–82.doi:10.1111/hex.12370pmid:http://www.ncbi.nlm.nih.gov/pubmed/25980690
      OpenUrlPubMed
    62. ↵
      1. van der Veer SN,
      2. Haller MC,
      3. Pittens CACM, et al
      . Setting Priorities for Optimizing Vascular Access Decision Making--An International Survey of Patients and Clinicians. PLoS One 2015;10:e0128228.doi:10.1371/journal.pone.0128228pmid:http://www.ncbi.nlm.nih.gov/pubmed/26151822
      OpenUrlCrossRefPubMed
    63. ↵
      1. Westby MD,
      2. Backman CL
      . Patient and health professional views on rehabilitation practices and outcomes following total hip and knee arthroplasty for osteoarthritis:a focus group study. BMC Health Serv Res 2010;10:119.doi:10.1186/1472-6963-10-119pmid:http://www.ncbi.nlm.nih.gov/pubmed/20459834
      OpenUrlCrossRefPubMed
    64. ↵
      1. Wilson EJ,
      2. Nasrin D,
      3. Banwell C, et al
      . Realities of practice. engaging parents and GPs in developing clinical practice guidelines. Aust Fam Physician 2000;29:498–503.pmid:http://www.ncbi.nlm.nih.gov/pubmed/10835790
      OpenUrlPubMed
    65. ↵
      1. Behets FMTF,
      2. Rasolofomanana JR,
      3. Van Damme K, et al
      . Evidence-Based treatment guidelines for sexually transmitted infections developed with and for female sex workers. Trop Med Int Health 2003;8:251–8.doi:10.1046/j.1365-3156.2003.01017.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/12631316
      OpenUrlCrossRefPubMedWeb of Science
    66. ↵
      1. Côté P,
      2. Yu H,
      3. Shearer HM, et al
      . Non-Pharmacological management of persistent headaches associated with neck pain: a clinical practice guideline from the Ontario protocol for traffic injury management (optima) collaboration. Eur J Pain 2019;23:1051–70.doi:10.1002/ejp.1374pmid:http://www.ncbi.nlm.nih.gov/pubmed/30707486
      OpenUrlPubMed
    67. ↵
      1. Groenen KHJ,
      2. van der Linden YM,
      3. Brouwer T, et al
      . The Dutch national guideline on metastases and hematological malignancies localized within the spine; a multidisciplinary collaboration towards timely and proactive management. Cancer Treat Rev 2018;69:29–38.doi:10.1016/j.ctrv.2018.05.013pmid:http://www.ncbi.nlm.nih.gov/pubmed/29870874
      OpenUrlPubMed
    68. ↵
      1. Hatemi G,
      2. Silman A,
      3. Bang D, et al
      . EULAR recommendations for the management of Behçet disease. Ann Rheum Dis 2008;67:1656–62.doi:10.1136/ard.2007.080432pmid:http://www.ncbi.nlm.nih.gov/pubmed/18245110
      OpenUrlAbstract/FREE Full Text
    69. ↵
      1. Hoch E,
      2. Batra A,
      3. Mann K
      . Interdisciplinary S3 practice treatment guidelines for substance use disorders. Sucht 2012;58:97–107.
      OpenUrl
    70. ↵
      1. Kiltz U,
      2. Smolen J,
      3. Bardin T, et al
      . Treat-to-target (T2T) recommendations for gout. Ann Rheum Dis 2017;76:632–8.doi:10.1136/annrheumdis-2016-209467pmid:http://www.ncbi.nlm.nih.gov/pubmed/27658678
      OpenUrlAbstract/FREE Full Text
    71. ↵
      1. Loeffen EAH,
      2. Kremer LCM,
      3. de Wetering MD, et al
      . Reducing pain in children with cancer: methodology for the development of a clinical practice guideline. Pediatr Blood Cancer 2019;66:e27698.doi:10.1002/pbc.27698
    72. ↵
      1. McInnes E,
      2. Cullum N,
      3. Nelson EA, et al
      . The development of a national guideline on the management of leg ulcers. J Clin Nurs 2000;9:208–17.doi:10.1046/j.1365-2702.2000.00369.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/11111612
      OpenUrlPubMed
    73. ↵
      1. Solari A,
      2. Giordano A,
      3. Sastre-Garriga J, et al
      . Ean guideline on palliative care of people with severe, progressive multiple sclerosis. Eur J Neurol 2020;27:1510–29.doi:10.1111/ene.14248pmid:http://www.ncbi.nlm.nih.gov/pubmed/32469447
      OpenUrlCrossRefPubMed
    74. ↵
      1. Wolfe AD,
      2. Frierdich SA,
      3. Wish J, et al
      . Sharing life-altering information: development of pediatric hospital guidelines and team training. J Palliat Med 2014;17:1011–8.doi:10.1089/jpm.2013.0620pmid:http://www.ncbi.nlm.nih.gov/pubmed/24971650
      OpenUrlPubMed
    75. ↵
      1. Gutman T,
      2. Lopez-Vargas P,
      3. Manera KE, et al
      . Identifying and integrating patient and caregiver perspectives in clinical practice guidelines for percutaneous renal biopsy. Nephrology 2019;24:395–404.doi:10.1111/nep.13406pmid:http://www.ncbi.nlm.nih.gov/pubmed/29797384
      OpenUrlPubMed
    76. ↵
      1. Armstrong MJ,
      2. Bloom JA
      . Patient involvement in guidelines is poor five years after Institute of medicine standards: review of guideline methodologies. Res Involv Engagem 2017;3:19.doi:10.1186/s40900-017-0070-2pmid:http://www.ncbi.nlm.nih.gov/pubmed/29062544
      OpenUrlPubMed
    77. ↵
      1. Ernstzen DV,
      2. Hillier SL,
      3. Louw QA,
      4. QJBmrm L
      . An innovative method for clinical practice guideline contextualisation for chronic musculoskeletal pain in the South African context. BMC Med Res Methodol 2019;19:1–12.doi:10.1186/s12874-019-0771-3
      OpenUrlCrossRef
    78. ↵
      1. Khodyakov D,
      2. Grant S,
      3. Denger B, et al
      . Practical considerations in using online Modified-Delphi approaches to engage patients and other stakeholders in clinical practice Guideline development. Patient 2020;13:11–21.doi:10.1007/s40271-019-00389-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/31544219
      OpenUrlPubMed
    79. ↵
      1. Arigo D,
      2. Pagoto S,
      3. Carter-Harris L, et al
      . Using social media for health research: methodological and ethical considerations for recruitment and intervention delivery. Digit Health 2018;4:205520761877175.doi:10.1177/2055207618771757pmid:http://www.ncbi.nlm.nih.gov/pubmed/29942634
      OpenUrlPubMed
    80. ↵
      1. Köpke S,
      2. Giordano A,
      3. Veronese S
      . Patient and caregiver involvement in formulation of guideline questions: findings from the EAN guideline on palliative care of people with severe multiple sclerosis. Multiple Sclerosis Journal 2018;24:843–450.doi:10.1111/ene.13760pmid:http://www.ncbi.nlm.nih.gov/pubmed/30035845
      OpenUrlPubMed
    81. ↵
      1. Wiercioch W,
      2. Akl EA,
      3. Santesso N, et al
      . Assessing the process and outcome of the development of practice guidelines and recommendations: PANELVIEW instrument development. Can Med Assoc J 2020;192:E1138–45.doi:10.1503/cmaj.200193
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @2weekSR, @rthomasEBP

    • Contributors EAB, RT, HG and AMS were responsible for the initial design of this study, developed the search strategy and eligibility criteria and data extraction criteria. EAB executed the search strategy. EAB conducted screening and data extraction with HG. AB led the writing of manuscript. EAB, RT, HG and AMS contributed to and approved the final version of this protocol. EAB is responsible for the overall content as guarantor and accepts full responsibility for the finished work and/or the conduct of the study, had access to the data, and controlled the decision to publish.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.