You are here

Article Text

Article menu

Download PDFPDF

XML
Research methods
Protocol
Development of a prioritisation tool for the updating of clinical guideline questions: the UpPriority Tool protocol
  1. Laura Martínez García1,
  2. Hector Pardo-Hernandez1,2,
  3. Ena Niño de Guzman1,
  4. Cecilia Superchi1,
  5. Monica Ballesteros1,
  6. Emma McFarlane3,
  7. Katrina Penman3,
  8. Margarita Posso1,
  9. Marta Roqué i Figuls1,
  10. Andrea Juliana Sanabria1,
  11. Anna Selva4,
  12. Robin WM Vernooij1,
  13. Pablo Alonso-Coello1,2,5
  1. 1 Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau (IIB Sant Pau), Barcelona, Spain
  2. 2 CIBER of Epidemiology and Public Health (CIBERESP), Barcelona, Spain
  3. 3 National Institute for Health and Care Excellence, Manchester, UK
  4. 4 Clinical Epidemiology and Cancer Screening. Corporació Sanitaria Parc Taulí, Sabadell, Barcelona, Spain
  5. 5 Clinical Epidemiology and Biostatistics Department, McMaster University, Hamilton, Canada
  1. Correspondence to Dr Laura Martínez García; laura.martinez.garcia{at}cochrane.es

Abstract

Introduction Due to a continuous emergence of new evidence, clinical guidelines (CGs) require regular surveillance of evidence to maintain their trustworthiness. The updating of CGs is resource intensive and time consuming; therefore, updating may include a prioritisation process to efficiently ensure recommendations remain up to date. The objective of our project is to develop a pragmatic tool to prioritise clinical questions for updating within a CG.

Methods and analysis To develop the tool, we will use the results and conclusions of a systematic review of methodological research on prioritisation processes for updating and will adopt a methodological approach we have successfully implemented in a previous experience.

We will perform a multistep process including (1) generation of an initial version of the tool, (2) optimisation of the tool (feasibility test of the tool, semistructured interviews, Delphi consensus survey, external review by CG methodologists and users and pilot test of the tool) and (3) approval of the final version of the tool.

At each step of the process, we will (1) calculate absolute frequencies and proportions (quantitative data), (2) use content analysis to summarise and draw conclusions (qualitative data) and (3) draft a final report, discuss results and refine the previous versions of the tool. Finally, we will calculate intraclass coefficients with 95% CIs for each item and overall as indicators of agreement among reviewers.

Ethics and dissemination We have obtained a waiver of approval from the Clinical Research Ethics Committee at the Hospital de la Santa Creu i Sant Pau (Barcelona). The results of the study will be published in peer-reviewed journal and communicated to interested stakeholders.

The tool could support the standardisation of prioritisation processes for updating CGs and therefore have important implications for a more efficient use of resources in the CG field.

  • Clinical guidelines
  • evidence-based medicine
  • methodology
  • updating
  • prioritisation

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-017226

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • To develop the tool, we will use the results and conclusions of a systematic review of methodological research on prioritisation processes for updating.

    • We will adopt a methodological approach we have successfully implemented in a previous experience.

    • We will collect views from clinical guidelines (CG) developers (semistructured interviews and external reviews), CG methodological experts (Delphi consensus survey) and CG users (semistructured interviews); these will allow us to pool different stakeholders’ opinions about CG updating prioritisation processes.

    • The principal limitation of the study is that we will not perform a formal validation of the tool.

    Introduction

    Clinical guidelines (CGs) are ‘statements that include recommendations intended to optimise patient care that are informed by systematic reviews (SRs) of evidence and an assessment of the benefits and harms of alternative care options’.1 Due to a continuous emergence of new evidence,2 3 CGs require regular surveillance of evidence to maintain their trustworthiness.4 5

    Several studies have assessed length of time that CGs and their recommendations remain valid.4–8 Based on this evidence, most CG developers have adopted updating strategies based on predetermined time frames.9

    An updating strategy involves different processes including the identification of new evidence; the assessment of the impact of new evidence on the current CG recommendations and whether an update is required and the update of the CG if needed.9 10 The updating of CGs is resource intensive and time consuming.11 In the current context of restricted resources, there is a growing interest in approaches that support decision-making for updating CGs.12

    We define the prioritisation process for updating of CGs as the methodology used to determine which CGs should be prioritised to ensure that resources are invested in updating the topics that are most relevant to different stakeholders.12 The prioritisation process includes two main stages: (1) assessment of CGs using prioritisation criteria (eg, availability of new evidence, clinical relevance or users’ interest) and (2) classification of CGs in groups according to priority for updating (eg, high, medium or low relevance for updating).12

    Different prioritisation processes could be implemented at different time points within an updating strategy. For example, a prioritisation process could be implemented to identify the CGs in greatest need of update (prioritisation across available CGs)13 14 or to identify the clinical questions in greatest need of update within a prioritised CG (prioritisation within a CG).15 16

    Until now, there is wide variability and suboptimal reporting of the methods used to develop and implement processes to prioritise updating of CGs.12

    Aims and objectives

    Primary objective

    To develop a pragmatic tool to prioritise clinical questions for updating within a CG.

    Secondary objectives

    • To identify the most important items required to prioritise clinical questions for updating within a CG.

    • To describe each item, establish a rating scale of items and provide a guidance on how to rate them.

    • To develop guidance on how to calculate and present priority scores to support decision-making for updating clinical questions within a CG.

    Methods and analysis

    To develop the UpPriority Tool, we will use the results and conclusions of a systematic review of methodological research on prioritisation processes for updating12 and will adopt a methodological approach we have successfully implemented in a previous experience.17 We will perform a multistep process including (1) generation of an initial version of the tool, (2) optimisation of the tool (feasibility test of the tool, semistructured interviews, Delphi consensus survey, external review by CG methodologists and users and pilot test of the tool) and (3) approval of the final version of the tool (table 1, figure 1).

    View this table:
    Table 1

    Characteristics of the multistep development process

    Figure 1
    Figure 1

    Multistep development process.

    Generation of the initial version of the tool

    Objective

    The objective is to develop the initial version of the tool (items, scoring calculation and summary report).

    Method

    The UpPriority Steering Group (UpSG) will participate in informal discussion and will approve the initial version of the tool.

    Participants

    UpSG.

    Optimisation of the tool

    Feasibility test of the tool

    Objective

    The objective is to explore the feasibility and refine the initial version of the tool.

    Study design

    Methodological survey.

    Participants

    A CG developed within the Spanish National Health System Clinical Guideline Program, published within the last 2 years and with <50 clinical questions.

    Main outcome

    Time to apply the tool.

    Other variables

    Response rate, characteristics of participants and workplace, characteristics of clinical questions, priority scores (single item and overall items) and overall assessment of the tool (table 2).

    View this table:
    Table 2

    Study variables in multistep development process

    Data collection

    Two reviewers from the original Guideline Development Group (GDG) and two reviewers from the UpSG will apply the initial version of the tool. We will use online software to design the survey and collect responses (www.digestepiclin.com).

    Bias

    To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers. To minimise observer bias, two reviewers from outside the UpSG will apply the tool.

    Study size

    Convenience sample.18

    Data analysis

    For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

    Semistructured interviews

    Objective

    The objective is to identify current practices in prioritisation processes for updating CGs and to refine the initial version of the tool.

    Study design

    Semistructured interviews (face-to-face, telephone or internet).

    Participants

    CG developers that (1) have experience in CG development and/or updating (defined as having participated in GDG and/or Guideline Updating Group (GUG) at least once in the past year) and (2) are fluent in English or Spanish. We will identify participants with the help of the UpSG. When someone does not respond or cannot participate, another contributor will be recruited.

    Main outcome

    Participants’ experiences with prioritisation processes for updating CGs.

    Other variables

    Characteristics of participants and workplace, current practices in prioritisation processes for updating CGs, assessment of each item, assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

    Data collection

    Interviews will be audiotaped and transcribed (each interview will last approximately 1 hour).

    Bias

    To minimise interviewer bias, semistructured interviews will be conducted using an interview guide.

    Study size

    We will recruit participants and collect data until information becomes repetitive and no new information emerges (sampling saturation).20 21

    Data analysis

    For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

    Delphi consensus survey

    Objective

    The objective is to reach a consensus about the included items and refine the initial version of the tool.

    Study design

    Delphi consensus survey.

    Before the first Delphi round, we will provide the results of previous methodological research to Delphi panel members.

    In the first Delphi round, we will ask participants to rate whether each item should be included in the tool and its clarity using a seven-point Likert scale (1=strongly disagree and 7=strongly agree).22 We will calculate the median score for inclusion of each item and will classify them as (1) excluded (median score of 0–3 points), (2) review, modify and retest (median score of 4–5 points or with substantial comments) and (3) included (median score of 6 to 7 points and without substantial comments).

    After each Delphi round, we will provide feedback to Delphi panel members (all responses will be anonymised prior to circulation). We will conduct additional Delphi rounds until consensus for inclusion or exclusion is reached and no more relevant comments were provided (two or three rounds, as needed).

    Participants

    CG methodological experts that (1) have methodological experience in CGs development and/or updating (defined as having participated in a CG technical team at least once in the past year and/or in methodological research) and (2) are fluent in English or Spanish. We will identify participants by contacting professionals associated with the Guidelines International Network (G-I-N) Updating Guidelines Working Group (http://www.g-i-n.net/working-groups/updating-guidelines) or authors of methodological research. Non-responders will not be invited to subsequent rounds.

    Main outcome

    Items considered important to prioritise clinical questions for updating within a CG.

    Other variables (per round)

    Characteristics of participants and workplace, assessment of each item (inclusion and understanding), assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

    Data collection

    We will use online software to design the survey and collect responses (www.digestepiclin.com).

    Bias

    To minimise selection bias of Delphi panel members, all G-I-N Updating Guidelines Working Group members will be invited to participate. To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers.

    Study size

    Twenty to 30 participants.23

    Data analysis

    For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

    External review

    External review with clinical guidelines developers

    Objective

    The objective is to assess the usefulness and understanding of each item and refine the initial version of the tool.

    Study design

    Survey.

    Participants

    CG developers that (1) have experience in CG development/updating (defined as having participated in GDG and/or GUG at least once in the past year) and (2) are fluent in English or Spanish. We will identify participants by contacting professionals associated with the G-I-N community (http://www.g-i-n.net).

    Main outcome

    Usefulness rating for each item of the tool.

    Other variables

    Characteristics of participants and workplace, assessment of each item (usefulness and understanding), assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

    Data collection

    We will use online software to design the survey and collect responses (www.digestepiclin.com).

    Bias

    To minimise selection bias of survey participants, all G-I-N members will be invited to participate. To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers. Furthermore, the questionnaire will be pilot tested to improve wording and layout.

    Study size

    Currently, about 250 organisations and individual members are registered in the G-I-N community (http://www.g-i-n.net/membership/members-around-the-world).

    Data analysis

    For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

    External review with clinical guidelines users

    Objective

    The objective is to assess the usefulness and understanding of each item and refine the initial version of the tool.

    Study design

    Semistructured interviews (face-to-face, telephone or internet).

    Participants

    CG users (defined as healthcare professionals that use CGs on a regular basis) who are fluent in English or Spanish. We will identify participants with the help of the UpSG. When someone does not respond or cannot participate, a new contributor will be recruited.

    Main outcome

    Participants’ views of prioritisation processes for updating CGs with the tool.

    Other variables

    Characteristics of participants and workplace, assessment of each item (usefulness and understanding), assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

    Data collection

    Interviews will be audiotaped and transcribed (each interview will last approximately 1 hour).

    Bias

    To minimise interviewer bias, semistructured interviews will be conducted using an interview guide.

    Study size

    We will recruit participants and collect data until information becomes repetitive and no new information emerges (sampling saturation).20 21

    Data analysis

    For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

    Pilot test of the tool

    Objective

    The objective is to explore the interobserver reliability of the final version of the tool and refine the initial version of the tool.

    Study design

    Methodological survey.

    Participants

    A CG developed within the Spanish National Health System Clinical Guideline Programme, published within the last 2 years and with <50 clinical questions.

    Main outcome

    Intraclass coefficient (ICC) with 95% CI for each item and overall.

    Other variables

    Response rate, characteristics of participants and workplace, characteristics of clinical questions and priority scores (single item) and overall assessment of the tool (table 2).

    Data collection

    Two reviewers from the original GDG and two reviewers from the UpSG will apply the initial version of the tool. We will use online software to design the survey and collect responses (www.digestepiclin.com).

    Bias

    To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers. To minimise observer bias, two reviewers from outside the UpSG will apply the tool.

    Study size

    Convenience sample; the results of the pilot test will inform the sample size calculation for a subsequent main study.24

    Data analysis

    For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will calculate the ICC with 95% CI for each item and overall as an indicator of agreement among reviewers. According to the scale proposed by Landis and Koch, the degree of agreement between 0.00 and 0.20 is poor, from 0.21 to 0.40 is fair, from 0.41 to 0.60 is moderate, from 0.61 to 0.80 is substantial and from 0.81 to 1.00 is almost perfect.25 We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

    Approval of the final version of the tool

    Objective

    The objective is to approve the final version of the tool (items, scoring calculation and summary report).

    Method

    The UpSG will participate in informal discussion and will approve the final version of the tool.

    Participants

    UpSG.

    Ethics and dissemination

    We have obtained a waiver of approval from the Clinical Research Ethics Committee at the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain), since this study will not involve patients or biological samples.

    The results of the study will be published in peer-reviewed journal and communicated to interested stakeholders (eg, via international conferences, electronic bulletin or website).

    We will develop the UpPriority tool through a comprehensive development process, including the use of previous methodological evidence,12 17 feasibility testing of the tool and engagement of the international CG community (semistructured interviews, Delphi consensus survey and external review) and finally a pilot testing of the tool.

    Previous SRs on CG updating strategies found limited evidence on processes that could inform the decision of which CGs should be prioritised for updating.9 10 26 There are, nevertheless, new studies that underscore the relevance of the prioritisation process in CG updating,13 27 coinciding with a growing interest among developers to shift from developing to updating CGs.28

    We recently systematically reviewed the available evidence on strategies to prioritise the updating of SRs, health technology assessments and CGs.12 We observed that there is wide variability and suboptimal reporting of the methods used to develop and implement such prioritisation processes. Therefore, developers may have difficulties selecting and implementing a prioritisation method to optimise the updating process of CGs.

    Agbassi et al 13 implemented an annual step-by-step prioritisation process of CGs for updating.13 The authors reviewed CGs using two questionnaires; the process requires evidence search, evidence review and review approval.13 We will build our proposal on this process while addressing some of its shortcomings. Following a comprehensive development process, we will develop a pragmatic survey based tool that will likely be less resource intensive and time consuming compared with formal approaches (based on step-by-step algorithm that generally includes literature searches). We will also publish detailed and explicit guidance to allow developers to implement the tool in their institutions and to adapt it, if needed, to their specific circumstances.

    We expect to develop a pragmatic tool (items, scoring calculation and summary report) that will be applicable to all clinical questions within a CG and should be easy to uptake by CG developers. The UpPriority Tool could support the standardisation of prioritisation processes for updating CGs and therefore have important implications for a more efficient use of resources in the CG field.

    References

    1. 1.↵
      2. Steinberg E ,
      3. Greenfield S ,
      4. Mancher M , et al
      . Clinical practice guidelines we can trust. National Academies Press, 2011.
    2. 2.↵
      2. Bastian H ,
      3. Glasziou P ,
      4. Chalmers I
      . Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 2010;7:e1000326.doi:10.1371/journal.pmed.1000326
    3. 3.↵
      2. Ioannidis JP
      . The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 2016;94:485–514.doi:10.1111/1468-0009.12210
      OpenUrlCrossRefPubMed
    4. 4.↵
      2. Martínez García L ,
      3. Sanabria AJ ,
      4. García Alvarez E , et al
      . The validity of recommendations from clinical guidelines: a survival analysis. CMAJ 2014;186:1211–9.doi:10.1503/cmaj.140547
      OpenUrlAbstract/FREE Full Text
    5. 5.↵
      2. Shekelle PG ,
      3. Ortiz E ,
      4. Rhodes S , et al
      . Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 2001;286:1461–7.
      OpenUrlCrossRefPubMedWeb of Science
    6. 6.↵
      2. Alderson LJ ,
      3. Alderson P ,
      4. Tan T
      . Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months. J Clin Epidemiol 2014;67:52–5.doi:10.1016/j.jclinepi.2013.07.012
      OpenUrlCrossRefPubMed
    7. 7.↵
      2. Lyratzopoulos G ,
      3. Barnes S ,
      4. Stegenga H , et al
      . Updating clinical practice recommendations: is it worthwhile and when? Int J Technol Assess Health Care 2012;28:29–35.doi:10.1017/S0266462311000675
      OpenUrlCrossRefPubMed
    8. 8.↵
      2. Neuman MD ,
      3. Goldstein JN ,
      4. Cirullo MA , et al
      . Durability of class I American College of Cardiology/American Heart Association clinical practice guideline recommendations. JAMA 2014;311:2092–100.doi:10.1001/jama.2014.4949
      OpenUrlCrossRefPubMedWeb of Science
    9. 9.↵
      2. Vernooij RW ,
      3. Sanabria AJ ,
      4. Solà I , et al
      . Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implement Sci 2014;9:3.doi:10.1186/1748-5908-9-3
      OpenUrlCrossRefPubMed
    10. 10.↵
      2. Martínez García L ,
      3. Arévalo-Rodríguez I ,
      4. Solà I , et al
      . Strategies for monitoring and updating clinical practice guidelines: a systematic review. Implement Sci 2012;7:109.doi:10.1186/1748-5908-7-109
      OpenUrlCrossRefPubMed
    11. 11.↵
      2. Martínez García L ,
      3. Pardo-Hernández H ,
      4. Sanabria AJ , et al
      . Continuous surveillance of a pregnancy clinical guideline: an early experience. 2017.
    12. 12.↵
      2. Martínez García L ,
      3. Pardo-Hernandez H ,
      4. Superchi C , et al
      . Methodological systematic review identifies major limitations in prioritization processes for updating. J Clin Epidemiol 2017;86:11–24. Epub ahead of print.doi:10.1016/j.jclinepi.2017.05.008
      OpenUrl
    13. 13.↵
      2. Agbassi C ,
      3. Messersmith H ,
      4. McNair S , et al
      . Priority-based initiative for updating existing evidence-based clinical practice guidelines: the results of two iterations. J Clin Epidemiol 2014;67:1335–42.doi:10.1016/j.jclinepi.2014.06.013
      OpenUrl
    14. 14.↵
      2. Jamshidi A ,
      3. Lamontagne M ,
      4. Ait-kadi D , et al
      . Developing a comprehensive Priority-based framework for updating clinical Practice guidelines (CPGs): A systematic review and international survey. Québec, Canada: Huitième Colloque Étudiant du CIRRIS, 2016.
    15. 15.↵
      2. Amos Q ,
      3. Chan W ,
      4. Tom G
      . Maximising efficiency in updating guidelines through prioritisation of clinical questions [abstract]. 10th G-I-N conference. San Francisco, California, 2013.
    16. 16.↵
      2. Theobald S ,
      3. Blanc-Vincent MP ,
      4. Farsi F , et al
      . 1999. The identification of questions in the updating process of clinical practice guidelines for oncology [abstract]. 15th Annual Meeting of the International Society of Technology Assessment in Health Care, Edinburgh (UK)
    17. 17.↵
      2. Vernooij RW ,
      3. Alonso-Coello P ,
      4. Brouwers M , et al
      . Reporting items for updated clinical guidelines: checklist for the reporting of updated guidelines (CheckUp). PLoS Med 2017;14:e1002207.doi:10.1371/journal.pmed.1002207
    18. 18.↵
      2. Arain M ,
      3. Campbell MJ ,
      4. Cooper CL , et al
      . What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol 2010;10:67.doi:10.1186/1471-2288-10-67
      OpenUrlCrossRefPubMed
    19. 19.↵
      2. Thomas J ,
      3. Harden A
      . Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8:45.doi:10.1186/1471-2288-8-45
      OpenUrlCrossRefPubMed
    20. 20.↵
      2. Grossoehme DH
      . Overview of qualitative research. J Health Care Chaplain 2014;20:109–22.doi:10.1080/08854726.2014.925660
      OpenUrl
    21. 21.↵
      2. Guetterman TC
      . Descriptions of sampling practices within five approaches to qualitative research in education and the health sciences. Forum Qual Soc Res 2015;16:25.
      OpenUrl
    22. 22.↵
      2. Likert R
      . A technique for the measurement of attitudes. Archives of psychology 1932.
    23. 23.↵
      2. Akins RB ,
      3. Tolson H ,
      4. Cole BR
      . Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol 2005;5:37.doi:10.1186/1471-2288-5-37
      OpenUrlCrossRefPubMed
    24. 24.↵
      2. Thabane L ,
      3. Ma J ,
      4. Chu R , et al
      . A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010;10:1.doi:10.1186/1471-2288-10-1
      OpenUrlCrossRefPubMed
    25. 25.↵
      2. Landis JR ,
      3. Koch GG
      . The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.doi:10.2307/2529310
      OpenUrlCrossRefPubMedWeb of Science
    26. 26.↵
      2. Becker M ,
      3. Neugebauer EA ,
      4. Eikermann M
      . Partial updating of clinical practice guidelines often makes more sense than full updating: a systematic review on methods and the development of an updating procedure. J Clin Epidemiol 2014;67:33–45.doi:10.1016/j.jclinepi.2013.06.021
      OpenUrlCrossRefPubMed
    27. 27.↵
      2. Lord J ,
      3. Willis S ,
      4. Eatock J , et al
      . Economic modelling of diagnostic and treatment pathways in National Institute for Health and Care Excellence clinical guidelines: the Modelling Algorithm Pathways in Guidelines (MAPGuide) project. Health Technol Assess 2013;17:1–192.doi:10.3310/hta17580
      OpenUrlCrossRefPubMedWeb of Science
    28. 28.↵
      2. Baker M
      . Maintaining the Currency of a Large Guidelines Portfolio; Necessity is the Mother of Invention. Abstracts directory of the 13th G-I-N Conference. Philadelphia, USA, 2016. http://www.g-in.net/document-store/g-i-n-conferences/philadelphia-2016/abstract-directory-2016.
    29. 29.
      2. van der Weegen S ,
      3. Verwey R ,
      4. Tange HJ , et al
      . Usability testing of a monitoring and feedback tool to stimulate physical activity. Patient Prefer Adherence 2014;8:311–22.doi:10.2147/PPA.S57961
      OpenUrlPubMed

    Footnotes

    • Contributors LMG and PAC were involved in conception and study design. LMG, HPH, ENG and CS were involved in drafting of the first version of the article. LMG, HPH, ENG, CS, MB, EM, KP, MP, MRF, AJS, AS, RWMV and PAC were involved in critical revision of the article for important intellectual content. LMG, HPH, ENG, CS, MB, EM, KP, MP, MRF, AJS, AS, RWMV and PAC were involved in final approval of the article.

    • Funding This study has been funded by Instituto de Salud Carlos III through the project 'PI15/00325' (Cofunded by European Regional Development Fund/European Social Fund, 'Investing in your future'). Laura Martínez García is funded by a Juan Rodés contract from the Instituto de Salud Carlos III (JR15/00004). Pablo Alonso-Coello is funded by a Miguel Servet research contract from the Instituto de Salud Carlos III (MSII15/00034).

    • Competing interests None declared.

    • Ethics approval Clinical Research Ethics Committee (Hospital de la Santa Creu i Sant Pau, Barcelona, Spain).

    • Provenance and peer review Not commissioned; externally peer reviewed.