Article Text

PDF

Development of a prioritisation tool for the updating of clinical guideline questions: the UpPriority Tool protocol
  1. Laura Martínez García1,
  2. Hector Pardo-Hernandez1,2,
  3. Ena Niño de Guzman1,
  4. Cecilia Superchi1,
  5. Monica Ballesteros1,
  6. Emma McFarlane3,
  7. Katrina Penman3,
  8. Margarita Posso1,
  9. Marta Roqué i Figuls1,
  10. Andrea Juliana Sanabria1,
  11. Anna Selva4,
  12. Robin WM Vernooij1,
  13. Pablo Alonso-Coello1,2,5
  1. 1 Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau (IIB Sant Pau), Barcelona, Spain
  2. 2 CIBER of Epidemiology and Public Health (CIBERESP), Barcelona, Spain
  3. 3 National Institute for Health and Care Excellence, Manchester, UK
  4. 4 Clinical Epidemiology and Cancer Screening. Corporació Sanitaria Parc Taulí, Sabadell, Barcelona, Spain
  5. 5 Clinical Epidemiology and Biostatistics Department, McMaster University, Hamilton, Canada
  1. Correspondence to Dr Laura Martínez García; laura.martinez.garcia{at}cochrane.es

Abstract

Introduction Due to a continuous emergence of new evidence, clinical guidelines (CGs) require regular surveillance of evidence to maintain their trustworthiness. The updating of CGs is resource intensive and time consuming; therefore, updating may include a prioritisation process to efficiently ensure recommendations remain up to date. The objective of our project is to develop a pragmatic tool to prioritise clinical questions for updating within a CG.

Methods and analysis To develop the tool, we will use the results and conclusions of a systematic review of methodological research on prioritisation processes for updating and will adopt a methodological approach we have successfully implemented in a previous experience.

We will perform a multistep process including (1) generation of an initial version of the tool, (2) optimisation of the tool (feasibility test of the tool, semistructured interviews, Delphi consensus survey, external review by CG methodologists and users and pilot test of the tool) and (3) approval of the final version of the tool.

At each step of the process, we will (1) calculate absolute frequencies and proportions (quantitative data), (2) use content analysis to summarise and draw conclusions (qualitative data) and (3) draft a final report, discuss results and refine the previous versions of the tool. Finally, we will calculate intraclass coefficients with 95% CIs for each item and overall as indicators of agreement among reviewers.

Ethics and dissemination We have obtained a waiver of approval from the Clinical Research Ethics Committee at the Hospital de la Santa Creu i Sant Pau (Barcelona). The results of the study will be published in peer-reviewed journal and communicated to interested stakeholders.

The tool could support the standardisation of prioritisation processes for updating CGs and therefore have important implications for a more efficient use of resources in the CG field.

  • Clinical guidelines
  • evidence-based medicine
  • methodology
  • updating
  • prioritisation

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Strengths and limitations of this study

  • To develop the tool, we will use the results and conclusions of a systematic review of methodological research on prioritisation processes for updating.

  • We will adopt a methodological approach we have successfully implemented in a previous experience.

  • We will collect views from clinical guidelines (CG) developers (semistructured interviews and external reviews), CG methodological experts (Delphi consensus survey) and CG users (semistructured interviews); these will allow us to pool different stakeholders’ opinions about CG updating prioritisation processes.

  • The principal limitation of the study is that we will not perform a formal validation of the tool.

Introduction

Clinical guidelines (CGs) are ‘statements that include recommendations intended to optimise patient care that are informed by systematic reviews (SRs) of evidence and an assessment of the benefits and harms of alternative care options’.1 Due to a continuous emergence of new evidence,2 3 CGs require regular surveillance of evidence to maintain their trustworthiness.4 5

Several studies have assessed length of time that CGs and their recommendations remain valid.4–8 Based on this evidence, most CG developers have adopted updating strategies based on predetermined time frames.9

An updating strategy involves different processes including the identification of new evidence; the assessment of the impact of new evidence on the current CG recommendations and whether an update is required and the update of the CG if needed.9 10 The updating of CGs is resource intensive and time consuming.11 In the current context of restricted resources, there is a growing interest in approaches that support decision-making for updating CGs.12

We define the prioritisation process for updating of CGs as the methodology used to determine which CGs should be prioritised to ensure that resources are invested in updating the topics that are most relevant to different stakeholders.12 The prioritisation process includes two main stages: (1) assessment of CGs using prioritisation criteria (eg, availability of new evidence, clinical relevance or users’ interest) and (2) classification of CGs in groups according to priority for updating (eg, high, medium or low relevance for updating).12

Different prioritisation processes could be implemented at different time points within an updating strategy. For example, a prioritisation process could be implemented to identify the CGs in greatest need of update (prioritisation across available CGs)13 14 or to identify the clinical questions in greatest need of update within a prioritised CG (prioritisation within a CG).15 16

Until now, there is wide variability and suboptimal reporting of the methods used to develop and implement processes to prioritise updating of CGs.12

Aims and objectives

Primary objective

To develop a pragmatic tool to prioritise clinical questions for updating within a CG.

Secondary objectives

  • To identify the most important items required to prioritise clinical questions for updating within a CG.

  • To describe each item, establish a rating scale of items and provide a guidance on how to rate them.

  • To develop guidance on how to calculate and present priority scores to support decision-making for updating clinical questions within a CG.

Methods and analysis

To develop the UpPriority Tool, we will use the results and conclusions of a systematic review of methodological research on prioritisation processes for updating12 and will adopt a methodological approach we have successfully implemented in a previous experience.17 We will perform a multistep process including (1) generation of an initial version of the tool, (2) optimisation of the tool (feasibility test of the tool, semistructured interviews, Delphi consensus survey, external review by CG methodologists and users and pilot test of the tool) and (3) approval of the final version of the tool (table 1, figure 1).

Table 1

Characteristics of the multistep development process

Figure 1

Multistep development process.

Generation of the initial version of the tool

Objective

The objective is to develop the initial version of the tool (items, scoring calculation and summary report).

Method

The UpPriority Steering Group (UpSG) will participate in informal discussion and will approve the initial version of the tool.

Participants

UpSG.

Optimisation of the tool

Feasibility test of the tool

Objective

The objective is to explore the feasibility and refine the initial version of the tool.

Study design

Methodological survey.

Participants

A CG developed within the Spanish National Health System Clinical Guideline Program, published within the last 2 years and with <50 clinical questions.

Main outcome

Time to apply the tool.

Other variables

Response rate, characteristics of participants and workplace, characteristics of clinical questions, priority scores (single item and overall items) and overall assessment of the tool (table 2).

Table 2

Study variables in multistep development process

Data collection

Two reviewers from the original Guideline Development Group (GDG) and two reviewers from the UpSG will apply the initial version of the tool. We will use online software to design the survey and collect responses (www.digestepiclin.com).

Bias

To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers. To minimise observer bias, two reviewers from outside the UpSG will apply the tool.

Study size

Convenience sample.18

Data analysis

For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

Semistructured interviews

Objective

The objective is to identify current practices in prioritisation processes for updating CGs and to refine the initial version of the tool.

Study design

Semistructured interviews (face-to-face, telephone or internet).

Participants

CG developers that (1) have experience in CG development and/or updating (defined as having participated in GDG and/or Guideline Updating Group (GUG) at least once in the past year) and (2) are fluent in English or Spanish. We will identify participants with the help of the UpSG. When someone does not respond or cannot participate, another contributor will be recruited.

Main outcome

Participants’ experiences with prioritisation processes for updating CGs.

Other variables

Characteristics of participants and workplace, current practices in prioritisation processes for updating CGs, assessment of each item, assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

Data collection

Interviews will be audiotaped and transcribed (each interview will last approximately 1 hour).

Bias

To minimise interviewer bias, semistructured interviews will be conducted using an interview guide.

Study size

We will recruit participants and collect data until information becomes repetitive and no new information emerges (sampling saturation).20 21

Data analysis

For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

Delphi consensus survey

Objective

The objective is to reach a consensus about the included items and refine the initial version of the tool.

Study design

Delphi consensus survey.

Before the first Delphi round, we will provide the results of previous methodological research to Delphi panel members.

In the first Delphi round, we will ask participants to rate whether each item should be included in the tool and its clarity using a seven-point Likert scale (1=strongly disagree and 7=strongly agree).22 We will calculate the median score for inclusion of each item and will classify them as (1) excluded (median score of 0–3 points), (2) review, modify and retest (median score of 4–5 points or with substantial comments) and (3) included (median score of 6 to 7 points and without substantial comments).

After each Delphi round, we will provide feedback to Delphi panel members (all responses will be anonymised prior to circulation). We will conduct additional Delphi rounds until consensus for inclusion or exclusion is reached and no more relevant comments were provided (two or three rounds, as needed).

Participants

CG methodological experts that (1) have methodological experience in CGs development and/or updating (defined as having participated in a CG technical team at least once in the past year and/or in methodological research) and (2) are fluent in English or Spanish. We will identify participants by contacting professionals associated with the Guidelines International Network (G-I-N) Updating Guidelines Working Group (http://www.g-i-n.net/working-groups/updating-guidelines) or authors of methodological research. Non-responders will not be invited to subsequent rounds.

Main outcome

Items considered important to prioritise clinical questions for updating within a CG.

Other variables (per round)

Characteristics of participants and workplace, assessment of each item (inclusion and understanding), assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

Data collection

We will use online software to design the survey and collect responses (www.digestepiclin.com).

Bias

To minimise selection bias of Delphi panel members, all G-I-N Updating Guidelines Working Group members will be invited to participate. To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers.

Study size

Twenty to 30 participants.23

Data analysis

For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

External review

External review with clinical guidelines developers

Objective

The objective is to assess the usefulness and understanding of each item and refine the initial version of the tool.

Study design

Survey.

Participants

CG developers that (1) have experience in CG development/updating (defined as having participated in GDG and/or GUG at least once in the past year) and (2) are fluent in English or Spanish. We will identify participants by contacting professionals associated with the G-I-N community (http://www.g-i-n.net).

Main outcome

Usefulness rating for each item of the tool.

Other variables

Characteristics of participants and workplace, assessment of each item (usefulness and understanding), assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

Data collection

We will use online software to design the survey and collect responses (www.digestepiclin.com).

Bias

To minimise selection bias of survey participants, all G-I-N members will be invited to participate. To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers. Furthermore, the questionnaire will be pilot tested to improve wording and layout.

Study size

Currently, about 250 organisations and individual members are registered in the G-I-N community (http://www.g-i-n.net/membership/members-around-the-world).

Data analysis

For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

External review with clinical guidelines users

Objective

The objective is to assess the usefulness and understanding of each item and refine the initial version of the tool.

Study design

Semistructured interviews (face-to-face, telephone or internet).

Participants

CG users (defined as healthcare professionals that use CGs on a regular basis) who are fluent in English or Spanish. We will identify participants with the help of the UpSG. When someone does not respond or cannot participate, a new contributor will be recruited.

Main outcome

Participants’ views of prioritisation processes for updating CGs with the tool.

Other variables

Characteristics of participants and workplace, assessment of each item (usefulness and understanding), assessment of the scoring calculation, assessment of the summary report and overall assessment of the tool (table 2).

Data collection

Interviews will be audiotaped and transcribed (each interview will last approximately 1 hour).

Bias

To minimise interviewer bias, semistructured interviews will be conducted using an interview guide.

Study size

We will recruit participants and collect data until information becomes repetitive and no new information emerges (sampling saturation).20 21

Data analysis

For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

Pilot test of the tool

Objective

The objective is to explore the interobserver reliability of the final version of the tool and refine the initial version of the tool.

Study design

Methodological survey.

Participants

A CG developed within the Spanish National Health System Clinical Guideline Programme, published within the last 2 years and with <50 clinical questions.

Main outcome

Intraclass coefficient (ICC) with 95% CI for each item and overall.

Other variables

Response rate, characteristics of participants and workplace, characteristics of clinical questions and priority scores (single item) and overall assessment of the tool (table 2).

Data collection

Two reviewers from the original GDG and two reviewers from the UpSG will apply the initial version of the tool. We will use online software to design the survey and collect responses (www.digestepiclin.com).

Bias

To minimise non-response bias, the survey will be available online for 1 month; weekly email reminders will be sent to reviewers. To minimise observer bias, two reviewers from outside the UpSG will apply the tool.

Study size

Convenience sample; the results of the pilot test will inform the sample size calculation for a subsequent main study.24

Data analysis

For quantitative data, we will calculate absolute frequencies and proportions. For qualitative data, we will use content analysis to summarise and draw conclusions (atlasti.com).19 Questionnaires with no response in over 20% of the items will be withdrawn. We will calculate the ICC with 95% CI for each item and overall as an indicator of agreement among reviewers. According to the scale proposed by Landis and Koch, the degree of agreement between 0.00 and 0.20 is poor, from 0.21 to 0.40 is fair, from 0.41 to 0.60 is moderate, from 0.61 to 0.80 is substantial and from 0.81 to 1.00 is almost perfect.25 We will draft a final report, discuss results and refine the initial version of the tool with the UpSG.

Approval of the final version of the tool

Objective

The objective is to approve the final version of the tool (items, scoring calculation and summary report).

Method

The UpSG will participate in informal discussion and will approve the final version of the tool.

Participants

UpSG.

Ethics and dissemination

We have obtained a waiver of approval from the Clinical Research Ethics Committee at the Hospital de la Santa Creu i Sant Pau (Barcelona, Spain), since this study will not involve patients or biological samples.

The results of the study will be published in peer-reviewed journal and communicated to interested stakeholders (eg, via international conferences, electronic bulletin or website).

We will develop the UpPriority tool through a comprehensive development process, including the use of previous methodological evidence,12 17 feasibility testing of the tool and engagement of the international CG community (semistructured interviews, Delphi consensus survey and external review) and finally a pilot testing of the tool.

Previous SRs on CG updating strategies found limited evidence on processes that could inform the decision of which CGs should be prioritised for updating.9 10 26 There are, nevertheless, new studies that underscore the relevance of the prioritisation process in CG updating,13 27 coinciding with a growing interest among developers to shift from developing to updating CGs.28

We recently systematically reviewed the available evidence on strategies to prioritise the updating of SRs, health technology assessments and CGs.12 We observed that there is wide variability and suboptimal reporting of the methods used to develop and implement such prioritisation processes. Therefore, developers may have difficulties selecting and implementing a prioritisation method to optimise the updating process of CGs.

Agbassi et al 13 implemented an annual step-by-step prioritisation process of CGs for updating.13 The authors reviewed CGs using two questionnaires; the process requires evidence search, evidence review and review approval.13 We will build our proposal on this process while addressing some of its shortcomings. Following a comprehensive development process, we will develop a pragmatic survey based tool that will likely be less resource intensive and time consuming compared with formal approaches (based on step-by-step algorithm that generally includes literature searches). We will also publish detailed and explicit guidance to allow developers to implement the tool in their institutions and to adapt it, if needed, to their specific circumstances.

We expect to develop a pragmatic tool (items, scoring calculation and summary report) that will be applicable to all clinical questions within a CG and should be easy to uptake by CG developers. The UpPriority Tool could support the standardisation of prioritisation processes for updating CGs and therefore have important implications for a more efficient use of resources in the CG field.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
View Abstract

Footnotes

  • Contributors LMG and PAC were involved in conception and study design. LMG, HPH, ENG and CS were involved in drafting of the first version of the article. LMG, HPH, ENG, CS, MB, EM, KP, MP, MRF, AJS, AS, RWMV and PAC were involved in critical revision of the article for important intellectual content. LMG, HPH, ENG, CS, MB, EM, KP, MP, MRF, AJS, AS, RWMV and PAC were involved in final approval of the article.

  • Funding This study has been funded by Instituto de Salud Carlos III through the project 'PI15/00325' (Cofunded by European Regional Development Fund/European Social Fund, 'Investing in your future'). Laura Martínez García is funded by a Juan Rodés contract from the Instituto de Salud Carlos III (JR15/00004). Pablo Alonso-Coello is funded by a Miguel Servet research contract from the Instituto de Salud Carlos III (MSII15/00034).

  • Competing interests None declared.

  • Ethics approval Clinical Research Ethics Committee (Hospital de la Santa Creu i Sant Pau, Barcelona, Spain).

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.