Article Text
PDF
Abstract
Objectives Guidelines that include antimicrobial recommendations should explicitly consider contextual factors that influence antimicrobial resistance and their downstream effects on resistance selection. The objectives were to analyse (1) how, and to what extent, tuberculosis, gonorrhoea and respiratory tract infection guidelines are considering antimicrobial resistance; (2) are of acceptable quality and (3) if they can be easily contextualised to fit the needs of specific populations and health systems.
Methods We conducted a systematic review and searched Ovid MEDLINE and Embase from 1 January 2007 to 7 June 2019 for tuberculosis, gonorrhoea and respiratory tract infection guidelines published in English. We also searched guideline databases, key websites and reference lists. We identified guidelines and recommendations that considered contextual factors including antimicrobial resistance, values, resource use, equity, acceptability and feasibility. We assessed quality of the guidelines using the Appraisal of Guidelines for Research and Evaluation II tool focusing on the domains scope and purpose, rigour of development, and editorial independence.
Results We screened 10 365 records, of which 74 guidelines met inclusion criteria. Of these guidelines, 39% (n=29/74) met acceptable quality scores. Approximately two-thirds of recommendations considered antimicrobial resistance at the population and/or outcome level. Five of the 29 guidelines reported all factors required for recommendation contextualisation. Equity was the least considered across guidelines.
Discussion Relatively few guidelines for highly prevalent infectious diseases are considering resistance at a local level, and many do not consider contextual factors necessary for appropriate antimicrobial use. Improving the quality of guidelines targeting specific regional areas is required.
PROSPERO registration number CRD42020145235.
- Antimicrobial resistance
- tuberculosis
- gonorrhoea
- respiratory tract infections
- guidelines
- recommendations
- contexts
- GRADE
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. No additional data available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Antimicrobial resistance
- tuberculosis
- gonorrhoea
- respiratory tract infections
- guidelines
- recommendations
- contexts
- GRADE
Strengths and limitations of this study
This is the first study to assess whether guidelines are considering local dimensions such as antimicrobial resistance.
We employed systematic methods and used established frameworks to assess the credibility of guidelines.
By focusing on three key Appraisal of Guidelines for Research and Evaluation (AGREE) II domains and a relatively low score we were is inclusive but we included only English language publications.
The use of the credibility cut-off score of 60% or greater for three of the six AGREE II domains is based on limited guidance on cut-off thresholds.
We used criteria of the Grading of Recommendations, Assessment, Development and Evaluation Evidence to Decision Frameworks that are fairly general as they apply to any interventions and may need to be complemented with specific criteria related to the antimicrobial field.
Introduction
Antimicrobials are essential to protecting human health. Their effectiveness is under threat due to antimicrobial resistance (AMR), resulting from misuse of antimicrobials over several decades. At the 2015 United Nations General Assembly, member states committed to address AMR by adopting national plans centred on five strategic objectives outlined in the WHO’s Global Action Plan.1 2 The fourth objective of this plan is to implement national and hospital treatment guidelines for the optimisation of antimicrobial medicines use.2 Guidelines are among AMR stewardship interventions intended to modify clinician behaviour by providing guidance on when, and how, to prescribe antimicrobials, integrating information on antimicrobial consumption, resistance surveillance, research and development and burden of resistance.3–5
Preservation of antimicrobials requires the consideration of how, and under what conditions, is it appropriate to recommend antimicrobials. However, only a small number of recently published guidelines considered epidemiological and resistance pattern data.6 There are also concerns that guidelines are not considering important contextual factors, including evidence on values, resource use, equity, acceptability and feasibility that go beyond resistance patterns and that may influence secular trends in AMR.7 8 For example, guideline recommendations are likely to better support effective use of antimicrobials in specific contexts when they account for how much people value the affected health outcomes (‘values’), AMR burden, public health infrastructure, local medicine policies for consistent access to safe, effective, affordable medicines and equitability of antimicrobial regimens.9 Considering these factors is also relevant for adapting and implementing. The failure to account for these factors likely results from the lack of formal guidance for developing recommendations that consider AMR and other local factors.
Incomplete reporting of evidence supporting recommendations, and the existing belief that guideline developers must develop their recommendations ‘from scratch’, results in additional challenges. Scientific societies and other organisations duplicate the same work to develop recommendations resulting in multiple guidelines on the same topic, confusion and loss of confidence by clinicians, and resource waste.8 10 However, guideline processes can become more effective, if they can be effectively adapted by others. This process requires transparent reporting of how the guideline development groups moved from evidence to recommendations, and properly include AMR. Formal processes for adaptation permit societies and organisations to capitalise on existing evidence evaluation and interpretation by considering important contextual factors, among which AMR is the most noticeable. This would reduce cost and redundancy.7
Our objectives were to analyse how, and to what extent, tuberculosis (TB), gonorrhoea and respiratory tract infection guidelines are considering AMR; are of acceptable methodological quality; and if they can be easily contextualised to fit the needs of specific populations.
Methods
Selection criteria and search strategy
We selected three types of infection: TB, gonorrhoea and respiratory tract infections, specifically otitis media, pharyngitis, sinusitis and community-acquired pneumonia. These infections are a public health priority because they are becoming increasingly harder to treat due to AMR and/or are treated inappropriately, leading to higher risk of toxicity or resistance development. Harder to treat drug-resistant TB strains are increasing and projected to account for a quarter of all TB deaths by 2050.11 Neisseria gonorrhoea is an urgent public health threat.12 The international spread of resistance to the last effective therapy, ceftriaxone and azithromycin, threatens sustained treatment of gonorrhoea.13 14 Otitis media, pharyngitis, sinusitis and community-acquired pneumonia are prevalent and Streptococcus pneumoniae (the main causal microorganism), was classified as a serious public health threat due to resistance observed by inappropriate use of antibiotics.12 15 All these syndromes have been prioritised by WHO as part of Access, Watch and Reserve (AWaRe)—a new classification system that supports a more nuanced approach to target inappropriate use of broad spectrum ‘Watch’ antibiotics.16
We included English language guidelines published between 2007 and 2019 on the above selected infections. We restricted to English language guidelines because, from a practical standpoint, English language publications would be the simplest to contextualise for most international groups and the major international organisations like WHO publish their guidelines at least in English. We marked the 2007 WHO decision to update its guideline development and using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach as a major change in methodology, representing a division of two eras.17 We limited the focus of our analyses to the era following this change.
We included guidelines with clearly articulated recommendations as defined by the Institute of Medicine Standards for Developing Trustworthy Clinical Practice Guidelines.14 After contacting guideline developers, we excluded guidelines with unobtainable online supplemental materials required for analysis (see online supplemental table 1S) for our guideline and recommendation selection outlined in PICAR format).
Supplemental material
We searched Ovid MEDLINE and Embase from inception to 7 June 2019 (detailed search strategies in online supplemental material). We conducted a second search in four guideline databases: Turning Research Into Practice (https://www.tripdatabase.com), Guidelines International Network (https://www.g-i-n.net/home), BIGG (http://sites.bvsalud.org/bigg/en/biblio/) and the Canadian Medical Association clinical practice guideline (CPG) Infobase (https://joulecma.ca/cpg/homepage). We finally searched key international websites (online supplemental table 2S) and reviewed references of included guidelines.
Independently and in pairs, reviewers (RSM, AB, AD, MV, GPM, SK and TB) screened titles and abstracts and the full text of potentially eligible guidelines. Disagreements were resolved by discussion or with a third reviewer (NS and HS).
Data extraction and quality assessment
We extracted data from guidelines, retrievable online supplemental materials and guideline development documents facilitated by pilot-tested forms and distillerSR (https://www.evidencepartners.com). Extractors (RSM, AB, AD, FS, GPM, MV and SK) recorded data independently and in pairs, and resolved disagreements.
Reviewers screened through recommendations classifying them as either considering AMR or not according to AMR dimensions (examples provided in table 1 and online supplemental table 3S). Although guidelines may have adopted different approaches to considering resistance with varying level of technicalities and detail, our operational definitions for considering a guideline ‘compliant’ were inclusive. We assumed that for each recommendation, there would be an opportunity to consider information pertaining to AMR at the population and outcome level, given that formulation of specific recommendations is guided by population, intervention, comparison and outcome (PICO) frameworks. Population-level considerations include recommendations for populations with some level of resistance, considerations of local resistance patterns, recommending the use of narrow-spectrum antimicrobials and recommending the watchful-waiting approach to prescribing. Outcome-level dimensions included considering prospects of AMR or the emergence of resistance as a consequence of antimicrobial use.
Satisfactory recommendations that consider antimicrobial resistance (AMR) dimensions
We considered a guideline satisfactory if it reports information on any of the above dimensions in either the recommendation, accompanying evidence summaries or PICO framework. Whereas guidelines that generally discussed AMR as an issue, without linking information pertaining to AMR to each recommendation were considered unsatisfactory.
We assessed a guideline’s quality using the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument focusing on three relevant domains: a well-defined scope and purpose (domain one), rigorous development including a systematic search for evidence, transparent reporting of methods, links between evidence and recommendations, external review and procedures for update (domain three), and editorial independence (domain six).18 We defined acceptable quality as guidelines that scored 60% or greater in these three domains a priori based on limited guidance on cut-off thresholds.3 19 Focusing on these three domains and selecting a relatively low score, allowed us to be inclusive.
We also abstracted information on values, resource use, equity, acceptability and feasibility from guidelines that met our acceptability cut-off (ie, 60%). Briefly, worldwide regions may differ in the accessibility of antimicrobials, the cultural view towards the use of antimicrobials, pharmaceutical costs and healthcare structures. We selected these dimensions as the transparent reporting of these factors is essential: in appraising the evidence for antimicrobials, guideline developers should be aware of the breadth of implications of their recommendations when used by decision-makers.7 10 20 21 Guidelines that ignore this wider agenda could provide narrow, misleading guidance.
Data synthesis and statistical analysis
We conducted descriptive statistics at the guideline and recommendation level, using counts and proportions (95% CI). We calculated the mean (SD) for AGREE II scores by region. We also compared the quality of guidelines from the WHO versus regional guidelines using scaled domain scores, mean difference, and a two-sided t-test. We calculated the frequency of guideline reporting of values, resource use, equity, acceptability and feasibility. All analyses were conducted in Microsoft Excel and R-Studio (RStudio Team (2016). RStudio: Integrated Development for R. RStudio, Boston, Massachusetts, USA, URL http://wwwrstudiocom/.).
This paper is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines and internally funded by the Michael G. DeGroote Cochrane Canada and McMaster GRADE centres.
Patient and public involvement
One of the authors is a patient with a rare disease affected by repeated infections and treatment related issues of resistance to antimicrobials and was involved in aspects of the design and data abstraction. We specifically looked for information about patient values and preferences and included this in our review. However, we did not make any additional specific efforts to involve the patient and public in other aspects of this systematic review.
Results
Our initial search identified 10 365 records. After screening, we retrieved 79 guidelines that had at least one recommendation on antimicrobial selection: (n=28 TB, n=13 gonorrhoea, n=38 respiratory tract infections). Of these, 78 guidelines had sufficient information for assessment—one gonorrhoea guideline was excluded because we were unable to retrieve supplemental materials (figure 1 and online supplemental table 4S).22
Flow diagram of the guideline selection process. BIGG, International database of GRADE guidelines; CPG infobase, Canadian Medical association CPG Infobase; out of scope, does not include recommendations on antibiotic selection or prescribing; does not have a significant section on tuberculosis, gonorrhoea, or respiratory tract infections. AMR, antimicrobial resistance; CPG, clinical practice guideline; G-I-N, Guidelines International Network; GRADE, Grading of Recommendations, Assessment, Development and Evaluation; PG, practice guideline; TRIP, Turning Research Into Practice.
Guideline recommendations considering AMR
After classifying recommendations, we found that 74 guidelines had at least one recommendation that considered AMR and four guidelines without such considerations (table 2).23–26 These were excluded from further assessment. Of the 74 guidelines, the majority were developed in North America (n=29),13 22 27–54 and Europe (n=26).44 55–78 A smaller portion were from Asia (n=7),79–85 South America (n=1),86 Africa (n=1)87 and Oceania (n=1).88 Nine guidelines were internationally developed by the WHO.89–97
Guidelines and recommendations with treatment recommendations with AMR* considerations
Within these 74 guidelines, we found that approximately two-thirds of recommendations (n=808/1198) considered AMR; that figure was 55.2% for TB recommendations (n=272), 84.7% for gonorrhoea recommendations (n=150) and 73.1% for respiratory tract infection recommendations (n=386). The majority of recommendations were regionally developed (n=736) (figure 2).
Number of regional guideline recommendations that consider antimicrobial resistance. AMR, antimicrobial resistance.
Most recommendations considered either population-level or outcome-level AMR dimensions, while fewer considered both simultaneously. Approximately 17.6% of recommendations (n=142/808) considered AMR at the population-level only while 34.7% (n=281/808) of recommendations considered resistance as an outcome only. Most notably, a majority of those considering AMR as an outcome were not explicitly stated in PICO format, but rather buried within evidence summaries. Clearly stated outcomes formally considered in PICO frameworks included: ‘acquired drug-resistance’, ‘antimicrobial in vitro resistance’, ‘bacterial antibiotic resistance’, and ‘emergence of drug-resistance’. Among respiratory tract infection recommendations, 6.9% (n=27/386) recommended no antimicrobial or back-up antimicrobial (ie, the watchful waiting approach), which is a population-level dimension, for example, recommendations for patients who likely have infections that are viral in nature or self-limiting.
Additionally, 47.6% (385/808) recommendations considered both population-level and outcome-level AMR dimensions simultaneously. For example, fully immunised infant or school-aged children with community-acquired pneumonia admitted to hospital are recommended to take ampicillin or penicillin G given that local epidemiologic data lacks a substantial high level of penicillin resistance for invasive Streptococcus pneumoniae.33 This recommendation is considering local resistance patterns (population-level dimension). It is also followed by an evidence summary the explains that lower costs of ampicillin or penicillin G need to be balanced by the increased possibility of emergence of resistance (outcome-level dimension) that may occur from prescribing broad-spectrum antimicrobials. About 22.5% (n=182/808) of recommendations considered local resistance patterns in a similar manner.
Credibility of international and regional guidelines with recommendations that consider AMR
Overall, only 39.2% (n=29/74) of all international and regional guidelines had scores of 60% or greater in scope and purpose, rigour of development, and editorial independence. Of the 29 guidelines that met our credibility cut-off, 10 were developed in North America,13 34–39 42 53 98 nine in Europe,44 55 64 67 68 72–75 87 and two were developed in Asia.80 82 When we compared international and regional guidelines, the majority of WHO guidelines performed significantly better than regional guidelines (table 3, online supplemental figure 1S). Guidelines that did not meet our credibility cut-off score and excluded from further assessment included: 19 from North America, 17 from Europe, 5 from Asia and 3 guidelines from South America, Africa and Oceania.
Performance of WHO versus regional guidelines with AMR considerations
Guidelines considering values, resource use, acceptability, feasibility and equity
Only 589 90 93 94 96 of the 29 guidelines reported all factors required for contextualisation: values, resource use, equity, acceptability and feasibility (online supplemental table 5S). The WHO was the only guideline developer to report on all five criteria in four TB guidelines and one gonorrhoea guideline.
Across all 29 guidelines, resource use was the most frequently considered (n=23 guidelines), followed by values (n=16 guidelines), acceptability (n=12 guidelines) and feasibility (n=12 guidelines). Equity was the least considered factor with only seven guidelines that made such considerations (figure 3): two were regionally and five were internationally developed. The WHO, the National Institute for Health and Care Excellence (NICE) and the US Preventative Task Force were the only organisations to consider equity.
Contextualisation of Grading of Recommendations Assessment, Development and Evaluation evidence to decision frameworks in current guidelines.
Regional guidelines tended to consider values, resource use, equity, acceptability and feasibility less than internationally developed guidelines (online supplemental figure 2S). Most regional guidelines considered one (n=6/21) or two (n=6/21) or three (n=4/21) or none (n=4/21) of the above contextual factors. Values and resource use were considered the most, while equity, acceptability and feasibility were less considered in regionally developed guidelines (figure 4).
Number of internationally and regionally developed guidelines with considerations of Grading of Recommendations Assessment, Development and Evaluation evidence to decision frameworks.
Discussion
Summary of findings
Over a 13-year period, relatively few guidelines on antibiotics for highly prevalent infectious diseases included AMR considerations. Approximately 60% of regionally developed guidelines were of low quality and reported less factors required for tailoring recommendations to specific contexts. International WHO guidelines had substantially higher quality scores than regional guidelines. International guidelines also consistently considered important information required for developing recommendations that are appropriate for specific contexts compared with regional guidelines.
There is an emerging consensus that reporting of Evidence to Decision dimensions is ethically and scientifically essential. Unfortunately, reporting these dimensions is not always seen in practice. Our review highlighted that some of the proposed dimensions seemed to be adopted by guideline developers (ie, values and resource use were most considered), while others were less so (ie, acceptability, feasibility and equity were the least considered). Further, the quality of these guidelines varied and there were inconsistencies between regions and guidelines promoted/sponsored by different entities.
The use of the GRADE Evidence to Decision framework by the WHO and NICE seems to positively influence the consideration of contextual factors in the guidelines we reviewed. A high proportion of WHO (n=5/7) and NICE (n=1/5) guidelines contained complete information necessary to provide optimal guidance on how to use antimicrobials in the considered syndromes. Other regional organisations provided limited information addressing contextual factors—most addressed one (n=6/21) or two (n=6/21) contextual factors and a good proportion did not address any (4/21).
Strengths and limitations
Our work has strengths. To our knowledge, this is the first study to assess the extent to which guidelines are considering local dimensions such as AMR, and to use established frameworks: AGREE II and GRADE Evidence to Decision. We also employed systematic methods to conduct our review and validated tools to measure the quality of guidelines.3 18
There are several limitations to our study. The use of a credibility score of 60% or greater for three of the six AGREE II domains is based on limited guidance on cut-off thresholds. However, by focusing on three domains and a low cut-off we were inclusive although we also focused on English language publications only.3 99 We used general criteria from the GRADE Evidence to Decision Frameworks that are applicable to various interventions, and not specific to antimicrobials. These general dimensions could be complemented with specific criteria related to the antimicrobial field such as providing guidance on the appropriate threshold for escalating empiric antimicrobials from narrower spectrum agents to broad-spectrum agents. In other words, the real test for antimicrobial guidelines may be whether they enable prescribers and the public to fully consider the potential implications of antimicrobial prescribing on resistance. This would lead to virtuous and parsimonious prescribing and consumption habits.
Context to other research
We previously found that about two thirds of respiratory tract infection recommendations on empirical antimicrobial use did not consider country-specific resistance patterns. The use of a broader framework and additional focus areas may have resulted in the larger number of recommendations that considered AMR uncovered by this study. Both studies support that there are inconsistencies in considering AMR in recommendation development and potential duplication of work among infectious disease guidelines.
Implications for practice
There are several implications for guideline developers. Given the suboptimal quality of guidelines in our sample, guideline methodology should improve particularly when recommendations move from global to regional levels. This includes improving the processes used in evidence syntheses and recommendation formulation, transparency and addressing potential unduly biases with competing interests. As far as regional guidelines need to incorporate contextual information when developing their recommendations, global guidelines need to provide information about how to contextualise recommendations for appropriate AMR considerations.
Guideline development can be done more efficiently and economically by using work done by other developers including the WHO. Rather developing guidelines from scratch, time and resources100 may be shifted towards refining AMR surveillance systems that provide national resistance data to support recommendations and appropriate antimicrobial use. Further, country-level participation of the Global Antimicrobial Resistance Surveillance System (GLASS) supports global monitoring of resistance trends, emerging resistance and the ability to evaluate the effectiveness of interventions.101 As of 2020, 94 countries are participating in GLASS.101 However, some countries lack public health infrastructure, national laboratory capacities, and data management which is essential for surveillance systems.6 102 In 2018, there was at least one country within each WHO regions with the ability to collect national resistance data.102 Regions facing unique challenges to antimicrobial stewardship capacities, may look to recommendations developed by other regions with similar resistance experiences. Finally, as new antimicrobial therapies become available, and the scientific community cumulates more evidence on resistance patterns and their implications for local prescribing, future infectious disease guidelines may require more frequent updating.
Implications for research
Although we focused on recommendations on antimicrobial selection and prescribing, there are many other approaches that could be assessed in future research (eg, rapid diagnostics to rule-out viral infections and resistant strains). In addition, research should also explore whether recommendations are appropriately guided by evidence, resistance data and the WHO’s Essential Medicines List and AWaRe Classification Database of Antimicrobials updates.103 With regard to contextualisation of infectious disease recommendations, we have developed transparent recommendation maps that facilitate use of recommendations across jurisdictions for TB (https://who.tuberculosis.recmap.org) and COVID-19 (https://COVID-19.recmap.org) where we apply some of our findings.
Conclusion
Our study offers information on how current infectious disease guidelines are considering contextual factors necessary to appropriately prescribe antimicrobials. We also present dimensions that can be considered by a formal AMR framework used in combination with GRADE Evidence to Decision Frameworks to facilitate amelioration of the cornerstones that are guiding current antimicrobial use. Improving the quality of guidelines targeting specific regional areas is required. This may help to protect the remaining and essential medicines we have left, and the future of new classes of antimicrobials.104
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. No additional data available.
Ethics statements
Acknowledgments
This work was supported by the Michael G. DeGroote Canada and McMaster GRADE centres. We would like to thank biostatistician Dr. Thuva Vanniyasingam for assisting with developing an analysis plan for our protocol.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors RSM, HS, NS, ML and TP designed the study protocol. RSM coordinated the study. RSM, AB, AD, GPM, MV, SK and TB assessed eligibility of records at title and abstract. RSM, AD and MV searched for unpublished guidelines in key websites. RSM, AB, AD, GPM, MV, and SK assessed eligibility of full text articles. RSM, AB, AD, FS, GPM, MV and SK extracted data and performed quality assessment using the AGREE II tool. NS and HS settled disputes. RSM analysed and interpreted the data with HS, NS and ML. RSM and HS drafted the manuscript, with writing contributions from NS, ML and LM. All authors interpreted and make edits to the manuscript.
Funding Michael G. DeGroote Cochrane Canada and McMaster GRADE centres.
Map disclaimer The depiction of boundaries on the map(s) in this article does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. The map(s) are provided without any warranty of any kind, either express or implied.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.