Article Text
PDF
Abstract
Objectives Cancer screening guidelines differ in their recommendations for or against screening. To be able to provide explicit recommendations, guidelines need to specify thresholds for the magnitude of benefits of screening, given its harms and burdens. We evaluated how current cancer screening guidelines address the relative importance of benefits versus harms and burdens of screening.
Data source We searched the Guidelines International Network, International Guideline Library, ECRI Institute and Medline. Two pairs of reviewers independently performed guideline selection and data abstraction.
Eligibility criteria We included all cancer screening guidelines published in English between January 2014 and April 2019.
Results Of 68 eligible guidelines, 25 included a statement regarding the trade-off between screening benefits versus harms and burdens (14 guidelines), or a statement of direction of the net effect (defined as benefits minus harms or burdens) (13 guidelines). None of these 25 guidelines defined how large a screening benefit should be to recommend screening, given its harms and burdens. 11 guidelines performed an economic evaluation of screening. Of these, six identified a key benefit outcome; two specified a cost-effectiveness threshold for recommending a screening option. Eight guidelines commented on people’s values and preferences regarding the trade-off between benefits versus harms and burdens.
Conclusions Current cancer screening guidelines fail to specify the values and preferences underlying their recommendations. No guidelines provide a threshold at which they believe the benefits of screening outweigh its harms and burdens.
PROSPERO registration number CRD42019138590.
- health policy
- protocols & guidelines
- quality in health care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This systematic survey adheres to the state-of-the-art systematic summary methodology.
The report states explicit definitions of key concepts, and detailed criteria for making judgements for setting a threshold of a key beneficial outcome, that ensures reproducible and accurate inferences.
The findings may be limited by under-reporting of methods in eligible guidelines.
The findings may only be applicable to guidelines published in English.
Introduction
Cancer is the second leading cause of death globally, with an estimated 9.6 million deaths in 2018.1 To reduce the mortality and burden of cancer, cancer screening programmes and associated guidelines for screening practice have been established.2 3 However, cancer screening recommendations implemented in countries with similar levels of healthcare spending differ.4 Even guideline panels using similarly rigorous methods for guideline development reach different conclusions.5 6 The differences are likely to stem from variation in how guideline panels value benefits, harms and burdens of screening, and what evidence they consider.4 People’s perspective on benefits-harm trade-offs might vary according to the importance individuals place on possible benefits versus harms and burdens of screening.7
Ideally, guidelines should entail explicit assumptions regarding values and preferences that underlie their judgements of the trade-off between screening benefits versus its harms and burdens.8 However, weighing these desirable and undesirable consequences, and clearly articulating the underlying values and preferences while simultaneously considering multiple outcomes, can be challenging. Researchers have developed frameworks for incorporating patients’ preferences regarding benefits and harms while simultaneously considered multiple outcomes into the evaluation of health interventions.9 10 Due to the complexity of the process, these approaches have not seen wide use.
Cancer screening guidelines have a potential advantage over guidelines in other areas of medicine because they often have one or two key benefit outcomes: reduction in cancer mortality or cancer incidence. Therefore, the central question guideline panels face may be framed as: given the harms and burdens of screening, what magnitude of its key benefits (effect on cancer incidence and/or mortality) would people require to undergo screening? In other words, what is the threshold above which people would undergo screening and below which they would not? Some may argue that the values and preferences regarding such threshold might vary among target population. However, guideline panel could always identify the distribution of individuals’ values and preferences and find the magnitude of benefit that majority of target population would require. By establishing such a threshold, a panel makes transparent, through a quantitative trade-off between benefits and harms or burdens, their assessment of the values and preferences of the target population.
We evaluated how current cancer screening guidelines address the relative importance of benefits versus harms and burdens. In particular, we evaluated:
The proportion of cancer screening guidelines that established a threshold for key benefit outcomes (reduction in cancer incidence or mortality) given the evidence of harms and burdens, above which people would undergo screening and below which they would not.
The proportion of cancer screening guidelines that qualitatively traded off the benefits versus the harms and burdens of screening.
The proportion of cancer screening guidelines that commented on the values and preferences of target population regarding the trade-off between the benefits versus the harms and burdens of screening.
Methods
Design overview
We conducted a systematic survey of cancer screening guidelines published between 1 January 2014 and 30 April 2019 using the standard methodology for systematic surveys.11
As defined by the WHO, screening is the presumptive identification of unrecognised disease in an apparently healthy, asymptomatic population by means of tests, examinations or other procedures that can be applied rapidly and easily.12 For this systematic survey, we defined cancer screening guideline as guidelines addressing recommendations of screening technologies for early detection or prevention of cancer, including guidelines that address the entire spectrum of screening, diagnosis and management. Ideally, guideline panels consider benefits, harms or burdens to make recommendations. Guideline panels may, if cost is in their scope, also consider cost-effectiveness to generate a cost-effectiveness ratio.13
For each identified screening guideline, we evaluated if the guideline included information about (1) an applied threshold for the key benefit outcome (ie, reduction in cancer mortality and/or incidence) required to balance harms or burdens of screening, and (2) an applied threshold for a beneficial cost-effectiveness ratio between benefits and costs of screening. We defined the threshold for a key benefit outcome as the threshold above which typical members of the target population would undergo screening and below which they would decline. We defined the threshold of cost-effectiveness as the magnitude of cost per unit of incremental benefit deemed acceptable to recommend a screening test.
Literature search
An experienced librarian developed a search strategy for the Guidelines International Network-International Guideline Library, ECRI Institute and Medline (online supplemental appendix 1).
Supplemental material
The inclusion criteria were cancer screening guidelines published in full text in English between January 2014 and April 2019 including new, updated or adapted guidelines, or consolidated guidelines. We defined consolidated guidelines as those that aggregate existing guidance addressing a disease or condition, and provide recommendations that have been evaluated and found to be up to date; such guidelines may contain new recommendations.14 We had no restriction with respect to the cancer type.
Two pairs of trained reviewers (LZ and FKN as a pair; LY and YW as another pair) independently performed the title and abstract screening of each citation for potential eligibility and full-text screening in duplicate. The reviewers abstracted data of eligible guidelines using a standardised, pilot-tested electronic form, attempted to resolve discrepancies by discussion and, if disagreement persisted, by discussion with arbitrators (LMH, RAS, GHG).
Data abstraction
We extracted information on year, institution, and country of publication, type of cancer and screening tests, target population and methods for grading quality of evidence or strength of recommendations.
Because a cancer screening guideline might address more than one key clinical question (eg, whether to recommend a particular screening test, or one test over another), we assumed that a guideline panel would apply the same approach for trading off the benefits and harms in each clinical question. Therefore, we abstracted data for one clinical question for each eligible guideline, with the following hierarchy of clinical questions: (1) recommendation for or against cancer screening; (2) a recommendation of one screening test over another(s); (3) a recommendation of a particular starting age for screening over other(s).
We established specific criteria to evaluate whether and how cancer screening guidelines defined a threshold for a key benefit outcome in the trade-off between benefits versus harms and burdens, and in cost-effectiveness evaluation (table 1). We also evaluated whether the guidelines qualitatively commented on the trade-off between benefits versus harms and burdens (eg, a statement of benefits over harms and burdens, or a statement of the magnitude of net effect) or commented on the target population’s values and preferences regarding the trade-off between benefits versus harms and burdens (ie, the relative importance of outcomes or health states of interest related to cancer screening) (online supplemental appendix 2).
Criteria for setting a threshold for the key benefit of cancer screening in the trade-off between benefits versus harms and burdens and in cost-effectiveness evaluation in cancer screening guidelines*
Analyses
We conducted descriptive analyses for all variables. We summarised categorical variables with frequencies and percentages. Using univariable and multivariable logistic regression analyses, we explored the association between characteristics of guidelines and (1) performing key steps of setting thresholds for a key benefit outcome (table 1), (2) considering the values and preferences of the target population regarding trade-off between benefits and harms or burdens, (3) qualitatively commenting on the trade-off between benefits and harms. We required at least 10 events per category of variables for conducting a regression analysis. We specified a priori the following hierarchy of independent variables (ie, guideline characteristics): (1) region of guideline development (North America vs other regions), (2) use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE) versus no GRADE. We presented associations using ORs and associated 95% CIs. For all analyses, we used R V.3.5.2.15 P<0.05 provided the threshold for statistical significance.
Patient and public involvement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination of our research.
Results
Characteristics of included guidelines
Our search resulted in 741 records, and 68 guidelines proved eligible for analyses (figure 1). Of these, 64 (94%) were new or updated guidelines, the remaining were adapted or consolidated guidelines; 42 (62%) were from North America, and 31 (45%) in breast or colorectal cancer (table 2). online supplemental appendix 3 presents the characteristics of the eligible guidelines.
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram representing the systematic literature search. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097
Characteristics of cancer screening guidelines
Setting threshold in trade-off between benefits and harms, burdens or in cost-effectiveness evaluation
None of the guidelines defined a threshold for the key benefit that would be required to recommend screening (or for a particular screening test or screening starting age), given screening harms and burdens. Twenty-two (32%) guidelines specified cancer mortality and/or cancer incidence as their key benefit outcome(s) (table 3). Among the 22 guidelines, 17 also specified key harms and burdens including diagnostic procedure-related harms (eg, invasive procedure, radiation exposure, infection), false-positive results, overdiagnosis, overtreatment, cost, life disruption and anxiety. Among the 17 guidelines that specified the key benefits and harms or burdens, nine specified the magnitude of the key harms and burdens, and 11 presented the magnitude of the key benefit in a quantitative way (eg, a small, moderate or large benefit).
Key steps for setting a threshold for the key benefit in cancer screening guidelines
Of the 68 guidelines, 25 (37%) included a qualitative statement of trade-off between benefits versus harms and burdens (14; 20%), or a statement of direction of net effect (defined as benefits minus harms or burdens) (13; 19%) (two of which stated both). However, none of the 25 guidelines described a quantitative approach for establishing a trade-off between benefits versus harms and burdens or a net effect of screening.
Eleven (16%) guidelines performed economic evaluation or identified economic evidence, among which six identified a key benefit outcome, and two specified a cost-effectiveness threshold required for recommending screening (table 3). Both of these two guidelines used the incremental cost-effectiveness ratio as the variable, and quantified the threshold at $100 000 per quality-adjusted life year gained.
Values and preferences
Eight (12%) guidelines commented on people’s values and preferences regarding the trade-off between benefits versus harms and burdens of screening (table 4). Of these, two reported important uncertainty or variability in people’s values and preferences (table 4). Because no guideline set thresholds for key benefit outcomes, and few considered the values and preferences regarding trade-off between benefits versus harms and burdens, the regression analysis was minimally informative (online supplemental appendix 4).
Considering values and preference in cancer screening guidelines
Discussion
An individual’s decision to participate in cancer screening is based on personal judgement of the magnitude of benefit versus harms and burdens. Our study reveals that current cancer screening guidelines lack transparency in trading off benefits and harms or burdens, hence lack transparency in the rationale for their recommendations. Fewer than half provided quantitative estimates of the harms and burdens associated with screening, and many did not quantify key benefits. Very few guidelines explicitly addressed the issue of people’s values and preferences, and none explicitly specified a threshold of key benefit given the magnitude of the harms and burdens. Few guidelines addressed economic issues. Of those that did, only two offered an explicit cost-effectiveness ratio that would justify screening.
Our study has several strengths. We used robust systematic survey methodology including explicit and reproducible broad eligibility criteria without restrictions on the type of cancers, thus ensuring generalisability; sensitive search strategies; and standardised forms for guideline screening and data abstraction. We developed explicit definitions of key concepts and detailed criteria for making judgements regarding the setting of a threshold of a key beneficial outcome. These definitions and criteria ensured reproducible and accurate inferences.
Our study also has limitations. Our evaluation was based on the reporting in guidelines. The findings may be vulnerable to under-reporting or selective reporting of guideline methods. To minimise this vulnerability, we included all published supporting documents for data abstraction. We included only guidelines published in full text in English. Generalisation to guidelines published in non-English languages is therefore limited. Because the primary aim of this study was to evaluate whether current cancer screening guidelines quantitatively trade off the benefits and harms or burdens of screening, we included only guidelines published in the last 5 years, and we did not address the qualitative approach the guideline panels used in the trade-off.
Commentators have frequently called for a requirement for a more systematic incorporation of patients’ preferences in guidelines,16–19 in particular when treatment is burdensome, benefits are limited or uncertain and harms may impact quality of life.20 Challenges for articulating values and preferences regarding the trade-off between benefits versus harms and burdens lie in how to simultaneously consider multiple outcomes and, given the time and resource limitation in most of the guideline development, how to efficiently address the values and preferences of the target population. These challenges, however, could also be considered as opportunities to develop new and better methods. A recent BMJ Rapid Recommendation addressing colorectal cancer screening has described the first application of a method to elicit, based on the evidence of key harms and burdens, panel members’ view on the target population’s threshold of magnitude of key benefit for undergoing screening. This Rapid Recommendation has shown the feasibility of the method being applied to guide a formal recommendation.21 22
Because values and preferences regarding the threshold benefit required will certainly vary among individuals in the target population, no threshold will be right for all patients (and thus the appropriateness of weak recommendations for virtually all screening tests).23 The goal for a guideline panel (which can only make recommendations for typical patients) is to identify the distribution of individuals’ values and preferences and find the magnitude of benefit the majority of the target population would require. By establishing such a threshold for the key benefit, a panel makes their assessment of the values and preferences of the target population transparent. The recommendations that follow from the threshold will, ideally, represent the starting point for a shared decision-making discussion between patients and clinicians.
Conclusions
Cancer screening guidelines, relative to many other areas, have the advantage of being able to identify one or two key benefit outcomes. This greatly facilitates an explicit specification of thresholds for the magnitude of benefit required to justify screening given the harms and burdens. Unfortunately, current guidelines do not use this approach.
Acknowledgments
We thank our librarian Rachel Couban for developing the search strategies and conducting the literature search.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors GHG, LZ, LMH and RAS drafted the study protocol and designed the data collection forms. LZ and FKN tested the data collection forms. LZ, FKN, YW and LY screened the literature and abstracted the data. LZ, LMH, FKN, YW, LY, RAS, MB and GHG drafted and revised the manuscript. All authors have read and approved the final version of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests LMH reports grants from South-Eastern Health Authority, during the conduct of the study. LMH is the first author of a guideline on colorectal cancer screening published in The BMJ 2019 (BMJ Rapid Recommendations). RAS is the coauthor of a guideline on colorectal cancer screening published in The BMJ 2019 (BMJ Rapid Recommendations). GHG is the co-founder and cochair of the GRADE working group, board member of MAGIC and the coauthor of a guideline on colorectal cancer screening published in The BMJ 2019 (BMJ Rapid Recommendations).
Patient consent for publication Not required.
Ethics approval Ethics approval for this study is not required. We will disseminate the results of this review in peer-reviewed publications and conference presentations.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.