World Health Organization recommendations are often strong based on low confidence in effect estimates

Paul E Alexander; Lisa Bero; Victor M Montori; Juan Pablo Brito; Rebecca Stoltzfus; Benjamin Djulbegovic; Ignacio Neumann; Supriya Rave; Gordon Guyatt

doi:10.1016/j.jclinepi.2013.09.020

World Health Organization recommendations are often strong based on low confidence in effect estimates

J Clin Epidemiol. 2014 Jun;67(6):629-34. doi: 10.1016/j.jclinepi.2013.09.020. Epub 2014 Jan 3.

Authors

Paul E Alexander¹, Lisa Bero², Victor M Montori³, Juan Pablo Brito⁴, Rebecca Stoltzfus⁵, Benjamin Djulbegovic⁶, Ignacio Neumann⁷, Supriya Rave⁸, Gordon Guyatt⁹

Affiliations

¹ Health Research Methods (HRM), Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada. Electronic address: pauleliasalexander@gmail.com.
² University of California, San Francisco, Suite 420, Box 0613, 3333 California Street, San Francisco, CA 94118, USA.
³ Healthcare Delivery Research Program, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA; Knowledge and Evaluation Research Unit, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA; Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA; Division of Health Care and Policy Research, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA.
⁴ Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA.
⁵ Global Health Program, Division of Nutritional Sciences, 120 Savage Hall, Cornell University, Ithaca, NY 14853, USA; Program in International Nutrition, Division of Nutritional Sciences, 120 Savage Hall, Cornell University, Ithaca, NY 14853, USA.
⁶ H Lee Moffitt Cancer Center, University of South Florida, Tampa, FL 33612, USA.
⁷ Health Research Methods (HRM), Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Department of Internal Medicine, School of Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile.
⁸ Blood, Tissues, Organs and Xenografts Unit, Health Canada, Toronto, Ontario M1P4R7, Canada.
⁹ McMaster University Health Sciences Centre, 1200 Main Street West, Room 2C12, Hamilton, Ontario L8S 4K1, Canada. Electronic address: guyatt@mcmaster.ca.

PMID: 24388966
DOI: 10.1016/j.jclinepi.2013.09.020

Abstract

Objectives: Expert guideline panelists are sometimes reluctant to offer weak/conditional/contingent recommendations. Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance warns against strong recommendations when confidence in effect estimates is low or very low, suggesting that such recommendations may seldom be justified. We aim to characterize the classification of strength of recommendations and confidence in estimates in World Health Organization (WHO) guidelines that used the GRADE approach and graded both strength and confidence (GRADEd).

Study design and setting: We reviewed all WHO guidelines (January 2007 to December 2012), identified those that were GRADEd, and, in these, examined the classifications of strong and weak and associated confidence in estimates (high, moderate, low, and very low).

Results: We identified 116 WHO guidelines in which 43 (37%) were GRADEd and had 456 recommendations, of which 289 (63.4%) were strong and 167 (36.6%) were conditional/weak. Of the 289 strong recommendations, 95 (33.0%) were based on evidence warranting low confidence in estimates and 65 (22.5%) on evidence warranting very low confidence in estimates (55.5% strong recommendations overall based on low or very low confidence in estimates).

Conclusion: Strong recommendations based on low or very low confidence estimates are very frequently made in WHO guidelines. Further study to determine the reasons for such high uncertainty recommendations is warranted.

Keywords: Clinical practice guidelines; Confidence in effect estimates; GRADE; High uncertainty; Public health guidelines; Strength of recommendation; World Health Organization.

Publication types

Review

MeSH terms

Evidence-Based Medicine*
Humans
Practice Guidelines as Topic*
Uncertainty*
World Health Organization*