You are here

  • Home
  • Archive
  • Volume 5, Issue 9
  • Reporting, handling and assessing the risk of bias associated with missing participant data in systematic reviews: a methodological survey

Article Text

Article menu
XML
Evidence based practice
Research
Reporting, handling and assessing the risk of bias associated with missing participant data in systematic reviews: a methodological survey
  1. Elie A Akl1,2,
  2. Alonso Carrasco-Labra2,3,
  3. Romina Brignardello-Petersen2,3,4,
  4. Ignacio Neumann5,
  5. Bradley C Johnston4,6,
  6. Xin Sun2,7,
  7. Matthias Briel2,8,
  8. Jason W Busse2,9,10,
  9. Shanil Ebrahim2,9,11,
  10. Carlos E Granados12,
  11. Alfonso Iorio2,13,
  12. Affan Irfan14,
  13. Laura Martínez García15,
  14. Reem A Mustafa2,16,
  15. Anggie Ramírez-Morera17,
  16. Anna Selva15,
  17. Ivan Solà15,
  18. Andrea Juliana Sanabria15,
  19. Kari A O Tikkinen2,18,
  20. Per O Vandvik19,20,
  21. Robin W M Vernooij15,
  22. Oscar E Zazueta15,
  23. Qi Zhou2,
  24. Gordon H Guyatt2,13,
  25. Pablo Alonso-Coello15
  1. 1Department of Internal Medicine, American University of Beirut, Beirut, Lebanon
  2. 2Department of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Ontario, Canada
  3. 3Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Santiago, Chile
  4. 4Institute of Health Policy, Management & Evaluation, University of Toronto, Toronto, Ontario, Canada
  5. 5Department of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
  6. 6Department of Anesthesia & Pain Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
  7. 7Clinical Research and Evaluation Unit, Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu, China
  8. 8Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel, Switzerland
  9. 9Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada
  10. 10The Michael G. DeGroote Institute for Pain Research and Care, McMaster University, Hamilton, Ontario, Canada
  11. 11Department of Medicine, Stanford Prevention Research Center, Stanford University, Palo Alto, California, USA
  12. 12Área de investigaciones, Facultad de Medicina, Universidad de La Sabana, Chía, Colombia
  13. 13Department of Medicine, McMaster University, Hamilton, Ontario, Canada
  14. 14Department of Cardiology, University of Louisville, Louisville, Kentucky, USA
  15. 15Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau-CIBER of Epidemiology and Public Health (CIBERESP- IIB Sant Pau), Barcelona, Spain
  16. 16Departments of Medicine, Nephrology and Biomedical and Health Informatics, University of Missouri-Kansas City, Kansas City, Missouri, USA
  17. 17Health Care Development Division, IHCAI Foundation & Central America Cochrane, San José, Costa Rica
  18. 18Departments of Urology and Public Health, Helsinki University Central Hospital and University of Helsinki, Helsinki, Finland
  19. 19Norwegian Knowledge Centre for the Health Services, Oslo, Norway
  20. 20Department of Medicine, Innlandet Hospital Trust, Gjøvik, Norway
  1. Correspondence to Dr Elie A Akl; ea32{at}aub.edu.lb

Abstract

Objectives To describe how systematic reviewers are reporting missing data for dichotomous outcomes, handling them in the analysis and assessing the risk of associated bias.

Methods We searched MEDLINE and the Cochrane Database of Systematic Reviews for systematic reviews of randomised trials published in 2010, and reporting a meta-analysis of a dichotomous outcome. We randomly selected 98 Cochrane and 104 non-Cochrane systematic reviews. Teams of 2 reviewers selected eligible studies and abstracted data independently and in duplicate using standardised, piloted forms with accompanying instructions. We conducted regression analyses to explore factors associated with using complete case analysis and with judging the risk of bias associated with missing participant data.

Results Of Cochrane and non-Cochrane reviews, 47% and 7% (p<0.0001), respectively, reported on the number of participants with missing data, and 41% and 9% reported a plan for handling missing categorical data. The 2 most reported approaches for handling missing data were complete case analysis (8.5%, out of the 202 reviews) and assuming no participants with missing data had the event (4%). The use of complete case analysis was associated only with Cochrane reviews (relative to non-Cochrane: OR=7.25; 95% CI 1.58 to 33.3, p=0.01). 65% of reviews assessed risk of bias associated with missing data; this was associated with Cochrane reviews (relative to non-Cochrane: OR=6.63; 95% CI 2.50 to 17.57, p=0.0001), and the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology (OR=5.02; 95% CI 1.02 to 24.75, p=0.047).

Conclusions Though Cochrane reviews are somewhat less problematic, most Cochrane and non-Cochrane systematic reviews fail to adequately report and handle missing data, potentially resulting in misleading judgements regarding risk of bias.

  • EPIDEMIOLOGY
  • STATISTICS & RESEARCH METHODS

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2015-009368

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.