You are here

Article Text

Article menu

Download PDFPDF

XML
Evidence based practice
Research
Search for unpublished data by systematic reviewers: an audit
  1. Hedyeh Ziai1,2,
  2. Rujun Zhang1,3,
  3. An-Wen Chan4,5,
  4. Nav Persaud3,6,7
  1. 1 Keenan Research Centre, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Toronto, Canada
  2. 2 Faculty of Medicine, University of Ottawa, Ottawa, Canada
  3. 3 Department of Family and Community Medicine, University of Toronto, Canada
  4. 4 Women’s College Research Institute, Women’s College Hospital, Toronto, Canada
  5. 5 Department of Medicine, University of Toronto, Toronto, Canada
  6. 6 Department of Family and Community Medicine, St Michael’s Hospital, Toronto, Canada
  7. 7 Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, Ontario, Canada
  1. Correspondence to Dr Nav Persaud; nav.persaud{at}utoronto.ca

Abstract

Objectives We audited a selection of systematic reviews published in 2013 and reported on the proportion of reviews that researched for unpublished data, included unpublished data in analysis and assessed for publication bias.

Design Audit of systematic reviews.

Data sources We searched PubMed and Ovid MEDLINE In-Process & Other Non-Indexed Citations between 1 January 2013 and 31 December 2013 for the following journals: Journal of the American Medical Association, The British Medical Journal, Lancet, Annals of Internal Medicine and the Cochrane Database of Systematic Reviews. We also searched the Cochrane Library and included 100 randomly selected Cochrane reviews.

Eligibility criteria Systematic reviews published in 2013 in the selected journals were included. Methodological reviews were excluded.

Data extraction and synthesis Two reviewers independently reviewed each included systematic review. The following data were extracted: whether the review searched for grey literature or unpublished data, the sources searched, whether unpublished data were included in analysis, whether publication bias was assessed and whether there was evidence of publication bias.

Main findings 203 reviews were included for analysis. 36% (73/203) of studies did not describe any attempt to obtain unpublished studies or to search grey literature. 89% (116/130) of studies that sought unpublished data found them. 33% (68/203) of studies included an assessment of publication bias, and 40% (27/68) of these found evidence of publication bias.

Conclusion A significant fraction of systematic reviews included in our study did not search for unpublished data. Publication bias may be present in almost half the published systematic reviews that assessed for it. Exclusion of unpublished data may lead to biased estimates of efficacy or safety in systematic reviews.

  • Review
  • Systematic Publication Bias Unpublished Data

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-017737

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Twitter @NavPersaud

    • Contributors HZ contributed to the conception and design of the project and the acquisition, analysis and interpretation of the data; drafted the work and revised it; approved the final version to be published; and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy and integrity are appropriately investigated and resolved. RZ contributed to the analysis and interpretation of the data; drafted the work and revised it; provided final approval of the version to be published; and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy and integrity are appropriately investigated and resolved. A-WC contributed to the conception of the project, reviewed it critically for important intellectual content, provided final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy and integrity are appropriately investigated and resolved. NP contributed to the conception and design of the project; the interpretation of the data; revised the work; approved the final version to be published; and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy and integrity are appropriately investigated and resolved.

    • Funding There was no specific funding for this project. NP is supported by a Physician Services Incoroporated Graham Farquharson Knowledge Translation Fellowship.

    • Disclaimer The views expressed in the submitted article are those of the authors and not an official position of the institution or funder.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement We have no further data to share.