Elsevier

Journal of Clinical Epidemiology

Volume 69, January 2016, Pages 225-234
Journal of Clinical Epidemiology

Guidelines
ROBIS: A new tool to assess risk of bias in systematic reviews was developed

https://doi.org/10.1016/j.jclinepi.2015.06.005Get rights and content
Under a Creative Commons license
open access

Abstract

Objective

To develop ROBIS, a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies).

Study Design and Setting

We used four-stage approach to develop ROBIS: define the scope, review the evidence base, hold a face-to-face meeting, and refine the tool through piloting.

Results

ROBIS is currently aimed at four broad categories of reviews mainly within health care settings: interventions, diagnosis, prognosis, and etiology. The target audience of ROBIS is primarily guideline developers, authors of overviews of systematic reviews (“reviews of reviews”), and review authors who might want to assess or avoid risk of bias in their reviews. The tool is completed in three phases: (1) assess relevance (optional), (2) identify concerns with the review process, and (3) judge risk of bias. Phase 2 covers four domains through which bias may be introduced into a systematic review: study eligibility criteria; identification and selection of studies; data collection and study appraisal; and synthesis and findings. Phase 3 assesses the overall risk of bias in the interpretation of review findings and whether this considered limitations identified in any of the phase 2 domains. Signaling questions are included to help judge concerns with the review process (phase 2) and the overall risk of bias in the review (phase 3); these questions flag aspects of review design related to the potential for bias and aim to help assessors judge risk of bias in the review process, results, and conclusions.

Conclusions

ROBIS is the first rigorously developed tool designed specifically to assess the risk of bias in systematic reviews.

Keywords

Evidence
Meta-analysis
Quality
Risk of bias
Systematic review
Tool

Cited by (0)

The ROBIS group members: Lars Beckmann (Institute for Quality and Efficiency in Health Care [IQWiG], Cologne, Germany), Patrick Bossuyt (University of Amsterdam), Deborah Caldwell (University of Bristol)*, Rachel Churchill (University of Bristol)*, Philippa Davies (University of Bristol)*, Kay Dickersin (US Cochrane Center), Kerry Dwan (University of Liverpool), Julie Glanville (York Health Economics Consortium, University of York), Julian Higgins (University of Bristol)*, Jos Kleijnen (KSR York Ltd and University of Maastricht)*, Julia Kreis (Institute for Quality and Efficiency in Health Care [IQWiG], Cologne, Germany), Toby Lasserson (Cochrane Editorial Unit)*, Fergus Macbeth (Wales Cancer Trials Unit), Silvia Minozzi (Lazio Regional Health Service), Karel GM Moons (University of Utrecht), Matthew Page (Monash University), Barney Reeves (University of Bristol)*, Nancy, Santesso (McMaster University), Jelena Savović (University of Bristol)*, Christopher H Schmid (Brown University), Beth Shaw (NICE), Beverley Shea (University of Ottawa)*, David Tovey (Cochrane Editorial Unit)*, Peter Tugwell (University of Ottawa), Meera Viswanathan (RTI International), Jasvinder Singh (University of Alabama at Birmingham & Birmingham Veterans Affairs Medical Center), Penny Whiting (University of Bristol)*, George Wells (University of Ottawa)*, Robert Wolff (KSR York Ltd.). The members designated with an asterisk (*) are also steering group members.

Conflicts of interest: None.

Funding: This work was funded by the Medical Research Council as part of the MRC–National Institute for Health Research (NIHR) Methodology Research Programme (Grant number MR/K01465X/1). B.C.R.'s time in contributing to the study was funded by the NIHR Bristol Biomedical Research Unit in Cardiovascular Disease. D.M.C. is supported by a Medical Research Council (UK) Population Health Science fellowship (G0902118).