Review Article
Summaries of findings, descriptions of interventions, and information about adverse effects would make reviews more informative

https://doi.org/10.1016/j.jclinepi.2005.12.011Get rights and content

Abstract

Background and Objective

To describe challenges when extracting and presenting relevant, consistent, and accessible information from systematic reviews.

Materials and Methods

We systematically selected comparisons and outcomes from 18 Cochrane reviews, evaluated the quality of evidence for each outcome using the GRADE system, and developed standardized patient information. We evaluated the information using patient, review author, researcher, and clinician feedback.

Results

Challenges included large numbers of comparisons and outcomes; missing information about treatments and adverse effects; and variations in how effect was measured and presented. By selecting comparisons and outcomes based on patient-relevance, quality, and nonredundancy, we halved the number of outcomes. We prepared information about treatments and adverse effects using other sources. We framed outcomes consistently and standardized the presentation of magnitude of effect.

Conclusions

The incorporation of summary of findings tables in reviews could address these challenges. Problems could also be reduced if review groups agreed upon standard outcomes; excluded less relevant outcomes; incorporated more information about interventions and adverse effects; and implemented clearer guidelines for the presentation of results.

Introduction

In an “evidence-informed patient choice,” a shared decision is taken after the person concerned has been given “research-based information about the effectiveness (likely outcomes, both benefits and risks) of at least two alternative interventions (which may include the option of no intervention)” [1].

Information that supports an evidence-informed choice should include reliable and unbiased information that covers treatments and outcomes that are of relevance to the patient in a language that the patient understands. But translating health care research into patient information presents a number of challenges.

A decade ago, the sheer volume of research as well as its varying quality would have represented one of the greatest challenges to this type of work. The development of systematic reviews represents a solution to these issues. Systematic reviews synthesize results of primary studies to answer a specific research question by using strategies to limit bias and random error [2]. Although the use of systematic reviews as a basis for the development of patient information is a good place to start, it may still pose a number of problems, either due to the reviews themselves or to the primary studies on which they are based.

The Backinfo project (www.backinfo.no) is one of a number of efforts to make the results of Cochrane reviews more accessible to people in decision-making situations by extracting and presenting key information [3]. BackInfo is an information Web site that presents the results of systematic reviews to Norwegian back pain sufferers about the effects of treatments for back pain.

Prior to the development of BackInfo's contents, one of us (C.G.) conducted a qualitative study of back pain sufferers' information needs [4], which showed that back pain sufferers wanted information about different treatment options, their possible benefits and harms, and the likelihood of these benefits and harms. Patient information guidelines [5], [6] and checklists (The DISCERN on the Internet Project; www.discern.org.uk/; 2005) recommend similar contents. When extracting and presenting this information from Cochrane reviews we aimed to achieve the following goals:

  • 1.

    Relevance: we excluded information regarded as not patient-relevant and presented information that was as specific as possible to specific patient subgroups and treatment types.

  • 2.

    Consistency: to reduce confusion and facilitate comparison across treatments, we presented information using a standardized format, and aimed for consistency with regard to population type, outcomes, and presentation of effect size.

  • 3.

    Comprehension: we chose language and presentation styles with the aim of increasing users' understanding of the material.

In this article we describe the challenges we encountered when aiming to achieve these goals. The implications of these challenges for trialists, review authors, and review groups are discussed.

Section snippets

Methods

The BackInfo team was made up of an international group of people with backgrounds in medicine, epidemiology, social anthropology, linguistics, physiotherapy, surgery, and nursing. One team member is also the coordinator of the Cochrane Back Group.

Selecting and creating comparisons

Eighteen reviews from the Cochrane Back Group and one review from the Cochrane Pregnancy and Childbirth Group covering 24 back pain treatments were used in BackInfo.

The reviews included 204 comparisons. The identification of comparisons in the reviews was complicated by the fact that comparisons were sometimes described differently in different parts of the reviews. It was also frequently not clear in reviews whether control groups received a placebo, usual care or “no treatment”; and what the

Discussion

A number of reasons lie behind our choice of Cochrane reviews as our primary source of research information. First, as contributors to the Cochrane Collaboration one of our motivations was to develop methods for making Cochrane reviews more accessible to the general public. Second, Cochrane reviews have been found to be of better quality than other published reviews on average, although they also have shortcomings. Third, and most importantly, Cochrane reviews are regularly updated and provide

Conclusions

By systematically selecting comparisons and outcomes, rephrasing and reframing outcome measures, and standardizing the presentation of treatment effects, we aimed to increase the usability and accessibility of the results of Cochrane reviews. One problem with this approach is that outcomes may already have been rephrased, reframed, and reanalyzed by the review authors after their selection from the primary studies. Because of the lack of information in many reviews about how outcomes were

Acknowledgments

This project was financed with the aid of EXTRA funds from the Norwegian Foundation for Health and Rehabilitation. The authors would also like to thank Timmo Koy and Eva Schonstein for their contributions to the BackInfo project.

References (25)

  • C.E. Wills et al.

    Patient comprehension of information for shared treatment decision making: state of the art and future directions

    Patient Educ Couns

    (2003)
  • V.A. Entwistle et al.

    Evidence-informed patient choice. Practical issues of involving patients in decisions about health care technologies

    Int J Technol Assess Health Care

    (1998)
  • D.J. Cook et al.

    Systematic reviews: synthesis of best evidence for clinical decisions

    Ann Intern Med

    (1997)
  • The Dissemination of Cochrane Evidence: an inventory of resources that use Cochrane reviews....
  • C. Glenton

    Developing patient-centred information for back pain patients

    Health Expect

    (2002)
  • A. Coulter et al.

    Informing patients. An assessment of the quality of patient information materials

    (1998)
  • M. Duman et al.

    The Poppi guide: practicalities of producing patient information

    (2000)
  • The GRADE Working Group

    Grading quality of evidence and strength of recommendations

    BMJ

    (2005)
  • K.B. Hagen et al.

    Bed rest for acute low back pain and sciatica (Cochrane Review). The Cochrane Library, Issue 1, 2003

    (2003)
  • R.A. Deyo et al.

    Outcome measures for low back pain research. A proposal for standardized use

    Spine

    (1998)
  • Alderson P, Green S, Higgins JPT, editors. Cochrane reviewers' handbook 4.2.2 [updated December 2003]. In: The Cochrane...
  • M. van Tulder et al.

    Updated method guidelines for systematic reviews in The Cochrane Collaboration Back Review Group

    Spine

    (2003)
  • Cited by (44)

    • Do clinicians want recommendations? A multicenter study comparing evidence summaries with and without GRADE recommendations

      2018, Journal of Clinical Epidemiology
      Citation Excerpt :

      However, this is not often achieved and a considerable gap between evidence and health care exists [2,3]. Key barriers for effective evidence implementation include the overwhelming amount of research literature and the lack of optimal evidence dissemination strategies [4–6]. One way of presenting evidence to clinicians in the course of their practice is through systematic reviews with summaries presenting tabular or narrative descriptions of best current evidence.

    • Knowledge Translation: The Bridging Function of Cochrane Rehabilitation

      2018, Archives of Physical Medicine and Rehabilitation
    • Reporting on interventions: Issues and guidelines for rehabilitation researchers

      2015, Archives of Physical Medicine and Rehabilitation
    • Reducing waste from incomplete or unusable reports of biomedical research

      2014, The Lancet
      Citation Excerpt :

      The adequacy of details can differ by clinical area and by stringency of the assessment. For example, only three (13%) of 24 reports of back pain trials were considered to supply sufficient information;12 only 30 (11%) of 262 reports of chemotherapy trials provided all ten items in a treatment checklist;13 and only 26 (17%) of 150 reports of patient education trials provided all ten items in their intervention checklist.14 Similarly, when the reporting of control conditions is poor, readers cannot draw correct inferences about the effect of the intervention (table).

    • Monitoring use of knowledge and evaluating outcomes

      2010, CMAJ. Canadian Medical Association Journal
    View all citing articles on Scopus
    View full text