GRADE Series
GRADE guidelines: 12. Preparing Summary of Findings tables—binary outcomes

https://doi.org/10.1016/j.jclinepi.2012.01.012Get rights and content

Abstract

Summary of Findings (SoF) tables present, for each of the seven (or fewer) most important outcomes, the following: the number of studies and number of participants; the confidence in effect estimates (quality of evidence); and the best estimates of relative and absolute effects. Potentially challenging choices in preparing SoF table include using direct evidence (which may have very few events) or indirect evidence (from a surrogate) as the best evidence for a treatment effect. If a surrogate is chosen, it must be labeled as substituting for the corresponding patient-important outcome.

Another such choice is presenting evidence from low-quality randomized trials or high-quality observational studies. When in doubt, a reasonable approach is to present both sets of evidence; if the two bodies of evidence have similar quality but discrepant results, one would rate down further for inconsistency.

For binary outcomes, relative risks (RRs) are the preferred measure of relative effect and, in most instances, are applied to the baseline or control group risks to generate absolute risks. Ideally, the baseline risks come from observational studies including representative patients and identifying easily measured prognostic factors that define groups at differing risk. In the absence of such studies, relevant randomized trials provide estimates of baseline risk.

When confidence intervals (CIs) around the relative effect include no difference, one may simply state in the absolute risk column that results fail to show a difference, omit the point estimate and report only the CIs, or add a comment emphasizing the uncertainty associated with the point estimate.

Introduction

What is new?

Key points

  1. Summary of Findings (SoF) tables provide succinct, easily digestible presentations of confidence in effect estimates (quality of evidence) and magnitude of effects.

  2. SoF table should present the seven (or fewer) most important outcomes—these outcomes must always be patient-important outcomes and never be surrogates, although surrogates can be used to estimate effects on patient-important outcomes.

  3. SoF table should present the highest quality evidence. When quality of two bodies of evidence (e.g., randomized trials and observational studies) is similar, SoF table may include summaries from both.

  4. SoF table should include both relative and absolute effect measures, and separate estimates of absolute effect for identifiable patient groups with substantially different baseline or control group risks.

The first 11 articles in this series introduced the GRADE approach to systematic reviews and guideline development [1], discussed the framing of the question [2], and presented GRADE’s concept of confidence in effect estimates [3] and how to apply it [4], [5], [6], [7], [8], [9]. In this 12th article, we describe the final product of a systematic review using the GRADE process, Summary of Findings (SoF) tables that present, for each relevant comparison of alternative management strategies, the quality rating for each outcome, the best estimate of the magnitude of effect in relative terms, and the absolute effect that one might see across subgroups of patients with varying baseline or control group risks. The focus of this article is on binary outcomes. Box 1 presents the seven elements recommended for SoF tables. Table 1, Table 2, Table 3, examples of SoF tables, highlight some of the issues in constructing such a table. Readers will find additional details in the Cochrane Handbook, Chapter 11 [10].

Section snippets

The seven elements of a SoF table

SoF tables include seven elements (Box 1). Uniformity of presentation is likely to facilitate readers’ familiarity and comfort with SoF tables and is therefore desirable and facilitated by the use of GRADEpro software [11]. Initial user testing with consumers of guidelines (clinicians and researchers) guided the format of Table 1 [12], [13]. In Table 1, putting what is most important first guided the order of the columns, and the presentation of absolute risks was guided by a finding that some

Choosing which outcomes to present

SoF tables should ideally present results of all patient-important outcomes—possibly noting which ones are critical—without, however, overwhelming the reader. GRADE suggests inclusion of no more than seven outcomes, including both benefits and harms. If there are more than seven outcomes that are judged important, reviewers should choose the seven most important. This number is based on our intuition about the amount of information users can grasp, and an informal survey of attendees at

Presentation of direct vs. indirect evidence

Sometimes, direct measures of the patient-important outcomes are unavailable or, as in Table 1, no events have occurred (for symptomatic venous thrombosis and pulmonary embolism). In such instances, reviewers should present their inferences regarding treatment effects on patient-important outcomes on the basis of the results of surrogate measures. That the inferences are coming from surrogates should be clearly labeled, and will almost certainly result in rating down the confidence in effect

Presentation of randomized controlled trials or observational studies

Randomized controlled trials (RCTs) usually provide higher-quality evidence than observational studies and, if RCTs are available, SoF tables should generally restrict themselves to reporting RCT results. On occasion, however, limitations of RCTs or particular strengths of observational studies may lead to conclusions that their confidence in effect estimates is similar, or that observational studies provide higher-quality evidence.

For instance, consider the use of octreotide to prevent

Dealing with analytic approaches that yield different results

Systematic reviews, in exploring sources of heterogeneity, may sometimes find that alternative analyses (“sensitivity analyses”) yield appreciably different results. For example, a systematic review of glucosamine for treating osteoarthritis found differences in pain reduction when including only trials with concealed allocation vs. all trials [20]. Presenting two rows, one summarizing each analytic approach, would have left the inevitably less-equipped readers with the decision about which

Measures of relative effect

Options for expressing relative measures of effect include the RR (synonym: risk ratio), odds ratio (OR), rate ratio, and hazard ratio [21], [22], [23]. ORs have advantageous statistical properties [24]. RRs, however, are more understandable intuitively, and easier to use for estimating absolute measures of effect in individual patients [21]. We find these advantages of RRs compelling (for more details, see Box 3). Meta-analysis can generate RRs or ORs from 2 × 2 tables using appropriate

Measures of absolute effect

As we have pointed out, relative measures tend to be consistent across risk groups, whereas absolute measures do not [22], [27], [28], [29]. Making management choices, however, focuses on trading off absolute effects on patient-important outcomes, therefore requiring both relative and absolute measures to appear in SoF tables.

The unrepresentativeness of patients in randomized trials, and the lack of consistency of absolute measures across risk groups and across individual trials argue against

Presentation of absolute effects

We suggest presenting the absolute effect—both benefits and harms—as natural frequencies (events per 10,000 patients in Table 1, although more frequent events can be presented as events per 1,000 or even per 100 patients) because this facilitates decision making [31], [32], [33], [34]. When events are sufficiently frequent, percentages may be as well, or marginally better, understood [35]. Although many clinicians prefer numbers needed to treat (NNTs), they may be more difficult to interpret

Absolute effects—confidence intervals

We further suggest reporting the CIs around the absolute risk in the intervention group (as in Table 1, Table 6) or around the difference between intervention and control groups (as in Table 2, Table 3, Table 4, Table 5). Just as one calculates the absolute risk in the intervention group on the basis of the absolute risk in the comparison group and the point estimate of the RR, the calculation of the CIs around the absolute risks in the intervention group is based on the absolute risk in the

Absolute effects—choice of time frame

In Table 1, the time frame for measurement of outcome is both obvious and short—symptomatic thrombosis, if it exists, will occur within days of a long flight. For conditions such as primary and secondary prevention of cardiovascular events, or cancer recurrence, there are options for choice of the duration of follow-up. Reviewers should therefore always indicate the length of follow-up to which the estimates of absolute effect refer. Note, this length of follow-up may not be the length of

Dealing with no events in either group

When no participant in any trial has suffered the outcome of interest, the trials provide no information about relative effects (and one can thus argue that there is no point in rating the quality of the evidence). However, particularly if there are large numbers of patients, the data may provide high-quality evidence that the absolute difference between alternative management strategies is small or very small. If reviewers believe this is the appropriate inference for an important or crucial

Uncertainty around estimates of baseline risk

Note that Table 1 provides estimates of risk in the intervention group based on the CIs around the RR. We do not, however, provide estimates of uncertainty regarding the estimates of baseline risk in high- and low-risk control groups. Not presenting such estimates reflects a high priority on simple presentations that clinicians and patients will find easily digestible.

Potentially, all the issues that raise uncertainty about estimates of absolute effects could raise uncertainty about estimates

What to do when there is no published evidence regarding an important outcome

We encourage systematic review authors and guideline developers to specify all important outcomes before commencing their reviews. If they do so, it is possible that they may find no published evidence regarding one or more outcomes (quality of life and rare side effects are two outcomes that may be subject to this problem). We suggest that if sufficiently important, such an outcome would warrant a row in the SoF table, with the confidence in effect estimates rating (and other cells aside from

Conclusion

The SoF table provides all the key information necessary for making decisions between competing health care management strategies [38]. Therefore, although not an absolute requirement for use of the GRADE approach, the SoF table is an invaluable tool for providing a succinct, accessible, transparent evidence summary for patients, health care providers, and policy makers.

References (38)

  • S.E. Rosenbaum et al.

    User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews

    J Clin Epidemiol

    (2010)
  • W.H. Geerts et al.

    Prevention of venous thromboembolism: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th edition)

    Chest

    (2008)
  • C.J. Fasano et al.

    Comparison of octreotide and standard therapy versus standard therapy alone for the treatment of sulfonylurea-induced hypoglycemia

    Ann Emerg Med

    (2008)
  • S.D. Walter

    Choice of effect measure for epidemiological data

    J Clin Epidemiol

    (2000)
  • M.H. Murad et al.

    Estimating risk difference from relative association measures in meta-analysis can infrequently pose interpretational challenges

    J Clin Epidemiol

    (2009)
  • H. Schünemann et al.

    Presenting results and ‘Summary of findings’ tables

  • Brozek J, Oxman A, Schünemann H. GRADEpro. [Computer program]. Version 3.2 for Windows. Available at...
  • M. Clarke et al.

    Compression stockings for preventing deep vein thrombosis in airline passengers

    Cochrane Database Syst Rev

    (2007)
  • J.W. Busse et al.

    Low intensity pulsed ultrasonography for fractures: systematic review of randomised controlled trials

    BMJ

    (2009)
  • Cited by (0)

    The GRADE system has been developed by the GRADE Working Group. The named authors drafted and revised this article. A complete list of contributors to this series can be found on the journal’s Web site at www.elsevier.com.

    View full text