• Statistics & Research Methods
• Epidemiology

Availability of high-quality estimates of the absolute difference in effectiveness between alternative treatment options is crucial to the application of evidence-based healthcare to populations of patients and corresponding decisions. One framework for assessing confidence in estimates of the effect of alternative management strategies on patient-relevant outcomes within the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system1 is summarised by Spencer et al.2 In this article we single out one of the five domains of the GRADE system, namely imprecision.

Often, the best available evidence for the absolute difference in effectiveness between a treatment under consideration and a standard regime does not come from a single study, but from two totally separate sources. Sometimes, an estimate of the relative risk (RR) of the outcome of interest between the two treatment options is available from a meta-analysis combining evidence from several randomised trials. Owing to the larger sample size available, this will in general have greater precision than an RR derived from a single study. In most contexts, estimates of relative effect of a therapy are more consistent across different baseline risks than absolute effect estimates.3 Consequently, it is a common practice in systematic reviews to report a pooled estimate of the RR, rather than the absolute risk difference (RD).2

To convert an RR into an absolute RD, we also require an estimate of the baseline risk (BR), the rate of occurrence of the event of interest when the standard treatment is used. The absolute RD is then calculated from the BR and RR using the formula RD = BR×(RR−1).

In most applications, the RR is below 1, representing a reduction in risk due to the intervention. The calculated RD is then negative. Sometimes, the RR may be greater than 1, representing an increase in risk due to the intervention. The calculated RD is then positive.

Spencer et al2 noted that the calculations currently performed under the GRADE framework take into account the imprecision of the RR estimate, but not that of the baseline risk estimate. They concluded that evaluating uncertainty in baseline risk, and its impact on confidence in absolute estimates of treatment effect, remained an important outstanding issue. The purpose of this article is to describe a simple, effective method that may be used to take both sources of uncertainty into account.

All the quantities we concerned to estimate, such as the BR, the RR or the RD, are derived from series of patients of finite size. A CI is normally used to display the resulting uncertainty of such an estimate. CIs convey information about magnitude and precision of effect simultaneously, keeping these two aspects of measurement closely linked.4 ,5 In the great majority of instances, researchers calculate 95% CIs, as a common metric to quantify sampling imprecision.

Confidence limits for the RD may be calculated from those for the BR and RR by a procedure called Method of Variance Estimates Recovery (MOVER). This is a general approach that may be used to calculate CIs for sums and differences of two independently estimated quantities. MOVER may be extended to apply to products or ratios, but greater care is required. Neither an approach using logarithms of BR and (RR−1)6 nor the version of MOVER developed specifically for ratios7 is guaranteed to yield meaningful results when the CI for the RR can span 1, for reasons explained by Newcombe.8 An enhanced version described by Newcombe8 is designed to accommodate datasets with RR and corresponding confidence limits below 1, representing benefit, as well as above 1, representing harm.

While it is simple to calculate the RD from the BR and RR, the formulae to derive confidence limits for the RD from those of the BR and RR are quite complicated.7 ,8 However, there is no need for the user to perform any part of the calculations. An Excel spreadsheet ‘RD from BR and RR.xls’ to perform all the calculations described in this article is freely downloadable from: http://medicine.cf.ac.uk/primary-care-public-health/resources/

The calculations in the spreadsheet start with estimates of the BR and RR and the corresponding CIs. The RD together with its CI is then derived from these figures. If 95% CIs are used for the BR and RR, the resulting CI for the RD is also a 95% CI. The spreadsheet is designed to be a highly user-friendly resource, though needless to say, great care is needed with the negative numbers used to represent benefits.

An example where the intervention is beneficial

Spencer et al2 reported calculations taken from Bates et al9 relating to the use of low-dose, low-molecular-weight heparin (LMWH) to prevent venous thromboembolic events in women undergoing assisted reproduction who develop severe ovarian hyperstimulation syndrome. The RR used here is 0.36 (95% CI 0.20 to 0.67), taken from a meta-analysis. The baseline risk is 2/49=0.041, with 95% CI 0.011 to 0.137 calculated with the Wilson score method10 taken from a small Czech study.11 Bates et al9 combined the RR and BR here to give RD=0.041×(0.36–1)=−0.026, indicating that use of low-dose LMWH is estimated to prevent 26 venous thromboembolic events per 1000 patients treated. Bates et al9 reported 95% confidence limits of 13 to 32 events prevented. However, these limits take account of the uncertainty of the RR only. Spencer et al2 observed that the uncertainty of the BR should also be taken into account here, and reported a CI of 4 to 110 for the number of events prevented per 1000 women. These figures are obtained by directly combining the 95% confidence limits for the RR and BR, which leads to an unnecessarily wide interval. The correct 95% CI calculated by MOVER-R is −0.089 to −0.006, indicating prevention of between 6 and 89 events per 1000 women.

We could equally well construct a CI representing the uncertainty of the BR only, −0.088 to −0.007 here. Figure 1 displays all four 95% CIs for the RD, expressed as a risk reduction. In this example, the correct MOVER-R interval is very similar to the interval derived from that for the BR. Here, the BR is the dominant source of imprecision.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Absolute risk reduction (expressed as events prevented per 1000 women) for effect of low-dose, low-molecular-weight heparin on venous thrombolic events, from Bates et al.9 Four 95% CIs are shown, (A) representing imprecision of relative risk (RR) only, (B) imprecision of baseline risk (BR) only, (C) a correct interval using MOVER-R and (d) calculated directly from upper and lower limits for both BR and RR.

An example where the interventionincreases risk

Arzola and Wieczorek12 evaluated the use of low-dose bupivacaine (≤8 mg) in spinal anaesthesia for elective caesarean section. While use of a low dose instead of the conventional dose (>8 mg) may help prevent hypotension resulting from spinal anaesthesia, it may compromise anaesthetic efficacy. In a meta-analysis, the need for analgesic supplementation during surgery was higher (RR=3.76, 95% CI 2.38 to 5.92) in women receiving the low dose compared with the conventional dose. The baseline risk of needing analgesic supplementation during surgery was taken as 10.9% based on Garry and Davies.13 This is derived from 175 occurrences among 1610 women, leading to a 95% CI for BR of 9.4% to 12.5%. The resulting RD is 0.109×(3.76–1)=0.301, with 95% CI 0.149 to 0.546. These results indicate that using low-dose bupivacaine leads to the need for analgesic supplementation in an additional 30% of women compared with the use of the conventional dose, with 95% CI 15% to 55%.

Figure 2 displays four 95% CIs for the RD, as in figure 1. Here, the RR is the dominant source of uncertainty, hence it is the interval representing the uncertainty of the RR only that approximates closely the MOVER-R interval.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Absolute risk difference (in %) for effect of low-dose bupivacaine on spinal anaesthesia efficacy, from Arzola and Wieczorek.12 Four 95% CIs are shown in figure 1.