Original Article
Uncertainty in the minimum event risk to justify treatment was evaluated

https://doi.org/10.1016/j.jclinepi.2008.09.017Get rights and content

Abstract

Objective

To derive expressions for the standard errors (SEs) and coefficients of variation (CV) of the threshold number needed to treat (NNTt) and the minimum target event risk for treatment (MERT).

Study Design and Setting

NNTt reflects the point at which the risks and costs of a clinical intervention balance the benefit. MERT defines the minimum target event risk at which the intervention is justified. Uncertainty in these measures has not previously been investigated.

Results

SEs for NNTt and MERT were derived. The corresponding CVs are particularly useful, because they decompose the variability of NNTt and MERT into the uncertainty in their components (the values of target and adverse events, and the adverse event risk [AER]). The precision required for these components to formulate treatment recommendations is, thereby, highlighted. These ideas were illustrated with data concerning warfarin treatment for atrial fibrillation.

Conclusion

Our expressions for uncertainty in NNTt and MERT inform the confidence one has in initiating a clinical intervention. In our example, a recommendation for treatment could be made for groups of patients whose risk exceeded the range of uncertainty in MERT. However, for lower-risk patients, a recommendation for or against treatment could not be made, mainly because of the limited data on AERs. Our methods can also be used to estimate how much additional data would be required to provide a firmer recommendation for such patient groups.

Introduction

What is new?

Clinical problem:

  1. Warfarin reduces the risk of stroke in patients with atrial fibrillation; however, it also causes important side effects. Thus, its use may be justified in groups of patients at high risk of stroke, but not in those at low risk.

What this article proposes to do:
  1. For treatments having both benefits and risks, we review expressions for the threshold NNT and the minimum level of risk for the target event (MERT) above which treatment is justified. The method requires data on the adverse event risk (AER), the values of the target and adverse events, and the relative risk reduction (RRR) caused by treatment on the target event. In our warfarin example, MERT for stroke was an annual risk of approximately 0.5%. We show how the uncertainty in the components of MERT can be used to derive a confidence interval (CI) for it.

What has been added:
  1. A recommendation to use, or not to use, a treatment which is effective but has important side effects, can be derived by comparing a patient's risk for the target event with the CI for MERT. The components of uncertainty in MERT can suggest how much additional data would be required, either on AER or values attached to clinical outcomes of treatment, to estimate MERT more precisely. In the example, therapy could be strongly recommended for patient groups whose annual stroke risk exceeded 1.25%, but considerably more data on adverse event rates would be needed to reach a recommendation for patient populations at lower risk.

The number of patients one needs to treat (NNT) to prevent one patient having the target event is a useful index for clinicians wishing to assess the benefit of a treatment [1]. NNT is defined as the inverse of the absolute risk reduction caused by treatment. Assuming that relative risk reduction (RRR) is reasonably constant across the range of risk of untreated patients, absolute risk reduction and NNT will vary systematically across that range [2]. For patients at low risk, NNT tends to be high, whereas for patients at high risk, NNT is lower.

The threshold number needed to treat (NNTt) is the value of NNT at which the expected benefit of treatment is exactly offset by the adverse effects of treatment. NNTt is determined by risks of adverse events caused by treatment, values (importance) attached to benefits and risks of treatment, and economic costs. We assume that the risk difference for adverse events is reasonably constant across the range of baseline risk for the target event.

The minimum event risk for treatment (MERT) is a related measure that incorporates NNTt, but also involves the RRR for the target event associated with the treatment vs. control. Clinicians should treat patient populations whose baseline risk is higher than MERT, because the expected benefits of treatment outweigh the expected negative consequences.

For example, consider the use of warfarin for the prevention of stroke in patients with atrial fibrillation. Warfarin is effective in reducing thromboembolic stroke in these patients. However, warfarin causes serious bleeding into the brain and gastrointestinal tract, and other minor bleeds. Thus, the question arises: even though warfarin is effective, should it be used in all patients? Or among these patients, is there a baseline risk below which the downsides of treatment (side effects caused, economic costs) outweigh its benefits? To examine this, data are needed on: the types of adverse events involved; their risk difference; the importance, or utility value, placed on the target event prevented and the adverse events caused; and economic costs. We will return to this example later.

Expressions for NNTt and MERT have been derived previously [3]. They involve a fuller model that incorporates clinical outcomes, values, and economic costs, or a simpler model that disregards costs. Another possible simplification is to formulate NNTt in terms of the relative values of the adverse and target events.

Figure 1 illustrates the relationships between NNT, its threshold value NNTt, MERT, and the baseline risk. The value of NNT increases as baseline risk decreases, calling into question the net benefit of treatment. The point at which the NNT curve crosses the threshold value NNTt defines MERT, is the minimum risk at which treatment can be recommended, taking benefits and harms into account. This article concerns the uncertainty associated with the estimated value of MERT. As shown in Fig. 1, if the baseline risk for a group of patients is minimally above MERT, a weak recommendation for treatment can be made, whereas if the risk is significantly above MERT, a strong recommendation for treatment applies. Similar considerations pertain to weak or strong recommendations against treatment for groups of patients whose baseline risk is minimally or substantially below MERT.

The effect on NNTt and MERT of uncertainty in the estimated treatment effect on the target outcome has been considered earlier [3]. However, we know of no previous work concerning the impact of uncertainty in the other determinants (specifically, values of benefits and harms, and risks of harms) of the variability of NNTt or MERT. Therefore, the goals of this study are to develop expressions for the standard errors (SEs) and coefficients of variation (CVs) of NNTt and MERT, and to apply them in an example. We consider the uncertainty in NNTt and MERT in models that include one or more kinds of adverse events, but we do not incorporate economic costs. Using the CV formulation facilitates the identification of the relative contributions to uncertainty in NNTt from each of its components. Uncertainty in MERT additionally depends on the uncertainty in the RRR for the target event.

Section snippets

Methods

We define and review the calculation of NNTt, allowing for the possibility of several types of adverse events, and the corresponding value of MERT. We then develop expressions for the SE of NNTt, which involve the variances of its components. An alternative formulation shows that when there is a single type of adverse event, the squared CV of NNTt equals the sum of squared CVs for its components; this approach is useful for interpretational purposes, and only requires estimates of the relative

Derivation of standard errors of threshold number needed to treat and minimum target event risk for treatment

We now develop expressions for the SEs of NNTt and MERT, and then apply them in a practical example.

Example

We now return to the warfarin example. A meta-analysis of five randomized trials showed that warfarin reduced the incidence of stroke in patients with atrial fibrillation, with RRR = 68% (95% confidence interval [CI]: 50%, 79%) over a mean follow-up of approximately 1.5 years [10]. RRR was fairly consistent across trials, even though the risk in the control groups varied substantially. Consequently, the absolute risk reduction and NNT varied considerably. The major adverse side effect of warfarin

Discussion

The NNT indicates the overall benefit of a given treatment in terms of how many patients should be treated to prevent one target event. The NNTt extends this concept by incorporating the notion of balancing potential benefit with potential harm from adverse events. We can consider patients with various levels of baseline risk, and determine if their individual NNT is below the threshold value, and hence, if a treatment intervention can be justified. The MERT index identifies those patients

Acknowledgments

The authors thank Dr. Gordon Guyatt (McMaster University) for helpful comments on a draft of this article, and Corinne Riddell for preparation of Fig. 1.

References (17)

There are more references available in the full text version of this article.

Cited by (9)

View all citing articles on Scopus
View full text