What is new?
Key findingMost methods used to ascertain the smallest worthwhile effect of interventions for low back pain do not reflect patients' opinions, do not weigh the costs and benefits of intervention, and are not expressed in terms of differences in outcomes between intervention and control groups.
What is the implication and what should change now?The benefit-harm trade-off method should be used to elicit estimates of the smallest worthwhile effect of health interventions. The method could be used routinely to inform the design and interpretation of randomized trials.
Although there is a high degree of consensus about many aspects of how randomized trials should be conducted (e.g., [1]) several important methodological issues remained unresolved. One of the most persistent issues concerns how to estimate the smallest worthwhile effect of intervention [2].
The smallest worthwhile effect of an intervention is the smallest beneficial effect of intervention that justifies the costs, risks, and inconveniences of that intervention. It defines a threshold effect above which intervention might be indicated. There are at least two important uses of estimates of the smallest worthwhile effect in design and analysis of randomized trials. First, in the planning of randomized trials, information about the smallest worthwhile effect can be used to inform sample size calculations. Trials can, and arguably should, be powered to detect the smallest worthwhile effect of intervention. Second, once a trial has been completed, interpretation of the trial’s findings should involve consideration of whether the estimated effects of intervention are large enough to justify use of the intervention in clinical practice. In large part, this involves determining if the estimated effect of intervention exceeds the smallest worthwhile effect [3].
In an important article published in 1989, Jaeschke et al. [4] defined the Minimum Clinically Important Difference as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient’s management.” They explained that their interest in this construct was motivated by the desire to evaluate the “clinical importance” of estimates of the effect of interventions estimated in particular randomized trials. This article was significant because it was one of the first attempts to obtain empirical estimates of the smallest worthwhile (“clinically important”) effects of intervention.
Since the publication of Jaeschke et al.’s article, there have been many reports which describe measurement of the Minimum Clinically Important Difference. Other reports describe measurement of quantities with similar names, such as the Minimal Clinically Important Difference [5], Minimum Important Difference [6], Minimum Worthwhile Reductions [7], or Minimal Important Change [8], [9]. It is not always clear what construct these measurements are intended to measure. However, the authors of the reports often indicate that they are interested in identifying “clinically important” or “clinically meaningful” effects of intervention, suggesting that these estimates could be used for sample size calculations or to interpret the findings of clinical trials.
Barrett et al. [10] have carefully reviewed methods used to estimate the smallest worthwhile effect (or “clinical significance”) of interventions. They argued convincingly that such estimates must satisfy two conditions. First, decisions about what constitutes a worthwhile effect must involve weighing the benefits of the intervention against its costs, risks, and inconvenience. An important implication is that the smallest worthwhile effect must be intervention-specific. Thus the smallest worthwhile effect is not a property of the outcome measure. Second, judgments about whether the benefits of intervention outweigh costs, risks, and inconvenience must be based on the perspective of patients who are to receive the intervention. It will usually not be reasonable to claim that the effects of intervention are worthwhile unless the patient judges that the intervention is worthwhile. Therefore, judgments about whether the effect of a particular intervention is large enough to be worthwhile must be made by potential recipients of the intervention (patients), not by clinicians or researchers.
We would add one further criterion: if an estimate of the smallest worthwhile effect is to be used to inform the design and interpretation of clinical trials, it must be expressed in terms of an effect rather than an outcome [11], [12]. The effect of an intervention on an individual is the difference in outcomes that would occur with and without intervention (alternatively, the effect of an intervention could be the difference in outcomes that would occur with two competing interventions). It is a hypothetical value because individual patients do not simultaneously experience and not experience the intervention. This means that the precise effect of intervention on an individual cannot usually be known. Nonetheless it is possible, in randomized trials, to estimate the mean effect of intervention because the difference in the mean outcome of the intervention and the control groups is equal to the mean effect of intervention [13], [14]. In contrast, treatment outcomes, or changes in outcome that occur over the course of treatment, do not provide a satisfactory measure of the effect of intervention because although they might be influenced by intervention, they might also be influenced by natural recovery, statistical regression, and placebo effects [15]. So estimates of the smallest worthwhile effect of intervention must be conceived in terms of the hypothetical difference in outcomes with and without intervention, rather than in terms of outcomes or changes in outcome over the course of treatment. The same point has been made by researchers associated with Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials [11].
Our impression, before conducting this review, was that although many studies use language that suggests they are interested in measuring the smallest worthwhile effect of an intervention, few studies use methods that would enable them to do so. Specifically, our impression was that most such measurements are not directly based on patients’ perceptions, are not intervention-specific, and are not formulated in terms of effects of intervention. Consequently we conducted a systematic review to explore how the smallest worthwhile effect of interventions has been measured. We focused on research into low back pain because many relevant studies have been conducted in this field. We sought to determine whether estimates of the smallest worthwhile effect were based on the opinions of patients, were intervention-specific, and were expressed in terms of effects of intervention.