CommentaryThe double-blind, randomized, placebo-controlled trial: Gold standard or golden calf?
Introduction
The double-blind randomized controlled trial (RCT) seeks to confer the ideal of scientific exactitude onto clinical experimentation in an effort to attain the objectivity of the laboratory model. A placebo-controlled RCT is considered medicine's most reliable method for “representing things as they really are” [1]. While random error is mathematically estimated, systematic error is minimized by the rigorous application of methodological safeguards, especially randomization and blinding. Randomization aims to eliminate both unconscious and deliberate human influence on the assignment of subjects to different groups. Blind assessment ensures that treatment and analysis of outcomes are not colored by prejudice. Without these precautions, according to the standard epidemiological rationale, deliberate subversions (albeit well intentioned) or “subtle and intangible…subconscious” processes will affect the work of even the most conscientious researcher [2]. Assumed to be stripped clean of human bias, the masked (blind) RCT is accepted as the gold standard and thus above scrutiny as a potential source of systematic error.
However, it may be that experiments on humans by humans cannot circumvent the distortions and subversion of human consciousness and subjectivity. Do we need to consider “the effect of the experiment on the subjects themselves?” [3]. Is there a possibility that we need to “begin to speak of a ‘Heisenberg Principle of [Human Experimental] Sciences,’ where the very act of setting up controls can alter the phenomenon sufficiently to yield quite different results?” [4]. The general adoption of the double-blind RCT was based on theoretical reasons and intuitive attractiveness rather than a compelling body of data 5, 6, and attempts to systematically investigate its assumed objectivity have been relatively scarce [7]. This article summarizes empirical evidence pertaining to the RCT's capacity to produce undistorted and objective information. Shortcomings of imperfect RCTs are rarely examined in this essay; its focus is on possible systematic errors intrinsic in even an ideal RCT. Concealment in placebo-controlled trials is especially discussed in detail. It may be that every research methodology has inherent and random artifacts and that “truth” lies buried underneath multiple approaches.
Section snippets
The discrepancy argument
The primary empirical evidence for the objectivity of the double-blind RCT lies in the differential outcomes it detects compared with other research designs. Until very recently, there was a widespread perception that the absence of the usual components of the masked RCT will “exaggerate estimates of treatment effects” [8]. Often called a “measure of bias,” this discrepancy among results achieved through different methodologies was accepted as evidence of the objectivity of a masked RCT [9]. It
A modified challenge to the discrepancy argument
The three recent comparisons of outcomes using different methodologies described above have slightly changed the discrepancy argument. Instead of consistently adjusting for bias in the direction of inflated estimates of effects, the methodological safeguards of randomization and blinding are now considered the “best protection against the unpredictability…of bias” [21]. All three studies comparing rigorous with less rigorous evidence agree that “a large, inclusive, fully blinded RCT…is likely
An overlooked aspect of the discrepancy debate: can an “unbiased” method produce its own distortions?
Overlooked in the discrepancy debate is that its logic is circular. It authenticates itself: “the truth is what we find out in such and such a way. We recognize it as truth because of how we find it out. And how do we know that the method is good? Because it gets at the truth” [42]. As one research team put it: “Unfortunately, there is no gold standard for judging the effectiveness of therapies apart from [double blind randomized] clinical trials” [43]. Is it possible that the exigencies of the
Testing the temper of the gold standard: is there a “masking bias”?
Any external standard used in order to be “more objective” and verify the validity of the masked RCT would have to erect an even higher barrier to the subversive threat of human subjectivity. One theoretical example of this hypothetical “platinum” standard would be a trial in which both the patients and the dispensing physician were unaware that they were involved in a blind RCT. If patients were randomized to either “platinum” or routine RCT, one could compare results and thus test the temper
“Investigator self-selection,” “preference” and other sources of bias in RCTs
Besides a possible “masking” bias, the RCT apparatus can generate other sources of potential bias. Some of these potential problems are rarely discussed. Others are well known (especially those that deal with circumstances that affect the external validity of the trial and are perhaps less pertinent in a discussion of an “ideal” RCT) and have been extensively described elsewhere (e.g., [67]). A very brief review of some of these potential internal and external confounders may be helpful in
Conclusion: is the double-blind RCT objective?
The masked RCT attempts to provide a method that can free medical research from the fallibility of the human mind. Some experimental evidence shows that masking cannot completely neutralize the potential distortions of human consciousness and subjectivity. Such bias may threaten the internal validity of information produced. Preference effects also have this potential. Human behavior, such as a patient's refusal to enter trials or the researcher's reluctance to randomize practitioners, can also
Acknowledgements
The critical feedback of Fred Mosteller, Al Fishman and John Emerson and the editorial assistance of June Cobb are gratefully appreciated. Also I wish to thank the Seminar on Effective and Affordable Health Care at Harvard University for allowing a presentation of an earlier version of this paper.
References (99)
- et al.
Placebo effect in double-blind clinical trialsa review of interactions with medications
Lancet
(1994) Rules of evidence and clinical recommendations on the use of antithrombotic agents
Chest
(1986)- et al.
A method for the evaluation of the effects of drugs on cardiac pain in patients with angina on effort. A study of Khellin (Visammin)
Am J Med
(1950) Powerful placebothe dark side of the randomized controlled trial
Lancet
(1998)- et al.
Randomized versus historical controls for clinical trials
Am J Med
(1982) - et al.
Randomization is important in studies with pain outcomessystematic review of transcutaneous electrical nerve stimulation in acute postoperative pain
Br J Med
(1996) - et al.
Assessment of the efficacy and safety of antiarrhythmic therapy for chronic atrial fibrillationobservations on the role of trial design and implications of drug related mortality
Am Heart J
(1992) - et al.
A meta-analysis of the therapeutic role of oil soluble contrast media at hysterosalpingographya surprising result?
Fertil Steril
(1994) Impact of random assignment on study outcomean empirical examination
Control Clin Trials
(1992)- et al.
Does the quality of randomized trials affect estimates of intervention efficacy reported in meta-analysis?
Lancet
(1998)
Patients' preferences and randomized trials
Lancet
Why randomize?
J Stat Plan Inf
Patients' willingness to enter clinical trialsmeasuring the association with perceived benefit and preference for decision participation
Soc Sci Med
Assessing non-consent bias with parallel randomized and nonrandomized clinical trials
J Clin Epidemiol
Meta-analysisstatistical alchemy for the 21st century
J Clin Epidemiol
Philosophy and the mirror of nature
Fundamentals of clinical trials
Allocation of subjects in medical experiments
N Engl J Med
Drug-set interactionthe effect of expectation on drug response in outpatients
Intentional ignorancea history of blind assessment and placebo controls
Bull Hist Med
Non-randomized controls in cancer clinical trials
N Engl J Med
Why are randomized controlled trials important?
Br Med J
Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials
J Am Med Assoc
Allocation of patients to treatment in clinical trials
Biometrics
Clinical research in psychiatry
J Ment Sci
A study of the influence of experimental design on clinical outcome in drug research
Am J Psychol
Glutamic acid and human intelligence
Psychol Bull
Research evaluating antidepressant medications on hospitalized mental patientsa survey of published reports during a five-year period
J Nerve Ment Dis
The present status of shunts for portal hypertension in cirrhosis
Gastroenterology
Indomethacina survey of clinical trials
Clin Pharm Ther
The unpredictability paradoxreview of empirical comparisons of randomized and non-randomized clinical trials
Br Med J
Evidence favoring the use of anticoagulants in the hospital phase of acute myocardial infarction
N Engl J Med
A comparison of randomized concurrent control groups with matched historical control groupsare historical controls valid?
J Clin Oncol
A review and meta analysis of hormonal treatment of cryptorchidism
J Clin Endocrinol Metab
Worldwide collaborative observational study and meta-analysis on allogenic leukocyte immunotherapy for recurrent spontaneous abortion
Am J Reprod Immunol
How study design affects outcomes in comparisons of therapy. Imedical
Stat Med
How study design affects outcomes in comparisons of therapy. IIsurgical
Stat Med
Interpreting the evidencechoosing between randomized and non-randomized studies
Br Med J
Comparison of effect size estimates derived from randomised and non-randomised studies
Randomized, controlled trials, observational studies, and the hierarchy of research designs
N Engl J Med
A comparison of observational studies and randomized, controlled trials
N Engl J Med
Randomized trials or observational tribulations
N Engl J Med
Bias in treatment assignment in controlled clinical trials
N Engl J Med
Statistical language, statistical truth and statistical reasonthe self-authentication of a style of scientific reasoning
Sensitivity and specificity of clinical trials. Randomized v historical controls
Arch Intern Med
Why we need observational studies to evaluate the effectiveness of health care
Br Med J
Cited by (319)
Preclinical evidence of the effect of quercetin on diabetic nephropathy: A meta-analysis of animal studies
2022, European Journal of PharmacologyEnacting a depoliticised alterity: law and traditional medicine at the World Health Organization
2022, International Journal of Law in ContextUse and Impact of Simulation in Family Caregiver Education: A Systematic Review
2024, Western Journal of Nursing ResearchConsiderations for peer research and implications for mental health professionals: learning from research on food insecurity and severe mental illness
2024, Journal of Psychiatric and Mental Health Nursing