Anecdotes that provide definitive evidence
BMJ 2006; 333 doi: https://doi.org/10.1136/bmj.39036.666389.94 (Published 14 December 2006) Cite this as: BMJ 2006;333:1267
- Jeffrey K Aronson, reader in clinical pharmacology1,
- Manfred Hauben, medical director of risk management strategy2
- 1Department of Clinical Pharmacology, Radcliffe Infirmary, Oxford University, Oxford OX2 6HE
- 2Risk Management Strategy, Pfizer, New York, USA
- Correspondence to: J K Aronson jeffrey.aronson{at}clinpharm.ox.ac.uk
- Accepted 30 October 2006
Many adverse drug reactions are first reported anecdotally. Anecdotal reports, by which we mean either individual cases or small case series, are generally regarded as providing poor quality evidence. They therefore usually require formal verification through robust epidemiological studies or clinical trials, although a minority are actually verified.1 However, we propose that some adverse drug reactions are so convincing, even without traditional chronological causal criteria such as challenge tests, that a well documented anecdotal report can provide convincing evidence of a causal association and further verification is not needed. Such reactions could serve as gold standards for use, for example, when validating pharmacovigilance systems or assessing the quality of systematic reviews of adverse drug reactions and the methods used to perform them. Specificity of an adverse drug reaction has previously been discussed as a concept2 but to our knowledge has never been fully developed.
Definitive adverse events
We have identified four types of spontaneously reported adverse events for which causal or contributory attribution to the drug is either irrefutable or demonstrable with a high level of confidence (table⇓):
-
Extracellular or intracellular tissue deposition of the drug or a metabolite
-
Specific anatomical location or pattern of injury
-
Physiological dysfunction or direct tissue damage that can be proved by physicochemical testing
-
Infection as a result of administration of a potentially infective agent or because of demonstrable contamination.
In each case the diagnosis can be established definitively, or with a high degree of certainty, in the individual patient. In some cases the diagnostic value of the event can be enhanced by further investigation, but the conclusion will always be related to the person affected.
Examples of definitive anecdotal adverse drug reactions
This analysis introduces a new perspective on anecdotal reports of adverse drug reactions. Our criteria are predicated on concepts that are not included in traditional procedures for assessing causality, which tend to emphasise chronological characteristics of reported associations, such as time courses of challenge, dechallenge, and rechallenge.2 However, no criticism of these traditional procedures is intended. Indeed, they may provide extremely cogent evidence for causation—for example, for reactions that occur immediately after the injection of a drug.
Extracellular or intracellular tissue deposition
The first group of definitive adverse events are those in which objective physicochemical testing shows that a pathological lesion is composed of the drug or metabolite. The lesion has to be accessible for biopsy or some form of in situ examination, and the event must not have been possible in the absence of the drug.
It is important to recognise the distinction between a lesion that is caused by the compound and one in which the compound is an innocent bystander. For example, calculi that are composed predominantly or exclusively of triamterene (that is, exceeding the solubility product in urine) qualify as definitive adverse events, but calculi in which triamterene is a minor component may reflect coprecipitation in the presence of pre-existing calculus disease and would not qualify as a definitive event.3 In well documented cases in which the calculus is composed entirely or predominantly of drug or metabolite, we can exclude confounding and infer either that an adverse drug reaction has occurred or at least that the drug had a major contributory role.
Specific anatomical location or pattern of injury
In the second category the location or pattern of injury is sufficiently specific to attribute the effect to the drug without the need for implicit judgment or formal investigation. The mechanism of injury can be related to physicochemical or pharmacological properties of the drug. Examples include extravasation reactions to cytostatic drugs and oral ulceration due to topical aspirin.
In some cases experimental evidence supports a multifactorial mechanism. For example, Nicolau's syndrome, a rare, necrotic, livedoid dermatitis reported with intramuscular injection of various drugs,9 10 11 is due to structural or functional vascular occlusion arising from inadvertent intra-arterial or paravascular injection, but the relative contributions of needle injury, volume effect, and physicochemical characteristics of the drug (such as fat emulsion or microcrystal deposition) have not been conclusively determined. Similarly, reports of injection site reactions with intramuscular and subcutaneous injections of drugs that are contained in bottles or cartridges with latex plungers and diaphragms may reflect latex allergy rather than a reaction to the drug.12 In these cases the event could be classified as a definitive adverse reaction to the whole formulation rather than to the drug itself.
Demonstrable physiological dysfunction or tissue damage
The third group includes adverse events that involve physiological dysfunction or tissue damage for which documentation by physicochemical testing is ethically and scientifically feasible. Some of the confirmatory test procedures may be operator or situation dependent. However, when properly done and interpreted, the tests can be confidently used to inform decision making in pharmacovigilance. An example is photopatch testing for photoallergy. False negative results are problematic, and positive results could theoretically reflect a local irritant effect of the drug interacting additively with a subclinical effect of the ultraviolet light source. Nevertheless, when drug versus vehicle and irradiated versus non-irradiated controls are properly used these tests have been reported as diagnostic in individual cases.
Infection
Adverse drug reactions related to infections can be due to contamination of the treatment or to a product that consists of live microbes. The infecting organism has to be proved to be the same as the organism contained in the product or contaminating the batch of product.
These four types of definitive adverse event are not mutually exclusive. For example, injection site reactions to vaccines that contain aluminium12 have characteristics that are consistent with more than one category, since they can show obvious anatomical contiguity (type 2) and electron microprobe analysis has shown features of aluminium crystal-storing histiocytosis (type 1). Fixed drug eruptions add an additional element of physical specificity that can further strengthen associations demonstrated with traditional provocative rechallenge.
Discussion
We have proposed a framework of types of associations between drugs and events that can be considered to be definitive adverse effects. We are not proposing a new classification of adverse drug reactions; we have merely described categories of reaction that can be regarded as definitive when described anecdotally. Our list is probably not exhaustive. In addition, although we have identified adverse effects that would not need confirmation in formal studies, studies with independent data sets would still be necessary to quantify the risk.
This framework can be conveniently expressed using a crime scene metaphor (table⇑). Type 1 events involve catching the suspect “red handed” at the scene of the crime (although sometimes the suspect may be an innocent bystander). With type 2 events the suspect is actually seen to have committed the crime. With type 3 events the association is substantiated by recreating the crime scene and showing that only the suspect could have committed the crime. With type 4 events the suspect's (DNA) fingerprints are all over the scene. In each case the proof is definitive. Other crime scenarios could suggest other categories of definitive adverse reactions that we have not described.
In all cases additional evidence—for example, a convincing time course—could be adduced to boost the strength of an association. Positive rechallenge can also contribute, in which case blinding and the use of placebo can further increase the evidentiary value of a single case. For example, the value of a positive rechallenge result is enhanced when the patient is unaware of the readministration13 or when placebo controlled rechallenge is used.14 The traditional methods of identification are thus independent but complementary.
Implications
The tenets of evidence based medicine include a hierarchy of evidence with systematic reviews of randomised clinical trials at the top and anecdotes near the bottom. Our observations show that anecdotes can, under the right circumstances, be of high quality and can serve as powerful evidence. In some cases other types of evidence may be more useful than a randomised controlled trial.15 And combining randomised trials with observational studies,16 or with observational studies and case series,17 can sometimes yield information that is not available from randomised trials alone.
Pharmacovigilance involves looking at single reports and heterogeneous case series of a wide variety of events and assessing the likelihood of there being adverse reactions. Definitive adverse reactions, such as we have described, could contribute to methods for testing the sensitivity of pharmacovigilance systems and tools by serving as “positive controls.” They could also be used to assess the quality of systematic reviews of adverse drug reactions and the methods used to perform them, by determining whether particular methods yield results that are consonant with this type of evidence. This is in the spirit of others who have discussed the tendency to undervalue observations from case reports.18
Summary points
-
Anecdotal reports of adverse drug reactions are generally regarded as providing poor evidence
-
Some anecdotal reports can be considered to describe definitive adverse reactions that do not need formal verification
-
Four categories of such adverse reactions are defined
-
These definitive adverse effects, which do not depend on time course, have important implications for pharmacovigilance
Footnotes
-
We thank Robin Ferner, Paul Glasziou, Yoon Loke, David Madigan, and Jan Vandenbroucke for reviewing draft versions of the manuscript and the resulting thoughtful discussion and valuable suggestions.
-
Contributors and sources: The authors have extensive experience of the reporting of adverse drug reactions, JKA as editor of Meyler's Side Effects of Drugs: the International Encyclopedia of Adverse Drug Reactions and Interactions (15th edition, Elsevier, 2006), and MH through his work in pharmacovigilance. The authors jointly formulated a categorical framework for “between the eyes” adverse drug reactions, based on prior knowledge and experience, deliberations, and discussions, and used this framework as the basis for exploring the published literature to locate examples to populate the proposed categories.
-
Competing interests: JKA is a member of the working party of the adverse events methods group of the Cochrane Collaboration; however, the views expressed here do not necessarily coincide with those of other members of the group. MH is an employee of Pfizer, which manufactures products referred to here or competing products in the same therapeutic classes.