What is new? The role of case series evidence in systematic reviews has been little studied. Our study, based on a review of catheter ablation for the treatment of atrial fibrillation, suggests that case series can strengthen the evidence base and increase the clinical credibility of reviews of emerging health technologies. These advantages must be balanced against the biases inherent in case series.
Systematic reviews of rapidly developing technologies face a number of challenges, particularly when the technology involved is a therapeutic procedure or device rather than a medical therapy [1]. In particular, the evidence base includes very few randomized controlled trials (RCTs). For surgical interventions, it has been estimated that RCTs comprise less than 10% of studies, with most of them being retrospective case series [2]. RCTs are important because they can provide reliable evidence of treatment effects. In contrast, case series rank lower in the hierarchy of evidence, because they are inherently susceptible to bias, and in the absence of a control group, causal relationships between interventions and outcomes cannot be definitely established.
Despite their limitations as evidence, case series for emerging interventions are frequently performed and published. In contrast to RCTs, such studies are relatively quick and easy to conduct and provide clinicians and patients with some information about the effects of a procedure. Indeed, by the time a technology has developed sufficiently to be evaluated in an RCT, evidence from uncontrolled studies may already have convinced clinicians of the effectiveness of the intervention, thus removing the degree of uncertainty (equipoise) that provides the ethical basis for randomizing patients to treatment [1]. Case series may also be useful sources of evidence on safety, because they often have relatively long follow-up and large sample size, and their inclusion criteria may be less strict than those of RCTs [3]. It follows that when conducting a systematic review or health technology assessment of a rapidly developing technology, there may be strong arguments for including case series [4]. However, the role of case series in reviews has been addressed in only a small number of studies.
Dalziel et al. and Stein et al. [4], [5] have assessed the use of case series in National Institute for Health and Clinical Excellence (NICE) health technology assessments (HTAs). They reported that 30% of HTAs included information from case series, but there was no consensus on which series to include, how to use them, or how to assess their quality. Comparison of case series and RCT data revealed a wider range of outcomes reported in case series, and estimates of comparative efficacy of different interventions obtained by comparing case series differed from those obtained from RCTs. However, these findings were based on only two examples, indicating a need for further research into this topic. Linde et al. [6] examined the contribution of nonrandomized studies (including case series) to a systematic review of acupuncture for treatment of chronic headache. In this case, the nonrandomized studies confirmed the findings of available RCTs, but provided little additional information of the sort case series might be expected to contribute, for example, data on long-term outcomes and adverse effects. In a systematic review of case series in pediatric surgery, Reeves et al. [7] reported little association between measures of reporting quality and outcomes reported in case series. They also noted that the outcomes reported in published series are likely to reflect the best achievable results rather than those normally achieved in clinical practice.
We recently performed a systematic review of the clinical effectiveness of radiofrequency catheter ablation (RFCA) for the treatment of atrial fibrillation (AF) as part of a health technology assessment of the procedure for both AF and typical atrial flutter [8]. RFCA is a relatively new intervention that involves the percutaneous insertion of catheters, which are guided by fluoroscopy to the heart. Small areas of tissue responsible for the propagation of abnormal electrical activity in the heart are selectively destroyed (ablated) using radiofrequency energy to restore normal sinus rhythm. Variations can relate to the type of catheter (standard or irrigated), type of mapping technique used to locate and guide the catheters within the heart (e.g., intracardiac electrogram, intracardiac echocardiography, electroanatomical mapping), and the ablation approach itself.
Over the last 5 years, the use of RFCA to treat AF has expanded rapidly. The procedure has changed over time because of changes in technology and increased knowledge of etiology. Although some RCTs are available to compare RFCA with other approaches to treating AF, the bulk of the evidence for evaluating the intervention came from case series.
The objective of this study was to investigate what the inclusion of case series data alongside data from RCTs contributed to the overall findings of the systematic review. In particular, we were interested in whether the case series supported the findings of the RCTs, whether they provided useful additional information, and whether the findings of the review of case series would be affected by using quality features as inclusion criteria.
The methods of the review have been described in more detail elsewhere [8]. Briefly, we searched a total of 25 electronic databases and Internet sources from inception to July 2006 for English language reports of clinical studies of any design of RFCA in patients with AF. Details of search strategies can be found in Appendix A (available on the journal's website at www.jclinepi.com) and in the full report [8]. Studies were assessed against the inclusion criteria by two reviewers acting
After screening over 4,800 abstracts and 482 full articles, we included eight controlled trials (six RCTs comparing RFCA with alternative treatment strategies and two nonrandomized trials) and 53 “case series”: 42 uncontrolled case series proper and 11 trials comparing ablation techniques, which we treated as case series (Fig. 1). Three RCTs comparing RFCA with continuing AAD therapy provided the main randomized evidence for evaluating the effectiveness of RFCA [14], [15], [16]. The value of
For the primary outcome of freedom from arrhythmia at 12 months, the findings of the case series were consistent with those from the three main controlled trials of RFCA in AF [8]. Compared with the controlled trials, the case series generally presented long-term follow-up data, but their reporting of mortality rates and other adverse outcomes was generally very limited. A similar finding was reported by Linde et al. [6] regarding acupuncture for chronic headache.
Most included case series were
In a typical systematic review of a rapidly developing technology, that is, RFCA for the treatment of AF, case series contributed substantially to the available evidence base, and their results complemented the limited evidence available from RCTs. Factors which may favor inclusion of case series in systematic reviews include:
The need to consider non-RCT evidence to ensure the clinical credibility of the review;
Potentially large or curative treatment effect of the intervention;
Suitable natural
We are grateful to Su Golder for carrying out the literature searches and to Dr. Chris Pepper and Dr. Derick Todd for acting as clinical advisors to the review.
This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment Programme (project number 06/13/01). The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the Department of Health.
The chosen studies have been fully checked for names of the first author, publication year, study design, population number of the study, demographic data (maternal age, gestational age, mode of injury, clinical features), patient’s management strategy (termination of birth timing; primary or secondary, neurosurgical treatment), maternal outcome, fetal outcome and follow-up. Quality of each study was done using case series quality assessment by Chambers, Rodgers, Woolacott’s methods of systematic reviews and meta-analysis, based on 2 previous systematic reviews that included case series [9–11], with published guidance [9,12]. There are 2,452 potentially relevant articles in total from Sciencedirect, PubMed, and other sources after the initial identification.
