Article Text
PDF
Abstract
Objectives To present a snapshot of experimental cardiovascular research with a focus on geographical and temporal patterns of early termination due to poor accrual.
Setting The Aggregate Analysis of ClinicalTrials.gov (AACT) database, reflecting ClinicalTrials.gov as of 27 March 2016.
Design The AACT database was searched for all cardiovascular clinical trials that started from January 2006 up to December 2015.
Results Thirteen thousand and seven hundred twenty-nine cardiovascular trials were identified. Of these, 8900 (65%) were classified as closed studies. Globally, 11% of closed trials were terminated. This proportion varied from 9.6% to 14% for trials recruiting from Europe and Americas, respectively, with a slightly decreasing trend (p=0.02) over the study period. The most common reason for trials failing to complete was poor accrual (41%). Intercontinental trials exhibited lower figures of poor accrual as the reason for their early stopping, as compared with trials recruiting in a single continent (28% vs 44%, p=0.002).
Conclusions Poor accrual significantly challenges the successful completion of cardiovascular clinical trials. Findings are suggestive of a positive effect of globalisation of cardiovascular clinical research on the achievement of enrolment goals within a reasonable time frame.
- Clinical trials
- EPIDEMIOLOGY
- STATISTICS & RESEARCH METHODS
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
To identify cardiovascular clinical trials terminated early because of poor accrual, this study relies on the most updated release of the Aggregate Analysis of ClinicalTrials.gov database and on automated and replicable text mining techniques.
By analysing early termination due to poor accrual by continent of recruitment over time, the study shows that poor accrual exhibits geographical heterogeneity with lower figures for intercontinental trials.
In interpreting the results it must be acknowledged that ClinicalTrials.gov is representative of registered trials recruiting from Europe and Americas. On the contrary, it cannot be ruled out that some selection bias occurred for trials recruiting from Asia and Oceania since they are increasingly registered in other regional registries.
Introduction
Clinical trials may terminate for a variety of reasons, some of which correctly envisaged in the study protocol1 2 and others unforeseen and attributable to failures in the trial conduct. Examples of appropriate reasons for terminating a trial prior to completion include unequivocal evidence of futility or harm.3 In this case early stopping may prevent additional patient exposure to ineffective or harmful treatments and limit further expenditure of resources on unsuccessful approaches. Conversely, premature termination due to poor accrual clearly reflects a failure of the trial process since trials completed with less than expected enrolment are usually delayed and unable to meet their intended objectives meaningfully.
Clinical trials that fail to recruit successfully or in a timely manner their target number of participants pose ethical, financial and statistical issues.4 They are a major concern in research areas where the acuity of the disease and the quickly evolving treatment paradigm render original research questions obsolete if not implemented in a timely manner. This partially explains why poor accrual and its consequences have been extensively investigated in some areas, mostly in oncology5–7 and less in others.
Unsuccessful recruitment has been recently acknowledged8 9 as the leading factor in cardiovascular trial failure, generating resource-consuming underpowered trials that provide only inconclusive data with no or little return on investments.
The past decade has seen a fast growth in cardiovascular research.10 Documenting impact of poor accrual on early termination of cardiovascular trials, also in terms of geographical and temporal trends, is important as it can provide insights to inform future trials.
The development of clinical trial registries, such as ClinicalTrials.gov, affords a remarkable opportunity to better understand the extent of early termination in cardiovascular trials. In the beginning, ClinicalTrials.gov was set to increase public awareness of clinical trials, and its systematic evaluation was hindered by a lack of access to the complete, annotated data. This registry now serves as a mandatory repository for information on most clinical studies run under US regulations, and registration with ClinicalTrials.gov is mandatory for publishing study results in many peer-reviewed journals. In addition, the database for Aggregate Analysis of ClinicalTrials.gov (AACT),11 a high-quality, searchable database of the information contained in ClinicalTrials.gov, is now publicly available.
By leveraging information available through the AACT database, reflecting ClinicalTrials.gov as of 27 March 2016, we present a snapshot of the last 10 years of cardiovascular research, with a focus on geographical and temporal patterns of early termination due to poor accrual.
Materials and methods
The AACT database, reflecting ClinicalTrials.gov as of 27 March 2016, is the data source. A comprehensive dictionary for the data elements in AACT is available online.12
The Medical Subject Headings (MeSH) code field in the AACT database was queried for at least one of the following MeSH codes: A07 cardiovascular system; C14 cardiovascular diseases; D27.505.954.411 cardiovascular agents (including antiarrhythmia/antihypertensive/cardiotonic/fibrinolytic/natriuretic/vasoconstrictor/vasodilator agents, calcium/potassium/sodium channel blockers, cardioplegic/sclerosing solutions and nitric oxide donors); E01.370.370 diagnostic techniques, cardiovascular (including angiography, angioscopy, blood circulation time, blood flow velocity, blood pressure/volume determination, capillary fragility, carotid intima-media thickness, heart function tests, laser Doppler flowmetry, microscopic angioscopy, plethysmography, pulse wave analysis and tilt-table test); E04.100 cardiovascular surgical procedures (E04.100.376 cardiac surgical procedures, E04.100.700 reperfusion and E04.100.814 vascular surgical procedures); G09.330 cardiovascular physiological phenomena (including ventricular function, myocardial contraction, blood circulation, neovascularisation, haemodynamics and cardiovascular deconditioning); and H02.403.429.163 cardiology, with the aim to identify all studies dealing with a cardiovascular condition. The final sample was limited to all such studies, started in the past decade — from January 2006 to December 2015 — and classified as ‘Interventional’, therein referred to as cardiovascular trials.
Study characteristics that have been considered are year of start date, phase (available only for drug and biologics trials), type of intervention, number of arms, primary purpose, endpoint classification, intervention model, masking, number and location of the facilities involved, and number of patients enrolled.
The rate of studies that have stopped recruiting or enrolling participants early and will not start again, defined as ‘terminated’ studies, was estimated on studies that are no longer recruiting participants because they have enough participants already, have ended or have been stopped for some reason, and referred to as ‘closed’ studies. The latter definition embraces also withdrawn studies, intended as studies closed with no patients enrolled.
Termination was used as the dichotomous outcome variable of a conditional inference tree built to detect associations with the study characteristics listed above.
Study duration was calculated as the difference between start date and completion date, only for closed studies and ‘actual’ completion date type.
To establish the reason for stopping, text mining was applied to the narrative field of reason for study termination. The following text preprocessing procedures were applied in the following order: conversion to lower case, removing numbers, removing punctuation, removing stop-words, stemming words, removing stemmed uninformative terms (‘studi’, ‘clinical’, ‘trial’, ‘patient’, ‘subject’, ‘approach’, ‘termin’, ‘stop’), treating the stemmed terms ‘accrual’ and ‘recruit’ as synonyms of ‘enrol’, stripping white space and building a sequence of two adjacent words from the text (bigrams).
A randomly selected sample of about 10% of terminated trials was extracted to assess the accuracy of automated text mining in identifying poor accrual as compared with manual review (gold standard).
AACT was downloaded in pipe delimited text format and managed in R V.3.3.0 software. The location of each facility, expressed as country in the AACT database, was ascribed to a region and to one of the five continents through R packages ‘countrycode’ and ‘rworldmap’. Text mining was performed with ‘tm’ R package. Tree-based models were fit with ‘party’ R package.
Results
In a 10-year time span, from 2006 to 2015, 13 729 cardiovascular trials were identified.
As shown in figure 1, cardiovascular trials were prevalently run in Northern America (41%), followed by Eastern and Western Europe (25%) and Eastern Asia (8%). The regions that together contributed to less than 0.5% were Middle, Eastern and Western Africa and Central Asia.
Geographical distribution of the facilities involved in cardiovascular trials, 2006–2015.
Single-centre studies accounted for the majority (60%) of cardiovascular trials. Among intercontinental trials (n=1089), the most likely collaborations were bilateral between Americas and Europe (32%), trilateral between Americas, Asia and Europe (14%), quadrilateral between Americas, Asia, Europe and Oceania (11%), and worldwide (8%).
Overall, the most common interventional model was parallel design (60%). Of studies reporting number of arms (n=13 419), 27% were single-armed and 57% had two arms. Multiarm studies (n=9735) were typically randomised (89%) and blinded (38% double-blinded and 20% single-blinded). Phase II and III trials together accounted for 4500 studies and 5052 recorded the phase as ‘not applicable’. The most frequently observed intervention was drug (n=6334), followed by device (n=2736) and procedure (n=1405).
Eight thousand and nine hundred cardiovascular trials (65%) were classified as closed studies. Of these, 11% were terminated. The reason for termination was missing for 141 studies.
According to the classification tree shown in figure 2, termination was significantly associated with facilities’ location (p<0.001). In addition, the type of intervention had an effect on the rate of early termination (lowest for behavioural and dietary supplement interventions as compared with all other types, p<0.001) for trials run in Americas and multinational trials run in Americas and Europe or in Americas, Asia and Europe. Among trials run in Europe, parallel trials were more likely to terminate early than trials with other intervention models (7.3% vs 11.7%, p=0.012).
Conditional inference tree predicting early termination. Retained variables: facility location (Americas (AM), Americas Europe (AM-EU), Americas Asia Europe (AM-ASIA-EU), Asia, Europe (EU), Other), year of start date, intervention type (behavioural (behav), biological (bio), device, dietary supplement (diet), drug, genetic (gene), procedure (proc), radiation (rad), other), and intervention model (crossover (Cr), factorial (Fac), parallel (paral), single group (sing)).
Low figures of early termination for trials run in Asia and for trials recently started should be interpreted with caution since they may suffer from a selection bias.
The single stemmed term ‘enrol’ occurred in 404 reasons for study termination and it was most associated with the adjectives ‘poor’, ‘low’ and ‘slow’, making poor accrual the most common reason for trials failing to complete. The second most common substantive was ‘fund’, with 85 occurrences and it was associated with the term ‘lack’, followed by ‘sponsor’, with 70 occurrences, and associated with the term ‘decis’. Table 1 reports the most frequently used bigrams in the reason for termination field. A network graph in figure 3 shows the most frequent relations (ie, correlation >0.15) between the (stemmed) terms ‘enrol’, ‘fund’, ‘sponsor’, ‘safety’ and ‘interim’, and the other words in the corpus of reason for termination text data.
Stemmed bigrams occurring in the reason for stopping field at least 10 times: terminated cardiovascular trials, 2006–2015
Characteristics of closed cardiovascular interventional trials – 2005–2015: proportion of terminated trials (% terminated) and proportion of trials terminated because of lack of accrual (% poor) by study characteristics
Network graph showing the most frequent relations (ie, correlation >0.15) between the (stemmed) terms ‘enrol’, ‘fund’, ‘sponsor’, ‘safety’ and ‘interim’, and the other words in the corpus of reason for termination text data. Solid lines identify associations with a correlation >0.15 and dotted lines identify tautological associations.
On a random sample of 85 terminated trials, where the reason for termination was manually reviewed by one of the authors (IB), the prevalent reason was poor accrual (47%). Stopping occurred for lack of funding and sponsor decision in 8% and 6% of the sampled trials, respectively. Appropriate reasons for termination based on internal or external evidence of futility or lack of safety accounted for 26% of all reasons. The sensitivity and specificity of the stemmed term ‘enrol’ in identifying poor accrual were 97.5% and 100%, respectively.
Terminated trials actually involved a total of 114 609 enrolled subjects (plus 44 513 scheduled) and more than 10 500 facilities worldwide. Eighteen thousand and eight hundred eight patients and 2319 facilities actually took part in trials terminated because of poor accrual.
The patterns of premature termination shown in figure 4 varied from 9.6% to 14% (p<0.001) among the two most contributing continents, Europe and Americas, and were slightly decreasing over the study period (p=0.02).
Proportion of early terminations by continents of recruitment over time. Cardiovascular trials, 2006–2015.
Intercontinental trials exhibited comparable figures of termination and lower figures of unsuccessful accrual as the reason for their early stopping, as compared with intracontinental trials (13% vs 11%, p=0.24, termination, of whom 28% vs 44%, p=0.002, were due to poor accrual, respectively). Figure 5 shows the time trend of poor accrual for intercontinental trials (p value for trend=0.02, p value for the difference=0.001).
Proportion of early terminations due to poor accrual for trials recruiting from a single continent or from different continents over time. Cardiovascular trials, 2006–2015.
A median actual duration of 24 months was observed both on closed trials and on the subset of terminated trials. Only a slight difference emerged in the third quartile, ranging from 38 to 39 months for closed and terminate trials, respectively. For this reason, time trends were truncated at the last biennium under study.
Poor accrual accounted for more than 40% of reasons for early stopping in trials recruiting from Americas and Europe. This proportion was by far inferior for all other continents of recruitment, although caution must be taken in interpreting these figures since they rely on small absolute numbers (table 2).
Conclusion
Clinical trials are a key step in advancing new therapeutic concepts for the management of chronic cardiovascular diseases from the research setting to the clinical practice.13 Despite this achievement, the successful recruitment of the targeted number of participants in a given time frame remains a significant challenge to clinical trials.14 Delayed and abandoned trials represent a waste of scarce human and economic resources, which may slow the advancement of medical progress or reduce its timely impact on patient health and well-being.
Therefore, a thorough understanding of the nature of trial enrolment patterns, from an overview on aggregate data through to the working of individual trials, is of paramount importance.
This study contributes to a growing body of research on early termination due to poor accrual in cardiovascular trials,2 3 9 providing further insights into geographical and temporal patterns.
Among study findings, about 11% of all closed trials were terminated, which is consistent with other cross-sectional studies using the ClinicalTrials.gov registry.9 15 When restricted to phase II and phase III trials together, the proportion of early termination (14%) was slightly inferior to that reported on a recent study4 based on the National Library of Medicine clinical trial registry (19%).
Consistent with existing literature,8 9 the most common reason for cardiovascular trials failing to complete was poor accrual (41%). Common logistical reasons were related to lack of funds and sponsor decision. Since reason for termination was automatically characterised by word patterns through text mining, we may just speculate that the concepts underlying ‘safety concerns’, ‘safety issue’ and ‘interim analysis’ pertain to appropriate termination, possibly on data safety and monitoring board advice, because of excessive toxicity or in accord with early stopping rules.
Almost all trials are dependent on the willingness of patients and professionals to give their time and effort to participate, and we estimated that the human dimension of poor accrual involved more than 8000 patients and the staff of 2000 sites.
The overall patterns of early termination and those concerned with poor accrual exhibited geographical variations. Cardiovascular trials recruiting in the two most contributing continents exhibited the highest and lowest proportion of early termination due to unsuccessful accrual in Americas and Europe, respectively, across all the study period.
Nevertheless, time trends should be interpreted with caution, since the decrease seen in the last biennium of the evaluation may be an artefact, reflecting both a delay in updating trial status in ClinicalTrials.gov and a selection of trials with a short duration and less likely to suffer from low accrual. However, comparisons between continents of recruitment per year appear to be appropriate and reliable.
It is interesting to notice that intercontinental trials, particularly those in bilateral collaboration with Americas and Europe, suffer more from early termination and less from termination due to lack of accrual, as compared with studies completely run in only one of the participating continents. Clearly, intercontinental trials are often resource-intensive, large-scale randomised controlled clinical trials, recruiting thousands of patients from large numbers of trial sites (ie, megatrials).
On the one hand, this probably allows to recognise the achievement of enrolment goals within a reasonable time frame as one of the numerous advantages of globalisation of cardiovascular clinical trials, often evolving into megatrials,16 but on the other hand it highlights that other organisational and financial issues challenge this process. Intercontinental trials represent a valuable attempt to harmonise the generation of clinical evidence across continents. Their implementation is particularly promising in device trials,17 where the comparison of procedural outcomes, using similar devices in different environments, makes it possible to disentangle the effects of practice patterns and procedural technique on clinical results.
Some study limitations must be acknowledged. First is the generalisability of study results. A recent investigation18 on trends in global clinical trial registration from 2005 to 2013, based on International Clinical Trials Registry Platform data, reports that trials conducted in Northern America and Latin America and Caribbean were almost exclusively registered in ClinicalTrials.gov. Also European trials were predominantly registered in ClinicalTrials.gov rather than in the EU Clinical Trials Register. Conversely, trials conducted in Oceania and Asia were increasingly registered in other regional registries such as the Australian New Zealand Clinical Trials Registry and the Japan Primary Registries Network, respectively.
As a consequence, study results concerning Europe and Americas can be regarded as representative of all registered trials recruiting from these continents. Moreover, differences in the extent of early termination and of poor accrual can be interpreted as genuine. On the contrary, it cannot be ruled out that some selection bias occurred for trials recruiting from Asia and Oceania. Therefore, the extent of early termination for trials run in these two continents may be underestimated.
Furthermore, the identification of trials terminated for poor accrual relied on an automated process with an estimated sensitivity of 97.5%; thus, the extent of poor accrual may be underestimated.
Poor accrual is a significant barrier to the successful and timely completion of clinical trials and to the advancement of medical knowledge. Although this issue may be addressed in a variety of ways, a global collaborative perspective on the planning and conduct of some cardiovascular clinical trials should be further encouraged.19
References
Footnotes
Contributors IB and PB designed the study and wrote the manuscript. CL and IB performed the statistical analysis. All authors contributed to results interpretation and approved the final manuscript.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data available.