Article Text
PDF
Abstract
Objective To examine how and when the results of COVID-19 clinical trials are disseminated.
Design Cross-sectional study.
Setting The COVID-19 clinical trial landscape.
Participants 285 registered interventional clinical trials for the treatment and prevention of COVID-19 completed by 30 June 2020.
Main outcome measures Overall reporting and reporting by dissemination route (ie, by journal article, preprint or results on a registry); time to reporting by dissemination route.
Results Following automated and manual searches of the COVID-19 literature, we located 41 trials (14%) with results spread across 47 individual results publications published by 15 August 2020. The most common dissemination route was preprints (n=25) followed by journal articles (n=18), and results on a registry (n=2). Of these, four trials were available as both a preprint and journal publication. The cumulative incidence of any reporting surpassed 20% at 119 days from completion. Sensitivity analyses using alternate dates and definitions of results did not appreciably change the reporting percentage. Expanding minimum follow-up time to 3 months increased the overall reporting percentage to 19%.
Conclusion COVID-19 trials completed during the first 6 months of the pandemic did not consistently yield rapid results in the literature or on clinical trial registries. Our findings suggest that the COVID-19 response may be seeing quicker results disclosure compared with non-emergency conditions. Issues with the reliability and timeliness of trial registration data may impact our estimates. Ensuring registry data are accurate should be a priority for the research community during a pandemic. Data collection is underway for the next phase of the DIssemination of REgistered COVID-19 Clinical Trials study expanding both our trial population and follow-up time.
- health policy
- medical ethics
- clinical trials
Data availability statement
Data are available in a public, open access repository. The DOI of the dataset is https://doi.org/10.5281/zenodo.4669937
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study examines a broad, global population of registered COVID-19 clinical trials for results dissemination patterns.
The protocol was preregistered, and all data and code are shared openly.
We used a comprehensive search strategy covering multiple databases and combining automated search methods with dual manual searches to discover potential results in the literature and on trial registries.
Our methods rely on registry data which varies considerably in its availability and accuracy; inaccurate or outdated data on registries compromise their value to the global research community.
We tested various alternate assumptions about trial completion in sensitivity analyses.
Introduction
Clinical trials drive evidence generation in medicine. Clinical trial registries support transparency, accountability and reducing bias in clinical research.1 The importance of these functions has been amplified during the COVID-19 pandemic. As global research efforts ramped up to address how best to treat and prevent infection with SARS-CoV-2, registrations became crucial to informing how the academic community discussed and anticipated the trajectory of clinical advancements. The rapid growth in COVID-19 research is notable: on 25 March 2020, the WHO International Clinical Trials Registry Platform (ICTRP) listed 668 registered COVID-19 studies; on 26 March 2021 that number had grown to 8798 studies. Similarly, over 3500 reviews of COVID-19 studies in humans are registered on the PROSPERO database as of March 2021.2
Expectations for sharing of trial results change during public health emergencies. The WHO noted in a 2015 consultation that ‘every researcher that engages in generation of information related to a public health emergency or acute public health event with the potential to progress to an emergency has the fundamental moral obligation to share preliminary results once they are adequately quality controlled for release’.3–5 Early on in the COVID-19 pandemic, various health publishers, funders and organisations signed a public statement reiterating their commitment to the WHO statement.6 Guideline 20 of the International ethical guidelines for health-related research involving humans reiterates the need to ‘generate knowledge quickly’ in emergency situations while maintaining ethical standards and public trust.7 Further development of principles for data sharing in public health emergencies has been published by the Global Research Collaboration for Infectious Disease Preparedness (GLOPID-R).8 9 However, none of these guidelines specify an exact time frame for reasonable dissemination timelines in emergencies. For instance, the WHO statement expressed that the usual expectation to report primary results within 12 months ‘should be greatly shortened’, and the GLOPID-R calls for timely sharing in a ‘logical, efficient and rapid manner’. Every public health emergency is unique and brings its own realities as to how, when and where results may become available; however, the need for rapid yet accurate dissemination remains.
Investigations of the 2009 H1N1 pandemic showed that just 29% of randomised trials examining H1N1 were reported in the literature within 18 months of trial completion, and just 12% of H1N1 vaccine trials published within a year of completion, rising to 30% within 2 years.10 11 During the H1N1, Ebola and Zika outbreaks, less than half (42%) of all trials met either of the WHO reporting standards for non-emergency situations (ie, 12 months for a registry, 24 months for journal publication).12 However, the global scale and scope of the COVID-19 pandemic, and its accompanying explosion in clinical research, distinguishes it from other modern disease outbreaks.13 The rapid rise in the usage of preprint servers for the clinical and biomedical sciences is also a distinct feature of the research response to the current pandemic.14–17
The DIssemination of REgistered COVID-19 Clinical Trials (DIRECCT) study was designed as a multiphase, living examination of results dissemination throughout the COVID-19 pandemic. Phase 1 of DIRECCT is part of the CEOsys (COVID-19 Evidence Ecosystem) project funded within the Network of University Medicine (Nationales Forschungsnetzwerk der Universitätsmedizin—NUM) by the Federal Ministry of Education and Research of Germany (Bundesministerium für Bildung und Forschung—BMBF). The objective of this project is to examine the results dissemination of registered clinical trials during the COVID-19 pandemic. Here we aimed to explore when and where detailed trial results are being made publicly available during the first six months of the pandemic. Understanding how and when results are shared in response to international demands for rapid, accurate data can help inform how the scientific community plans for and approaches future public health crises.
Methods
In this first phase of the DIRECCT study, we examined trials completed during the first 6 months of the pandemic (ie, through 30 June 2020). The full protocol for this study, with additional methods details, was preregistered on the Open Science Framework (https://osf.io/26yuj/).
Trial population
The WHO ICTRP maintains a list of registered COVID-19 studies across 18 global registries (table 1). This curated COVID-trial database was downloaded on 1 July 2020 (last updated 29 June 2020), and data on trial completion from all 18 registries were collected separately as this is not available in the ICTRP dataset. Registry data were retrieved via custom web scraping tools or by manual extraction during the first week of July 2020. Trials data taken from the ICTRP were processed via automated code to clean and standardise certain data elements across various registries (eg, phase, study type) and combined with data scraped from registries. Known cross-registrations, taken from the COVID-19 TrialsTracker project (run by NJD), were collapsed to single trial record, preferring a ClinicalTrials.gov entry when available, unless another registration is deemed more complete by the study team, per protocol.
ICTRP data provider registries
Inclusion/exclusion criteria
Inclusion and exclusion criteria for trials were assessed two times. First using automated methods on data fields extracted directly from the ICTRP and registries between 30 June 2020 and 5 July 2020, and then via a manual assessment of each trial passing automated screening. We note two additional post hoc exclusion criteria that arose from coding reconciliation discussions; as we are interested in trials on the direct treatment or prevention of acute infection with COVID-19, we excluded trials focused exclusively on treating side-effects, for example, reducing pain or mental distress experienced during or because of COVID-19, or those on post-COVID-19 experiences, for example, rehabilitation.
Inclusion criteria
Trial assessed an intervention for treatment or prevention of COVID-19 infection and subsequent acute disease.
Trial included a completion date, or primary completion date, on or before 30 June 2020.
Exclusion criteria
Registration was found, at any time, to indicate that the trial was withdrawn before enrollment and therefore never occurred.
Registration prior to 1 January 2020.
Trial exclusively on symptomatic treatment of COVID-19 disease side-effects only (post hoc).
Trial on rehabilitation after acute disease (post hoc).
Search strategy
Automated searches
On 30 June 2020, we searched PubMed using a PRESS Peer Reviewed search strategy for COVID-19 trial publications from the COVID-evidence project18 and downloaded the XML records for all results. Next, we downloaded the CORD-19 database which includes full-text and metadata for a comprehensive collection of open access coronavirus-related literature, including the PubMed Commons (PMC).19 20 We limited both collections to only those publications first available on or after 1 January 2020. We further limited our sample to publications that either (1) matched a regular expression pattern for an ICTRP-approved primary registry ID, prefix and/or name in either the abstract, metadata or full text (full-text available for the CORD-19/PMC sample only), or (2) was designated as a clinical trial ‘publication type’ in PubMed. Following de-duplication, we manually screened all hits to determine whether they represented bona fide primary trial results that matched the registered trial characteristics, like trial name and ID, investigators, treatment, enrollment and dates All hits were screened by two reviewers. All included results had to achieve consensus agreement between reviewers for inclusion as trial results in addition to reasonably matching registered characteristics. Issues not resolved by reviewer pairs were referred to the full study team for a final determination as needed.
Manual searches
Following automated exclusion of non-interventional, pre-2020 and non-completed clinical trials, we manually reviewed all remaining clinical trials to assess their inclusion status. If a trial was included, we adapted a search strategy from Wieschowski and colleagues21 and searched all registry entries along with PubMed, Europe PMC, Google Scholar and Google in a stepwise fashion for results. In contrast, Wieschowski and colleagues searched Web of Science and not Europe PMC and Google; we removed Web of Science so that all databases used for searches were openly accessible and added Europe PMC and Google for better coverage of preprint publications.22 We searched all databases using, first, the trial ID(s) and, second, keywords derived from the trial registration. Per protocol, each searcher built keyword searches using the title/abbreviation of the trial, investigator names and affiliations and the intervention/population under study; searchers also had discretion to search additional relevant keywords from the registry as appropriate. Rather than using repeated searches with fixed keyworks, as in Wieschowski et al, we felt this approach would maximise coverage of our searches. Searches were further filtered by date ranges as necessary, and the first 30 results of a given database were checked for relevant results. Searches took place between 21 October 2020 and 18 January 2021. If a full-results preprint and journal publication were located for a given record, no additional searches beyond a review of the registry entry were performed. All searches were performed by at least two reviewers. For each manual search, we recorded any results located, the date of publication and whether the publication contained a ‘last patient, last visit’ date for the primary outcome. As with automated searches, publications were matched to registry entries on key study criteria and discrepancies were resolved by consensus between either the two reviewers or the full study team. A methods flowchart is available as online supplemental figure 1.
Supplemental material
Outcomes
Outcome definitions
When a results publication was located, we recorded it as either a journal publication, preprint publication, registry result or other type of result. Other results types (eg, secondary analyses, conference abstracts, grey literature) were not systematically searched for but recorded if incidentally located for completeness of our overall dataset. These were not included in any outcome assessments. In addition, we recorded whether the publication included the complete results of the primary outcome(s) for all participants, or if the results were interim results without complete follow-up. The primary outcome was assessed as described in the publication, and not compared with the registered primary outcome(s). Our main results are based on publication of complete, non-interim results.
Outcome reporting
Per protocol, our outcome assessments are based on registry data as it stood on 30 June 2020. Subsequent updates to the completion date are not reflected in our main analysis. The only exception is if a results’ publication clearly included a specific ‘last patient, last visit’ date that contradicted the registered completion date. If this occurred, we used the published completion date in all analyses.
For all trials included in our per-protocol analysis, we report overall results availability in any format by 15 August 2020 (6 weeks after our cut-off) and each specific dissemination route. We also report these results broken down by registry and by trial country. Additionally, we fit cumulative incidence curves using the Kaplan-Meier method for time to any results availability and time to journal publication censoring follow-up on 15 August 2020. Cumulative incidence for time to preprint publication was fit using the Aalen-Johansen method with journal publication as a competing risk and ties broken by nominal offsets.23 For trials with multiple publications, we selected the earliest publication date for the dissemination route(s) relevant to each analysis. We separately calculated the median and IQR for time to publication among only those trials with a result.
Sensitivity analyses
Following data collection, we assessed the robustness of our per-protocol analyses to changes in completion date and results definitions. We assessed how the overall reporting rate changes when we: used completion dates extracted from our later manual searches rather than from 30 June 2020; used full completion dates rather than primary completion dates when available; and expanded our definition of results to include interim results. As our searches did not start until October 2020, we can also incidentally report results availability beyond our 6-week minimum follow-up cut-off.
Protocol deviations
In addition to the post hoc sensitivity analyses described above, we report the following protocol deviations. We decided to delay manual searches in order to provide all completed trials at least six full weeks to report (ie, by 15 August 2020). This exact cut-off was not specified in our protocol. For our first round of searches, we began searches 15 weeks following our cut-off due to extensive development and piloting of our search and data extraction methods. We plan to maintain this 6-week search buffer as a minimum for results searches moving forward.
We also noted that we would continuously check inter-rater reliability for each reviewer pair. This was difficult and impractical to implement into our eventual workflow for assessing results availability. Therefore, we developed an alternate system in which each reviewer-pair would attempt to reconcile any differences on study inclusion or results availability and categorisation through consensus. If consensus could not be reached, the issue was referred to the larger study team for discussion, a final consensus decision, and if necessary a specific rule addressing similar situations moving forward. We created two post hoc exclusion criteria based on these adjudications as detailed above. For future data extraction tasks (eg, determining directionality of results, extracting intervention information), we may reintroduce inter-rater reliability calculations as necessary.
Software, code and data
ICTRP and registry data collection, management and parts of the analysis were performed in Python V.3.8.1 (Python Software Foundation, Wilmington, Delaware USA). Additional data preparation and analysis was performed in R V.4.0.2 (R Foundation for Statistical Computing, Vienna, Austria). All code and data for this project are available on GitHub and Zenodo.24–26 Manual data extraction was conducted via Qualtrics forms,27 and reviewer reconciliation via Numbat Systematic Review Manager.28
Patient and public involvement
Patients and the public were not involved in developing this study.
Results
As of 30 June 2020, the ICTRP COVID-19 database contained information on 3844 study registrations. Following all automated and manual exclusions, our final analysis dataset included 285 completed clinical trials. Details of all exclusions are available in figure 1. Brief descriptive details of the 285 trials are included in table 2.
Flow chart for trial inclusion. ICTRP, International Clinical Trials Registry Platform.
Characteristics of included trials
Per-protocol analysis
Among the 285 trials registered as completed on 30 June 2020 or earlier, we located 41 trials (14%) with results available by 15 August 2020 spread across 47 individual results publications. Minimum follow-up for unreported trials was 46 days (ie, six full weeks from our cut-off) and the longest trial follow-up time from completion was 196 days. Details on the screening of included results are included in online supplemental file 1. The breakdown of results by dissemination route is detailed in figure 2. Figure 3 shows the reporting of results by registry.
COVID-19 clinical trial results by dissemination route.
Result reporting rate per clinical trial registry.* *The count of registrations is inclusive of known cross-registrations for included trials, therefore the denominators will sum to >285. ANZCTR, Australian New Zealand Clinical Trials Registry; ChiCTR, Chinese Clinical Trial Registry; CRiS, Clinical Research Information Service (South Korea); CTRI, Clinical Trials Registry—India; EudraCT, European Union Drug Regulation Authorities Clinical Trials Database; IRCT, Iranian Registry of Clinical Trials; JapicCTR, Japan Pharmaceutical Information Center Clinical Trial Registry; PACTR, Pan African Clinical Trial Registry; RPCEC, Cuban Public Registry of Clinical Trials; SLCTR, Sri Lanka Clinical Trials Registry; UMIN-CTR, University Hospital, Medical Information Network Clinical Trial Registry.
Figure 4A–C shows cumulative incidence plots for time-to-publication and 95% CIs. Trials with a publication date prior to the available completion date (across all results, n=8 of 41 trials, 19.5%) were considered reported at time 0 (except for the Aalen-Johanssen plot, which used nominal offsets to break ties at 0). If a full results publication included a different completion date (ie, last patient, last visit), this replaced the registered completion date in the dataset (n=4 of 41 trials, 9.8%). We show the cumulative incidence of (a) first publication across any dissemination route, (b) earliest journal publication and (c) earliest preprint publication. The medians for all cumulative incidence plots were undefined as no curve crossed 50%. The cumulative incidence of any results’ availability, accounting for censorship, surpassed 20% at 119 days from completion.
(A–C) Time to results across dissemination routes.
As only two trials reported results to a registry, we did not generate a plot for this route. Summary results for these two trials were published on Chinese Clinical Trial Registry 3 and 20 days following completion. For the four trials that had both a preprint and a journal publication, the median time from preprint publication to journal publication was 24.5 days (IQR: 14.2–36.2). For the 21 trials that had only a preprint, preprints had been published a median of 90 days (IQR: 76–136) without a matched publication by completion of follow-up.
Trials with only a preprint (n=21) published the preprint with a median of 32 days of completion (IQR: 10–53), whereas trials with both a preprint and a journal article (n=4) published the preprint with a median of 31 days (IQR: 19.5–31); trials with only a journal article (n=14) published the article with a median of 45.5 days after completion (IQR: 0.8–83.5), whereas trials with both a preprint and a journal article published the article with a median of 46 days after completion (IQR: 24.2–67.2). Due to the small number of trials with both preprint and journal publications, we did not undertake any statistical comparisons.
Sensitivity analyses
Each sensitivity analysis changed only a single aspect of the main analysis; the changes were not cumulatively applied. First, we recorded a new completion date for 48 trials during manual searches, 33 of which moved the completion date post-June 2020. Using this updated trial population (n=252) shows a reporting percentage of 16% with a minimum of 6 weeks of follow-up. Second, restricting our original sample to only those trials that reached full completion by 30 June 2020 showed 38 of 212 (18%) trials reported with a minimum of 6 weeks of follow-up. Third, adding interim results to our per-protocol findings yielded only one additional results publication for a reporting percentage of 15%.
Lastly, extending our 6-week minimum follow-up window to 3 months (ie, counting any results reported by 1 October 2020), aligned with when the searches for this phase of our project actually began, added an additional 17 results for a reporting rate of 54/285 (19%). Figure 5 includes a cumulative incidence plot for any results with expanded follow-up time.
Time to results dissemination from registered completion date—extended follow-up.
Discussion
Summary of findings
We identified 285 registered trials for the treatment or prevention of COVID-19 completed during the first 6 months of the pandemic. Of these trials, 14% had a result available in either a preprint, journal article, or posted to a clinical trials registry after a minimum follow-up time of 6 weeks. Sensitivity analyses using alternate dates and definitions of results for assessments did not appreciably change the reporting percentage. Extending the minimum follow-up time to 3 months yielded 19% of registered trials with results. Preprints have played an important role in results dissemination with the most results of any route (n=25). These preprint results do not appear to be rapidly making it into the peer reviewed literature, as just four preprints in our sample had a matched journal publication. There was low use (5%) of clinical trial registries for results dissemination.
Strengths and limitations
This work had a number of strengths. We used a broad definition of interventional clinical trials, beyond just randomised controlled trials, and included COVID-19 trials from the full ICTRP database, including trials from 18 registries, in our potential population. Searches for publications were performed using multiple strategies and databases, with dual-coded manual verification throughout. Our findings were also robust to a number of alternate assumptions for judging completion and results availability.
This work also has limitations. Missing, incomplete, inaccurate or outdated data all likely impacted our sample to varying degrees that are difficult to quantify. Deviations from best practices for discoverability (eg, inclusion of registration IDs in abstracts and metadata), publications in non-indexed journals or preprint servers and articles not available in English language may have made some results difficult to locate despite our extensive search strategy. Lastly, our methods may under-represent some aspects of how results have been shared during the pandemic. For instance, large adaptive trials like the Randomised Evaluation of COVID-19 Therapy (RECOVERY) trial or master protocol trials like Solidarity may contain numerous distinct research studies under one broad registration, produce numerous publications and will only be represented as a single datapoint in our analysis that remains ongoing until the end of follow-up for the final arm.29 30
Research in context
Registered and reported COVID-19 trials have been well-described in the literature.13 31–35 Our results add to these examinations of the design, focus and outcomes of the COVID-19 research response. We are aware of one similar analysis to date that attempted to quantify the dissemination of registered COVID-19 clinical trials. Janiaud and colleagues searched a single database for results of all registered COVID-19 Randomised controlled trials from the first 100 days of 2020 and conducted outreach to trial contacts in October 2020. They located results for 53 of the 516 trials (10.3%) but did not limit their analysis to only registered completed trials. Of the 516 trials in their sample, 155 (30%) had not started or were discontinued, per either the registry or trial team outreach.36
Reporting for COVID-19 trials may be accelerated compared with other trial populations even if the overall reporting percentage remains very low. A meta-analysis from 2014 showed a pooled reporting percent of registered trials across studies of 54.2% with a minimum of 24 months of follow-up but with substantial heterogeneity. Reporting percentages in the individual studies, often examining trials in a single specialty, ranged from 23% to 76%. Five studies with time to publication data showed a 30% pooled probability of reporting at 24 months of follow-up.37 Subsequent large examinations of reporting registered trials from academic medical centres in the USA and Germany showed 35.9% and 39% of results published in the literature within 2 years of completion, and overall reporting did not surpass 20% in either study until well after 6 months of follow-up.21 38 Low reporting rates have also been seen in previous pandemics; vaccine trials during the H1N1 pandemic showed <20% probability of reporting within 5 months of completion.11 Jones and colleagues examined the extent and timeliness of reporting during the H1N1, Zika and Ebola outbreaks. As in our findings, registries were less commonly used to disseminate results compared with journal publications (47% to 68% available at 24 months). Overall, there was a median delay from completion of 42 months (IQR: 16–76) for results posting to registries and 21 months (IQR: 9–34) for journal publication. H1N1 trials were substantially slower to publish results compared with Ebola and Zika trials; however, Zika trials showed the lowest overall reporting percentage across both routes. No negative findings were reported among 44 non-vaccine drug or biologic trials.12
Preprints grew modestly during prior outbreaks;14 however, their rise has been substantial since the COVID-19 pandemic began, including for trials.16 39 Preprints represented the most popular dissemination route, and often the only route, in which results were available in our sample. The low conversion of preprints to journal articles is consistent with other research on preprints during COVID-19.40 41 The time to journal publication for preprinted trials requires additional follow-up and consideration alongside emerging evidence on the relationship between preprints and the peer-reviewed COVID-19 academic literature.17 42
Implications for policy and practice
The deluge of trial registrations also reflects the fragmentation of the global COVID-19 response. As others have noted, this rush to register research without coordination or consideration of the work of others may compromise progress towards answering crucial questions.43 A disparate research environment may lead to considerable research waste as unnecessary duplication, disjointed outcomes and competition for participants slows evidence generation.44–47 It is essential that we understand the COVID-19 research response and how and why it evolved as it did.
A complete assessment of COVID-19 trial results dissemination would require accurate registry data on both trial completion and the overall status of the trial. The availability of completion dates in registries ranged from never (eg, Brazilian Clinical Trials Registry), to mixed (eg, Australian New Zealand Clinical Trials Registry), to always (eg, ClinicalTrials.gov). Other registries, like the EU Clinical Trials Register, make completion dates available only retrospectively. Despite being a Consolidated Standards of Reporting Trials (CONSORT) checklist reporting element, explicit dates for trial completion are rarely included in clinical trial publications, which would aid checks for accuracy and provide context for results.48 Issues with missing, incomplete and outdated data on registries have been detailed in prior investigations.49–51 Registries were also rarely used as a dissemination route for clinical trial results. The US National Institutes of Health have since called for the rapid publication of COVID-19 study results on ClinicalTrials.gov, the largest global registry, and instituted expedited reviews of results submissions moving forward.52 Our future work will examine whether this has led to more results availability on registries.
It is difficult to conclude from these preliminary results whether our findings show true potential for publication bias from unpublished results, poor management of trial registrations or some combination of both. The stakes of a global pandemic only amplify the importance of minimising reporting issues that may impact evidence synthesis, guideline development and ultimately clinical practice. Failure to update registry entries compromises an important tool for transparency and accountability in the COVID-19 research response. Searching for and anticipating results that may simply never come and should not be expected hampers crucial efforts to collect and examine the evidence around COVID-19.
Future research
Data collection for the remainder of the DIRECCT project is currently underway, extending our population and follow-up into 2021. The continued examination of publication rates over time will help place the COVID-19 results reporting data in further context and inform expectations for reasonable dissemination timelines in public health emergencies. We plan to expand our data collection to include outreach to authors and study teams, as well as to examine factors associated with the timely reporting of COVID-19 clinical trials. Future studies may also wish to examine additional forms of dissemination, such as press releases or other grey literature findings.
We hope that our openly available dataset of results matched to registrations will be useful to other groups investigating the COVID-19 research response. Our data form one part of a robust landscape of curation, examination and synthesis of COVID-19 research and literature, through platforms like COVID-NMA, Cochrane, COVID-evidence and Epistimonikos.18 53–55 These efforts allow for rapid evidence synthesis and the ability to efficiently examine pandemic research output in order to inform strategies for improved management and coordination of the global scientific community both during global health emergencies and otherwise.
Conclusion
Many of the trials registered, and apparently completed, during the first 6 months of the pandemic failed to yield rapid results in the literature or on clinical trial registries. Nevertheless, our findings suggest that the COVID-19 response may be seeing quicker results disclosure compared with non-emergency conditions. Issues with the accuracy and timely updates of trial registration data are likely to obfuscate a more precise estimate of non-publication, and time-to-publication, of clinical trials. Maintaining accuracy in clinical trial registrations should be a priority, especially during global public health emergencies, when collaboration and research prioritisation is key to the efficient advancement of knowledge. Among registered trials that did report results, preprints were the most common dissemination route, however, very few had converted to a full journal publication within the timeframe of this study. Results were rarely made available on clinical trial registries. Better, more specific guidance, aligned to the real-world learnings about how, when and where results from clinical trials should be expected during pandemic situations, may be warranted.
Data availability statement
Data are available in a public, open access repository. The DOI of the dataset is https://doi.org/10.5281/zenodo.4669937
Ethics statements
Patient consent for publication
Ethics approval
This study was exempted from ethics approval at the University of Oxford as it solely used non-disclosive, publicly available data.
Acknowledgments
We would like to thank Lars Hemkens (University of Basel) and Ben Goldacre (University of Oxford) for their support and advice in developing this project; Benjamin Gregory Carlisle for extensively supporting our use of the Numbat Systematic Review Manager software and making adaptations for this project; and Kerstin Rubarth from the Charité Institut für Biometrie und Klinische Epidemiologie for her consultation on the protocol and statistical analysis plan. We also thank our colleagues who provided useful comments on our study protocol and/or manuscript: James Smith (University of Oxford), Florian Naudet (CHU Rennes), and members of the DataLab at the University of Oxford and QUEST Center for Transforming Biomedical Research at the Berlin Institute of Health at Charité Universitätsmedizin zu Berlin.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @maiameta, @PeterRolandG
MS-H, PG and NJD contributed equally.
Contributors All authors contributed to study concept and design. MS-H, NJD, PG, MP-J contributed to acquisition, analysis or interpretation of data. NJD contributed to drafting of manuscript. All authors contributed to critical revision of the manuscript for important intellectual content. MS-H, NJD contributed to data curation, software, statistical analysis. PG, DS contributed to administrative, technical or material support. DS, NJD contributed to study supervision. DS contributed to funding acquisition. NJD contributed as guarantor.
Funding The CEOsys project is funded under a scheme issued by the Network of University Medicine (Nationales Forschungsnetzwerk der Universitätsmedizin—NUM) by the Federal Ministry of Education and Research of Germany (Bundesministerium für Bildung und Forschung—BMBF). FKZ: 01KX2021. During the course of this project NJD was also employed on a grant from The Good Thinking Society to work on trials transparency research. The funders had no involvement in the study design, analysis or writing of the manuscript nor the decision to submit.
Competing interests All authors declare that they have no direct conflict of interest related to this work. Outside of the submitted work, NJD has received a doctoral studentship from the Naji Foundation, grant support from the Fetzer-Franklin Memorial Fund, and been employed on grants from the Laura and John Arnold Foundation and Good Thinking Society. MS-H is employed as a researcher under additional grants from the German Bundesministerium für Bildung und Forschung (BMBF). PG received grant funding from the BMBF and Mercator Foundation and consults for scite inc. MP-J received grant funding from the BMBF. DS serves as a member of the Sanofi Advisory Bioethics Committee and receives an honorarium for contributing to meetings.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.