Intended for healthcare professionals

  1. Research
  2. Prospective...
  3. Prospective registration and reporting of trial number in randomised clinical trials: global cross sectional study of the adoption of ICMJE and Declaration of Helsinki recommendations
CCBYNC Open access
Research

Prospective registration and reporting of trial number in randomised clinical trials: global cross sectional study of the adoption of ICMJE and Declaration of Helsinki recommendations

BMJ 2020; 369 doi: https://doi.org/10.1136/bmj.m982 (Published 14 April 2020) Cite this as: BMJ 2020;369:m982
  1. Mustafa Al-Durra, PhD student12,
  2. Robert P Nolan, associate professor, director, and adjunct professor345,
  3. Emily Seto, assistant professor and research scientist67,
  4. Joseph A Cafazzo, associate professor and executive director8910
  1. 1Centre for Global eHealth Innovation, Techna Institute, University Health Network, Toronto General Hospital, Toronto, ON, M5G 2C4 Canada
  2. 2Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, ON, Canada
  3. 3Psychiatry Department and Institute of Medical Sciences, University of Toronto, ON, Canada
  4. 4Cardiac eHealth and Behavioural Cardiology Research Unit, Peter Munk Cardiac Centre, University Health Network, Toronto, ON, Canada
  5. 5Department of Psychology, University of York, ON, Canada
  6. 6Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, ON, Canada
  7. 7Centre for Global eHealth Innovation, Techna Institute, University Health Network, ON, Canada
  8. 8Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, ON, Canada
  9. 9Centre for Global eHealth Innovation, Techna Institute, University Health Network, ON, Canada
  10. 10Institute of Biomaterials and Biomedical Engineering, University of Toronto, ON, Canada
  1. Correspondence to: M Al-Durra maldurra{at}ehealthinnovation.org
  • Accepted 27 February 2020

Abstract

Objectives To evaluate the compliance with prospective registration and inclusion of the trial registration number (TRN) in published randomised controlled trials (RCTs), and to analyse the rationale behind, and detect selective registration bias in, retrospective trial registration.

Design Cross sectional analysis.

Data sources PubMed, the 17 World Health Organization’s trial registries, University of Toronto library, International Committee of Medical Journal Editors (ICMJE) list of member journals, and the InCites Journal Citation Reports.

Study selection criteria RCTs registered in any WHO trial registry and published in any PubMed indexed journal in 2018.

Results This study included 10 500 manuscripts published in 2105 journals. Overall, 71.2% (7473/10500) reported the TRN and 41.7% (3013/7218) complied with prospective trial registration. The univariable and multivariable analyses reported significant relations (P<0.05) between reporting the TRN and the impact factor and ICMJE membership of the publishing journal. A significant relation (P<0.05) was also observed between prospective trial registration and the registry, region, condition, funding, trial size, interval between paper registration and submission dates, impact factor, and ICMJE membership of the publishing journal. A manuscript published in an ICMJE member journal was 5.8 times more likely to include the TRN (odds ratio 5.8, 95% confidence interval 4.0 to 8.2), and a published trial was 1.8 times more likely to be registered prospectively (1.8, 1.5 to 2.2) when published in an ICMJE member journal compared with other journals. This study detected a new form of bias, selective registration bias, with a higher proportion (85.2% (616/723)) of trials registered retrospectively within a year of submission for publication. Higher rates of retrospective registrations were observed within the first three to eight weeks after enrolment of study participants. Within the 286 RCTs registered retrospectively and published in an ICMJE member journal, only 2.8% (8/286) of the authors included a statement justifying the delayed registration. Reasons included lack of awareness, error of omission, and the registration process taking longer than anticipated.

Conclusions This study found a high compliance in reporting of the TRN for trial papers published in ICMJE member journals, but prospective trial registration was low.

Introduction

The selective or incomplete reporting of outcomes of randomised controlled trials (RCTs) is a major concern and leads to biased interpretation of clinical evidence—through overstating benefits or understating adversity in trial results.123456

In 1986, Simes explained how the implementation of a trial registry would mitigate bias in selective reporting of published trials.7 Trial registries would equip researchers with another source of trial information to verify if the reported outcomes in the published trials were indeed the a priori measures submitted in the trial registration records.18 As such, the prompt registration of clinical trials mitigates bias associated with modification of predefined trial outcome measures based on preliminary analysis of the trial results.910

Following Simes’s proposition, several national and international initiatives (box 1) were introduced to promote the registration of clinical trials.

Box 1 Key trial registration initiatives

  • 1986 Simes7

    • Trial registration to mitigate publication bias in clinical trials

  • 2004 International Committee of Medical Journal Editors (ICMJE)

    • Trial registration in a public trial registry before the enrolment date as a prerequisite for accepting trial publication in any ICMJE member journal11

  • 2005 World Health Organization

    • Standardise the trial registration process across multiple international trial registries, with emphasis on the trial registration number (TRN)12

  • 2008 Seventh revision of the Declaration of Helsinki

    • Emphasis on prospective trial registration before the enrolment date1314

  • 2015 International Clinical Trials Registry Platform

    • Established by WHO and includes 17 international trial registries. WHO introduced a statement signifying the importance of reporting TRN in trial publications1516

In several studies, compliance with inclusion of a TRN in trial publications was between 26.0% and 95.8%.217181920212223

Compliance with the prospective registration of RCTs was reported to be between 3.6% and 77.0% in a number of studies.21822232425262728293031323334353637383940 These studies were, however, limited in scope and included trials published in a specific group of journals, trials of a specific treatment or condition, or trials registered in a specific trial registry or a small group of trial registries. Because of these limitations, these studies provided a wide range of results that were neither conclusive nor generalisable.

We performed an empirical appraisal of the current state of compliance with the ICMJE recommendation 10 years after the adoption of the seventh revision of the Declaration of Helsinki. Across all the World Health Organization trial registries we investigated compliance with the inclusion of the TRN in published RCTs and compliance with the prospective registration of RCTs. We also sought to understand the rationale behind late registration of RCTs and validate our own hypothesis of a potential new form of bias, which we refer to as selective registration bias. This bias is defined by the tendency of investigators to selectively register their trials only when intending to publish the manuscript in a peer reviewed journal.

Methods

The study cohort comprised 10 500 RCTs published in 2018 across 2105 journals indexed in PubMed and registered across the WHO trial registries. We developed specialised software to download, screen, and extract the registration and publication information of published RCTs from the PubMed database and WHO trial registries.

Data sources

Our data were obtained from five different sources. Firstly, we searched PubMed for trial papers and downloaded abstracts and meta-data of RCTs that were published in 2018.41 Secondly, we utilised WHO’s International Clinical Trials Registry Platform (ICTRP), which includes 17 trial registries, to identify and download registration records of published trials.424344454647484950515253545556575859 Thirdly, we accessed the University of Toronto’s library resources to download the full text manuscripts of the included trial papers to help extract the TRN.60 Fourthly, we used the International Committee of Medical Journal Editors (ICMJE) website to identify ICMJE member journals and journals that follow the ICMJE’s recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals.6162 Lastly, we considered the InCites Journal Citation Reports 2017 to cross reference the impact factor of the publishing journal of our included trial papers.63 We accessed the InCites Journal Citation Reports 2017 through the University of Toronto’s online library resources.64

Tools for statistical analysis and data preparation

We used Pearson’s χ2 test when considering a single independent variable and binary logistic regression for multivariable analyses of our results. SPSS Statistics version 24 (IBM, NY) was used for statistical analyses. When the expected cell value was less than 5 we used Fisher’s exact test instead of Pearson’s χ2 test. Fisher’s exact test was performed in STAT Statistics version 14.2 (StataCorp, TX).

To aid in the data preparation process we also used other tools: Microsoft Excel (Microsoft Office 365 ProPlus Version 1906), Microsoft SQL Server 2017, Microsoft .NET 4.5/C#, Microsoft Visual Studio 2017, NuGet package iTextSharp.5.5.13, and PaperPile (Paperpile, MA).

Inclusion and exclusion criteria

To be included, publications had to be RCTs that were published in 2018 in all journals or databases indexed by PubMed. For our trial registration analysis, we considered all clinical trial registries that are included in the WHO Registry Network.

Publications were excluded if they concerned registered, but not randomised, controlled trials or were secondary analyses, editorials, letters, erratums, corrigendums, reviews, meta-analyses, and design papers.

Search terms and methodology

Search terms applied to the title field in PubMed were “Trial” OR “RCT” OR “Randomized” OR “Randomised.”

Our study provides a critical evaluation of the registration of RCTs. Although it is not a systemic review or meta-analysis, we followed the recommendations of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement in the identification, screening, and inclusion of the search results.65

Data extraction

Identification of trial registration number

From PubMed we downloaded the publication metadata for the trial papers in Extensible Markup Language (XML) format (see supplementary appendix A for a sample of the XML file export). We used three different steps to identify the TRN in the papers of published trials. Firstly, we obtained the TRN and trial registry name from two XML data elements: “AccessionNumber” and “DataBankName”, respectively.66 Secondly, when the TRN was not indicated in these two XML data elements, we searched for the TRN in the paper’s abstract provided in the “Abstract” XML data element. Lastly, and for the remaining papers where the TRN was still not identified, we downloaded the full text manuscripts as PDF files from the online library resources at the University of Toronto to complete our search. To help download the full text manuscripts as PDF files we used a commercially available reference management software, PaperPile. We utilised an existing open source programming library, NuGet package iTextSharp.5.5.13, to serialise and parse the full text content from the PDF files. We developed our own computer program, in Microsoft .NET/C# programming language and used advanced text pattern search techniques (also known as regular expressions) to identify potential matches of TRN. Supplementary appendix B provides further details about the search methodology to identify the TRN in published papers. Supplementary appendix C provides details on the text patterns of WHO trial registries.

Classification of publishing journals

The type of publishing journal was classified on the basis of matching the journal name provided in the PubMed metadata of the published trial papers with the information provided in the ICMJE website:

ICMJE member journals—if the publishing journal of a matched trial was one of the 12 ICMJE member journals.61

Journals that follow the ICMJE’s recommendations—if the publishing journal of a matched trial was not one of the ICMJE member journals, and if the journal was one of the 5061 journals of which the editors or publishers have explicitly indicated compliance with the ICMJE’s recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals.6267

None—if the publishing journal of a matched trial was neither enlisted as an ICMJE member journal nor as a journal that follows ICMJE recommendations.

The publishing journals were also classified into quarters based on their impact factors as reported by the 2017 InCites Journal Citation Report.63 The quarters are divided by the 25th, 50th, and 75th centile.

Trial registration information

Using the WHO search portal we downloaded the trial registration records of included trials with an identified TRN. The records were downloaded in XML format and the XML file was then transformed and imported into a SQL server database for further analysis.

From the respective trial registries we extracted the trial condition and funding information from the trial registration records. Supplementary appendix D shows the categorisation and grouping of the trial conditions and appendix E of the funding information.

Prospective trial registration

The ICMJE guidelines require clinical trials to be publicly and prospectively registered to be considered for publication within an ICMJE member journal—that is, investigators of clinical trials need to register their trials in one of the public trial registries recognised by WHO before enrolment of the first participant.6267 We evaluated the prospective registration of our included trials by comparing the trial registration and enrolment dates provided in the trial registration records. The trial registration was categorised to be prospective if the trial registration date was earlier than, or the same as, the trial enrolment date. Otherwise, we categorised the trial registration as retrospective.

Patient and public involvement

This research was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

Results

Supplementary appendix F shows the number of trials identified from using the search terms and phrases in the title field of the PubMed advanced search portal on 13 January 2019.

The titles of 11 857 trial papers identified during the preliminary search were screened to exclude editorials, erratums, and design papers. After a review of the journal publication dates in the XML export from PubMed, 19 trials were excluded because the journal publication date was not in 2018. Supplementary appendix G provides more details on the screening process for the journal publication dates. Overall, 575 papers were excluded because the TRN was not reported in the abstract or PubMed publication metadata and the full text manuscript could not be accessed through the University of Toronto online library resources. After screening, 10 500 trial papers were included (fig 1).

Fig 1
Fig 1

Screening and identification of trial registration number (TRN)

Initially, the TRN was identified in the PubMed publication metadata for 3779 papers and in the full text manuscript for 3790 papers. Supplementary appendix B provides further details on the search methodology to identify the TRN in published papers.

A total of 7390 unique TRNs were identified across 7569 published papers. It was possible to download 7257 trial registration records from the WHO search portal, and 24 directly from the respective trial registry when the trial registration record was not found in the WHO search portal (see supplementary appendix H). For the remaining 109 identified TRNs in published papers no match was found in trial registration records in either the WHO search portal or the respective trial registries.

To evaluate the rate and characteristics of prospective trial registration, trial registration records were only included if they provided trial registration and participant enrolment dates. These two data fields were required to determine if the trial was registered prospectively. Overall, 63 trial registration records with missing trial registration or enrolment dates were excluded.

Reporting of TRN

A valid TRN was identified for 71.2% (7473/10 500) of the included papers. Overall, 7281 unique and matching trial registration records that were registered in 16 different WHO trial registries from 127 countries were downloaded. Table 1 shows the relation between the reporting of a valid TRN in trial papers and the impact factor as well as the ICMJE membership of the publishing journals.

Table 1

Relation between characteristics of journal and reporting of trial registration number (TRN)

View this table:

The test results of Pearson’s χ2 test and the binary logistic regression test indicated significant relations (P<0.05) between the characteristics of publishing journals and the reporting of a valid TRN in the published papers. Supplementary appendix I provides further analysis of the reporting of the TRN within the group of ICMJE member journals. No significant relation was found between the reporting of the TRN in published trials and whether the trial paper was published in a specialty ICMJE journal. The result of a test for interaction between journal type and journal impact factor was also not significant (P=0.20).

Prospective registration of clinical trials

A total of 7218 unique trial registration records with valid registration dates and participant enrolment dates were identified and downloaded. Overall, 287 trials were reported in 612 papers—that is, when a trial is published in more than one paper. To extract the journal impact factor, ICMJE membership, submission, and publication dates only one paper needed to be considered for any given trial—the paper with the earlier publication date was chosen because it signifies the first publication of the registered trial.

As such, 228 trials were reported in 114 papers—that is, a small set of papers provided a report for two registered trials; 28 papers reported on two different trials and 86 papers reported on a single trial that was registered in two different registries. When reporting on the journal impact factor, ICMJE membership, and the interval between the registration and paper submission dates in the current study, only one trial registration was accounted for in every published paper. After including the earliest trial registration for 63 papers with both reported trials registered prospectively or retrospectively and excluding 51 papers with one of the two trials registered prospectively, and the other trial registered retrospectively, the study included 7053 unique papers and trial registration records.

In total, 41.7% (3013/7218) of the trials were registered prospectively. Using Pearson’s χ2 test and binary logistic regression, the relation between the prospective registration of clinical trials and the trial registry, region, condition, funding, trial size, interval between registration and paper submission dates, impact factor, and ICMJE membership of the publishing journal were found to be significant (P<0.05). Supplementary appendix J provides the details.

Supplementary appendix K provides a further analysis of the prospective registration of trials published within the group of ICMJE member journals. A published trial was 2.4 times more likely to be registered prospectively (P<0.05, odds ratio 2.4, 95% confidence interval 1.7 to 3.3) when published in a specialty ICMJE member journal compared with a trial paper published in a general ICMJE member journal.

In this study it was not feasible to evaluate the modifications of trial registration records, especially if the predefined trial outcomes were modified after the enrolment date of the 3031 prospectively registered trials. Modifications of trial registrations were only provided through the web pages of the registered trials, with no option to export or download the modifications dataset for further analysis (see supplementary appendix L).

Authors’ explanations for delayed registration

The ICMJE recommendations emphasise that in the exceptional cases of retrospective registration of clinical trials, investigators should explicitly indicate the rationale behind the delayed registration in the paper publication.67 The ICMJE recommendations also signify that editors of ICMJE member journals should publish a statement explaining why an exception was made to accept a publication for a retrospectively registered trial.67

In the current study, 286 trials were published in an ICMJE member journal and were registered retrospectively. To investigate the compliance of the submitting authors and the journal editors with the ICMJE recommendations, the full text content of these trials publications was screened and only 2.8% (8/286) of the authors explained the reason for delayed registration of the trial—that is, registering the trial retrospectively after the enrolment of the first participant. Reasons for delayed registration were lack of awareness or error of omission in five papers, that the registration process took longer than anticipated in two papers, and that in one paper the feasibility of recruitment was to be tested before registration.6869707172737475 Appendix M provides details of this analysis. No published statement was identified from the editors of the ICMJE member journals.

Based on the authors’ explanations, some were committed to register their trial prospectively but the registration was not completed until during, or after, enrolment because of an extended revision process with the registry or attempts being made to validate the feasibly of recruitment before registration.6869

The interval between enrolment date and date at which the trial was registered was measured for the 4205 retrospectively registered trials in the current stud, with a weekly breakdown (fig 2, see supplementary appendix N for source data).

Fig 2
Fig 2

Delayed trial registration (weeks) after enrolment

A clear trend was observed for a higher number of trials registering in the early weeks after enrolment. In particular, the number of trials registered retrospectively in the first three weeks after enrolment was in the top 95th centile of the number of trials registered retrospectively at any given week up to 52 weeks after enrolment.

The US Food and Drug Administration requires all submitted trials to be registered no later than 21 days after enrolment.7677 The increased trend in retrospective trial registration within the first three weeks after enrolment might be associated with this requirement for US based trials. Retrospective trial registrations within three weeks after enrolment were compared between US based trials and trials from other countries, with the assumption that US based trials are more likely to acknowledge and follow the FDA requirements. Compared with international trials, those from the US were more likely (P<0.05) to be registered retrospectively within the first three weeks after enrolment: 25.5% (178/699) of US based trials compared with 18.1% (633/3506) of international trials.

To determine if delayed registration occurred during or after the enrolment period, the interval between the enrolment start date and the enrolment end date was measured in 330 trials that were registered retrospectively in the ISRCTN registry. This sample was chosen because the WHO trial registration dataset does not include the enrolment end date.78 The enrolment start date and end date are provided in the ISRCTN trial registration dataset. Overall, 70.9% (234/330) of the retrospectively registered trials were registered within the enrolment interval. Using Pearson’s χ2 test statistically significant (P<0.05) differences were found between the enrolment interval (mean 693.5 (SD 910.18)) and the journal’s impact factor (table 2).

Table 2

Relation between enrolment interval, journal types, and journal impact factors of trials registered retrospectively in ISRCTN registry

View this table:

Discussion

Our study showed higher overall compliance (71.2%) with reporting the trial registration number (TRN) in published trial papers compared with seven other studies reporting a compliance from 26% to 68%.2171819202122 These studies were limited in scope, with a substantially smaller size; between 54 and 317 included trials (see supplementary appendix O).

We also found that the compliance with the inclusion of TRN in published RCTs was higher (95.5%) for papers published in ICMJE member journals compared with papers published in journals that followed ICMJE recommendations (76.5%) and other journals (66.0%). Our results concur with those of a previous study that reported nearly the same compliance (95.8%) with the inclusion of TRN in 698 papers of trials that were registered in the ClinicalTrials.gov registry and were published in a purposive set of five ICMJE journals.23

The authors of nearly half (48.9%) of the RCT papers with a unique and valid TRN included the TRN in the full text manuscript and not in the abstract or the PubMed metadata. Indicating the TRN in the paper abstract is important for systematic reviews and meta-analysis, as future research could automate the screening and pre-identification of the trial registration records without the need to skim over the full text manuscript.

Prospective registration of clinical trials

Compliance in prospective registration of our included and unique randomised trials with valid registration and enrolment dates was 41.7%. Our results differ from those of other studies, which reported between 3.6% and 77% compliance in prospective trial registration.21822232425262728293031323334353637383940

The European Clinical Trials Register (EU CTR) reported the highest compliance, with prospective trial registration at 93.1% (81/87). Trials that were registered in the EU CTR were 10.8 times more likely to be registered prospectively before the enrolment date compared with trials registered in other WHO trial registries. The EU CTR has a two step registration process, with the investigator initiating the registration process and the member state regulatory authority completing and authorising the trial with details from the respective ethics committees.79 Furthermore, the EU CTR provides a detailed explanation of this registration and authorisation process and how it might result in a seemingly delayed registration date owing to its technical implementation—over which the investigators have no control—although registration should still be validated as a prospective registration in accordance with the ICMJE guidelines.80 The significantly higher compliance in prospective trial registration in the EU CTR registry compared with the rest of the WHO trial registries, might be attributed to the 2014 legal requirements for registration and publication of the results of clinical trials in the European Union, and to the rigor of the registration process in the EU CTR registry.81 In fact, the authors of a publication of a retrospectively registered trial included in our study reported that they were able to register successfully in ClinicalTrials.gov after their initial trial registration was declined at the EU CTR.70 This process might also explain the tendency for European trials not to register in the EU CTR—only 87 of 2086 European trials identified in our study were registered in the EU CTR.

Given the scale, investment, and impact of large and global trials, investigators would likely practice more diligence in the design phase of these trials. Trial with study locations across multiple countries are 2.9 times more likely to be registered prospectively.

We observed a wide range of compliance in prospective trial registration between different study conditions, from oral health trials at 15.2% (29/191) to addiction trials at 53% (71/134). Addiction interventions face unique challenges to recruitment and retention, such as staff scepticism of the efficacy of research, lack of patient motivation, and fear of randomisation into a placebo group.8283 We postulate that these recruitment challenges could lead to increased awareness and thoughtfulness of the investigators in adhering to best practices during the design phase of addiction RCTs. This postulate might explain why investigators of such trials are more inclined to comply with the ICMJE recommendation and register their trials prospectively.

A 2015 study of oral health trials reported 9% compliance of prospective trial registration in 15 high impact factor journals.2 None of the 15 journals was an ICMJE journal and only two were journals that followed ICMJE recommendations. In our study, none of the 191 oral health trials were published in an ICMJE journal. We also confirm that none of the 13 ICMJE member journals is focused on dentistry or oral health studies. The lack of ICMJE member journals and the under-representation of journals that follow ICMJE recommendations in the cohort of journals in which oral health trials are published might explain the low compliance in prospective registration of oral health trials.

The odds of prospective trial registration showed a trend towards significance for industry funded trials, with a 2.5 times greater likelihood of trial being registered prospectively before enrolment; however, the trend did not reach statistical significance. Aligned with findings from similar studies, our finding suggests that investigators of industry funded trials are more adherent to the ICMJE trial registration recommendations.848586 This adherence could be driven by editors and peer reviewers exercising more rigor when reviewing submissions from industry funded trials, because of potential funding source bias toward favourable results.

Compliance within the ICMJE and high impact factor journals

Not all ICMJE journals or journals that followed ICMJE recommendations in our study complied with the ICMJE and WHO requirements. Interestingly, among those journals that do comply with the guidelines, the rate of compliance is statistically meaningful. A manuscript published in an ICMJE member journal was 5.8 times more likely to include the TRN, and a published trial was 1.8 times more likely to be registered prospectively when published in an ICMJE member journal compared with other journals.

In this study, the high compliance (95.5% (721/755)) in reporting the TRN in papers published in an ICMJE member journal is aligned with two other studies from 2012 and 2013 reporting a 97% and 95.8% compliance, respectively.1723 Our reported compliance of 61.3% (410/669) in prospective registration of trials published in ICMJE member journals is lower than two other studies reporting the compliance at 77% and 72%.2338 Compared with our study, these two studies were different in size and scope and allowed for a grace period in the prospective registration of RCTs. We included over 10 times more trials in our study.2338 The study by Gopal et al was limited to RCTs published in the 10 highest impact US medical specialty society journals and considered a trial registration prospective if registered up to one month after the enrolment date of the first participant.38 The study by Huser et al was limited to RCTs published in five ICMJE journals and considered a trial registration to be prospective if registered up to two months after the enrolment date of the first participant.23 These differences could have contributed to lower compliance in prospective trial registration in our study. Given the resources and advocacy of the ICMJE community in promoting compliance in reporting TRN and prospective trial registration, the higher compliance within its group of journals would not be surprising but would nevertheless be reassuring to set the course for other scholarly journals to follow suit.

Compared with trial papers published in ICMJE member journals, we reported lower compliance in reporting the TRN (76.5%) and in prospective trial registration (41.4%) in papers published in journals that followed ICMJE recommendations. Our finding is aligned with a 2014 study that surveyed editors of journals that followed ICMJE recommendations and showed that at least half of these journals did not adhere to the ICMJE recommendation and that 67% of the editors would consider the publication of retrospectively registered trials.87 The same study also reported that only 18% of surveyed journal editors crosschecked submitted papers against registered trial records to identify any discrepancies, which might explain why we could not locate a trial registration record in the trial registry for 109 trial publications in our study with a reported TRN.

Trials with papers published in a journal with an impact factor in the top fourth were 1.5 times more likely to be registered prospectively, and the published papers were 3.7 times more likely to include the TRN. Our finding could be explained not only by the rigorous revision process of journals with higher impact factors, but also by findings from a 2011 study indicating that journals with higher impact factors are more likely to provide editorial advice and recommendations in the author instructions about trial registration.34

Selective registration bias

The adoption of the ICMJE recommendations for trial registration among editors of medical journals might have introduced another form of bias within the researcher community of clinical trials. We hypothesise that investigators of clinical trials could be biased to selectively register their trials when intending to submit their paper for publication. Retrospective and late registration could increase the acceptance chances of the submitted manuscript by both journal editors and peer reviewers.

We examined the relation between timely trial registration and the interval between trial registration date and the date at which the trial paper was submitted for publication.

We found a statistically significant relation between retrospective trial registration and the interval between registration and submission. Overall, 85.2% of the trials registered within one year from manuscript submission were registered retrospectively (see supplementary appendix J), for which we propose the new term, selective registration bias. Of the 616 trials that were registered retrospectively within one year from journal submission, only three were from the US and were registered within the first 21 days after the enrolment date. This does not, however, change the conclusion or the importance of our analysis for selective registration bias.

Selective registration bias is distinct and different from other types of biases that are empirically reported in the clinical research literature—namely, publication, outcome reporting, and registration bias. Publication bias measures the extent to which trial results are published, irrespective of registration, whereas registration bias measures the extent to which trials are registered, irrespective of publication of results. Reporting bias measures the differences in the outcome measures reported between the trial registration record and the published paper.

Delayed registration of clinical trials

The ICMJE and WHO recommendations do not allow for any grace period for prospective registration (ie, a trial is considered to have been registered prospectively if it was registered after enrolment within a defined period). Based on the analysis of authors’ explanations for delayed registration, two of the eight studies emphasised that the registration occurred during the enrolment process and before the collection of any outcome data.6869 We identified four other studies that reported prospective trial registration and considered a trial registration as prospective up to two months after enrolment, and in one study even up to the date of primary endpoint ascertainment (see supplementary appendix O).23383940 Allowing a grace period for prospective registration after enrolment is also recommended by the US Food and Drug Administration and the UK national health service through its health research authority, which require trial registration no later than three and six weeks after enrolment, respectively.767788

To help determine if a grace period was appropriate, we analysed the interval from enrolment to registration of the retrospectively registered trials and observed a clear trend of more trial registrations in the first three weeks after enrolment. We postulated that this trend might be explained by the three week grace period in the FDA requirements. To support this hypothesis, we found that US based trials were more likely to be registered within the first three weeks after enrolment compared with international trials, with the assumption that US based RCTs would adhere more to the FDA requirements.

The trend of higher trial registration continued until the eighth week after enrolment. We suspect that this eight week delay might be driven by trial record registration, revision, and ethical approvals in multisite trials. These delays are likely to be beyond the control of investigators.6880 Therefore, our findings would support the notion of perhaps including a grace period of between three weeks and two months to the existing ICMJE definition of prospective trial registration.

We determined whether late registration occurred within the enrolment period. Of the retrospectively registered trials in the ISRCTN registry, 70.9% were registered before the end of the enrolment date and were more likely to be published in a high impact journal. However, we could not validate if the trials were registered before any primary outcome data are collected. The start date for collection of primary outcome data is not a designated field in either the WHO trial registration dataset or the major trial registries: ClinicalTrials.gov and ISRCTN.78 This date is only provided voluntarily in line within the textual description of the primary outcome field. With 4205 retrospective trials included in our study, such analysis would be extensive and not feasible with our limited resources.

If investigators register a trial after collection of the primary outcome data has started, they would have the chance to adjust the trial record based on the preliminary finding of the collected data. However, registering the trial before collection of the primary outcome data would eliminate this type of bias. Perhaps the definition of prospective trial registration based on registering the trial before data collection would be more meaningful. However, we foresee a few challenges with this definition. Firstly, it requires schema modifications to the WHO trial registration dataset to include a dedicated and mandatory start date field for data collection. Secondly, the primary outcome might not be captured within a predefined time point for all study subjects—for example, surgery related trials or trials with outcomes measured by frequency of relapses or episodes. Thirdly, it does not preclude the bias for modification of the trial registration record after enrolment that might be driven by other factors, such as feasibly of recruitment, resources, and pre-intervention baseline measures.3878

Limitations of this study

This study has a few limitations. Our inclusion criteria were limited to randomised trials published in PubMed indexed journals and not other relevant journals, such as npj Digital Medicine, not indexed in PubMed.89

The specificity of our search terms might present a limitation. Although we acknowledge that not all RCTs contain the words “Randomized”, “Randomised”, or “Trial” in their titles, given the size and the scope of our study we expect the effect of this limitation to be marginal.

Because we based our trials registration on the WHO registry network, we only searched the 17 registries. We did not report on publications of trials that might be registered in a non-WHO registry, such as the Philippine Health Research Registry and the Federal Office of Public Health’s portal for human research in Switzerland.9091

We were unable to report on trial registration of 3027 published papers that did not include the TRN. Some of these trials might have been registered but the authors were not aware of the importance of including the TRN in the published paper. One study reported that 16% of the trials that did not include the TRN in the published papers were registered.27 A manual search for multiple thousands of potentially matching trials was not feasible with our scarce research resources. Hence we were not able to report on the rate of non-registration of clinical trials or the rate of registered trials that were published without the TRN in the published paper.

The qualitative analysis of the reasons behind delayed registration was based on explanations by the authors of eight published trial manuscripts. This is a small sample size and as such the results might not be representative of authors of retrospectively registered trials.

Analysis of trials characteristics, registration, and enrolment dates was based on trial registration records that we downloaded from the WHO trial registries. Investigators provide the information in the trial registration records manually—and in many cases voluntarily—when registering their trials. Some discrepancies in the trial registration records might exist owing to entry error or lack of awareness and resources of the researcher team to maintain and update the trial registration record. These discrepancies could have impacted the internal validity of our finding.

Conclusion

Compliance with including the TRN in published trials was 71.2% overall and significantly higher (95.5%) for papers published in an ICMJE member journal. These journals should be commended for adherence to their mandate and driving the adoption within the broader community of scholarly medical journals.

We found low compliance with prospective trial registration, as defined by the ICMJE (ie, when a trial is registered before the enrolment of the first participant) at 41.7% overall and 61.3% for trials published in one of the ICMJE member journals. The empirically low adoption, even within the ICMJE member journals, 10 years after the seventh revision of the Declaration of Helsinki, questions the adaptability and viability of this recommendation.

We detected a new form of bias, referred to as selective registration bias, with a statistically significant high proportion (85.2%) of investigators registering their trials retrospectively within one year of submission to a journal.

Within the cohort of trials that were registered retrospectively and published in one of the ICMJE member journals, we measured author compliance with publishing a statement justifying the late registration and the reasons why journal editors accept submissions of retrospectively registered trials. No published editor’s statement was identified, and only 2.8% of the authors included a statement justifying the delayed registration. Reasons for late registration were lack of awareness, error of omission, or because the registration process took longer than anticipated. Some authors emphasised that late registration was completed before any data analysis or collection.

This assessment of the adoption of the ICMJE and WHO recommendations for the prospective registration of clinical trials should help to inform the future development of publication and registration guidelines of RCTs for researchers, policy makers, and journal editors.

What is already known on this topic

  • Prospective registration of randomised controlled trials (RCTs) mitigates selective reporting of trial results

  • Existing research has shown compliance with prospective registration of RCTs in several studies of between 4% and 77%

  • These studies were limited in scope and included trials published in specialised journals, trials of a specific treatment or condition, or trials registered in a particular trial registry or small group of registries

What this study adds

  • This study found low compliance with the prospective registration of RCTs 10 years after the adoption of the seventh revision of the Declaration of Helsinki; 42% overall and 61% for RCTs published in ICMJE member journals

  • A new type of bias, selective registration bias, was identified

  • A statistically significant higher proportion (85%) of investigators registered their RCTs retrospectively within a year of submitting their manuscript for publication

Acknowledgments

Henry Potts (chartered statistician at the Royal Statistical Society, United Kingdom), reviewed and appraised the statistical methods.

Footnotes

  • Contributors: The lead author (MAD) and senior author (JC) conceived the study and developed the study design, with contributions from all authors. MAD ran the search queries, extracted data, analysed the results, and drafted the manuscript. All authors interpreted the results, were involved in reporting the study and reviewing and editing the manuscript, and approved the final version. MAD takes responsibility for the work and is the guarantor. The authors followed the recommendations of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement in the identification, screening, and inclusion of our search results. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: None received.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work in the previous three years; no financial relationships with any organisations that might have an interest in the submitted work; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Not required.

  • Data sharing: The dataset for this study is available on request.

  • The lead author (MAD) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

  • Dissemination to participants and related patient and public communities: The results of this study will be disseminated through the institutional websites. There is also a plan to present these findings at national and international scientific meetings.

http://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References

  1. ↵
    1. Harrison BA,
    2. Mayo-Wilson E
    . Trial Registration. Res Soc Work Pract2014;24:372-6. doi:10.1177/1049731513512374.
    OpenUrlCrossRefWeb of Science
  2. ↵
    1. Smaïl-Faugeron V,
    2. Fron-Chabouis H,
    3. Durieux P.
    Clinical Trial Registration in Oral Health Journals. J Dent Res2015;94:8S-13S. doi:10.1177/0022034514552492.
    OpenUrlCrossRefPubMed
  3. ↵
    1. Hopewell S,
    2. Loudon K,
    3. Clarke MJ,
    4. Oxman AD,
    5. Dickersin K
    . Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev2009;(1):MR000006. doi:10.1002/14651858.MR000006.pub3.
    OpenUrlCrossRef
  4. ↵
    1. Chan A-W,
    2. Hróbjartsson A,
    3. Haahr MT,
    4. Gøtzsche PC,
    5. Altman DG
    . Empirical Evidence for Selective Reporting of Outcomes in Randomized Trials. JAMA2004;291:2457. doi:10.1001/jama.291.20.2457.
    OpenUrlCrossRefPubMedWeb of Science
  5. ↵
    1. Melander H,
    2. Ahlqvist-Rastad J,
    3. Meijer G,
    4. Beermann B
    . Evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ2003;326:1171-3. doi:10.1136/bmj.326.7400.1171.
    OpenUrlAbstract/FREE Full Text
  6. ↵
    1. Wager E
    . The need for trial identifiers. Curr Med Res Opin2004;20:203-6. doi:10.1185/030079903125002883.
    OpenUrlCrossRefPubMed
  7. ↵
    1. Simes RJ
    . Publication bias: the case for an international registry of clinical trials. J Clin Oncol1986;4:1529-41. doi:10.1200/JCO.1986.4.10.1529.
    OpenUrlAbstract/FREE Full Text
  8. ↵
    1. Dwan K,
    2. Altman DG,
    3. Arnaiz JA,
    4. et al
    . Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias. PLoS One2008;3:e3081. doi:10.1371/journal.pone.0003081.
    OpenUrlCrossRefPubMed
  9. ↵
    1. Antes G,
    2. Chalmers I
    . Under-reporting of clinical trials is unethical. Lancet2003;361:978-9. doi:10.1016/S0140-6736(03)12838-3.
    OpenUrlCrossRefPubMedWeb of Science
  10. ↵
    1. Dyer C
    . UK clinical trials must be registered to win ethics committee approval. BMJ2013;347:f5614. doi:10.1136/bmj.f5614.
    OpenUrlFREE Full Text
  11. ↵
    1. De Angelis C,
    2. Drazen JM,
    3. Frizelle FA,
    4. et al
    . Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors. N Engl J Med2004;351:1250-1. doi:10.1056/NEJMe048225.
    OpenUrlCrossRefPubMedWeb of Science
  12. ↵
    1. Gülmezoglu AM,
    2. Pang T,
    3. Horton R,
    4. Dickersin K
    . WHO facilitates international collaboration in setting standards for clinical trial registration. Lancet2005;365:1829-31. doi:10.1016/S0140-6736(05)66589-0.
    OpenUrlCrossRefPubMedWeb of Science
  13. ↵
    The World Medical Association. Declaration of Helsinki - Ethical Principles for Medical Research Involving Human Subjects. https://www.wma.net/wp-content/uploads/2018/07/DoH-Oct2008.pdf. Accessed April 29, 2019.
  14. ↵
    1. Krleza-Jerić K,
    2. Lemmens T
    . 7th revision of the Declaration of Helsinki: good news for the transparency of clinical trials. Croat Med J2009;50:105-10. doi:10.3325/cmj.2009.50.105.
    OpenUrlCrossRefPubMedWeb of Science
  15. ↵
    1. Moorthy VS,
    2. Karam G,
    3. Vannice KS,
    4. Kieny M-P
    . Rationale for WHO’s New Position Calling for Prompt Reporting and Public Disclosure of Interventional Clinical Trial Results. PLoS Med2015;12:e1001819. doi:10.1371/journal.pmed.1001819.
    OpenUrlCrossRefPubMed
  16. ↵
    World Health Organization. WHO Statement on Public Disclosure of Clinical Trial Results. https://www.who.int/ictrp/results/reporting/en/. Published 2017. Accessed April 28, 2019.
  17. ↵
    1. van de Wetering FT,
    2. Scholten RJPM,
    3. Haring T,
    4. Clarke M,
    5. Hooft L
    . Trial Registration Numbers Are Underreported in Biomedical Publications. PLoS One2012;7:e49599. doi:10.1371/journal.pone.0049599.
    OpenUrlCrossRefPubMed
  18. ↵
    1. Rokhsefat S,
    2. Morra DE,
    3. Offringa M,
    4. Askie LM,
    5. Kelly LE
    . Trial registration in pediatric surgery trials. J Pediatr Surg2018;53:1273-9. doi:10.1016/j.jpedsurg.2017.10.049.
    OpenUrlCrossRef
  19. ↵
    1. Ross A,
    2. George D,
    3. Wayant C,
    4. Hamilton T,
    5. Vassar M
    . Registration Practices of Randomized Clinical Trials in Rhinosinusitis: A Cross-sectional Review. JAMA Otolaryngol Head Neck Surg2019;145:468-74. doi:10.1001/jamaoto.2019.0145.
    OpenUrlCrossRef
  20. ↵
    1. McGee RG,
    2. Su M,
    3. Kelly PJ,
    4. Higgins GY,
    5. Craig JC,
    6. Webster AC
    . Trial Registration and Declaration of Registration by Authors of Randomized Controlled Trials. Transplantation2011;92:1094-100. doi:10.1097/TP.0b013e318232baf2.
    OpenUrlCrossRefPubMedWeb of Science
  21. ↵
    1. Rayhill ML,
    2. Sharon R,
    3. Burch R,
    4. Loder E
    . Registration status and outcome reporting of trials published in core headache medicine journals. Neurology2015;85:1789-94. doi:10.1212/WNL.0000000000002127.
    OpenUrlCrossRefPubMed
  22. ↵
    1. Jones CW,
    2. Platts-Mills TF
    . Quality of Registration for Clinical Trials Published in Emergency Medicine Journals. Ann Emerg Med2012;60:458-64.doi:10.1016/j.annemergmed.2012.02.005.
    OpenUrlCrossRefPubMed
  23. ↵
    1. Huser V,
    2. Cimino JJ
    . Evaluating adherence to the International Committee of Medical Journal Editors’ policy of mandatory, timely clinical trial registration. J Am Med Inform Assoc2013;20(e1):e169-74. doi:10.1136/amiajnl-2012-001501.
    OpenUrlCrossRefPubMed
  24. ↵
    1. Harriman SL,
    2. Patel J
    . When are clinical trials registered? An analysis of prospective versus retrospective registration. Trials2016;17:187. doi:10.1186/s13063-016-1310-8.
    OpenUrlCrossRefPubMed
  25. ↵
    1. El-Boghdadly K,
    2. Wiles MD,
    3. Atton S,
    4. Bailey CR
    . Adherence to guidance on registration of randomised controlled trials published in Anaesthesia. Anaesthesia2018;73:556-63. doi:10.1111/anae.14103.
    OpenUrlCrossRef
  26. ↵
    1. Bradley HA,
    2. Rucklidge JJ,
    3. Mulder RT
    . A systematic review of trial registration and selective outcome reporting in psychotherapy randomized controlled trials. Acta Psychiatr Scand2017;135:65-77. doi:10.1111/acps.12647.
    OpenUrlCrossRef
  27. ↵
    1. Farquhar CM,
    2. Showell MG,
    3. Showell EAE,
    4. et al
    . Clinical trial registration in fertility trials – a case for improvement?Hum Reprod2017;32:1827-34. doi:10.1093/humrep/dex251.
    OpenUrlCrossRef
  28. ↵
    1. Mann E,
    2. Nguyen N,
    3. Fleischer S,
    4. Meyer G
    . Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012. Age Ageing2014;43:872-6. doi:10.1093/ageing/afu086.
    OpenUrlCrossRefPubMed
  29. ↵
    1. Cybulski L,
    2. Mayo-Wilson E,
    3. Grant S
    . Improving transparency and reproducibility through registration: The status of intervention trials published in clinical psychology journals. J Consult Clin Psychol2016;84:753-67. doi:10.1037/ccp0000115.
    OpenUrlCrossRef
  30. ↵
    1. Reveiz L,
    2. Bonfill X,
    3. Glujovsky D,
    4. et al
    . Trial registration in Latin America and the Caribbean’s: study of randomized trials published in 2010. J Clin Epidemiol2012;65:482-7. doi:10.1016/j.jclinepi.2011.09.003.
    OpenUrlCrossRefPubMed
  31. ↵
    1. Huić M,
    2. Marušić M,
    3. Marušić A
    . Completeness and Changes in Registered Data and Reporting Bias of Randomized Controlled Trials in ICMJE Journals after Trial Registration Policy. PLoS One2011;6:e25258. doi:10.1371/journal.pone.0025258.
    OpenUrlCrossRefPubMed
  32. ↵
    1. Østervig RM,
    2. Sonne A,
    3. Rasmussen LS
    . Registration of randomized clinical trials - a challenge. Acta Anaesthesiol Scand2015;59:986-9. doi:10.1111/aas.12575.
    OpenUrlCrossRef
  33. ↵
    1. Solaymani-Dodaran M,
    2. Khalili D,
    3. Hosseini H,
    4. et al
    . Iranian Registry of Clinical Trials two years on and the timing of registrations. J Evid Based Med2011;4:168-71. doi:10.1111/j.1756-5391.2011.01146.x.
    OpenUrlCrossRef
  34. ↵
    1. Kunath F,
    2. Grobe HR,
    3. Keck B,
    4. et al
    . Do urology journals enforce trial registration? A cross-sectional study of published trials. BMJ Open2011;1:e000430. doi:10.1136/bmjopen-2011-000430.
    OpenUrlAbstract/FREE Full Text
  35. ↵
    1. Papageorgiou SN,
    2. Antonoglou GN,
    3. Sándor GK,
    4. Eliades T
    . Randomized clinical trials in orthodontics are rarely registered a priori and often published late or not at all. PLoS One2017;12:e0182785. doi:10.1371/journal.pone.0182785.
    OpenUrlCrossRef
  36. ↵
    1. Scott A,
    2. Rucklidge JJ,
    3. Mulder RT
    . Is Mandatory Prospective Trial Registration Working to Prevent Publication of Unregistered Trials and Selective Outcome Reporting? An Observational Study of Five Psychiatry Journals That Mandate Prospective Clinical Trial Registration. PLoS One2015;10:e0133718. doi:10.1371/journal.pone.0133718.
    OpenUrlCrossRefPubMed
  37. ↵
    1. Boccia S,
    2. Rothman KJ,
    3. Panic N,
    4. et al
    . Registration practices for observational studies on ClinicalTrials.gov indicated low adherence. J Clin Epidemiol2016;70:176-82. doi:10.1016/j.jclinepi.2015.09.009.
    OpenUrlCrossRef
  38. ↵
    1. Gopal AD,
    2. Wallach JD,
    3. Aminawung JA,
    4. et al
    . Adherence to the International Committee of Medical Journal Editors’ (ICMJE) prospective registration policy and implications for outcome integrity: a cross-sectional analysis of trials published in high-impact specialty society journals. Trials2018;19:448. doi:10.1186/s13063-018-2825-y.
    OpenUrlCrossRef
  39. ↵
    1. Dechartres A,
    2. Ravaud P,
    3. Atal I,
    4. Riveros C,
    5. Boutron I
    . Association between trial registration and treatment effect estimates: a meta-epidemiological study. BMC Med2016;14:100. doi:10.1186/s12916-016-0639-x.
    OpenUrlCrossRef
  40. ↵
    1. Dal-Ré R,
    2. Ross JS,
    3. Marušić A
    . Compliance with prospective trial registration guidance remained low in high-impact journals and has implications for primary end point reporting. J Clin Epidemiol2016;75:100-7. doi:10.1016/j.jclinepi.2016.01.017.
    OpenUrlCrossRefPubMed
  41. ↵
    National Center for Biotechnology Information USNL of M. Advanced search - PubMed - NCBI. https://www.ncbi.nlm.nih.gov/pubmed/advanced. Accessed April 24, 2019.
  42. ↵
    World Health Organization. International Clinical Trials Registry Platform (ICTRP) Search Portal. http://apps.who.int/trialsearch/. Accessed April 24, 2019.
  43. ↵
    Australian New Zealand Clinical Trials Registry (ANZCTR). http://www.anzctr.org.au/. Accessed April 24, 2019.
  44. ↵
    The Chinese clinical test registration center. The Chinese Clinical Trial Registry (ChiCTR). http://www.chictr.org.cn/enIndex.aspx. Accessed April 24, 2019.
  45. ↵
    U.S. National Library of Medicine. Advanced Search - ClinicalTrials.gov. https://clinicaltrials.gov/ct2/search/advanced. Accessed April 24, 2019.
  46. ↵
    The European Medicines Agency. The European Union Clinical Trials Register. https://www.clinicaltrialsregister.eu/ctr-search/search. Accessed April 24, 2019.
  47. ↵
    BioMed Central Ltd. Part of Springer Nature. ISRCTN Registry. https://www.isrctn.com/. Accessed April 24, 2019.
  48. ↵
    Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht. The Netherlands National Trial Register. https://www.trialregister.nl/trials. Accessed April 24, 2019.
  49. ↵
    Fundação Oswaldo Cruz. The Brazilian Clinical Trials Registry (ReBec). http://www.ensaiosclinicos.gov.br/. Accessed April 24, 2019.
  50. ↵
    National Institute of Medical Statistics. The Clinical Trials Registry - India (CTRI). http://ctri.nic.in/Clinicaltrials/login.php. Accessed April 24, 2019.
  51. ↵
    The Korea Centers for Disease Control and Prevention (KCDC). Clinical Research Information Service (CRIS) - Republic of Korea. https://cris.nih.go.kr/cris/en/use_guide/cris_introduce.jsp. Accessed April 24, 2019.
  52. ↵
    The National Coordinating Center of Clinical Trials (CENCEC). The Cuban Public Registry of Clinical Trials (RPCEC). http://registroclinico.sld.cu/en/home. Accessed April 24, 2019.
  53. ↵
    Bundesministerium für Bildung und Forschung (Federal Ministry of Education and Research). The German Clinical Trials Register (DRKS). https://www.drks.de. Accessed April 24, 2019.
  54. ↵
    The Ministry of Health and Medical Education (MOHME). The Iranian Registry of Clinical Trials (IRCT). https://www.irct.ir/. Accessed April 24, 2019.
  55. ↵
    The National Institute of Public Health. Japan Primary Registries Network (JPRN) - NIPH Clinical Trials Search of Japan. https://rctportal.niph.go.jp/en/. Accessed April 24, 2019.
  56. ↵
    Cochrane South Africa (Cochrane SA) at the South African Medical Research Council. (SAMRC). The Pan African Clinical Trial Registry (PACTR). https://pactr.samrc.ac.za/Default.aspx. Accessed April 24, 2019.
  57. ↵
    Sri Lanka Medical Association. The Sri Lanka Clinical Trials Registry (SLCTR). https://slctr.lk/. Accessed April 24, 2019.
  58. ↵
    The Clinical Research Collaboration Network (CRCN). The Thai Clinical Trials Registry (TCTR). http://www.clinicaltrials.in.th/. Accessed April 24, 2019.
  59. ↵
    The National Institute of Health. The Peruvian Clinical Trials Registry (REPEC). https://ensayosclinicos-repec.ins.gob.pe/en/. Accessed April 24, 2019.
  60. ↵
    The University of Toronto. University of Toronto Libraries. https://onesearch.library.utoronto.ca/. Accessed April 24, 2019.
  61. ↵
    International Committee of Medical Journal Editors (ICMJE) | About ICMJE | FAQ | ICMJE Membership. http://icmje.org/about-icmje/faqs/icmje-membership/. Accessed April 1, 2019.
  62. ↵
    International Committee of Medical Journal Editors (ICMJE). Journals stating that they follow the ICMJE Recommendations. http://icmje.org/journals-following-the-icmje-recommendations/. Accessed April 24, 2019.
  63. ↵
    Clarivate Analytics. InCites Journal Citation Reports. https://incites.clarivate.com. Accessed April 24, 2019.
  64. ↵
    The University of Toronto. The Gerstein Science Information Centre. https://gerstein.library.utoronto.ca/. Accessed April 24, 2019.
  65. ↵
    1. Moher D,
    2. Liberati A,
    3. Tetzlaff J,
    4. Altman DG,
    5. Group TP
    . Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med2009;6:e1000097. doi:10.1371/journal.pmed.1000097.
    OpenUrlCrossRefPubMed
  66. ↵
    U.S. National Library of Medicine. MEDLINE®/PubMed® Data Element (Field) Descriptions. https://www.nlm.nih.gov/bsd/mms/medlineelements.html#si. Accessed May 23, 2019.
  67. ↵
    International Committee of Medical Journal Editors (ICMJE). Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. http://www.icmje.org/icmje-recommendations.pdf. Accessed April 24, 2019.
  68. ↵
    1. Thorn BE,
    2. Eyer JC,
    3. Van Dyke BP,
    4. et al
    . Literacy-Adapted Cognitive Behavioral Therapy Versus Education for Chronic Pain at Low-Income Clinics: A Randomized Controlled Trial. Ann Intern Med2018;168:471-80. doi:10.7326/M17-0972.
    OpenUrlCrossRef
  69. ↵
    1. Geldsetzer P,
    2. Francis JM,
    3. Sando D,
    4. et al
    . Community delivery of antiretroviral drugs: A non-inferiority cluster-randomized pragmatic trial in Dar es Salaam, Tanzania. PLoS Med2018;15:e1002659. doi:10.1371/journal.pmed.1002659.
    OpenUrlCrossRef
  70. ↵
    1. Koch G,
    2. Bonnì S,
    3. Casula EP,
    4. et al
    . Effect of Cerebellar Stimulation on Gait and Balance Recovery in Patients With Hemiparetic Stroke. JAMA Neurol2019;76:170. doi:10.1001/jamaneurol.2018.3639.
    OpenUrlCrossRef
  71. ↵
    1. Ogedegbe G,
    2. Plange-Rhule J,
    3. Gyamfi J,
    4. et al
    . Health insurance coverage with or without a nurse-led task shifting strategy for hypertension control: A pragmatic cluster randomized trial in Ghana. PLoS Med2018;15:e1002561. doi:10.1371/journal.pmed.1002561.
    OpenUrlCrossRef
  72. ↵
    1. Källander K,
    2. Alfvén T,
    3. Funk T,
    4. et al
    . Universal versus conditional day 3 follow-up for children with non-severe unclassified fever at the community level in Ethiopia: A cluster-randomised non-inferiority trial. PLoS Med2018;15:e1002553. doi:10.1371/journal.pmed.1002553.
    OpenUrlCrossRef
  73. ↵
    1. White DB,
    2. Angus DC,
    3. Shields A-M,
    4. et al
    . A Randomized Trial of a Family-Support Intervention in Intensive Care Units. N Engl J Med2018;378:2365-75. doi:10.1056/NEJMoa1802637.
    OpenUrlCrossRef
  74. ↵
    1. Rabin JC,
    2. Karunathilake N,
    3. Patrizi K
    . Effects of Milk vs Dark Chocolate Consumption on Visual Acuity and Contrast Sensitivity Within 2 Hours. JAMA Ophthalmol2018;136:678. doi:10.1001/jamaophthalmol.2018.0978.
    OpenUrlCrossRef
  75. ↵
    1. Goralnick E,
    2. Chaudhary MA,
    3. McCarty JC,
    4. et al
    . Effectiveness of Instructional Interventions for Hemorrhage Control Readiness for Laypersons in the Public Access and Tourniquet Training Study (PATTS): A Randomized Clinical Trial. JAMA Surg2018;153:791-9. doi:10.1001/jamasurg.2018.1099.
    OpenUrlCrossRef
  76. ↵
    U.S. National Library of Medicine - ClinicalTrials. gov. FDAAA 801 and the Final Rule. https://clinicaltrials.gov/ct2/manage-recs/fdaaa. Accessed April 30, 2019.
  77. ↵
    U.S. Food and Drug Administration. Section 801 of the Food and Drug Administration Amendments Act of 2007. https://www.govinfo.gov/content/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf. Accessed April 27, 2019.
  78. ↵
    World Health Organization. Trial Registration Data Set (Version 1.3.1). WHO. https://www.who.int/ictrp/network/trds/en/. Published 2019. Accessed April 27, 2019.
  79. ↵
    EU Clinical Trials Register. National medicine regulatory authorities. https://www.clinicaltrialsregister.eu/natauthorities.html. Accessed April 26, 2019.
  80. ↵
    EU Clinical Trials Register. Clarification note on the date of registration in the register. https://www.clinicaltrialsregister.eu/doc/14-03-26%20FLS%20-%20Response%20to%20Paolo%20Daniele%20Siviero%20-%20to%20whom%20it%20may%20concern.pdf. Accessed April 26, 2019.
  81. ↵
    1. Flear ML
    . The EU Clinical Trials Regulation: key priorities, purposes and aims and the implications for public health. J Med Ethics2016;42:192-8doi:10.1136/medethics-2015-103258.
    OpenUrlAbstract/FREE Full Text
  82. ↵
    1. Thomson C,
    2. Morley K,
    3. Teesson M,
    4. Sannibale C,
    5. Haber P
    . Issues with recruitment to randomised controlled trials in the drug and alcohol field: a literature review and Australian case study. Drug Alcohol Rev2008;27:115-22. doi:10.1080/09595230701829561.
    OpenUrlCrossRefPubMedWeb of Science
  83. ↵
    1. Ashery RS,
    2. McAuliffe WE
    . Implementation issues and techniques in randomized trials of outpatient psychosocial treatments for drug abusers: recruitment of subjects. Am J Drug Alcohol Abuse1992;18:305-29. https://www.ncbi.nlm.nih.gov/pubmed/1329493doi:10.3109/00952999209026069.
    OpenUrlCrossRefPubMedWeb of Science
  84. ↵
    1. Zarin DA,
    2. Tse T,
    3. Williams RJ,
    4. Rajakannan T
    . Update on Trial Registration 11 Years after the ICMJE Policy Was Established. N Engl J Med2017;376:383-91. doi:10.1056/NEJMsr1601330.
    OpenUrlCrossRefPubMed
  85. ↵
    1. Ajithkumar TV,
    2. Gilbert DC
    . Modern Challenges of Cancer Clinical Trials. Clin Oncol (R Coll Radiol)2017;29:767-9. doi:10.1016/j.clon.2017.10.006.
    OpenUrlCrossRef
  86. ↵
    1. Hunter KE,
    2. Seidler AL,
    3. Askie LM
    . Prospective registration trends, reasons for retrospective registration and mechanisms to increase prospective registration compliance: descriptive analysis and survey. BMJ Open2018;8:e019983. doi:10.1136/bmjopen-2017-019983.
    OpenUrlAbstract/FREE Full Text
  87. ↵
    1. Hooft L,
    2. Korevaar DA,
    3. Molenaar N,
    4. Bossuyt PMM,
    5. Scholten RJPM
    . Endorsement of ICMJE’s Clinical Trial Registration Policy: a survey among journal editors. Neth J Med2014;72:349-55. https://www.ncbi.nlm.nih.gov/pubmed/25178768.
    OpenUrlPubMed
  88. ↵
    National Health Service - Health Research Authority. Research registration and research project identifiers. https://www.hra.nhs.uk/planning-and-improving-research/research-planning/research-registration-research-project-identifiers/. Accessed April 30, 2019.
  89. ↵
    1. Springer Nature Publishing AG
    . NPJ Digital Medicine.Nature Publishing Group, 2018.
  90. ↵
    The Philippine Health Research Registry (PHRR). http://registry.healthresearch.ph/. Accessed April 27, 2018.
  91. ↵
    The Federal Office of Public Health’s (FOPH) portal for human research in Switzerland. https://www.kofam.ch/en/snctp-portal/searching-for-a-clinical-trial/. Accessed April 27, 2018.
Back to top