You are here

Article Text

Article menu

Download PDFPDF

XML
Oncology
Research
Publication of interventional phase 3 and 4 clinical trials in radiation oncology: an observational study
  1. Jaime Pérez-Alija1,
  2. Pedro Gallego1,
  3. Isabel Linares2,
  4. Eva Ambroa3,
  5. Agustí Pedro1
  1. 1 Radioterapia i Oncología, Hospital Plato, Barcelona, Spain
  2. 2 Radiotherapy, Institut Catala d' Oncologia, Barcelona, Spain
  3. 3 Medical Physics, Consorci Sanitari de Terrassa, Barcelona, Spain
  1. Correspondence to Dr Pedro Gallego; pedro.gallego{at}hospitalplato.com

Abstract

Objectives Clinical trials produce the best data available for decision-making in modern evidence-based medicine. We aimed to determine the rate of non-publication of interventional phase 3 and 4 clinical trials involving patients with cancer undergoing radiotherapy.

Setting The ClinicalTrials.gov database was searched for interventional phase 3 and 4 trials in radiotherapy with a primary completion date before 1 January 2013. We determined how many of these registry entries have not published the compulsory deposition of their results in the database and performed a systematic search for published studies in peer-reviewed journals.

Results Of 576 trials, 484 (84.0%) did not deposit a summary result in the registry. In addition, 44.9% of them did not publish their results in a peer-reviewed journal. Similar percentages were found for most cancer subtypes: brain (41%), breast (38%), cervical (66%), colorectal (38%), lung (48%), prostate (45%), bladder (56%), head and neck (56%) and lymphoma (33%).

Conclusion Our results show that most trials in radiation oncology did not report the results in the registry. Almost half of these trials have not been published in the biomedical literature. This means that a large number of study participants were exposed to the risks of trial participation without the supposed benefits that sharing and publishing of results would offer to future generations of patients.

  • oncology
  • radiation oncology
  • clinical trials
  • health economics
  • medicine evidence based

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

http://dx.doi.org/10.1136/bmjopen-2017-016040

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • We have considered and analysed the higher levels of evidence-based radiation oncology.

    • Each trial meeting the inclusion criteria was independently searched by at least two authors in order to assess its publication in a peer-reviewed journal.

    • ClinicalTrials.gov is, by far, the largest trial registry in the world. Any applicable medical device trial or medical drug trial planned to be market in the USA has to be registered in this registry.

    • Insufficient statistical power and lack of data to test for hypotheses giving a plausible explanation of non-publication in radiation oncology.

    Background

    Clinical trials produce the best data available for decision-making in modern evidence-based medicine. All this evidence should be both published and available, since withholding results skews the evidence and therefore dangerously distorts it. Publication of all trials conducted in radiation oncology is needed to fully determine the benefits and risks of treatments currently in use in our clinics.

    Since 2005, the International Committee of Medical Journal Editors has required prospective registration of all interventional clinical studies prior to publication. It does not, however, require authors to report the results of registered trials.1 On the other hand, the US federal law, the Food and Drug Administration Amendments Act of 2007 (FDAAA 801),2 requires responsible parties of all interventional trials to submit summary results to the ClinicalTrials.gov database 12 months after the primary completion date (PCD); PCD is the term used at ClinicalTrials.gov for the ‘completion date’, as defined in FDAAA 801. Furthermore, this summary must be made publicly available, keeping with the Declaration of Helsinki, which makes it an ethical obligation to make the results of all medical research involving human subjects publicly available.3

    In this work, we answered two important questions regarding the state of the evidence in radiation oncology. The first was ‘Were the trials conducted in radiation oncology in compliance with the US law and therefore did they make their results publicly available?’ The second was ‘How many of the trials conducted in radiation oncology have published their results in a peer-reviewed journal (PRJ)?’ The answers to both questions are vital to our patients, to our healthcare system (independently of the model a country has chosen as its own), and to the state of evidence we have within our reach as practitioners (are our treatments really based on evidence?).

    Methods

    Data source

    ClinicalTrials.gov is a clinical trial registry and results database that provides the public with access to registrations and summary results information for clinical studies. This registry is maintained by the National Library of Medicine at the National Institutes of Health (NIH). As is often stated, this registry represents nowadays the most comprehensive source for information about ongoing and completed trials within and outside the USA, and we consequently chose it to conduct this research.4

    Database search

    We searched the ClinicalTrials.gov database for trials in radiotherapy as of 6 May 2016 that had a PCD between 1 January 2008 and 1 January 2015. We chose this date because we had to allow a minimum 12-month period for publication of the compulsory summary results in the registry (16 months in our case). When a PCD was missing, we instead used the completion date field. We used the ‘Advanced Search’ form to broaden our search (box 1).

    Box 1

    Search terms in clinical trial registry

    • Search terms: “radiotherapy” OR “radiation therapy” OR brachytherapy” OR “IMRT” OR “SBRT” OR “IMPT” OR “radiation oncology” (IMRT stands for intensity-modulated radiation therapy; SBRT stands for stereotactic body radiation therapy; IMPT stands for intensity-modulated proton therapy)

    • Study type: interventional studies

    • Study results: all studies

    • Recruitment: all studies

    • Additional criteria → phase: no phase was ticked since phase 3 or 4 trials concerning radiation therapy were also registered as trials without phase

    For this study and within the aforementioned date range, we considered all clinical trials that met the criteria shown in box 2.

    Box 2

    Search terms in Clinical Trial Registry II

    • Study type: interventional studies

    • Interventions: radiotherapy as standard treatment or primary focus in oncology

    • Phase: phase 3; phase 4

    Trials with a ‘Withdrawn’ status were excluded because these trials have ended early before enrolling the first patients.

    Each trial registered on ClinicalTrials.gov has a unique identification code, ‘NCT’, followed by an eight-digit number. This identifier is commonly known as the NCT number. We used this NCT number to avoid trial duplicates within our final set. In order to avoid false positives, for each trial, we extracted all the information provided by ClinicalTrials.gov’s application programming interface (see table 1). We also used the uniform resource locator field in order to access all the trial information registered in the database. Two researchers (JP-A and PG) independently reviewed the information displayed by using the same search protocol and decided for each trial whether the criteria mentioned above were fully met, with a consensus discussion in case of disagreement. If they failed to reach a consensus, a third researcher (IL) took a final decision after taking into account both arguments.

    View this table:
    Table 1

    Information extracted for each interventional phase 3 and 4 trials

    Finally, we analysed the ‘Study Results’ field and differentiated between those studies with a ‘Has Results’ tag from those with a ‘No Results Available’ tag (see figure 1).

    Figure 1
    Figure 1

    Database search. PCD, primary completion date.

    Publication search in a PRJ

    Because our query on ClinicalTrials.gov was conducted on 6 May 2016, we allowed a minimum of 24 months after the latest possible PCD (6 May 2014) for journal submission, peer review and editorial process until the trial was finally published in a PRJ. For those trials published electronically ahead of print, we used the date on which online publication occurred. Trials with a ‘Suspended’ or ‘Terminated’ status were excluded from this search (see figure 2).

    Figure 2
    Figure 2

    Publication search in a peer-reviewed journal (PRJ). PCD, primary completion date.

    Our clinical trial set was divided into four subsets. Each subset was given to a particular researcher (JP-A, PG, IL and EA). A trial was considered published if it met the criteria shown in box 3.

    Box 3

    Criteria listed for peer-reviewed journal (PRJ) search

    • The trial was published in a PRJ

    • Results reported in the publication were a primary outcome measure or a secondary outcome measure, or both

    • No abstract, poster, oral communication or private communication of a trial result was considered as a valid publication

    Each author searched PubMed, Google Scholar and Google by using the following characteristics: NCT number, other identification numbers provided by ClinicalTrials.gov, author names, institutions, title, official title and keywords. Matches were evaluated according to title, trial design, sample size, intervention, location, dates of recruitment and completion, study hypotheses, and primary and/or secondary outcome measures, as described in the ClinicalTrials.gov database. Matches found by each researcher were always checked by a second researcher. We then categorised our data into subsets by cancer subtype.

    ClinicalTrials.gov also displayed publication citations at the bottom of the ‘Full Text View’ tab of a study record, under the ‘More Information’ heading. These citations are either submitted by sponsors or investigators, or are automatically indexed by ClinicalTrials.gov. Citations submitted by sponsors or investigators may provide background information instead of information about results. We also reviewed this linked information to evaluate whether or not the information provided by sponsors or indexed by ClinicalTrials.gov was relevant to our study. We applied the same methodology as explained in the previous paragraph.

    In order to look for publication bias, we took into account all trials with results in the registry that qualified for a search in a PRJ. This set was further divided into two subsets: the first contained all trials with a summary result reported in the registry and no publication in a PRJ; the second contained all trials with a summary result reported in the registry and a publication in a PRJ. For each subset, we further analyse positive and negative result frequencies. A positive finding was defined as a result rejecting the null hypothesis in favour of the experimental arm; a negative finding, on the other hand, was defined as a result that either confirmed the null hypothesis or rejected it in favour of the control arm.

    Statistical analysis

    We used the χ2 test to compare publication rates in the registry between trials grouped by funding type. p Values of <0.05 were considered statistically significant. We also used the χ2 test to compare publication rates in a PRJ between trials grouped by funding type. To test for the effect of this variable on publication, we used adjusted binary logistic regression (non-publication vs publication), which produced an OR and a 95% CI; an OR larger than 1.0 indicated a greater likelihood of trial publication in this group. The main explanatory variable was funding status adjusted for number of patients in the trial and the country of the principal investigator (American Institution vs Other). These analyses was prespecified and undertaken to evaluate whether or not industry funding, enrolment or country had an impact on patterns of publication. Statistical analyses were performed by using R version V.3.3.15

    Results

    Overall, 583 interventional phase 3 and 4 clinical trials met the inclusion criteria. Of these 583 trials, 7 had a ‘Withdrawn’ status and were consequently excluded. Fifty-one were phase 4 trials with the remaining 525 phase 3 trials. A total of 484 (84.0%) of all the interventional phase 3 and 4 clinical trials did not publish the compulsory summary results in the ClinicalTrials.gov registry. NIH funding was significantly associated with a higher likelihood of reporting results (OR 3.23, 95% CI 1.89 to 5.57; p<0.001). Industry funding was likewise significantly associated with a higher likelihood of reporting results in the registry (OR 3.43, 95% CI 1.93 to 6.08; p<0.001). No statistically significant differences were found between NIH-funded trials and industry-funded trials (OR=1.14, 95% CI 0.64 to 2.04, p=0.66) (see table 2 and figure 3). Although we had focus in funding as our explanatory variable, we have also observed that ‘being from an American Institution’ (in principal investigator variable) was significantly associated with a higher likelihood of reporting results when adjusted by funding type and enrolment (see table 3).

    View this table:
    Table 2

    Number of trials with results not posted on ClinitalTrials.gov registry. Funded feature is not an exclusive one: trials might have been funded by a combination of the three possible options (National Institutes of Health (NIH), industry and other)

    Figure 3
    Figure 3

    Distribution of trials is described in table 2. NIH, National Institutes of Health.

    View this table:
    Table 3

    Adjusted binary logistic regression (non-publication vs publication in ClinicalTrials.gov) by funding type, adjusted for the country of the principal investigator and enrolment

    When categorised by phase, 46 (90.2%) phase 4 trials and 438 (83.4%) phase 3 trials did not publish a deposition of their results in the registry, although this percentage difference was not significant (OR 1.75, 95% CI 0.68 to 5.99; p=0.301).

    Overall, 463 interventional phase 3 and 4 clinical trials met the criteria for searching a publication in a PRJ (43 phase 4 trials and 420 phase 3 trials). A total of 255 (55.1%) trials each had at least one publication of their results in a PRJ, but 208 (44.9%) trials remained unpublished. Median and mean time to publication was 60 months. NIH funding was significantly associated with a higher likelihood of published results (OR 3.17, 95% CI 1.85 to 5.55; p<0.001). Industry funding was not significantly associated with a higher or lower likelihood of publishing results in a PRJ (OR 1.14, 95% CI 0.67 to 1.98; p=0.63) (see table 4 and figure 4). ‘Being from an American Institution’ was not significantly associated with a lower or higher likelihood of publishing results when adjusted by funding type and enrolment (see table 5).

    View this table:
    Table 4

    Number of trials with results not published on a peer-reviewed journal (PRJ). As given in table 1, the funded feature is not exclusive, and there might be trials which were funded by a combination of the three possible options (National Institutes of Health (NIH), industry and other)

    Figure 4
    Figure 4

    Distribution of trials is described in table 4. NIH, National Institutes of Health.

    View this table:
    Table 5

    Adjusted binary logistic regression (non-publication vs publication in peer-reviewed journal) by funding type, adjusted for the country of the principal investigator and enrolment

    Taking into account the trial phase, 27 (62.8%) phase 4 trials and 181 (43.1%) phase 3 trials remained unpublished. This difference between phase 3 and phase 4 trials was statistically significant (OR=2.23, 95% CI 1.18 to 4.34; p=0.02).

    Of these 463 trials, when taking into account cancer subtype, we found the following percentages for unpublished results in a PRJ (total number of unpublished trials is shown in parentheses): 41.2% for brain (14 of 34), 37.9% for breast (25 of 66), 61.1% for cervical (11 of 18), 37.9% for colorectal (11 of 29), 33.3% for endometrial (3 of 9), 75% for oesophagus (3 of 4), 62.5% for eye (5 of 8), 37.5% for gastric (3 of 8), 55.6% for head and neck (47 of 84), 100.0% for kidney (2 of 2), 36.0% for leukaemia (9 of 25), 50.0% for liver (4 of 8), 48.1% for lung (25 of 52), 100.0% for melanoma (1 of 1), 66.7% for myeloma (2 of 3), 80% for metastasis (4 of 5), 36.4% for pancreatic (4 of 11), 45.2% for prostate (19 of 42), 55.6% for bladder (5 of 9), 33.3% for lymphoma (7 of 21), 33.3% for sarcoma (4 of 12), 61.5% for other (8 of 13). For all subgroups, we ran a significance test to determine whether these percentages were different from the global non-publication tendency. As can be seen in table 6, no statistically significant difference was found in any of them with the exception of head and neck which showed slightly worse numbers.

    View this table:
    Table 6

    Number of trials with results not published in a peer-reviewed journal (PRJ) by cancer subtype. For those subgroups with at least 16 trials, we run a significant test in order to see if these percentages were different from the global non-publication tendency. For each cancer subtype, OR were calculated taking as reference the global set minus this cancer subtype subset

    For publication bias, only 67 trials (14.4%) met the criteria: 18 trials reported a summary result but were not published in a PRJ, and 49 trials reported a summary result and were published in a PRJ. For our first subset, 8 of 18 trials (44.4%) showed a positive finding and the remaining 10 (55.6%) a negative finding; the second subset showed a similar pattern: 24 of 49 (49.0%) had a positive finding and the remaining 25 (51%) a negative finding (table 7).

    View this table:
    Table 7

    Number of trials meeting the inclusion criteria for analysing the publication bias

    Discussion

    Clinical trials produce the best data available for decision-making in modern evidence-based medicine. All evidence should be both published and available because withholding the results skews the evidence and therefore dangerously distorts it. When evidence is not published, those who make decisions about potential treatments do not have complete information about the outcome and the entire set of benefits and risks that a particular treatment might involve. The importance of publishing negative results has not been stressed strongly enough6; publishing these results reduces biases regarding the efficacy of a treatment and plays a huge role in helping science to move forward. Perhaps the most famous example of a negative result was the historic paper published by Michelson and Morley in 1883,7 which led a young physicist working at a patent office in Bern 22 years later, in 1905, to completely change our notion of space and time—a notion that almost one hundred years later turned out to be an essential feature in the GPS system. This young physicist was Albert Einstein. Despite the importance of knowing whether there is publication bias in radiation oncology, the present work confirms that it is not possible to assess such bias because of a massive lack of data: a mere 15% of the trials registered at ClinicalTrials.gov had published the compulsory summary result and only 45% of all trials conducted had been published in a PRJ. Rates of publication in radiation oncology were nonetheless higher than those previously reported 3 years ago in a cross-sectional analysis of large randomised clinical trials in medicine, although comparisons are hard to make because our work is an observational study in a specific medical field with substantially different inclusion criteria.8

    As our results showed, a large number of interventional phase 3 and 4 trials in radiation oncology have been conducted but have not published their results. Thus, 45% of all evidence collected in our field is seemingly lost forever and raises the question about the extent to which the treatments being offered to patients are really evidence based. This problem of representation does not only concern radiation oncology, but it has also been a distinctive issue in medicine. Even if our findings are consistent with previously observed rates of non-publication in other clinical scenarios, our results add to existing work by showing that this representation problem is an essential feature of interventional phase 3 and 4 trials in radiation oncology, since studies assessing non-publication did not analyse interventional radiotherapy trials separately.9–20

    It is worth noting that trials funded by NIH and industry showed a higher rate of reporting results in the registry than did other trials, even though nearly 65% of NIH-funded and industry-funded trials did not report anything in ClinicalTrials.gov. In addition, there was no statistically significant difference between trials funded by private companies or by NIH. One way to improve these reporting rates would be to apply economic sanctions against sponsors who do not comply with the regulation (such sanctions already exist in the USA by the Food and Drug Administration, although they have rarely been applied); however, economic sanctions against clinical investigators or companies might prevent them from deciding to begin a new trial if sanctions are a possibility. Having fewer trials could be damaging to the health system as a whole, as well as to future patients. A potential solution would be to institute a system whereby if clinical investigators apply for public funding, they have to disclose results of all previously conducted trials; for privately funded trials, results from all previous studies would have to be made available before the new trial could be registered.

    Recently, it has been reported that fewer than half of the trials funded by NIH were published in a PRJ.4 We found a far better publishing rate within the radiation oncology field, since almost 75% of all trials with NIH funding published their results in a PRJ. We found that publication rates for industry-funded trials, on the other hand, were far worse, with 60% of them remaining unpublished. An important consideration is that, leaving aside NIH-funded trials, although this 50% rate of non-publication was higher in industry-funded than in non-industry-funded trials, the differences were not statistically significant. This result is opposite to what has been sometimes reported in the medical literature.21

    We would like also to mention that principal investigators from an American Institution were more likely to report results on ClinicalTrials.gov registry and this might be because the law enforcing the registration and reporting of clinical trial results was an American one.

    A study design limitation should be considered when interpreting these results. Although we allowed a minimum 24 months for publication in a PRJ, but we did not know if this period was long enough for an assessment of publication. Since all trials analysed in this study should have reported results after a 12-month period, we decided to allow for another 12 months for publishing in a PRJ. Phase 3 and 4 clinical trials provide strong evidence and are more easily accepted for publication in a PRJ. Although a 24-month period might not seem sufficient to our purposes, we have to emphasise that these 24 months was a minimum and most trials analysed in our study were given much more time to publish their results, with a median and mean ‘time to publication’ of 60 months.

    It is hard to fathom the reasons underlying this non-publication. One reason might be that we are living in a ‘publish or perish’ era and most clinicians and researchers are willing to participate in a trial without questioning what is really happening with these data globally (there are more ongoing trials than ever before and, as a consequence, it is easy for investigators to participate in multiple trials at the same time; the paradox might rest on the fact that when one of those trials remain unpublished, little attention is paid to it). Another potential reason is publication bias, although it was not possible to assess it in this study. A final possibility is ‘the planning fallacy,22 23: people tend to make terrible predictions of task completion times and what once looked like a feasible trial becomes a longer and much more difficult project to undertake. Given these possibilities, it is important to highlight initiatives such as the 2013 ‘Restoring Invisible and Abandoned Trials’ statement, which was supported by a number of important journals, giving trialists an amnesty of 1 year to publish the results of previously unreported trials.24

    As it has been previously stated in the Methods section, we chose ClinicalTrial.gov registry because this registry represented the most comprehensive source for information about ongoing and completed trials within and outside the USA. However, as large and important as this registry is, many trials conducted in radiotherapy have been registered in other registries. Therefore, it should be taken into account that our dataset did not represent the entire population of interventional phase 3 and 4 trials conducted in radiotherapy. On the other hand, we assumed most phase 3 and 4 trials conducted in radiotherapy would be willing to apply their results on the USA soil and therefore have to comply with the FDAAA 801.

    There are additional limitations concerning our described search method in ClinicalTrials.gov registry. ClinicalTrials.gov search engine allows the user to focus its search through multiple search fields. Searching for the word ‘Radiotherapy’ did not account for all trials conducted in radiotherapy and produced an enormous amount of false-positive results. To account for all these false-negative and false-positive results, we had to extend our search terms further, including radiotherapy-related terms such as ‘radiation oncology’, ‘radiation therapy’ or ‘IMRT’. This strategy broadened the initial search and lowered considerably false-negative results in our final set, but it is likely that not all phase 3 and 4 interventional clinical trials were capture by our search strategy. On the other hand, false-positive results were easily handled performing a double check on every item at our final set.

    In summary, non-publication means poor use of financial resources from funders, host institutions and commissioning bodies. It also means loss of knowledge through hidden data, makes medical practice less evidence based and risks biasing the evidence in important ways. Moreover, it means that a large number of study participants were exposed to the risks of trial participation without the supposed benefits that sharing and publishing of results would offer to future generations of patients. This ethical issue should be at the heart of our current medical practice.

    References

    1. 1.↵
      2. De Angelis C ,
      3. Drazen JM ,
      4. Frizelle FA , et al
      . Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med 2004;351:1250–1.doi:10.1056/NEJMe048225
      OpenUrlCrossRefPubMedWeb of Science
    2. 2.↵
      Section 801 of the Food and Drug Administration Amendments Act. STAT. 904 PUBLIC LAW, 2007;121:110–85.
    3. 3.↵
      World Medical Association. Declaration of Helsinki: ethical principles for medical research involving human subjects. WMA General Assembly, 1–5.
    4. 4.↵
      2. Ross JS ,
      3. Tse T ,
      4. Zarin DA , et al
      . Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ 2012;344:d7292.doi:10.1136/bmj.d7292
      OpenUrlCrossRefPubMed
    5. 5.↵
      R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria, 2016. URL https://www.R-project.org/
    6. 6.↵
      2. Unger JM ,
      3. Barlow WE ,
      4. Ramsey SD , et al
      . The Scientific Impact of Positive and Negative Phase 3 Cancer Clinical Trials. JAMA Oncol 2016;2:875–81.doi:10.1001/jamaoncol.2015.6487
      OpenUrl
    7. 7.↵
      2. Michelson AA ,
      3. Morley EW
      . On the relative motion of the Earth and the luminiferous ether. Am J Sci 1887;34:333–45.doi:10.2475/ajs.s3-34.203.333
      OpenUrl
    8. 8.↵
      2. Jones CW ,
      3. Handler L ,
      4. Crowell KE , et al
      . Non-publication of large randomized clinical trials: cross sectional analysis. BMJ 2013;347:f6104.doi:10.1136/bmj.f6104
      OpenUrlAbstract/FREE Full Text
    9. 9.↵
      2. Turner EH ,
      3. Matthews AM ,
      4. Linardatos E , et al
      . Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252–60.doi:10.1056/NEJMsa065779
      OpenUrlCrossRefPubMedWeb of Science
    10. 10.↵
      2. Turner EH ,
      3. Knoepflmacher D ,
      4. Shapley L
      . Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLoS Med 2012;9:e1001189.doi:10.1371/journal.pmed.1001189
    11. 11.↵
      2. Krzyzanowska MK ,
      3. Pintilie M ,
      4. Tannock IF
      . Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 2003;290:495–501.doi:10.1001/jama.290.4.495
      OpenUrlCrossRefPubMedWeb of Science
    12. 12.↵
      2. Bourgeois FT ,
      3. Murthy S ,
      4. Mandl KD
      . Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med 2010;153:158–66.doi:10.7326/0003-4819-153-3-201008030-00006
      OpenUrlCrossRefPubMedWeb of Science
    13. 13.↵
      2. Ross JS ,
      3. Mulvey GK ,
      4. Hines EM , et al
      . Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 2009;6:e1000144.doi:10.1371/journal.pmed.1000144
      OpenUrlCrossRefPubMed
    14. 14.↵
      2. Chan AW ,
      3. Krleza-Jerić K ,
      4. Schmid I , et al
      . Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004;171:735–40.doi:10.1503/cmaj.1041086
      OpenUrlAbstract/FREE Full Text
    15. 15.↵
      2. von Elm E ,
      3. Röllin A ,
      4. Blümle A , et al
      . Publication and non-publication of clinical trials: longitudinal study of applications submitted to a research ethics committee. Swiss Med Wkly 2008;138:197–203.doi:2008/13/smw-12027
      OpenUrlPubMedWeb of Science
    16. 16.↵
      2. Lee K ,
      3. Bacchetti P ,
      4. Sim I
      . Publication of clinical trials supporting successful new drug applications: a literature analysis. PLoS Med 2008;5:e191.doi:10.1371/journal.pmed.0050191
      OpenUrlCrossRefPubMed
    17. 17.↵
      2. Schmucker C ,
      3. Schell LK ,
      4. Portalupi S , et al
      . Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One 2014;9:e114023.doi:10.1371/journal.pone.0114023
    18. 18.↵
      2. Shiles C ,
      3. Sinclair J
      . It’s never too late to publish an abandoned trial. F1000Res 2015;4:120.doi:10.12688/f1000research.6519.1
      OpenUrl
    19. 19.↵
      2. Brænd AM ,
      3. Straand J ,
      4. Jakobsen RB , et al
      . Publication and non-publication of drug trial results: a 10-year cohort of trials in Norwegian general practice. BMJ Open 2016;6:e010535.doi:10.1136/bmjopen-2015-010535
    20. 20.↵
      2. Tompson AC ,
      3. Petit-Zeman S ,
      4. Goldacre B , et al
      . Getting our house in order: an audit of the registration and publication of clinical trials supported by the National Institute for Health Research Oxford Biomedical Research Centre and the Musculoskeletal Biomedical Research Unit. BMJ Open 2016;6:e009285.doi:10.1136/bmjopen-2015-009285
    21. 21.↵
      2. Chapman SJ ,
      3. Shelton B ,
      4. Mahmood H , et al
      . Discontinuation and non-publication of surgical randomised controlled trials: observational study. BMJ 2014;349:g6870.doi:10.1136/bmj.g6870
      OpenUrlAbstract/FREE Full Text
    22. 22.↵
      2. Kahneman D ,
      3. Tversky A
      . Intuitive prediction: biases and corrective procedures. TIMS Studies in Management Science 1979;12:313–27.
      OpenUrl
    23. 23.↵
      2. Buehler R ,
      3. Griffin D ,
      4. Ross M
      . Exploring the “Planning Fallacy”: Why people underestimate their task completion times. J Pers Soc Psychol 1994;67:366–81.doi:10.1037/0022-3514.67.3.366
      OpenUrlCrossRefWeb of Science
    24. 24.↵
      2. Doshi P ,
      3. Dickersin K ,
      4. Healy D , et al
      . Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 2013;346:f2865.doi:10.1136/bmj.f2865
      OpenUrlFREE Full Text

    Footnotes

    • Contributors JP-A and PG conceptualised and designed the study. JP-A and PG wrote the first draft of the manuscript. IL, EA, JP-A and PG conducted and analysed registry and peer-reviewed journal searches. AP reviewed the manuscript and helped with the interpretation of the data. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement All data used in this research are publicly available from Clinicaltrials.gov, with the inclusion criteria cited in the text.