Article Text
PDF
Abstract
Objectives To establish the rates of publication and reporting of results for interventional clinical trials across Polish academic medical centres (AMCs) completed between 2009 and 2013. We aim also to compare the publication and reporting success between adult and paediatric trials.
Design Cross-sectional study.
Setting AMCs in Poland.
Participants AMCs with interventional trials registered on ClinicalTrials.gov.
Main outcome measure Results reporting on ClinicalTrials.gov and publishing via journal publication.
Results We identified 305 interventional clinical trials registered on ClinicalTrials.gov, completed between 2009 and 2013 and affiliated with at least one AMC. Overall, 243 of the 305 trials (79.7%) had been published as articles or posted their summary results on ClinicalTrials.gov. Results were posted within a year of study completion and/or published within 2 years of study completion for 131 trials (43.0%). Dissemination by both posting and publishing results in a timely manner was achieved by four trials (1.3%).
Conclusions Our cross-sectional analysis revealed that Polish AMCs fail to meet the expectation for timely disseminating the findings of all interventional clinical trials. Delayed dissemination and non-dissemination of trial results negatively affects decisions in healthcare.
- medical ethics
- health policy
- clinical trials
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Our analysis is the first study on the dissemination rates of clinical trial findings in Poland, an important clinical trial spot in the Central and Eastern Europe.
For each included academic medical centre (AMC) trial, search for a publication was done independently by two researchers in a 4-step process involving ClinicalTrials.gov, PubMed, Google Scholar and Web of Science.
We show the dissemination rates following both primary completion date and completion date.
Included studies may represent only a fraction of clinical trials performed in Poland—our results may be underestimated.
For identifying studies conducted in Polish AMCs we relied on the names of AMCs and teaching hospitals provided on the ClinicalTrials.gov website.
Background
The results of completed clinical trials are crucial for decision-making in evidence-based medicine.1–5 They also inform patients,5 6 clinicians, researchers, policy-makers,5 impact future research5–7 and play an important role in health technology assessment.8 9 Non-dissemination or delayed dissemination of trial findings not only negatively affects decisions in healthcare, but is also unethical as the results of all research involving human subjects must be publicly available regardless of whether they are considered positive or negative.6 10 11 Not reporting trials’ results is unfair to trial participants who often put themselves at risk and burden to contribute to scientific knowledge. Paediatric trials are particularly challenging as they recruit children, thus additional protections are required to avoid their exploitation in the research.12–14
The statement on public disclosure of results from clinical trials published in 2015 by WHO defines reporting timeframes and calls for publication of the results of still unpublished trials. The key outcomes of clinical trials are to be posted in the results section of the clinical trial registry within 12 months of primary study completion (the last visit of the last subject for collection of data on the primary outcome) and the main findings of clinical trials should be published in a peer-reviewed journal (preferably free to access) at most within 24 months of study completion.15 However, compliance with this and similar requirements16–19 has been poor.20 21 Still about 50% of completed trials remain unreported22 or delay sharing their results.20 In the European Union, 89% of trials completed between 2004 and 2018 sponsored by academic institutions were not reported within a year of the trial’s end20 which indicates that the academic community failed to meet requirements of the European Commission guideline17 and the Clinical Trial Regulation EU No.536/2014.19
Even though the problem of under-reporting has been widely discussed and many attempts have been made to urge dissemination of trial findings at the international level, the concept of benchmarking the reporting of clinical trials across academic institutions at the country level is emerging.21 23–25
In this study, we aim to establish the rates of publication and reporting of results for interventional clinical trials across Polish academic medical centres (AMCs) completed between 2009 and 2013. We also want to compare the publication and reporting success between adult and paediatric trials.
Methods
Our protocol was prospectively posted on the Open Science Framework (OSF) website (https://osf.io/w4pfj/). We adapted methods used previously in similar studies.21 23
Retrieval of trials
Interventional clinical trials conducted across Polish AMCs completed between 2009 and 2013 were identified in ClinicalTrials.gov database. The complete Aggregate Analysis of ClinicalTrials.gov (AACT) dataset was downloaded from http://aact.ctti-clinicaltrials.org/ on 10 August 2018. We performed the search of ClinicalTrials.gov instead of EU Clinical Trials Register as our preliminary search for the same inclusion criteria resulted in 3505 records identified at the ClinicalTrials.gov and only 1312 at the EU Clinical Trials Register. Moreover, EU Clinical Trials Register unfortunately do not allow to perform a search including primary completion date (CD) and CD of registered trials.
Inclusion and exclusion criteria
We define an AMC as a medical university or an academic medical institution with clustered teaching hospital/s. There are nine medical universities, three universities with a medical college and one Center of Postgraduate Medical Education, all affiliated with at least one teaching hospital in Poland. The search terms used for identification of clinical trials across 13 Polish AMCs are available on the OSF (https://osf.io/bsp3r/).
An R script was used to combine datasets and restrict our resulting dataset to studies with a CD between 2009 and 2013 as well as to exclude observational studies. Only studies with the status ‘Completed’, ‘Terminated’, ‘Suspended’ or ‘Unknown’ were included. After the automatic filtering for the AMCs and city names, the correct assignment of studies to the AMCs was verified manually by two researchers independently (KS, LZ).
A given trial was assigned to an AMC if the AMC was either mentioned as the responsible party, lead sponsor or collaborator or if the principal investigator, study chair or study director was affiliated with the AMC. The AMC was then considered a ‘lead’ contributor in these trials. If AMC was mentioned only as a facility or a study was conducted in an academic hospital or the principal investigator, study chair or study director was affiliated only with an academic hospital without the name of the AMC, then AMC was considered a ‘facility’ contributor in these trials. One trial could be counted for multiple AMCs. The flow chart presenting the trial selection process with reasons for exclusion is shown in online supplementary figure 1.
Supplemental material
Data extraction
An R script was used to extract the trial characteristics from the AACT dataset. For included studies researchers checked ClinicalTrials.gov whether summary results were posted or submitted as of 4 December 2018 (KS, MTW).
Publication search
For each of the included studies, search for a publication was done independently by two researchers (KS, MTW) in a 4-step process between 3 December 2018 and 7 February 2019 on ClinicalTrials.gov, PubMed, Google Scholar and Web of Science (figure 1). We defined a publication as an article with at least 400 words. In case of multiple results publications, we chose the earliest publication. When a study contained the results of two or more trials but reported the results of each trial separately, it was included. Abstracts, study design publications without results, reviews and other background literature were excluded. If there was a disagreement whether to include or exclude a publication, the third person (an arbiter, MW) was involved. The flow chart showing the publication search with reasons for exclusion is presented in online supplementary figure 2. When a publication was identified, we extracted first publication date, PubMed ID and DOI (if applicable). Only if all searches stayed without results, the study was characterised as ‘no publication found’.
Supplemental material
Publication search strategy.
Paediatric trials
We classified a study as paediatric when all or most participants (over 50%) were less than 18 years. If the study enrolled both adult and paediatric participants but it was impossible to determine whether the majority of study participants were above or under 18 years of age, we classified the trial as mixed population study. Unclear studies lacked data on average/median age of participants, thus it was impossible to assign them to either of these groups. The information on participants’ age was searched on: ClinicalTrials.gov website, in publications or in other sources such as other registries (eg, European Clinical Trials Register) or sponsor websites if any additional study identification numbers were provided on ClinicalTrials.gov.
Patient and public involvement
No patient involved.
Statistical analysis
We used a logistic regression analysis to identify explanatory variables with a possible effect on the timely publication rates (online supplementary material). We started with a univariate model, testing all variables individually to identify the variable that leads to the largest increase in the log-likelihood. Then the regression model was built stepwise by including the variable with the largest log-likelihood increase in each step until there was no more variable that could be added to substantially increase the log-likelihood. There were however no strict rules for variable inclusion, as this was an exploratory analysis.
Supplemental material
Results
We identified 1267 interventional clinical trials registered on ClinicalTrials.gov with a CD between 2009 and 2013, conducted in a city with an AMC. Of these, we excluded 962 mainly because there was no name of the AMC or the academic hospital, leaving 305 trials across 13 academic institutions enrolling 119 490 anticipated participants. Among the trials, 259 (84.9%) were randomised, 248 (81.3%) were adult trials, 241 (79.0%) were multicentre trials, 223 (73.1%) were industry-sponsored, 209 (68.5%) tested drugs and 75 (24.6%) were lead trials (table 1). For more information, see table 1.
Characteristics of all and lead trials across 13 Polish academic medical centres completed between 2009 and 2013
Results reporting
Of the 305 trials completed between 2009 and 2013, 120 (39.3%) posted summary results on ClinicalTrials.gov and 23 (7.5%) did it within 12 months after trial CD (table 2). Still, more than 5 years after all the trials ended, 175 (57.4%) have not posted their results. Among 75 lead trials, 5 (6.7%) posted results on the registry website, leaving 67 (89.3%) trials without results posted or submitted.
Dissemination of clinical trial results following trial CD for all, lead, paediatric and adult trials
Publication rates
Of the 305 trials, 218 (71.5%) had published results as a journal publication as of 7 February 2019 (table 2). Results were published before trial ended or within 2 years of trial completion for 123 trials (40.3%). Furthermore, 57 trials (76.0%), in which Polish AMC took the lead published results and 39 (52.0%) did it before trial completed or within 2 years of study end.
Overall dissemination
Overall, 243 of the 305 trials (79.7%) had been published or posted their summary results (table 2). Results were posted within a year of study completion and/or published within 2 years of study completion for 131 trials (43.0%). Dissemination by both posting summary results on the registry website within a year of study CD and publication within 2 years of study CD was achieved for 15 trials (4.9%). The percentage of non-disseminated trials over time is presented in online supplementary figure 3.
Supplemental material
Paediatric versus adult studies
There was no significant difference in dissemination of findings in paediatric trials compared to adult trials, 73.6% (95% CI 59.7% to 84.7%) vs 81.0% (95% CI 75.6% to 85.7%), p=0.22. Among paediatric trials, 21 (39.6%) posted summary results on ClinicalTrials.gov within a year of study completion and/or published them within 2 years of CD (see table 2). Among adult trials, the percentage of posted or published results was 43.5%. There was no significant difference in posting summary results within 12 months of CD comparing adult and paediatric trials 7.3% (95% CI 4.4% to 11.2%) vs 9.4% (95% CI 3.1% to 20.7%), p=0.80; neither in disseminating results by both posting them on the website and publication 4.4% (95% CI 2.2% to 7.8%) for adult vs 7.5% (95% CI 0.4% to 14.7%) for paediatric trials, p=0.55.
Timely dissemination
In table 3, we presented results in the light of the WHO criteria of timely dissemination.15 Of the 305 investigational clinical trials completed between 2009 and 2013 across Polish AMCs, four trials (1.3%) would meet both criteria to post summary result within 12 months of primary study completion and publish them as a journal publication within 24 months of study completion. For lead trials, neither of 75 trials would meet these both criteria.
Dissemination of clinical trial results following trial primary completion date for all, lead, paediatric and adult trials
Dissemination rates between academic institutions
Rates of results reporting within 12 months of PCD ranged from 0.0% (0/60) to 16.0% (4/25) across AMCs (table 4). The proportion of clinical trials with results published within 24 months of PCD ranged from 0.0% (0/1) to 58.3% (14/24). Rates of trials meeting both criteria of timely dissemination ranged from 0.0% (0/60) to 8.0% (2/25). The overall rate of dissemination across institutions ranged from 72.1% (31/43) to 100% (9/9).
Dissemination of clinical trial results following trial PCD and CD across 13 academic medical centres
Subgroup analyses
Only the variable ‘phase’ lead to a strong reduction in the log-likelihood in the logistic regression model (see online supplementary material). The effect of the ‘phase’ variable on timely reporting was as the following: while the early phase trials (especially phase 2) had low timely publication rates (24% for phase 2), the later phase trial publication rates were on average above 50% (55% for phase 3, 62% for phase 4).
Discussion
Our cross-sectional analysis of all Polish AMCs revealed low performance rates for timely disseminating the findings of interventional clinical trials completed between 2009 and 2013. Only 36.1% (110/305) of trials were published as a journal article within 2 years of primary study completion or reported results on ClinicalTrials.gov within a year of PCD. Still 20.3% (62/305) of trials remain without their results disseminated. Delayed dissemination and non-dissemination of trial results, among others, negatively affect decisions in healthcare and leave recruited participants without information whether the therapy actually works or not.26 Posting summary results on the registry website allows the research findings to be accessible to everyone. Moreover, the results are presented in the same format, which reduces reporting bias and limitations arising from journals’ various requirements on maximum number of tables or article length. Despite the fact, that sharing study results via ClinicalTrials.gov is not as complicated and time consuming as publishing a peer-reviewed journal article, our analysis showed that the rates of timely reporting of summary results in the clinical trial registry had been particularly low between Polish AMCs, ranging from 0.0% across 4 Polish AMCs to 16.0% (Ludwik Rydygier Collegium Medicum in Bydgoszcz). Overall, 3.9% (12/305) of all trials and 1.3% (1/75) of lead trials had met the criterion of posting the trial findings in the results section of the clinical trial registry within 12 months of primary study completion.15 16 18 Other research confirms such low reporting rates.20–25 This may be due to researchers’ anxiety that posting summary results in the results section on ClinicalTrials.gov will deprive them the chance to publish the same results in the journal. Another possible reason is that researchers in academia are required to publish in peer-reviewed journals and not necessarily in registries and databases. These could be improved by various initiatives at AMCs motivating researchers to disseminate results more broadly. Another solution that could increase reporting in the clinical trial registry would be that all journals do not accept for publication unregistered trials and trials without their results at least submitted. Moreover, study sponsors should enforce timely results reporting.27 When a principal investigator applies for new funding, they may be asked to provide a list of all previous trials and their reporting status.15 The motivation to disseminate the study results may be also the effect of the extensive actions such as public debate, training, conferences, pressure from the scientific community and other. AMCs may also formulate clear and efficient rules that will apply to the persons responsible for disseminating the results of the research.
Despite the broad consensus that all research results should be disseminated, a suggested scope and suggested dissemination time varies.11 13 15–19 Thus in our study we provided the rates of results dissemination following both the study’s primary CD (PCD) and CD. We found that rates of reporting of results following CD are as low as the rates of results reporting following PCD.
We decided to analyse the dissemination results in paediatric trials separately because of a special status of the paediatric population. These trials are required to offer an additional protection by imposing a strict risk threshold or allowing paediatric research only when there is a prospect of direct benefit for the participants. The ethical justification of trials with vulnerable populations hinges also on a social value they are able to offer, that is, the ability to provide generalisable scientific and medical knowledge. If the results of a trial are not available publicly, the trial has no way of providing generalisable knowledge or changing clinical practice, thus robbing the trial of its social value component. Our analysis showed that there was no significant difference in dissemination of results between paediatric and adult trials.
Our analysis has several limitations. First, we searched ClinicalTrials.gov instead of EU Clinical Trials Register as it contained more registered clinical trials conducted in Poland in a specific search period. Thus, we possibly not included a fraction of trials registered only at the EU Clinical Trials Register. Second, our results may be underestimated as in 2009–2013 about 450 new clinical trials were conducted in Poland annually for both academic and non-academic sites, giving a total of 2243 new clinical trials over 5 years.28 We captured 1267 completed trials and excluded almost 76% of them mainly because the name of the research site was not provided (also see online supplementary figure 1). Third, we relied on the recruitment status found in the database which might not had been updated, meaning that some active or recruiting trials could in fact be completed.29 We also relied on the names of AMCs and teaching hospitals provided on the website. We added the city name as the additional search criterion to include all studies conducted by AMCs to avoid misspellings of names and shortcuts on ClinicalTrials.gov. We classified studies as lead trials only when it was clearly reported that AMC was a lead contributor in the trial. If there was a name of the teaching hospital without the AMC name, the trial was then considered as a facility trial. Fourth, we did not asses the accuracy of the dates of study start and completion in the ClinicalTrials.gov database. In some clinical trials the reporting of results and publication dates preceded the PCD and CD. We counted those results as reported within a year of study completion which may overestimate our results. Fifth, we defined publication as an article with at least 400 words. Such publication may be only a hint that such a study was conducted and may not demonstrate full methodology and results. Nevertheless, every journal has different requirements, word and table limits; therefore, it is challenging to present entire study results of large multicentre trials. There is also a well-known problem of the selective results reporting.30 Sixth, despite extensive publication search by two researchers independently we could have missed some relevant publications. Seventh, we did not assess the quality of results reporting and we did not compare protocols and outcomes, we also did not assess the consistency of results reporting in the ClinicalTrials.gov database compared to those published in journal publications. Finally, we could follow trials completed in 2009 for more than 9 years whereas trials competed in 2013 we could follow only for 5 years.
Conclusion
Our cross-sectional analysis revealed that Polish AMCs fail to meet the expectation for timely disseminating the findings of all interventional clinical trials. Despite the fact, that eventually about 80% trial results were disseminated, some of them are hard to find, for example, published in non-indexed journals or not linked clearly with National Clinical Trial (NCT) identifier number. Overall dissemination rates outperformed the rates in similar studies in the USA and Germany.19 23 Nevertheless, timely reporting on the registry still remains very poor and the dissemination by both posting and publishing results was achieved by four trials (1.3%). Our findings illustrate that after quite successful efforts in promoting a timely dissemination among industry, it is time to urge AMCs and other public institutions to fulfil this important ethical requirement.
Footnotes
Twitter @KStrzebonska, @m_waligora
Contributors MW, DS, SW developed the concept for the research study. NR retrieved trials and performed statistical analyses. KS, MTW, LZ performed the manual review. KS drafted the manuscript and was responsible for coordination of all aspects of the work. All the authors critically revised the manuscript for important intellectual content and approved the final manuscript.
Funding Funding was provided by the National Science Center, Poland UMO-2015/18/HS1/00354 (KS, MTW, MW).
Competing interests KS, MTW and MW were funded by Narodowe Centrum Nauki. DS and MW report personal fees from Advisory Bioethics Council, Sanofi outside the submitted work.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available in a public, open access repository. All datasheets with raw data are available on the Open Science Framework (https://osf.io/8xz29/). Additional data are available upon request.