Objectives Transparent reporting of trials is necessary to assess their internal and external validity. Currently, little is known about the quality of reporting in antibiotics trials. Our study investigates the reporting of adverse events, conflicts of interest and funding information in trials of penicillins, cephalosporins and macrolides.
Design A secondary analysis of trials included in a convenience sample of three systematic reviews.
Methods All randomised controlled trials included in the systematic reviews were included, although duplicates were removed. Eligible trials compared the specified antibiotics to placebo, for any indication. Author pairs independently extracted the data on reporting of adverse events from parent reviews, and data on funding and conflict of interest information from the trial reports. We calculated the overall proportion of trials reporting adverse events, conflict of interest information and funding information, and their proportion before and after the publication of the Consolidated Standards of Reporting Trials (CONSORT) 2001 Statement.
Results We included 432 trials. Overall, 62% of trials reported adverse events of any kind, although reporting of deaths or antibiotic resistance was less frequent (20% and 37%, respectively). Conflict-of-interest information was provided in 26% of the trials, and funding information was provided in 66% of the trials. There was no significant difference in reporting of adverse events before and after the publication of CONSORT 2001 Statement (62% vs 62%, p=0.92). Conflict of interest statements were provided more frequently (2% vs 55%, p<0.001) and conflict was present more often (0% vs 14%, p<0.001). There was no difference in the provision of the information about trial funding before (62%) and after (70%) CONSORT 2001 publication.
Conclusions Information about adverse events, conflict of interest and funding, remains under-reported in trials of antibiotics.
- adverse events
- primary care
- public health
Data availability statement
The full list of RCTs analysed is provided in online supplemental appendix 1. Further data are available from the authors on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
We analysed the reporting of adverse events, conflicts of interests and funding information in published antibiotic trials, overall, and before and after publication of the Consolidated Standards of Reporting Trials 2001 Statement.
The dataset consists of 432 randomised controlled trials of antibiotics, conducted across a period of 50 years, without language or publication restrictions.
Because the data set are limited to three classes of antibiotics (macrolides, cephalosporins and penicillins), the conclusions may not be generalisable to other classes of antibiotics.
Randomised controlled trials (RCTs) are considered the gold standard in evaluating the effectiveness of new healthcare interventions.1 2 However, RCTs have frequently been criticised for poor reporting of harms or adverse events.3 4 Concerns have also been raised about the inadequacy of reporting of conflicts of interests and funding information, as those may influence—or appear to influence—trial design, its conduct, reporting of results, their interpretation and conclusions,5–12 in turn jeopardising public trust.13
A clear and transparent reporting of studies is therefore needed to assess both their internal and external validity.14 To improve the quality of RCT reporting, the Consolidated Standards of Reporting Trials (CONSORT) Statement was first published in 1996,15 with two revisions, in 200116 and in 2010,14 to reflect the evolving consensus on the importance of reporting of elements of RCTs.
The 1996 CONSORT Statement did not include items requiring the reporting of adverse events, funding or conflicts of interest.15 The 2001 update included an addition of an adverse events item on the reporting checklist, recommending that ‘all important adverse events or side effects in each intervention group’ be reported.16 Although neither funding nor conflict of interest (COI) information was not listed among the reporting items, information about sources of funding of the trial was identified in the article as highly desirable, and its inclusion in an RCT report was encouraged.16
Although several studies have assessed the quality of reporting funding resources and conflicts of interests,17–19 little is known about the quality of reporting of antibiotics trials. In this study, we, therefore, investigated the reporting in trials of three commonly used antibiotic classes: penicillins, cephalosporins and macrolides. We focused on the reporting of adverse events, conflicts of interests and funding information, both overall, and before and after the CONSORT 2001 Statement.
This is a secondary analysis of RCTs included in a convenience sample of three systematic reviews conducted by our research group. We choose these reviews as their main objective was to analyse the adverse events associated with the use of the most commonly used antibiotics. These systematic reviews included trials of penicillin, cephalosporin, or macrolides antibiotics, that compared each class alone against a placebo arm, for any indication (excluding trials with combined therapy).20–22 The cephalosporin review protocol was withdrawn as the review authors did not finish it during the expected time frame, along with some administrative issues just before publication. However, as three of this study authors were coauthors of the review, they retrieved all the RCTs included in this review (final search date was January 2019). Data were extracted from the review and the authors used it for this study (eg, reporting of adverse events and resistant data).
All RCTs included in the systematic reviews were eligible for this study, although duplicates (ie, the same RCT included in more than one systematic review) were excluded. The Participant, Intervention, Comparator and Outcome characteristics of the included systematic reviews and trials are summarised in table 1.
Author pairs (KY, RN, AMS, MB and JB) independently extracted and entered data into a prepiloted data extraction form. All discrepancies were resolved by reference to a third author. Data on reporting of adverse events were extracted from the systematic reviews, which categorised adverse events as: reported, not reported or unclear. For the present analysis, we considered ‘unclear’ reporting of adverse events as not reporting.
Data on COI and funding statements were extracted directly from the trial reports. As defined per the International Committee of Medical Journal Editors (ICMJE), COIs included: current or former board membership, industry employment, consultancy work, grants (financial or in-kind), royalties, stock, travel reimbursement or other relations with relevant pharmaceutical companies.23 Funding statements included such information—whether financial or in kind (eg, supply of drug)—found in the article. Conventional funding statements (eg, ‘this study was funded by a grant from an XYZ organisation’) were included; statements about in-kind provision of pharmaceuticals for the trial by its manufacturer, for example, were also included.
Data were analysed using Microsoft Excel (Microsoft, Redmond, Washington, USA) and descriptive statistics were calculated. We calculated the proportion of trials reporting or not reporting adverse events, which: (1) provided a COI statement; (2) identified a present COI; (3) provided a funding statement; (4) stated whether the trial was funded; (5) provided information on the source of trial funding and (6) identified the funder’s role in the trial.
We calculated the proportion of trials up to and including 2001 (the year of publication of CONSORT 2001), and the proportion of trials published from 2002, which reported: (1) any adverse events (other than deaths or antibiotic resistance); (2) deaths; (3) antibiotic resistance; (4) COI statement; (5) identified an existing COI among the trial authors and (6) provided information on the source of trial funding. We chose CONSORT 2001 rather than the ICMJE disclosure form publication in 2009 as the cut-point for the before and after analysis, as the former is specific to issues in reporting of RCTs, while the latter is study type independent.
The χ2 test was used to test for significant differences in reporting before and after CONSORT Statement.
Patient and public involvement
Neither patients nor public were directly involved in the conduct or writing of this work.
Summary of reporting characteristics
Overall, 62% of the 432 RCTs reported adverse events; reporting was less common specifically for deaths (20%) and antibiotic resistance (37%). COI statement was provided by 26% of all trials, and a COI was present in 7%. Funding information was provided in 66% of trials, and most commonly, the funding was provided by the industry (43%). Statement about the funder’s role in the trial was provided by nearly half (49%) of all trials (table 2).
Among trials that did report any type of adverse events (n=344), 27% provided a COI statement and COI was present in 7%. Most trials provided a funding statement (69%), and 66% were funded—most frequently by the industry (45%). Just over half (52%) of the trials indicated the funder’s role in the trial (table 2).
Of the trials that did not report any type of adverse events (n=98), 22% provided a COI statement, and COI was present in 4% of the trials. Funding statement was provided by 53% of trials, and 51% of the trials were funded, most commonly by non-industry (36%). Thirty-four per cent of trials provided information about the funder’s role in the trial. (table 2)
Reporting before and after CONSORT 2001
The reporting of adverse events—overall, or for deaths or antibiotic resistance specifically—did not change after the publication of the CONSORT 2001 Statement (table 3). Provision of the COI statement increased from 2% to 55% of the trials after 2001 (p<0.001), and COIs became more pervasive, increasing from 0% to 14% after 2001 (p<0.001). There was a non-significant increase in provision of the funding statement, from 62% to 70% after 2001 (p=0.077) (table 3).
We found that overall, nearly 40% of antibiotics trials did not report any adverse events, and there was no change in reporting of adverse events before and after the publication of the CONSORT Statement in 2001. Reporting of antibiotic resistance slightly (although non-significantly) decreased. Nearly three-quarters of the trials failed to provide the COI statement (74%). While 45% of the included trials were industry funded, only 7% reported a COI. Although there was a significant increase in its provision after the publication of the CONSORT Statement. Funding statements were provided by two-thirds of the trials overall, although no increase was observed after CONSORT.
Our findings are consistent with analyses of COI reporting in other areas. A study of 444 RCTs of surgical interventions found, similarly, that 79% of trials did not provide a COI statement, and there was a trend towards increase on its provision from 0% (of the trials conducted between 1985 and 1994) to 33% (of the trials conducted between 2005 and 2014).10 Fewer than half of 848 studies in supportive and palliative oncology provided COI information, although there was an increase in reporting, from 39% of studies in 2004 to 55% in 2009.7 However, funding information was provided in only 41% of the studies. An analysis of 374 studies in critical care also showed a trend towards increased reporting of conflicts of interest (from 4% of studies in 2001 to 84% in 2016),6 and in reporting of funding (from 17% to 59%, respectively). Why funding information provision in supportive and palliative oncology, or critical care, is lower than in trials of antibiotics (66%) is unclear.
Our finding of poor reporting of death is consistent with the results of a similar study that analysed a random sample of trials.24 This study of 500 randomly selected records in Clinical Trial registry found, similarly, that only 123 records (25%) reported the number of deaths.
We are unaware of previous analyses of trends around the reporting of adverse events of antibiotics over time. However, their continued under-reporting is particularly concerning, in light of the estimates that deaths attributable to antibiotic resistance may rise to 10 million per annum by 2050.25
Our study has several limitations. We analysed only the published trials previously included in our convenience sample of three systematic reviews, potentially limiting generalisability to other classes of antibiotics and other non-published trials. However, the included RCTs trialled antibiotics for a large variety of indications and in a wide range of settings. We also relied on the original systematic reviews’ reporting of adverse events in the original trials, which may have introduced errors, although two of the present authors (CDM and AMS) were also involved in conducting those systematic reviews. Although it would have been interesting to analyse the COI and funding reporting by individual journals, it was impossible to do so as their policies change over time and it would have been difficult to identify which policy was in effect at each trial’s publication date. Finally, the results from the macrolides trials (n=183) may bias the results towards better reporting of COI and funding as they are a newer class of antibiotics, compared with cephalosporins and penicillin. The strengths include the large number of included trials (n=432) covering three of the most commonly prescribed antibiotic classes, and a 50-year period covered by the trials (1969–2018). The trials included in the analysis were not restricted by language or publication (eg, lower vs higher impact factor journals). Extraction of data on COI and funding from the included RCTs was conducted independently by two pairs of authors, and accuracy was checked by a third author.
Our study suggests that information about adverse events, conflicts of interest and funding remains under-reported in antibiotics trials. The lack of change in reporting of adverse events, and the small decrease in reporting of antibiotic resistance are concerning, although may shift in the coming years by adequately reporting the protocols of clinical trials by using Standard Protocol Items for Randomised Trials checklist,26 27 and by the adoption of checklists specific to reporting antibiotic resistance in prospective studies of antibiotic use, such as the checklist developed by several authors of the present study.28 Pervasiveness of conflicts of interest and funding in trials of antibiotics is unsurprising, as RCTs are expensive to conduct, and public funding sources are limited. However, their presence needs to be transparently reported, so that physicians, patients and other stakeholders can consider that information in assessing the evidence. An additional benefit would be a positive flow-on effect on systematic reviews of primary studies, which currently infrequently report the funding sources for included RCTs,11 despite requirement that this information be provided for studies included in the review.29 Much work remains to be done by research funders, journal editors, clinical trial registries and other research outlets to require and clearly convey this information to consumers of research, although the trends towards increased reporting over time in antibiotics and other areas are encouraging.
Data availability statement
The full list of RCTs analysed is provided in online supplemental appendix 1. Further data are available from the authors on reasonable request.
As the data source for this analysis consists of published trials and is publicly available, we did not seek human research ethics approval.
Contributors AMS and CDM conceived and designed the project. MB, JB, RN, KY and AMS extracted the data. MJ conducted the analyses. All authors (MB, MJ, JB, RN, KY, CDM and AMS) contributed to writing and critical revisions of drafts of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.