Article Text
PDF
Abstract
Objective To determine abstracts’ adherence to the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) statement and to explore the factors associated with reporting quality.
Design An observational study.
Setting Abstracts of randomised controlled trials published between 2010 and 2019, found searching the MEDLINE database.
Participants A total of 451 abstracts of the clinical trials on Helicobacter pylori infections were included.
Primary and secondary outcome measures Abstracts’ reporting quality was determined by assessing their adherence to 17-item CONSORT-A checklist, with overall score being calculated as the sum of items that were adequately reported for each abstract. Additional factors that might influence the reporting quality of the abstracts were analysed, with univariate and multivariate linear regression used to determine how those factors influenced the overall reporting quality.
Results Included abstracts had an overall median quality score of 8/17 (IQR 7–9). Large proportions of abstracts adequately reported interventions, participants, objectives, numbers randomised and conclusions (97.1, 99.3, 89.1. 94.7 and 98.4% of abstracts, respectively). Trial design, randomisation, blinding and funding were severely under-reported with only 8.0, 2.7, 11.0 and 2.0% of abstracts reporting each item. Overall quality scores for H. pylori abstracts were higher in association with CONSORT-A endorsement (B=5.698; 95% CI 1.781 to 9.615), pharmacological interventions (B=4.063; 95% CI 0.224 to 7.902), multicentre settings (B=5.057; 95% CI 2.370 to 7.743), higher numbers of participants (B=3.607; 95% CI 1.272 to 5.942), hospital settings (B=4.827; 95% CI 1.753 to 7.901) and longer abstracts (B=3.878; 95% CI 0.787 to 6.969 for abstracts with 251–300 words and B=7.404; 95% CI 3.930 to 10.878 for abstracts with more than 300 words).
Conclusions The overall reporting quality of abstracts was inadequate. The endorsement of CONSORT-A guidelines by more journals might improve the standards of reporting.
- gastroenterology
- medical education & training
- statistics & research methods
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study investigating the reporting quality of randomised controlled trial abstracts regarding Helicobacter pylori treatment, a trending topic in gastroenterology research.
Study period included a relatively broad time frame and a large sample size in which every eligible abstract was included.
Univariate and multivariate linear regression were used to determine which additional factors had influenced the reporting quality.
Only abstracts published after the Consolidated Standards of Reporting Trials for Abstracts statement and indexed in MEDLINE were included in the analysis which could limit the findings.
Introduction
Recent epidemiological studies report that Helicobacter pylori infects up to 50% of the population in highly industrialised nations and up to 80% of people in less-developed countries.1 H. pylori infection is highly associated with gastrointestinal diseases, including gastric inflammation, peptic ulcer disease, gastric carcinoma and gastric mucosa-associated lymphoid-tissue lymphoma.2–5 As a result of the ever changing epidemiological conditions (eg, immigration and climate changes), pathogenicity, pathogen evolution, population genetics, changing antibiotic resistance and newly discovered knowledge relating to the eradication of pathogen, the treatment of H. pylori is a constantly changing and challenging task which requires regular reassessment.6 7 Over the last 30 years, numerous national and international recommendations and guidelines on the diagnosis and treatment of H. pylori infection have been issued based on the best current available evidence at the time.8–10 The amount of research about H. pylori and its eradication is growing, with new clinical trials bringing potential new advances in this field of medicine. To improve the visibility and critical appraisal of the new research findings, it is imperative to report adequately those trials, so those of the highest quality could be rapidly and successfully used in practice.
The Consolidated Standards of Reporting Trials (CONSORT) encompasses various initiatives developed to alleviate the issues arising from inadequate reporting of data from randomised controlled trials (RCTs). The main product of the CONSORT is the CONSORT statement, an evidence-based, minimum set of recommendations for reporting data from RCTs.11 12 It offers a way for authors to organise reports of trial findings, facilitating their complete and transparent reporting, and aiding their critical analysis and interpretation.13 An addition to the CONSORT statement was developed and it gives a list of essential elements that authors should include when describing the main outcomes of a randomised trial in a journal or conference abstract—CONSORT-A.14 Those elements include recognising study as an RCT to allow indexing in databases, as well as description of the trials design, with contact details of a corresponding author to ask for additional information or clarification. Methods’ elements describe eligibility criteria, setting, intervention, objective, outcome measures, allocation and randomisation of the participants and whether the blinding was used. Those data should aid the determination of validity and applicability of the trial results for the readers. Results’ items allow the description of the validity and the quality of the trial, as well as to describe the findings. They include status of the trial, numbers of participants randomised and analysed in each group, summary of results for those groups, including any harms done by the interventions. Final two items are trials registration, to help curb the selective reporting, and source of funding to assess the potential bias of results towards sponsors.14
The abstract of published research enables communication from scientists towards clinicians and improves the translation of scientific research into clinical practice. Moreover, abstracts are the most likely part of articles to be read, and most often the only part that clinicians read because abstracts allow clinicians to quickly peruse articles for applicability to their own patients. If abstracts are of high quality, they provide clinicians with information about articles’ methodology and results. Furthermore, high-quality abstracts allow clinicians to accurately assess if the published research is relevant to their field or could improve their practice. Therefore, scientists should increase the quality of abstracts reporting data from RCTs in order to enable efficient article screening by clinicians. It should also be noted that busy clinicians lack the time to read entire articles. Additionally, they do not have the skills for the critical evaluation of articles so they often subscribe to abstracting services from which they get information. Furthermore, full texts are frequently unavailable outside of subscription services.15–17
As the treatment of H. pylori becomes ever more challenging, more research, including RCTs, are needed to provide better understanding of the disease. Recent studies had shown more obstacles to successful management of the disease, such as insufficient knowledge of H. pylori guidelines among primary care physician and medical students.18 Additional problem was a poor accordance between treatment regimens and drug pack sizes.19 Those challenges could contribute to antimicrobial resistance, affect adherence, lead to more medication errors and worse outcomes for the patients.18 19 Another obstacle to improved H. pylori eradication could be the poor integration of the latest research into the practice due to insufficient reporting quality. So far, the quality of published RCT abstracts in the field of gastroenterology, the cornerstone of evidence based medicine practice, remained unknown. Therefore, the aim of the study was to assess the abstracts' adherence to the CONSORT-A statement and to explore the factors associated with reporting quality.
Methods
Search strategy and study selection
An observational study of RCT abstracts indexed in MEDLINE/PubMed about the topic of H. pylori infections relevant to the field of gastroenterology was conducted. RCTs were included if they had a control group with random allocation of the participants. RCTs were included regardless of their design type. The included studies compared a treatment with placebo, an active treatment or no treatment. Studies were not excluded due to the outcome measures used. Studies with comorbid diagnoses were also not excluded. Abstracts of non-clinical trials, observational studies with no intervention, follow-up studies of previously published trials, reviews, protocols, letters to editors and comments were excluded. Abstracts describing trials with exclusively H. pylori negative patients were excluded. Studies with no relevance to H. pylori infection in field of gastroenterology were excluded (eg, oral H. pylori infections, peri-implantitis etc.). Only studies published in years including and between 2010 and 2019 were included. We chose the year 2010 as the start date of our search so that the authors of RCTs would have had 2 years to incorporate CONSORT for abstracts guidelines, as those were published in 2008. The following search strategy was used on MEDLINE/PubMed: (“helicobacter pylori”[MeSH Terms] OR (“helicobacter”[All Fields] AND “pylori”[All Fields]) OR “helicobacter pylori”[All Fields]) AND ((randomizedcontrolledtrial[Filter]) (AND (2010:2019[pdat])). The full list of the extracted abstracts is available on request to the authors.
Data extraction
The reporting quality of the included abstracts was determined by assessing their adherence to the 17-item CONSORT-A checklist. Each item was given a binary grade (0 or 1) depending on whether the item was adequately reported or not.20 The overall reporting quality of an abstract was determined by calculating an overall score, a method which was adapted from previous research.21–24 The overall reporting quality score was defined as the number of items achieved for each abstract, on a scale from 0 to 17. The score was also presented as a percentage of the number of items achieved in regard to the total number of items.
We have also included data about additional factors as potential predictors of reporting quality. Included variables were journals’ impact factor and quartile, study sample size (<100 or≥100 participants included), pharmacological intervention, study centres (single or multicentre), significance of the results (whether the results favoured the experimental or control treatment), presence of the CONSORT statement’s endorsement on journal websites, funding by industry, hospital setting, number of authors, abstract structure and abstracts’ length defined as their word count.20 21 25 The impact factor and quartile were identified according to the Thomson Reuters Journal Citation Report of the year in which the study was published. The significance of the results was considered for the primary outcome measure, indicated by p values (p<0.05). The result was considered significant when the primary outcome results favoured the experimental group. In case of a non-inferiority trial design, no statistical difference in comparison to the control group was considered as a significant result. In case of multiple outcome measures, result was considered significant if at least one of the specified primary outcome measures reached statistical significance.
Two authors; a gastroenterologist with experience in conducting RCTs (PVC) and an experienced research professional with a background in public health and biomedicine (SP) independently screened and assessed the extracted abstracts. Disagreement between the two aforementioned authors was resolved through discussion with the third author, an experienced research professional with a background in RCT conduction and pharmacological sciences.
Statistical analysis
Interobserver agreement between the authors for rating the abstracts for quality was determined using the Cohen κ coefficient and was considered sufficient for the kappa point estimates higher than 0.6.26 Data were presented as overall number and proportion (%), mean and the SD, mean and 95% CI or median and IQR, where applicable. Linear regression analysis was performed to determine the factors associated with higher reporting quality.20–22 Univariate analysis was performed for each variable, with the overall quality score serving as a dependent variable. Multivariate regression analysis was further conducted by including factors that were significantly associated with a higher quality score in univariate analysis (p<0.05). Change in overall quality score in time was assessed by comparing scores of abstracts published in five 2-year periods using Kruskal-Wallis test with Dunn post hoc analysis. Statistical analysis was conducted using SPSS (V.16.0, IBM) and Prism six software (GraphPad Software, La Jolla, California, USA).
Patient and public involvement
Not applicable as the study did not involve human participants.
Results
Characteristics of included Abstracts
The flow diagram summarises the search strategy and eligibility testing (figure 1). The previously described search strategy found 551 abstracts, which were subjected to further screening to exclude those not in accordance to the inclusion criteria. A hundred abstracts were excluded. Out of those hundred, fifty included only H. pylori negative participants. Twenty-three abstracts were not RCTs, of which three were in vitro trials, one was a correction and one was a response to a letter to the editor while the rest were observational studies without an intervention. Eighteen abstracts were excluded, as they were not related to H. pylori infections relevant to the field of gastroenterology. Six abstracts only described protocols for RCTs. Finally, abstracts for three trials were not available.
Flow diagram with search strategy and study selection. CONSORT, Consolidated Standards of Reporting Trials.
The study characteristics are described in table 1. Only 31/451 (6.9%) of the included abstracts were published in journals that endorsed the use of the CONSORT guidelines for abstracts. Abstracts predominantly reported the results of pharmacological trials (422/451, 93,6%). Trials were mainly single centre (374/451, 82.9%) and included more than 100 participants (353/451, 78,3%). Most reported results were statistically significant (323/451, 71.6%). The included abstracts were predominantly structured (400, 88.7%). The average impact factor of the journals in which the abstracts were published was 2.99 (SD=6.28) and had a median of 8 authors (IQR 5–11).
Characteristics of included abstracts
Quality of individual consort for Abstract items
The Cohen κ values for all items were above 0.6, indicating substantial interobserver agreement (table 2).
Interobserver agreement for abstract reporting items
Table 3 shows the adherence of each item to the CONSORT for abstracts guideline. Less than half of the abstracts (202/451, 44.8%) included ‘randomised controlled’ in the title. The contact details for the corresponding author were given in 177/451 abstracts (39.2%). An adequate description of trial design was shown in only 36/451 abstracts (8.0%).
In regard to the study methodology, interventions, objectives and outcomes, they were predominately well reported, with 438 (97.1%), 448 (99.3%) and 402 (89.1%) abstracts adequately reporting each item, respectively. On the other hand, randomisation was described in merely 12/451 (2.7%) abstracts. Blinding was mentioned in 50/451 (11.1%), while participants’ inclusion criteria were described in 93/451 (20.6%) trials.
Quality of individual consort for abstract items
The number of participants randomised to each group was included in 427/451 (97.1%) abstracts, yet the number of participants included in the analysis were not reported in similar proportions (298/451, 64.1%). The adequate reporting of primary outcomes, with both effect sizes and measurement precision, was found in 338/451 (74.9%) abstracts. Side effects and adverse events were described in 250/451 (55.4%) abstracts.
Almost all abstracts gave a meaningful conclusion (444/451, 98.4%). Funding statement and trial registry information was included by 9 (2.0%) and 75 (16.6%) out of 451 included abstracts, respectively.
Overall reporting quality
Abstracts had a median of 8 (IQR 7–9) out of 17 (47.1%) adequately reported items. None of the included abstracts reported all 17 items. The maximum number of reported items was 16/17 (94.1%) and was achieved by two abstracts (2/451, 0.4%). The minimum number was 3/17 (17.6%) and was achieved by three abstracts (3/451, 0.7%). The scores indicating the overall quality of reporting are shown in table 4.
Overall reporting quality score
The quality score for each study characteristic is presented in table 5.
Overall reporting quality score for each study characteristic
Overall quality scores of the last three periods were significantly higher than the score for the first, 2010–2011 interval (8.0±3.0 vs 9.0±3.0, p<0.01 for 2014–15; vs 8.0±2.5 p<0.05 for 2016–17 and 8.5±2.0, p<0.01 for 2019–19, data expressed as median ±IQR). Scores for each time period are presented in figure 2.
Overall quality scores of abstracts for 2-year periods. *P<0.05 (Kruskal-Wallis test with Dunn post hoc test, difference in comparison to years 2010–11). **P<0.01 Kruskal-Wallis test with Dunn post hoc test, difference in comparison to years 2010–2011) data are presented as median values with IQR.
Reporting quality predictors
Results of the linear regression analysis are shown in table 6. The CONSORT endorsement (p<0.001), pharmacological intervention (p<0.01), multicentre setting (p<0.001), higher number of participants (p<0.001), hospital setting (p<0.001), impact factor of journal above 3 (p<0.001), journal in the first quartile (p<0.001), longer abstracts with 251–300 (p<0.01) and more than 300 words (p<0.001), as well as number of authors between 7 and 10 (p<0.05) and above 10 (p<0.001) were associated with a significantly improved reporting quality in a univariate model. No significant association was found for the significance of results, funding or abstract structure and for those reasons, these predictors were omitted from the multivariate analysis. In the multiple regression model, a higher overall quality score remained associated with CONSORT endorsement (p<0.01), pharmacological interventions (p<0.05), multicentre settings (p<0.001), higher numbers of participants (p<0.01), hospital settings (p<0.01) and longer abstracts with 251–300 (p<0.05) and more than 300 words (p<0.001).
Linear regression derived estimates and 95% CI with dependent variable defined as mean overall quality score shown as a percentage
Discussion
Our study constitutes an effort to evaluate the reporting quality of published RCT abstracts for H. pylori, based on the CONSORT-A checklist. We have reviewed 451 abstracts of H. pylori RCTs and their overall reporting quality could be rated as inadequate, as the median value of reported items was 8 (IQR 7–9) (47.1%), meaning that a half of the included abstracts reported less than half of the checklist items. Reporting of principal methodological aspects, such as randomisation and blinding, was poor, as only 2.7% of abstracts described the randomisation and 11.1% of them reported blinding. The rest of the items in the methods section were more adequately reported, as more than 90% of the abstracts reported about the participants, interventions, objectives, and outcomes. Moreover, the randomisation item was among the least frequently reported CONSORT-A items. The lowest number of abstracts reported funding, only 2%, followed by the aforementioned randomisation item and trial design item, which was reported in 8.0% of abstracts. Furthermore, among the least reported items was trial registration information, which was included in 16.6% of abstracts. Items with respect to the results section were more sufficiently reported, in comparison to items in the methods section. The main under-reported item in the results section was the harms item, defined as reported adverse events or side effects, which was reported in merely half of the included abstracts. Overall quality score showed improvement over time. However, the change, although statistically significant, was only marginal, signifying slow uptake of CONSORT-A guidelines among the scientists working in the field of gastroenterology. Previous research showed similar trends in other fields of research.27
Our results are consistent with the results of previous studies that reported suboptimal adherence to the CONSORT-A across different journals and fields of medicine.28 29 It should be noted that omission of essential RCT information could lead to inaccurate interpretation of study results and improper application into clinicians’ daily practice. Previous research showed that funding was poorly reported in RCT abstracts. This proportion oscillated from 0% in studies by Xie et al,30 Gallo et al31 and Speich et al32 to 9% in a study by Germini et al.21 Funding information is relevant to the reader, as it is known that funding by industry could be associated with positive results of RCTs.33 Furthermore, the results of the study by Germini et al also showed that the methods of randomisation, blinding, funding and trial registration are the most frequently omitted items in RCT abstracts, all of which were reported in less than 20% of the abstracts.21 Partial reporting of methodological items has the potential to mask sources of bias that could have an influence on the internal validity of an RCT.31 Moreover, similar to our results, results of studies by Chow et al and Gallo et al showed that randomisation was rarely described in the abstracts, with a frequency of merely 2%.31 34 Another poorly reported item was recruitment, as only 14.6% of abstracts explicitly stated whether the trial was completed, terminated early or still ongoing. This item is considered more important for conference abstracts and therefore its omittance form RCT abstracts was not surprising.14
A higher overall quality score of H. pylori abstracts was associated with CONSORT-A endorsement, pharmacological intervention, multicentre setting, higher number of participants, a hospital setting and abstracts’ length. It seems reasonable that journals that endorsed CONSORT-A had higher reporting quality. Furthermore, it can be assumed that reviewers of these journals are given instructions to evaluate abstracts according to CONSORT-A checklists and this encourages authors to improve compliance with the checklist. Unfortunately, only 6.9% of included abstracts were published in journals who advocated the use of CONSORT-A guidelines. On the other hand, endorsement of reporting guidelines might not be sufficient as the editors and reviewers might not strictly enforce them. For this reason, some authors proposed involving a reporting guideline expert in a review process.35 The association of reporting quality with the number of authors was previously established, but no such correlation was found in this study. Further, in a study by Germini et al, abstracts of RCTs in the field of emergency medicine that included pharmacological interventions had a significantly higher reporting quality, in comparison to RCTs of non-pharmacological interventions. The authors concluded that this finding can be explained because authors of RCTs with pharmacological interventions more frequently apply strict methods, probably for regulatory issues required for drug approvals. The same results were observed in a study by Mbuagbaw et al.21 28 30 Another interesting result was the lack of a relationship between the quality score and impact factor. The journal ranking was also not associated with better reporting. The impact factors’ poor relationship with reporting quality could be explained by the previously described lack of comparability between impact factors of journals from different disciplines.36 The abstracts in this study were mainly published in gastroenterology journals but some were published in other fields such as pharmacology or general medicine, which might have influenced the results of the linear regression. The authors of the CONSORT-A statement found that abstracts with 250–300 words should be sufficient to address all the items of the checklist.14 Our results were in accordance with their recommendations as the reporting quality was higher for abstracts with more than 250 words while shorter abstracts had lower reporting quality.
This study has some limitations. First, the study period was from years including and between 2010 and 2019 and we excluded studies published before the CONSORT guidelines for abstracts were issued. Moreover, the second limitation was that we used only MEDLINE/PubMed for the identification and selection of the abstracts. However, it should be noted that this search engine is the only one publicly available to all clinicians globally. Therefore, our study included abstracts of articles that are freely available on the internet and can be used as guidance to clinicians. Other search engines, such as the Web of Science and Scopus, are not freely available outside the scientific community. Finally, we only compared the reporting quality between structured and unstructured abstracts without investigating the effects of the structure format. Abstracts with highly specified format were found to have more complete reporting in comparison to simpler ones which could lead to difference between formats in our study as well.37 However, the aim of this study was not to investigate the formats of structured abstracts but solely to compare them with unstructured abstracts. Our study, however, has several strengths. First, we did not add an option for the authors that assessed the abstracts to evaluate incomplete reporting. For instance, 0.5 points for the participants’ item if abstracts included information about eligibility but not information about the setting, as this approach seemed arbitrary. Moreover, our methods are reproducible and we allowed a wide time frame for our study, from 2010 to 2019. Furthermore, we have included the CONSORT-A item for contact details of corresponding authors, as we believe that an available email address is important to enable communication between authors and readers. Finally, our interobserver agreement measured by Cohen’s kappa was sufficiently high throughout all the checklist items.
Conclusions
The results of this study showed the subpar overall reporting quality of RCT abstracts.
Regarding individual items, inconsistency was observed as some basic information, such as the trials’ design, description of the included participants, blinding and randomisation were not adequately described while the other items were reported by the vast majority of abstracts.
More transparency is needed in regards to the reporting of the funding and adverse events.
Longer abstracts of RCTs with pharmacological interventions, performed in a hospital setting, with more than 100 included participants and published in the journals who endorsed the use of CONSORT-A guidelines had better reporting quality.
The non-endorsement of the CONSORT-A guidelines by the majority of journals was a major obstacle in improving the reporting standards.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
References
Footnotes
Contributors JBu and DM were responsible for the study conception, design and protocol. SP, PVC and DM were responsible for abstract search, evaluation, scoring and data gathering. DL, ASP and DR analysed the data. JBo, JV and SP were responsible for the data interpretation. JBu, DL, SP and PVC were responsible for the initial draft of the manuscript. DM, ASP, DR, JV and JBo revised the manuscript. All authors gave a final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. DM was responsible for the overall content as guarantor and accepts full responsibility for the finished work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.