Article Text

Download PDFPDF

Protocol for a scoping review to support development of a CONSORT extension for randomised controlled trials using cohorts and routinely collected health data
  1. Linda Kwakkenbos1,
  2. Mahrukh Imran2,
  3. Kimberly A McCord3,
  4. Margaret Sampson4,
  5. Ole Fröbert5,
  6. Chris Gale6,
  7. Lars G Hemkens3,
  8. Sinead M Langan7,
  9. David Moher8,
  10. Clare Relton9,
  11. Merrick Zwarenstein10,11,
  12. Eric I Benchimol12,13,14,
  13. Isabelle Boutron15,16,17,
  14. Marion K Campbell18,
  15. David Erlinge19,
  16. Sena Jawad6,
  17. Philippe Ravaud15,16,17,
  18. Danielle B Rice2,20,
  19. Maureen Sauve21,22,
  20. Tjeerd P van Staa23,24,
  21. Lehana Thabane25,
  22. Rudolf Uher26,
  23. Helena M Verkooijen27,28,
  24. Edmund Juszczak29,
  25. Brett D Thombs2,20,30,31,32,33
  1. 1 Behavioural Science Institute, Clinical Psychology, Radboud University, Nijmegen, Gelderland, Netherlands
  2. 2 Lady Davis Institute for Medical Research, Jewish General Hospital, Montréal, Quebec, Canada
  3. 3 Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
  4. 4 Library Services, Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada
  5. 5 Department of Cardiology, Faculty of Health, Örebro University, Örebro, UK
  6. 6 Section of Neonatal Medicine, Department of Medicine, Imperial College London, London, UK
  7. 7 Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
  8. 8 Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  9. 9 Centre for Clinical Trials and Methodology, Barts Institute of Population Health Science, Queen Mary University, London, UK
  10. 10 Department of Family Medicine, Western University, London, UK
  11. 11 Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
  12. 12 Department of Pediatrics and School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
  13. 13 Institute for Clinical Evaluative Sciences, Ottawa, Ontario, Canada
  14. 14 Division of Gastroenterology, Hepatology and Nutrition, Children’s Hospital of Eastern Ontario, Ottawa, Ontario, Canada
  15. 15 Sorbonne Paris Cité Epidemiology and Statistics Research Center, INSERM, UMR1153, Paris, France
  16. 16 Centre d’Épidémiologie Clinique, Hôpital Hôtel Dieu, Assistance Publique–Hôpitaux de Paris, Paris, France
  17. 17 Faculté de Médecine, Université Paris Descartes, Sorbonne Paris Cité, Paris, France
  18. 18 Health Services Research Unit, University of Aberdeen, Aberdeen, UK
  19. 19 Department of Cardiology, Clinical Sciences, Lund University, Lund, Sweden
  20. 20 Department of Psychology, McGill University, Montréal, Québec, Canada
  21. 21 Scleroderma Society of Ontario, Hamilton, Ontario, Canada
  22. 22 Scleroderma Canada, Hamilton, Ontario, Canada
  23. 23 Health e-Research Centre, School of Health Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
  24. 24 Division of Pharmacoepidemiology and Clinical Pharmacology, Faculty of Science, Utrecht University, Utrecht, The Netherlands
  25. 25 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
  26. 26 Department of Psychiatry, Dalhousie University, Halifax, Nova Scotia, Canada
  27. 27 Department of Epidemiology, University Medical Center Utrecht, Utrecht, The Netherlands
  28. 28 Department of Epidemiology, University of Utrecht, Utrecht, The Netherlands
  29. 29 NPEU Clinical Trials Unit, National Perinatal Epidemiology Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK
  30. 30 Department of Psychiatry, McGill University, Montreal, Quebec, Canada
  31. 31 Departments of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, Quebec, Canada
  32. 32 Department of Medicine, McGill University, Montreal, Quebec, Canada
  33. 33 Departments of Educational and Counselling Psychology, McGill University, Montreal, Quebec, Canada
  1. Correspondence to Dr Brett D Thombs; brett.thombs{at}mcgill.ca

Abstract

Introduction Randomised controlled trials (RCTs) conducted using cohorts and routinely collected health data, including registries, electronic health records and administrative databases, are increasingly used in healthcare intervention research. The development of an extension of the CONsolidated Standards of Reporting Trials (CONSORT) statement for RCTs using cohorts and routinely collected health data is being undertaken with the goal of improving reporting quality by setting standards early in the process of uptake of these designs. To develop this extension to the CONSORT statement, a scoping review will be conducted to identify potential modifications or clarifications of existing reporting guideline items, as well as additional items needed for reporting RCTs using cohorts and routinely collected health data.

Methods and analysis In separate searches, we will seek publications on methods or reporting or that describe protocols or results from RCTs using cohorts, registries, electronic health records and administrative databases. Data sources will include Medline and the Cochrane Methodology Register. For each of the four main types of RCTs using cohorts and routinely collected health data, separately, two investigators will independently review included publications to extract potential checklist items. A potential item will either modify an existing CONSORT 2010, Strengthening the Reporting of Observational Studies in Epidemiology or REporting of studies Conducted using Observational Routinely collected health Data item or will be proposed as a new item. Additionally, we will identify examples of good reporting in RCTs using cohorts and routinely collected health data.

Ethics and dissemination The proposed scoping review will help guide the development of the CONSORT extension statement for RCTs conducted using cohorts and routinely collected health data.

  • cohort
  • consort
  • randomized controlled trials
  • reporting guideline
  • routinely collected health data
  • rcts

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Our scoping review will be conducted using rigorous methods, with peer-reviewed searches developed by a research librarian that will comply with Institute of Medicine standards and are not limited by language.

  • Due to the novelty of randomised controlled trials using cohorts and routinely collected health data, we anticipate identifying only a limited number of methods and reporting articles in our scoping review.

  • To supplement articles on methods and reporting, we will review primary trial protocols and reports to identify elements that need reporting and to identify examples of good reporting.

Introduction 

Randomised controlled trials (RCTs), when well designed and conducted, are widely acknowledged to be the gold standard for evaluating the effectiveness and harms of medical interventions.1–3 Important concerns exist, however, about many RCTs, including limitations related to difficulty recruiting sufficiently large and representative samples, limited real-world generalisability and prohibitive costs.4–12 To attempt to address these and other challenges, trial designs have been developed in which RCTs are conducted within the frameworks cohorts4 and routinely collected health data. Routinely collected health data are defined as data collected for administrative and clinical purposes, without specific a priori research questions,13 and include registries,14 electronic health records15 and health administrative databases.16

Biomedical research reporting guidelines have been developed to assist authors to report research studies as accurately, transparently and completely as possible. Reporting guidelines typically describe a minimum set of information that should be clearly reported, provide examples of guideline-consistent reporting and include a checklist to facilitate compliance.17 18 Multiple existing reporting guidelines include items that are potentially applicable to RCTs conducted using cohorts and routinely collected health data. In addition to the CONsolidated Standards of Reporting Trials (CONSORT) statement for reporting of parallel group RCTs,19 reporting guidelines with the most direct overlap include the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for the reporting of observational studies, generally,20 and the REporting of studies Conducted using Observational Routinely collected Data (RECORD) guideline21 which address reporting specific to observational studies conducted using routinely collected health data.

The development of an extension of the CONSORT statement for RCTs conducted using cohorts and routinely collected health data is being undertaken with the goal of improving long-term reporting quality by setting standards early in the process of uptake of these trial designs.22 To develop this CONSORT extension, information is needed to understand which items from CONSORT, STROBE and RECORD can be used without modification and which should be included with adaptations, as well as aspects of reporting of RCTs conducted using cohorts and routinely collected health data that are not covered adequately in these reporting guidelines and that require new reporting items. In addition, examples of complete and transparent reporting of different aspects of these RCTs are needed.

Relatively little guidance has been published on the methods and reporting of RCTs conducted using cohorts and routinely collected health data. To account for this, the proposed scoping review will identify articles on the methods or reporting of RCTs conducted using cohorts, registries, electronic health records and health administrative databases, as well as examples of protocols and reports of results from these types of RCTs. The objectives of the scoping review are to (1) determine which items from an initial long list of items based on CONSORT, STROBE and RECORD that are being considered for possible inclusion in the CONSORT extension can be included without modification, identify items from the initial list that need adaptation and identify additional reporting considerations to develop new items; and (2) identify examples of complete and transparent reporting of different aspects of these types of RCTs that can be used to support the CONSORT extension.

Methods

The scoping review will be conducted following the approach described by Arksey and O’Malley23 and will be reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses: extension for Scoping Reviews (PRISMA-ScR) guidelines.24

Database searches

In separate searches, we will seek publications that describe aspects of methods or reporting or that describe protocols or results from RCTs (including cluster RCTs) using (1) cohorts, (2) registries, (3) electronic health records and (4) health administrative databases. Ovid MEDLINE Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Ovid MEDLINE Daily and Ovid MEDLINE and EBM Reviews—Cochrane Methodology Registry (Final issue, third Quarter 2012) will be searched by an experienced librarian familiar with knowledge synthesis for publications on methods or reporting of these types of RCTs and for examples of these types of RCTs. MEDLINE strategies for the searches were developed by a research librarian with input from the project team and were peer reviewed using the Peer Review of the Electronic Search Strategy standard.25 The MEDLINE strategy was then adapted for the Cochrane Library Methodology Register which includes methodological research available up to its last update in July 2012.

Search strategies comply with Institute of Medicine standards and are not limited by language.26 We will search for articles on methods and reporting and examples of RCTs published in the last 10 years (2008–2018) which will allow us to identify relatively recent reporting practices and focus on challenging aspects of reporting. See online supplementary file 1 for detailed search strategies. In addition to the database searches, references of included studies will be reviewed for additional eligible studies, a web search will be conducted and members of the project team with experience in each type of trial will be consulted to provide additional studies that were not identified in our search.

Supplementary file 1

Study selection

For each search, separately, results will be downloaded into the citation management database RefWorks, and duplicate references will be removed. Following this, references will be transferred into the systematic review software DistillerSR (Evidence Partners, Ottawa, Canada). A coding manual based on eligibility criteria has been developed, and a pilot test of the coding manual will be performed prior to the study’s inception. The initial coding manuals for inclusion and exclusion for all four types of trial designs are shown in online supplementary file 2 . Because the trial designs that will be included in the CONSORT extension reflect relatively recent developments, we anticipate that we will identify only a small number of articles on their methodology and reporting. Thus, we will also include publications of trial protocols and results.

Supplementary file 2

We will assess the eligibility of each publication through a two-stage process. In the first stage, two reviewers will independently screen titles and abstracts to identify potentially relevant studies. We will use a liberal accelerated method27 to screen titles and abstracts, meaning that articles deemed eligible by one of the reviewers will be included in full-text review, and only excluded articles will be screened by a second reviewer. Since title and abstract screening is done randomly and concurrently, reviewers will not know if the other reviewer has excluded the reference or not. In the second stage, two investigators will independently conduct a full-text review. Disagreements after full-text review will be resolved by consensus, with a third investigator consulted as necessary. Translators will be consulted to evaluate titles and abstracts and full-text articles for languages other than those for which team members are fluent, if any. See online supplementary file 3 for the preliminary PRISMA flow of studies figures for the four types of trial designs.

Supplementary file 3

Data extraction and verification

To develop a preliminary ‘long list’ of items to consider for the CONSORT extension checklist, as an initial step, items from the CONSORT 2010 will be examined to identify items where modifications will be needed for RCTs conducted using cohorts and routinely collected health data, and items from the STROBE and RECORD reporting guidelines will be examined to identify additional items to complement CONSORT items. Two investigators will independently review these reporting guidelines, and any item deemed possibly relevant to RCTs using cohorts and routinely collected health data by either or both investigators will be included in the ‘long list’. Additional preliminary ‘long-list’ items will be provided by other members of the project team.

For each of the four types of RCTs conducted using cohorts and routinely collected health data, separately, two investigators will independently review included publications to extract additional potential items for the ‘long list’. A potential item will either modify an existing CONSORT 2010, STROBE or RECORD item that has been included in the ‘long list’ or will be proposed as a new item. Potential items will be identified from publications that report information relevant to conducting RCTs using cohorts and routinely collected health data, but that were not included in our initial ‘long list’. In addition, potential items will be suggested based on gaps in reporting identified from primary trial protocols or reports. Data will be extracted and collected in DistillerSR using a standardised data-extraction form. The long list of items will evolve dynamically as potential modifications and new items are added based on the review of publications identified from our literature search using the DistillerSR Dynamic Question function. Thus, reviewers will add a potential item only once to the long list, after which it becomes visible for all reviewers. Reviewers will not duplicate items already provided by other reviewers. This will be done to avoid redundancy, as we expect potential gaps in reporting to occur in multiple publications that will be reviewed. In addition to each proposed item modification or new item, reviewers will add a brief explanation of why the suggested modification or new item is deemed important.

In addition to identifying gaps in reporting, for each item on our long list, we will attempt to identify examples of complete and transparent reporting in RCTs using cohorts, registries, electronic health record and health administrative databases. When examples of complete and transparent reporting for a particular item on the long list are identified, text corresponding to reporting of that item will be inserted in the data-extraction form in DistillerSR.

Prior to data extraction from included studies, all reviewers will assess a sample of trial reports. The results will be compared and discussed among the reviewers in order to ensure consistent application of the data-extraction process.

Patient and public involvement

One of the members of our extension to the CONSORT statement, Maureen Sauvé, is a patient organisation leader. She has been involved in working with researchers to establish a cohort of patients living with the rare disease, scleroderma, which supports RCTs of trials of online rehabilitation, self-management and psychological intervention programmes.28

Conclusion

This scoping review will gather previously published methods and recommendations for the reporting of RCTs using cohorts and routinely collected health data, as well as identify gaps in reporting of these studies. We will identify potential modifications or clarifications of CONSORT 2010, STROBE and RECORD items, as well as potential additional items to develop an extension to the CONSORT statement for reporting RCTs using cohorts and routinely collected health data. Following the scoping review, identified items will be vetted using a three-stage Delphi approach29 and a face-to-face meeting, after which the reporting checklist and explanation and elaboration documents for the CONSORT extension will be finalised. The resulting CONSORT extension will promote transparency, clarity, reduce research waste and provide guidance to researchers on appropriate and consistent reporting of RCTs using cohorts and routinely collected health data.

Ethics and dissemination

The proposed scoping review will help guide the development of the CONSORT extension statement for RCTs conducted using cohorts and routinely collected health data. The findings will be disseminated through peer-reviewed publications and conference presentations.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.

Footnotes

  • Contributors LK, MI, EJ, LGH, OF, CR, CG, MZ, SML, DM, MS and BDT were involved in initial phases of study conception, design of the search strategy and development of conceptual frameworks. KAMcC, EIB, IB, MKC, DE, SJ, PR, DBR, MS, TPvS, LT, RU and HMV provided regular feedback on each of these steps. LK, MI and BDT were responsible for the first draft of the manuscript. All authors made substantive intellectual contributions to the development of this protocol and approved the final version.

  • Funding The development of this CONSORT extension has been funded by grants from the Canadian Institutes of Health Research (BDT, OF, EJ, LK, CR; Grant #PJT-156172), and from the United Kingdom National Institute of Health Research (NIHR) Clinical Trials Unit Support Funding - Supporting efficient / innovative delivery of NIHR research (EJ, co-applicant CG). BDT is supported by a Fonds de recherche du Québec - Santé researcher salary award. CG is supported by the United Kingdom Medical Research Council through a Clinician Scientist Fellowship. SML is supported by a Wellcome Senior Clinical Fellowship in Science (205039/Z/16/Z). RU is supported by the Canada Research Chairs Program (Award #231397). EIB is supported by a New Investigator Award from the Canadian Institutes of Health Research, Canadian Association of Gastroenterology and Crohn’s and Colitis Canada, and the Career Enhancement Program of the Canadian Child Health Clinician Scientist Program. DBR is supported by a Vanier CIHR Graduate Scholarship.

  • Disclaimer The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

  • Competing interests None declared.

  • Patient consent Not required.

  • Ethics approval This study does not require ethics approval, as required data will be collected through the review of published literature.

  • Provenance and peer review Not commissioned; peer reviewed for ethical and funding approval prior to submission.