Article Text

Download PDFPDF

Protocol for a systematic scoping review of reasons given to justify the performance of randomised controlled trials
  1. Brian Dewar1,
  2. Mark Fedyk2,
  3. Lucas Jurkovic1,
  4. Stephanie Chevrier1,
  5. Rosendo Rodriguez3,
  6. Simon C Kitto4,
  7. Raphael Saginur1,
  8. Michel Shamy1
  1. 1Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  2. 2Philosophy, Faculty of Arts, Mount Allison University, Sackville, New Brunswick, Canada
  3. 3Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
  4. 4Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
  1. Correspondence to Dr Michel Shamy; mshamy{at}toh.ca

Abstract

Introduction Randomised controlled trials (RCTs) are widely viewed to generate the most reliable medical knowledge. However, RCTs are not always scientifically necessary and therefore not always ethical. Unfortunately, it is not clear when an RCT is not necessary or how this should be established. This study seeks to systematically catalogue justifications offered throughout the medical and ethics literature for performing randomisation within clinical trials.

Methods and analysis We will systematically search electronic databases of the medical literature including MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, Cochrane Clinical Trials Register, Web of Science Proceedings, ClinicalTrials.gov; databases of philosophical literature including Philosopher’s Index, Phil Papers, JSTOR, Periodicals Archive Online, Project MUSE, National Reference Centre for Bioethics; the library catalogue at the University of Ottawa; bibliographies of retrieved papers; and the grey literature. We will also pursue suggestions from experts in the fields of medical ethics, philosophy and clinical trial methodology. Article screening, selection and data extraction will be performed by two independent reviewers based on prespecified inclusion/exclusion criteria. A third reviewer will be consulted to resolve any discrepancies. We will then extract the reasons given to justify randomisation using methodology established to extract data in a defensible, systematic manner. We will track the reasons given, their frequency of use and changes over time. Finally, using grounded theory, we will combine the reasons into broader themes. These themes will form the foundation of our subsequent analysis from qualitative and quantitative perspectives. This review will map existing arguments that clinicians, ethicists and philosophers use to ethically justify randomisation in clinical trials.

Ethics and dissemination No research ethics board approval is necessary because we are not examining patient-level data. This protocol complies with the reported guidance for conducting systematic scoping reviews. The findings of this paper will be disseminated via presentations and academic publication. In a subsequent phase of this research, we hope to engage with stakeholders and translate any recommendations derived from our findings into operational guidelines.

  • medical ethics
  • clinical trials
  • ethics (see medical ethics)
  • systematic scoping review

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

View Full Text

Statistics from Altmetric.com

Strengths and limitations of this study

  • This study will ask a novel research question with broad applicability for investigators, research ethics boards, funding agencies and government regulators.

  • We will apply a rigorous, quantitative approach to an ethics problem, which has resisted a standardisable solution.

  • We intend to produce a scoping review which in its data analysis will draw on methods typically associated with qualitative systematic reviews.

  • The diversity of data included in the relevant literature presents a potential challenge from the perspective of interrater reliability and consistency in analysis.

Introduction

In modern medicine, randomised controlled trials (RCTs) are considered to generate the highest form of medical knowledge.1 However, an RCT is not required to answer every clinical question, nor is every RCT ethical. For example, an RCT that deprives patients of a standard, proven therapy could be both scientifically unnecessary and unethical. There are, of course, less obvious examples in many fields of study that have generated controversy, including the well-described case of extracorporeal membrane oxygenation for neonatal respiratory failure.2 The crux of the problem relates to the use of randomisation as an experimental technique, in which patients are allocated to one of several treatment strategies, which may have varying degrees of support and which may not all be in the best interest of the patient.

In order to be able to determine when an RCT is justified, researchers, regulators and funders should be able to refer to a set of criteria that render RCTs scientifically and ethically justifiable. However, it is not entirely clear what criteria are currently used or what the optimal criteria should be. Our study aims to systematically review the bioethical literature to determine what justifications are being offered for randomisation within specific RCTs or for RCTs in general. As such, this review incorporates studies from ethics (ie. issues of values), epistemology (ie. issues of knowledge) and clinical research.

Much of the literature on the epistemic-ethical review of RCTs has focused on the principle of ‘equipoise’,3 a term introduced by American legal scholar Charles Fried in 1974.4 Fried argued that the criterion justifying a therapeutic trial was ‘equipoise’, meaning uncertainty on the part of the enrolling physician about which of the treatments is superior. Though Fried’s definition has since been criticised as an inconsistent and non-evidence–based standard, it remains in use, often under the name of the ‘uncertainty principle'.5,6

In 1987, McGill bioethicist Benjamin Freedman argued that the justification for RCTs should depend on the existence of ‘clinical equipoise’, which he defined as ‘honest, professional disagreement’ within the medical community.7 For Freedman, individual clinicians would be expected to disagree about the optimal treatment of a patient with a particular disease, and this disagreement was the condition that justified conducting an RCT.

Freedman’s ‘clinical equipoise’ remains the modern standard by which most RCTs are ethically evaluated.8 Equipoise likely remains popular because its central notion—that there should be uncertainty about which treatment is best for a therapeutic RCT to be ethically justifiable—is deeply appealing to clinicians. In situations where equipoise is said to exist, clinicians can honestly say that they do not know which treatment is best and therefore cannot feel that a patient is being given an inferior treatment within the context of the RCT.

However, equipoise has also been subject to significant criticism from ethicists,9 trialists10 and clinicians.11,12 Translating the concept of equipoise into real-world practice has always been a challenge. For example, how much disagreement is necessary to justify a trial? How is this disagreement to be measured? Whose opinion is relevant to any given question? If disagreement exists, should the basis for that disagreement—for example, science, economics, politics, influence— matter? Moreover, is the mixture of opinion within a medical community an accurate reflection of the state of scientific knowledge on a given topic?

We suspect that ‘clinical equipoise’ incompletely describes the conditions that justify RCTs. From medical history, we know that a lack of disagreement does not mean a question has been adequately answered.13 Moreover, the need for knowledge translation activities demonstrates how the presence of disagreement does not mean that a clinical question remains unanswered.14 As RCT design becomes increasingly complex (eg, with cluster and pragmatic trials),15,16 our conception of RCT ethics also needs to become more sophisticated. Therefore, we believe it is necessary to capture all the reasons offered in the medical, dental and other literature indexed on major health databases for justifying randomisation and RCTs. If a uniform set of reasons can be extrapolated, then these reasons may form the basis for the development of a standardisable, transparent set of criteria to be used by researchers, regulators and funders when evaluating proposed RCTs.

Objectives

This systematic scoping review explores the reasons given to ethically justify the performance of therapeutic randomised clinical trials, in which patients are randomised to receive one of several different treatments.17,18 In this project, we are concerned with elaborating the conditions that are taken to render RCTs ethically justifiable,19 by which we mean describing the features of the state of knowledge surrounding a research question that make it suitable to conduct an RCT. We have chosen to do a scoping review instead of a systematic review for several reasons: (1) the complexity and heterogeneity of the sources; (2) the need for including heterogeneous concepts and arguments to make this review more sensitive and inclusive; (3) the need for mapping these concepts first to identify broad and narrow reasons for justification and subsequently to apply an inductive process to achieve a more sensible framework.

The central question of this research project is as follows:

  1. What reasons are given to ethically justify the performance of RCTs?

Secondary questions include:

  1. How often is ‘equipoise’ offered as a reason, and what definition of equipoise is implied and/or given?

  2. Are there differences in how clinicians, ethicists and philosophers justify RCTs?

  3. How have the reasons offered changed over time?

Methods and analysis

This protocol complies with the reported guidance for conducting systematic scoping reviews.20 We will perform this study between October 2017 and September 2019 and will generally reflect three stages: article search and selection, data extraction and analysis (figure 1).

Our data extraction process has been adapted from a methodology developed by Strech and Sofaer for the systematic review of the bioethical literature.21 This method is intended to address a heterogeneous literature by focusing exclusively on the reasons given for particular decisions or stances. As defined in Strech’s methodology, a reason can generally be defined as an explanation given for the views that a position is or is not correct. 22 In our case, these positions relate to the decisions to perform or not perform an RCT. As outlined in the Strech methodology, we will not be performing an analysis of the quality of the papers in which the reasons appear or of the strengths of the arguments. This decision is made in an attempt to limit the impact of bias in the assessment process by restricting our analysis (as much as possible) to the original texts.

Our analytical methods are closely aligned with the techniques of a qualitative systematic review (QSR), which is a recognised method for integrating findings from qualitative studies.23 We chose to follow a QSR method because our question relates to arguments rather than to quantitative data, and our sample includes literature from the bioethics, philosophy and medical domains. The QSR approach was selected to achieve methodological rigour and apply a technique that would be familiar to medical researchers. Like other systematic reviews, a QSR applies a standard methodology to ensure that all relevant scholarship is captured. The main analytical process is not a quantitative meta-analysis but rather an integrative process that identifies common themes and concepts. For example, our analysis will be based on a grounded theory approach, in which iterations of analysis and discussion lead to the determination of themes through consensus. Grounded theory refers to a general research method in which the analysis of data begins with an inductive process (ie, based on prior knowledge and experiences) that is then modified during the course of the study as determined by the research findings. In this way, the theory that emerges is grounded in the data collected.

We anticipate that the reasons we will identify may include concepts related to equipoise, personal uncertainty, group uncertainty and medical necessity, among others. However, these concepts will not be defined at the data extraction stage as we anticipate there to be inconsistencies in how individual words or phrases are used. For example, ‘equipoise’ may be used in reference to various different versions of the concept. Additionally, defining these concepts at the data extraction phase may bias data extraction. We anticipate that a key component of the data synthesis stage will be the organic development of an organisational hierarchy of reasons and subsequent themes generated through a grounded theory methodology.

Patient and public involvement

Patients were not explicitly consulted in the design of this study, as the process under investigation—namely, how RCTs are ethically justified—usually occurs before and without patient participation. In future phases of this research, we will seek to engage patients on questions of RCT ethics.

Eligibility criteria

Study characteristics

This systematic scoping review will include observational studies, interventional trials, philosophical scholarship, narrative reviews, editorials, commentaries and systematic reviews pertaining to the reasons given to justify randomisation in clinical trials.

Setting and time frame

This review will only include papers produced in 1948 or later, as that was the date of the first modern RCT. The dates of papers selected for inclusion will be captured during data extraction to enable the assessment of trends in the justifications offered for RCTs.

Report characteristics

Articles written in English or French will be included, reflecting the authors’ language facility. We will only include complete articles that have been published. Abstracts or articles in press will be excluded.

Information sources

We will search electronic databases, grey literature and consult bibliographies and authors of selected papers. We will search electronic databases of the medical literature including MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, Cochrane Clinical Trials Register, Web of Science Proceedings, ClinicalTrials.gov and databases of philosophical literature including Philosopher’s Index, Phil Papers, JSTOR, Periodicals Archive Online, Project MUSE and the National Reference Centre for Bioethics. We will also search the library catalogue of the University of Ottawa and solicit suggestions from experts in the fields of medical ethics and other philosophy and clinical trial methodologies. A number of related search terms such as ethics, epistemology, clinical trials, randomisation and equipoise were combined to generate the original set of search criteria. Please see online supplementary appendix 1 for the full search terms.

We will identify seminal papers and then review our search results to ensure the search methodology is appropriately inclusive.

Data items

We will collect the following bibliographic information from included studies: the first author, title of the paper, the journal in which it was published, year of publication and the classification of the first author (defined as being either a physician, ethicist/philosopher, regulator or historian). We will also categorise the article type into five broad categories derived from the National Library of Medicine classification: type 1: case reports, observational studies or clinical trials; type 2: commentaries, editorials, personal narratives or reviews; type 3: historical articles; type 4: philosophical articles; and type 5: government publications. We will collect the in-text citation for reasons given as justifications for RCTs. Because there are nearly infinite ways for authors to state a reason, data extractors will be instructed to look for formulations that followed the pattern of ‘a RCT is justified if X’ or ‘If X does not apply, then an RCT is unethical’ and variations on those structures. Please see table 1 for the data extraction form.

Table 1

Data extraction form

As outlined in the methodology developed by Strech and Sofaer for a systematic review of an ethics literature, we will not be explicitly performing a risk of bias assessment in the manner typical of quantitative systematic reviews. The reasons for this are four-fold. First, we will not include the clinical outcomes of any clinical studies (which would be vulnerable to traditional biases), but only the justifications offered for those studies. Second, argumentative and qualitative data do not lend themselves to the risk of bias assessment common in quantitative systematic reviews, in that an argument will always be intentionally and overtly biased by the position of the author. Third, we chose not to interpret the risk of bias assessment as a broad judgement on the quality of the manuscript because we are including many different types of manuscripts whose evaluations would have to be type-specific, and once again this was not felt to be relevant to our question. Fourth, we were concerned that quality assessments would be very difficult to standardise between reviewers and hence would inject unnecessary bias and heterogeneity into our own analyses.

Data collection and analysis

Citations will be imported into the Covidence systematic review management software and screened for selection by two reviewers using titles and abstracts; this process will be documented in a flow diagram according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses criteria. Then, data will be extracted independently by two reviewers (with no blinding to authors or journal) using a standardised form described above. We will test the data extraction form with 10 studies. The results of these extractions will be compared for homogeneity, and then new instructions will be given to increase interrater reliability. On attaining high levels of agreement, we will proceed to full data extraction. Should any disagreement arise, discrepancies will be resolved by discussion, with involvement of a third reviewer when necessary. For instances where agreement cannot be reached, we will note the potential arguments of each party and include them in the final data capture.

Data synthesis

On completion of data extraction, principal researchers will meet to combine the extracted reasons using a grounded theory approach based on methods elaborated in Heath and Cowley.24 Reasons will be grouped by themes. These themes will then be placed within the broader context of medical bioethics. Both narrative and quantitative reviews will be produced. Specifically, we will quantify the number of reasons given, the relative frequency of their use, the relationship between reasons and author or paper types and any obvious changes over time. Interpretation will be qualitative, in that we will gather references to themes. The identification of themes will lead to reconsideration of the relevant literature, further refinement of themes and the generation of conclusions. Additionally, we will assess both the quality of our own search strategy as well as the quality of the thematic analyses produced.25 When coding themes, unresolvable disagreements will be noted and discussed.

Presentation of Results

We intend to present a list of the justifications offered, analysed by frequency of use, use over time and associations with author type. We will organise these justifications into themes and will address variations on those themes. These data will then be used to draw conclusions about the scope of the field of ethical justification for RCTs and will inform future work on developing ethical guidelines and translating them into standardisable practices.

Ethics and dissemination

We will seek to publish our results in leading medical journals (eg, BMJ) and in leading bioethics journals (eg, Bioethics) with a focus on open access publication. Moreover, we plan to present the results, as well as the methodology of this systematic scoping review:

  • to the local medical community through vehicles such as Grand Rounds;

  • to the local clinical trials community, for example, in a seminar with the Department of Epidemiology and Community Medicine;

  • to the local bioethics and philosophy communities, for example, through the Ottawa Hospital bioethics and philosophy of medicine rounds;

  • and to national and international research communities through presentations at major meetings.

We will engage with research ethics board (REB) leaders and national regulators to identify collaborators willing to participate in a future, synthetic phase of this project with the aim of developing a revised ethics framework for clinical trial evaluation. We are also exploring the central question of this review through a series of interviews with trialists, philosophers and REB chairs. We hope to integrate this separate analytical method with the results of the systematic scoping review described here. Until the results of these two studies are known, further plans for knowledge translation would be premature.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
View Abstract

Footnotes

  • Contributors MS is the guarantor of the review. MS conceived of the presented idea. MS, BD, MF, LJ and SC assisted in research design. RR, SK and RS assisted in verifying analytical methods. MS provided supervision of the design process.

  • Funding This work was supported by the Canadian Institutes for Health Research Catalyst Grant in Ethics, grant number 385778. It was also supported by a grant from the University Medical Research Fund, Department of Medicine, University of Ottawa.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Patient consent for publication Not required.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.