Objective Post-randomisation exclusions in randomised controlled trials are common and may include participants identified as not meeting trial eligibility criteria after randomisation. We report how a decision might be reached and reported on, to include or exclude these participants. We illustrate using a motivating scenario from the BREATHE trial (Trial registration ClinicalTrials.gov, NCT02426112) evaluating azithromycin for the treatment of chronic lung disease in people aged 6–19 years with HIV in Zimbabwe and Malawi.
Key points Including all enrolled and randomised participants in the primary analysis of a trial ensures an unbiased estimate of the intervention effect using intention-to-treat principles, and minimises the effects of confounding through balanced allocation to trial arm. Ineligible participants are sometimes enrolled, due to measurement or human error. Of 347 participants enrolled into the BREATHE trial, 11 (3.2%) were subsequently found to be ineligible based on lung function criteria. We assumed no safety risk of azithromycin treatment; their inclusion in the trial and subsequent analysis of the intervention effect therefore mirrors clinical practice. Senior trial investigators considered diurnal variations in the measurement of lung function, advantages of retaining a higher sample size and advice from the Data Safety and Monitoring Board and Trial Steering Committee, and decided to include these participants in primary analysis. We planned and reported analyses including and excluding these participants, and in our case the interpretation of treatment effect was consistent.
Conclusion The decision, by senior investigators, on whether to exclude enrolled participants, should reflect issues of safety, treatment efficacy, statistical power and measurement error. As long as decisions are made prior to finalising the statistical analysis plan for the trial, the risk of exclusions creating bias should be minimal. The decision taken should be transparently reported and a sensitivity analysis can present the opposite decision.
- statistics & research methods
- paediatric thoracic medicine
- education & training (see medical education & training)
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
- statistics & research methods
- paediatric thoracic medicine
- education & training (see medical education & training)
In an individually randomised placebo-controlled trial (registered ClinicalTrials.gov, NCT02426112) of the impact of azithromycin on treatment of chronic lung disease in children and adolescents born with HIV in Zimbabwe and Malawi, one eligibility criterion was a measurement cut-off for lung function (using forced expiratory volume in 1 second (FEV1) z-score).1 2 After enrolment was complete, and prior to data analysis, inconsistencies were identified with the FEV1 inclusion criteria. Specifically, height (an input variable for the reference equations on the European Respiratory Society/Global Lung function Initiative 2012 spreadsheet to compute the FEV1 z-score,3) measured at screening did not always align with height measurements from later study visits. A review of practices undertaken found that in one country, different models of stadiometers were used at different screening centres resulting in inconsistencies in height measurements. It was decided to recalculate the z-score using a mean of height from two later study visits in that country, and in the other country to use a mean height from screening and two later study visits (up to 2 weeks after randomisation). These recalculations meant that 11/347 (3.2%) participants fell outside the lung function cut-off for inclusion into the trial and a debate ensued among the trial investigators as to how to proceed. The first stage was to unmask these participants to the local study physician, report this protocol violation to the Data Safety and Monitoring Board (DSMB), and the Trial Steering Committee (TSC) and to the relevant ethics committee(s).
At this stage, 7 of the 11 ineligible participants had completed their course of study medication and the remaining four participants had between 1 and 3 weeks of study medication remaining. The trial drug, azithromycin, is considered safe,4 so the potential for harm in continuing the four participants on treatment was considered low. The initial suggestion from the investigators was to withdraw these participants from the trial and stop treatment, but the DSMB and TSC advised that they should be included as the primary analysis, with a sensitivity analysis excluding them. The reason for this was (1) lung function may vary by time of day,5 so if lung function had been retested at a different time even on the same day, eligibility may have been different, (2) the low risk of treatment-related adverse events and (3) the advantage in retaining a greater sample size (original power calculations required 300 for primary outcome at 12 months). Further, a prespecified subgroup analysis was included in the statistical analysis plan to investigate effect modification by baseline FEV1 measurement, and provide estimates of treatment effect at different severities of baseline lung function. We also considered the possible adverse impact on statistical power if ineligible participants were less responsive to azithromycin treatment than eligible participants (potentially resulting in greater variability of treatment effect). On balance, this was outweighed by reasons to include ineligible participants.
The intention-to-treat (ITT) principle underpins the analysis of randomised controlled trials as a means of obtaining balance between arms on potential confounding factors, and preventing estimates of the intervention effect from being biased.6 7 ITT in its purest form estimate the intervention effect for all enrolled randomised participants, based on the initial arm allocation. In practice, the trial outcomes may not be measured or analysed for all randomised participants and this may impact on the ITT principle. One reason for not measuring outcomes in all enrolled participants is attrition, causing missing outcome data. If outcome data are missing not at random the ITT principle can become compromised.8 A second reason, which compromises the ITT principle, but is common nonetheless,9 is that some randomised participants may be excluded from analysis post-randomisation. Reasons for such exclusions might be that participants (1) have incomplete baseline or outcome data, (2) did not receive the intervention allocated or (3) were found to be ineligible post-randomisation. In this communication, we summarise issues to consider when deciding whether to exclude enrolled, but ineligible, participants during the analysis of the intervention effect (table 1).
There is conflicting evidence as to whether post-randomisation exclusions of enrolled participants produce bias.10–13 Bias can be considered a potential issue where decisions about exclusions are influenced by the trial sponsor or conflicts of interest of the investigators.14
The statistical power of the trial may be affected in either direction when ineligible individuals are enrolled incorrectly. Including the ineligible enrolled participants in analysis will retain a larger sample size, while excluding them may increase the variance of the estimated intervention effects (if those ineligible were to respond differently to treatment than eligible participants).
The type of inclusion/exclusion criteria must be considered. For example, in a drug treatment trial providing treatment for a certain infection, if it was found post-randomisation that an enrolled participant was uninfected it is best to exclude that participant from analysis and withdraw them immediately from the trial. Decisions are less clear if (1) the criteria include a cut-off used for inclusion (eg, body mass index), and there is error in the measurement of this, or (2) exclusion criteria include the presence of a clinical condition for which screening tests were not available at enrolment and only become apparent during follow-up.
Depending how quickly it became apparent that a participant did not meet eligibility criteria, trial outcome data may have been collected on ineligible enrolled participants and a decision must be made whether to include them in primary analysis. If excluded, a ‘modified ITT’ may be performed.15–17
Reporting and reflection on motivating example
The primary outcome was analysed for 308 participants, 11 of whom were ineligible based on FEV1 inclusion criteria. By chance, differences were observed between trial arms in age and sex distributions and with HIV-related characteristics. Primary analyses were therefore prespecified, prior to unmasking of outcome data, to adjust for site, age, sex and HIV viral load. Once-weekly administration of azithromycin did not improve lung function measured by FEV1 z-score after 48 weeks in ITT analysis (adjusted mean difference (aMD) 0.06%, 95% CI −0.10% to 0.21%) and in sensitivity analysis excluding those who did not meet eligibility criteria (aMD 0.07%, 95% CI −0.08% to 0.23%).2 The prespecified per-protocol analysis suggested weak evidence for an effect of azithromycin, with an aMD in z-scores of 0.14 (95% CI −0.02 to 0.29) favouring azithromycin. Those not meeting eligibility criteria were more likely to be in the azithromycin arm, in Malawi, younger, of male sex and have HIV viral suppression (table 2).
The study was powered to detect a 0.32 z-score difference between trial arms with 300 participants assuming a mean z-score of −2.04 (SD 0.82) in the placebo arm. The primary outcome was assessed in 308 participants, with a mean of −1.95 (SD 0.91) in the placebo arm. Effectively, with a sample size of 146 in the placebo arm and 162 in the azithromycin arm, the study had 80% power to detect a 0.29 z-score difference between trial arms; excluding ineligible participants gave the same z-score difference.
In practice, the inclusion of ineligible participants did not change the interpretation of the trial results, likely due to their low numbers and/or because the adjustments used for primary analysis (to account for baseline imbalance) were also associated with ineligibility (and being assessed for the primary outcome). The study remained sufficiently powered. Sensitivity analyses were prespecified in a formal statistical analysis plan, shared with reviewers and reported in the publication of the trial findings for transparency and to maintain research integrity.
There is not a one-size-fits-all approach to deciding on post-randomisation exclusions and in fact, there is evidence to suggest that more trials tend to make post-randomisation exclusions than do not.9 Consideration should be given to safety, assessment of treatment effects, statistical power and measurement error (table 1). We recommend that the decision is made after a joint discussion among senior trial investigators in conjunction with the TSC and DSMB. Others may advise, but the final decision falls to the senior investigators of the trial who should not be influenced by the trial sponsor or conflicts of interest, financial or otherwise. To further reduce bias, a decision should be made prior to finalising the statistical analysis plan for the trial, and for transparency, reported explicitly when publishing the trial results. Justification for including or excluding the participants who were found not to meet inclusion criteria after randomisation should be presented for scrutiny by the scientific community and it may be appropriate to consider a sensitivity analysis using the opposite decision. The aim of any decision is to remain as close to ITT principles as possible and present an unbiased estimate of the treatment effect.
Contributors AMR prepared the first draft of manuscript. AMR, RF, EA, VS, GM and HAW were involved in the preparation and conception of the manuscript, critically reviewed the manuscript and approved the submitted version.
Funding The BREATHE trial was funded by the Global Health and Vaccination Research (GLOBVAC) Programme of the Medical Research Council of Norway. RF is funded by the Wellcome Trust (Grant Number 206316/Z/17/Z). HAW, AMR and VS were supported in part by a grants from the Medical Research Council (MRC) and the Department for International Development (DFID UK) under the MRC/DFID Concordat (K012126/1) and is also part of the EDCTP2 programme supported by the European Union (Grant Number MR/R010161/1).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.