Motivating example

In an individually randomised placebo-controlled trial (registered ClinicalTrials.gov, NCT02426112) of the impact of azithromycin on treatment of chronic lung disease in children and adolescents born with HIV in Zimbabwe and Malawi, one eligibility criterion was a measurement cut-off for lung function (using forced expiratory volume in 1 second (FEV₁) z-score).1 2 After enrolment was complete, and prior to data analysis, inconsistencies were identified with the FEV₁ inclusion criteria. Specifically, height (an input variable for the reference equations on the European Respiratory Society/Global Lung function Initiative 2012 spreadsheet to compute the FEV₁ z-score,3) measured at screening did not always align with height measurements from later study visits. A review of practices undertaken found that in one country, different models of stadiometers were used at different screening centres resulting in inconsistencies in height measurements. It was decided to recalculate the z-score using a mean of height from two later study visits in that country, and in the other country to use a mean height from screening and two later study visits (up to 2 weeks after randomisation). These recalculations meant that 11/347 (3.2%) participants fell outside the lung function cut-off for inclusion into the trial and a debate ensued among the trial investigators as to how to proceed. The first stage was to unmask these participants to the local study physician, report this protocol violation to the Data Safety and Monitoring Board (DSMB), and the Trial Steering Committee (TSC) and to the relevant ethics committee(s).

At this stage, 7 of the 11 ineligible participants had completed their course of study medication and the remaining four participants had between 1 and 3 weeks of study medication remaining. The trial drug, azithromycin, is considered safe,4 so the potential for harm in continuing the four participants on treatment was considered low. The initial suggestion from the investigators was to withdraw these participants from the trial and stop treatment, but the DSMB and TSC advised that they should be included as the primary analysis, with a sensitivity analysis excluding them. The reason for this was (1) lung function may vary by time of day,5 so if lung function had been retested at a different time even on the same day, eligibility may have been different, (2) the low risk of treatment-related adverse events and (3) the advantage in retaining a greater sample size (original power calculations required 300 for primary outcome at 12 months). Further, a prespecified subgroup analysis was included in the statistical analysis plan to investigate effect modification by baseline FEV₁ measurement, and provide estimates of treatment effect at different severities of baseline lung function. We also considered the possible adverse impact on statistical power if ineligible participants were less responsive to azithromycin treatment than eligible participants (potentially resulting in greater variability of treatment effect). On balance, this was outweighed by reasons to include ineligible participants.

Statistical considerations

The intention-to-treat (ITT) principle underpins the analysis of randomised controlled trials as a means of obtaining balance between arms on potential confounding factors, and preventing estimates of the intervention effect from being biased.6 7 ITT in its purest form estimate the intervention effect for all enrolled randomised participants, based on the initial arm allocation. In practice, the trial outcomes may not be measured or analysed for all randomised participants and this may impact on the ITT principle. One reason for not measuring outcomes in all enrolled participants is attrition, causing missing outcome data. If outcome data are missing not at random the ITT principle can become compromised.8 A second reason, which compromises the ITT principle, but is common nonetheless,9 is that some randomised participants may be excluded from analysis post-randomisation. Reasons for such exclusions might be that participants (1) have incomplete baseline or outcome data, (2) did not receive the intervention allocated or (3) were found to be ineligible post-randomisation. In this communication, we summarise issues to consider when deciding whether to exclude enrolled, but ineligible, participants during the analysis of the intervention effect (table 1).

There is conflicting evidence as to whether post-randomisation exclusions of enrolled participants produce bias.10–13 Bias can be considered a potential issue where decisions about exclusions are influenced by the trial sponsor or conflicts of interest of the investigators.14

The statistical power of the trial may be affected in either direction when ineligible individuals are enrolled incorrectly. Including the ineligible enrolled participants in analysis will retain a larger sample size, while excluding them may increase the variance of the estimated intervention effects (if those ineligible were to respond differently to treatment than eligible participants).

The type of inclusion/exclusion criteria must be considered. For example, in a drug treatment trial providing treatment for a certain infection, if it was found post-randomisation that an enrolled participant was uninfected it is best to exclude that participant from analysis and withdraw them immediately from the trial. Decisions are less clear if (1) the criteria include a cut-off used for inclusion (eg, body mass index), and there is error in the measurement of this, or (2) exclusion criteria include the presence of a clinical condition for which screening tests were not available at enrolment and only become apparent during follow-up.

Depending how quickly it became apparent that a participant did not meet eligibility criteria, trial outcome data may have been collected on ineligible enrolled participants and a decision must be made whether to include them in primary analysis. If excluded, a ‘modified ITT’ may be performed.15–17

Reporting and reflection on motivating example

The primary outcome was analysed for 308 participants, 11 of whom were ineligible based on FEV₁ inclusion criteria. By chance, differences were observed between trial arms in age and sex distributions and with HIV-related characteristics. Primary analyses were therefore prespecified, prior to unmasking of outcome data, to adjust for site, age, sex and HIV viral load. Once-weekly administration of azithromycin did not improve lung function measured by FEV₁ z-score after 48 weeks in ITT analysis (adjusted mean difference (aMD) 0.06%, 95% CI −0.10% to 0.21%) and in sensitivity analysis excluding those who did not meet eligibility criteria (aMD 0.07%, 95% CI −0.08% to 0.23%).2 The prespecified per-protocol analysis suggested weak evidence for an effect of azithromycin, with an aMD in z-scores of 0.14 (95% CI −0.02 to 0.29) favouring azithromycin. Those not meeting eligibility criteria were more likely to be in the azithromycin arm, in Malawi, younger, of male sex and have HIV viral suppression (table 2).

The study was powered to detect a 0.32 z-score difference between trial arms with 300 participants assuming a mean z-score of −2.04 (SD 0.82) in the placebo arm. The primary outcome was assessed in 308 participants, with a mean of −1.95 (SD 0.91) in the placebo arm. Effectively, with a sample size of 146 in the placebo arm and 162 in the azithromycin arm, the study had 80% power to detect a 0.29 z-score difference between trial arms; excluding ineligible participants gave the same z-score difference.

In practice, the inclusion of ineligible participants did not change the interpretation of the trial results, likely due to their low numbers and/or because the adjustments used for primary analysis (to account for baseline imbalance) were also associated with ineligibility (and being assessed for the primary outcome). The study remained sufficiently powered. Sensitivity analyses were prespecified in a formal statistical analysis plan, shared with reviewers and reported in the publication of the trial findings for transparency and to maintain research integrity.