Justification of margin | Who is included in analysis | CI | Missing data | Sensitivity analyses | |
---|---|---|---|---|---|
CONSORT 20061 | ‘Margin should be specified and preferably justified on clinical grounds’ | ‘Non-ITT analyses might be desirable as a protection from ITTs increase in type I error. There is greater confidence in results when the conclusions are consistent’. ITT: ‘Analysing all patients within their randomised groups, regardless of whether they completed allocated treatment is recommended’ PP: ‘Alternative analyses that exclude patients not taking allocated treatment or otherwise not protocol-adherent could bias the trial in either direction. The terms on-treatment or PP analysis are often used but may be inadequately defined’. | ‘Many non-inferiority trials based their interpretation on the upper limit of a one-sided 97.5% CI, which is the same as the upper limit of a two-sided 95% CI’. ‘Although one-sided and two-sided CIs allow for inferences about non-inferiority, we suggest that two-sided CIs are appropriate in most non-inferiority trials. If a one-sided 5% significance level is deemed acceptable for the non-inferiority hypothesis test (a decision open to question), a 90% two-sided CI could then be used’. | ||
CONSORT 20129 | ‘Should be indicated if conclusions are related to PP analysis, ITT analysis or both and if the conclusions are stable between them’. | ‘The two-sided CI provides additional information, in particular for the situation in which the new treatment is superior to the reference treatment’ | Sensitivity analysis is discussed through an example: ‘Study endpoints were analysed primarily for the PP population and repeated, for sensitivity reasons, for the ITT population’. | ||
Draft FDA 20102 | ‘Whether M1 (the effect of the active control arm relative to placebo) is based on a single study or multiple studies, the observed (if there were multiple studies) or anticipated (if there is only one study) statistical variation of the treatment effect size should contribute to the ultimate choice of M1, as should any concerns about constancy. The selection of M2 (the largest clinically acceptable difference of the test treatment compared to the active control) is then based on clinical judgment regarding how much of the M1 active comparator treatment effect can be lost. The exercise of clinical judgment for the determination of M2 should be applied after the determination of M1 has been made based on the historical data and subsequent analysis’ | ‘It is therefore important to conduct both ITT and “as-treated” analyses in non-inferiority studies’. ITT: ‘preserve the principle that all patients are analysed according to the treatment to which they have been randomised even if they do not receive it’ | ‘Typically, the one-sided type I error is set at 0.025, by asking that the upper bound of the 95% CI for control treat be less than the NI margin. If multiple studies provide very homogeneous results for one or more important endpoints, it may be possible to use the 90% lower bound rather than the 95% lower bound of the CI to determine the active control effect size’ | ‘Poor quality can reduce the drug's effect size and undermine the assumption of the effect size of the control agent, giving the study a “bias towards the null”’. | |
ICH E912 | ‘This margin is the largest difference that can be judged as being clinically acceptable’ | ‘In confirmatory trials, it is usually appropriate to plan to conduct an analysis of the full analysis set and a PP analysis. In an equivalence or non-inferiority trial, use of the full analysis set is generally not conservative and its role should be considered very carefully’. ITT: ‘participants allocated to a treatment group should be followed up, assessed and analysed as members of that group irrespective of their compliance to the planned course of treatment’. Full analysis set: ‘The set of participants that is as close as possible to the ideal implied by the ITT principle. It is derived from the set of all randomised participants by minimal and justified elimination of participants’. PP: ‘The set of data generated by the subset of participants who complied with the protocol sufficiently to ensure that these data would be likely to exhibit the effects of treatment, according to the underlying scientific model. Compliance covers such considerations as exposure to treatment, availability of measurements and absence of major protocol violations’. | ‘For non-inferiority trials, a one-sided interval should be used. The choice of type I error should be a consideration separate from the use of a one-sided or two-sided procedure’. | ‘Imputation techniques, ranging from LOCF to the use of complex mathematical models, may be used to compensate for missing data’ | ‘An investigation should be made concerning the sensitivity of the results of analysis to the method of handling missing values, especially if the number of missing values is substantial’. |
ICH E1013 | ‘The determination of the margin in a non-inferiority trial is based on statistical reasoning and clinical judgment’ | ||||
SPIRIT14 | Use an example where ‘non-inferiority would be claimed if ITT and PP analyses show conclusions of NI’. ITT: ‘To preserve the unique benefit of randomisation as a mechanism to avoid selection bias, an “as randomised’ analysis retains participants in the group to which they were originally allocated. To prevent attrition bias, outcome data obtained from all participants are included in the data analysis, regardless of protocol adherence’. PP and mITT: ‘Some trialists use other types of data analyses (commonly labelled as “mITT” or “PP”) that exclude data from certain participants—such as those who are found to be ineligible after randomisation or who deviate from the intervention or follow-up protocols. This exclusion of data from protocol non-adherers can introduce bias, particularly if the frequency of and the reasons for non-adherence vary between the study groups’. | ‘Multiple imputation can be used to handle missing data although relies on untestable assumptions’ | ‘Sensitivity analyses are highly recommended to assess the robustness of trial results under different methods of handling missing data’ | ||
EMEA 200611 | ‘The choice of delta must always be justified on clinical and statistical grounds’ | ‘A two-sided 95% CI (or one-sided 97.5% CI) is constructed. The interval should lie entirely on the positive side of the margin. Statistical significance is generally assessed using the two-sided 0.05 level of significance (or one-sided 0.025)’ | |||
EMEA 200010 | ‘ITT and PP analyses have equal importance and their use should lead to similar conclusions for robust interpretation’ | ‘A two-sided CI should lie entirely to the right of delta. If one-sided confidence is used then 97.5% should be used’ | ‘It will be necessary to pay particular attention to demonstrating the sensitivity of the trial by showing similar results for the full analysis set and PP analysis set’ |
ITT, intention to treat; LOCF, last observation carried forward; mITT, modified intention to treat; NI, non-inferiority; PP, per-protocol.