We investigate the chance of demonstrating Ebola vaccine efficacy in an individually randomised controlled trial implemented in the declining epidemic of Forécariah prefecture, Guinea.

We extend a previously published dynamic transmission model to include a simulated individually randomised controlled trial of 100 000 participants. Using Bayesian methods, we fit the model to Ebola case incidence before a trial and forecast the expected dynamics until disease elimination. We simulate trials under these forecasts and test potential start dates and rollout schemes to assess power to detect efficacy, and bias in vaccine efficacy estimates that may be introduced.

Under realistic assumptions, we found that a trial of 100 000 participants starting after 1 August had less than 5% chance of having enough cases to detect vaccine efficacy. In particular, gradual recruitment precludes detection of vaccine efficacy because the epidemic is likely to go extinct before enough participants are recruited. Exclusion of early cases in either arm of the trial creates bias in vaccine efficacy estimates.

The very low Ebola virus disease incidence in Forécariah prefecture means any individually randomised controlled trial implemented there is unlikely to be successful, unless there is a substantial increase in the number of cases.

Timely estimates of chance of success of individually randomised controlled trials (RCTs) in the declining Ebola epidemic.

Determination and explanation of bias introduced to vaccine RCTs by exclusion of cases that occur shortly after vaccination.

This model can only account for RCTs conducted in the declining phase of the epidemic.

Since 2013, the largest epidemic of Ebola virus disease (EVD) to date has been ongoing in West Africa, with over 25 000 cases and 10 000 deaths reported as of 7 July 2015. There is no licensed vaccine or treatment for EVD, and the case fatality rate is around 70%.

Some areas have continued transmission, however, and thus remain potential candidate locations for a large-scale Ebola vaccine trial.

To investigate the dynamics of EVD in the prefecture of Forécariah (population 245 000), we fitted a stochastic Susceptible-Exposed-Infectious-Recovered (SEIR) transmission model to the weekly incidence of confirmed and probable cases published by the WHO

Parameter descriptions and values

Parameter | Description | Value |
---|---|---|

β_{t} | Time varying transmission rate | Estimated |

1/ɛ | Average latent period | 9.4 days |

1/ν | Average infectious period | 11.8 days |

_{t} | Time-varying reproduction number | β_{t}/ν |

1/κ | Average time between vaccination and protection | 14 days |

1/γ | Average duration of vaccine protection | 1 year |

σ | Vaccine efficacy | 0%, 50%, 70%, 90% |

We used published estimates of 9.4 days for the mean latent period and 11.8 days for the mean infectious period._{t}), for example, variation in population behaviour, or epidemic control measures, we assumed that the transmission rate could change over time. Therefore, the change in β_{t} would also absorb any effective change in the infectious period during the epidemic. The extent and direction of rate change was estimated during the model-fitting procedure.

After fitting the transmission parameters of the model, we projected the model forward in time, to simulate the potential future trajectories of the epidemic. More precisely, we simulated 200 000 epidemic trajectories without a vaccine trial from 7 June 2015 until 1 May 2016. This corresponds to 40 stochastic simulations of 5000 samples from the posterior distribution of the parameters and model states inferred on 7 June 2015. We restricted the forecast to those parameter sets for which more than 25% of the 40 simulated epidemics go extinct before 1 May 2016, that is, assuming that elimination of EVD will be achieved within 10 months. We kept 3542 (71%) of the 5000 parameter sets. Epidemic trajectories resulting from these parameter sets are summarised in _{t} for forecasted epidemics is shown in _{t} below the epidemic control threshold, that is, we assume that the epidemic will remain under control until elimination. This is a reasonable assumption, given the low incidence in Forécariah.

Model fit (blue) to the incidence data in Forécariah (red points) and forecast (grey) based on the posterior distribution at the latest data point (A). The solid line corresponds to the median estimate and the shaded areas to the 50% and 95% credible intervals. Fitted (blue) and forecasted (grey) values for the time-varying reproduction number _{t} (B). Posterior distribution of _{t} on 7 June 2015 (blue) and distribution corresponding to the trajectories used for forecasting (grey)(C). Mechanistic model for the vaccine trial (D). Susceptible participants are recruited into the trial at rate, rt. Before the trial begins, rt equals zero, and the model reduces to an SEIR model. Those entering the vaccine arm pass through a period of immune development, V_{s}, during which they are susceptible to Ebola virus disease infection. Following onset of protective immunity, they enter V_{p} and experience reduced susceptibility, σ, equal to the vaccine efficacy. Protective immunity is lost at rate γ, and individuals become susceptible again. Participants enter the control arm at the same rate as vaccinated participants and are separated between early (C_{1}) and late (C_{2}) control to match the delay in acquiring immunity in the vaccine arm. For biological realism, the distribution of durations of E, V_{s} and V_{p} follow an Erlang distribution with shape parameter two. Similarly, to match the vaccine arm, the same distribution is assumed for the compartment C_{1}.

To model the vaccine trial, we extend the stochastic SEIR transmission model to include the recruitment of two arms of an individually randomised controlled trial

In the model, the vaccine is delivered in one dose, and protective immunity begins 2 weeks later. We also conducted a sensitivity analysis by using 1 week delay, based on the intermediate results of the rVSV ring-vaccination trial in Guinea.

In a primary analysis of a randomised controlled vaccine trial, the vaccine efficacy,

For each trial simulation, we computed

Extinction probability—the probability that the epidemic has gone extinct by time t, which is the proportion of extinct simulations at time t.

Measured vaccine efficacy—the median value of

False-positive rate—the probability that a positive or negative vaccine effect can be detected, given that the vaccine has no efficacy and the epidemic is non-extinct at that time. Calculated as the proportion of simulations with a positive or negative vaccine efficacy when true efficacy is 0%, and the epidemic is non-extinct at t.

Power to detect vaccine efficacy—the probability that a positive vaccine effect can be detected given that the vaccine is efficacious and the epidemic is non-extinct. We use the proportion of simulations with a positive effect among the non-extinct simulations at time t.

Power adjusted by extinction probability—the probability that the epidemic is non-extinct and vaccine efficacy is detected. The power at time t is multiplied by 1-extinction probability at time t, and this therefore represents the chance of success of the trial.

Some trial protocols exclude participants who develop symptoms shortly after vaccination, that is, before the vaccine becomes immunoprotective, under the assumption that the participant became infected before recruitment or before the vaccine could generate an immune response in the host. Other trial protocols also exclude control participants who develop symptoms within this period,

For each vaccine efficacy tested, the extinction probability quickly increases through time, with more than 50% chance of extinction by October 2015 (

Detection of vaccine efficacy for a trial starting on 1 July with immediate vaccination. Extinction probability (A), power to detect efficacy (B) and power to detect efficacy adjusted by extinction probability (C), for assumed efficacy values 50, 70 and 90%.

For a model with immediate recruitment on 1 July, and 70% vaccine efficacy, the highest power is achieved by excluding only early cases in the vaccine arm (

Effect of group definition on trial outcomes. The X-axis shows the probability that the epidemic goes extinct before the value on the Y-axis is reached. Advancing time moves left-to-right, as the extinction probability increases. Power to detect efficacy (A), false-positive rate (B), and measured vaccine efficacy (where 70% is assumed) (C). The three group definitions (1) no early cases excluded (blue), (2) early cases in the vaccine arm excluded (black), and (3) cases in both arms excluded (red).

Including all cases reduces the false-positive rate below 5% but also decreases the trial power, and leads to underestimates of vaccine efficacy (

In practice, it may be difficult to find the appropriate exclusion period, where the period over which immunity is developed is unknown. Reducing both the protection delay and exclusion period from 2 to 1 week leads only to a slightly earlier and higher peak in power due to greater sample sizes and number of cases included in the analysis. In addition, shorter delay increases herd immunity effect, leads to faster extinction of the epidemic and thus reduces the adjusted power at later time (

Effect of start date on trial success. The figure shows immediate administration of a 70% efficacious vaccine to all participants (round points) for trials starting on the 1 July, 1 August and 1 September. In addition, the gradual recruitment of participants (triangles) is shown for the 1 July start date. The dashed line shows the power when the assumed delay from administration of vaccine until protective immunity is 1 week. All other results are for a 2-week delay.

Under an ideal scenario of immediate recruitment of 100 000 participants, the later the trial starts the lower the probability to detect vaccine efficacy (

Here we modelled the implementation of an individually randomised control vaccine trial in Forécariah prefecture, Guinea using an extended version of a previously published dynamic transmission model for EVD. We showed that if an RCT were to start later this year in Forécariah prefecture it would have a very limited chance of detecting any vaccine efficacy, because the epidemic is likely to go extinct before enough cases have occurred in participants. In addition, in realistic rollout scenarios of 10 000 participants per month, the chance that the epidemic persists until enough participants are recruited and the trial is able to detect efficacy is very low, for example, below 2% for a trial beginning on 1 July 2015 with a 70% efficacious vaccine. We note that this adjusted power is probably an overestimate since our model operates at the population level and does not account for clustering effect at small scales.

We also demonstrated that exclusion of early cases in the group definition for the vaccine arm of a trial (ie, individuals vaccinated but not yet protected) inflates the power but also the false-positive rate due to the declining risk of infection over time. Ideally, the group definition for a primary analysis in a declining epidemic should consider excluding early cases in the control and intervention arms to maximise the power to detect vaccine efficacy while keeping the false-positive rate below 5%. Alternatively, more advanced statistical analyses accounting for time-varying risk of infection should be considered to circumvent the necessity of excluding early control cases.

Overall, our analysis is an example of how real-time mathematical models can be used to design trials more efficiently during an epidemic, and assess feasibility of planned trials, although models are infrequently utilised to this end. More realistic models accounting for network structure could be even more precise given that the majority of transmission events may be seen in clusters formed at confined space (eg, hospital or household) and also at a small spatial scale.

The authors acknowledge funding from the Research for Health in Humanitarian Crises (R2HC) Programme, managed by Research for Humanitarian Assistance (Grant 13165) (AC, SF, AJK), the Innovative Medicines Initiative 2 (IMI2) Joint Undertaking, under grant agreement 115854 (RME, WJE) and UK Medical Research Council grant MR/J003999/1 (CHW). IMI2 receives support from the European Union's Horizon 2020 research and innovation programme and the European Federation of Pharmaceutical Industries and Associations (EFPIA). The authors also thank Stefan Flasche and John Ojal for helpful technical discussion.

Stefan Flasche; John Ojal.

AC, RME, SF, CHW, AJK and WJE developed the model. AC implemented the analysis. AC, RME, SF and WJE interpreted the results. AC and RME wrote the first draft. All authors contributed to the final version of the manuscript.

Research for Health in Humanitarian Crises Programme from Research for Health in Humanitarian Crises (Grant 13165). Innovative Medicines Initiative 2 Joint Undertaking (grant agreement 115854). Medical Research Council (grant MR/J003999/1).

AC, WJE and CHW are co-investigators on the Ebola ça suffit ring vaccination trial in Guinea, have acted as unpaid advisors to the WHO on Ebola vaccination and report travel and accommodation paid for by the WHO to attend meetings. WJE is a co-investigator on, and RME is funded by, the European Commission Innovative Medicines Initiative-funded EBOVAC trial of the Johnson & Johnson prime-boost Ebola vaccine candidate. WJE's partner is an epidemiologist at GlaxoSmithKline, in a role unrelated to the company's development of an Ebola vaccine. AC and CHW have acted as unpaid advisors to the EBOVAC trial, for which CHW reports travel and accommodation paid for by the EBOVAC consortium to attend a meeting.

Not commissioned; externally peer reviewed.

No additional data are available.