Elsevier

Vaccine

Volume 31, Issue 48, 19 November 2013, Pages 5634-5642
Vaccine

Review
Vaccine effects and impact of vaccination programmes in post-licensure studies

https://doi.org/10.1016/j.vaccine.2013.07.006Get rights and content

Highlights

  • Vaccine effectiveness and vaccination programme impact are not consistently defined and are widely confused.

  • We delineate these effects based on the exposure considered and the population compared.

  • Vaccine studies measuring them are based on a number of assumptions.

  • Most of these assumptions are not fully respected in observational studies.

  • Potential confounding and biases must be minimized in study design and analyses.

Abstract

Once a vaccine is licensed and introduced in the population, post-licensure studies are required to measure vaccine effectiveness and impact of vaccination programmes on the population at large. However, confusion still prevails around these concepts, making it difficult to discern which effects are measured in such studies and how their findings should be interpreted. We review from the public health evaluation perspective the effects of vaccine-related exposures, describe the methods used to measure them and their assumptions.

We distinguish effects due to exposure to individual vaccination from those due to exposure to a vaccination programme, as the latter depends on vaccine coverage, other population factors and includes indirect effects as well. Vaccine (direct) effectiveness is estimated by comparing vaccinated and unvaccinated individuals exposed to the same vaccination programme. The impact of a vaccination programme, defined here as the population prevented fraction when exposure is the programme, is measured by comparing populations with and without a vaccination programme, most commonly the same population before and after vaccination. These designs are based on a number of assumptions for valid inference. In particular, they assume that vaccinees and non-vaccinees do not differ in terms of susceptibility and exposure to the disease or in ascertainment of vaccination and disease status. In pre and post-vaccination design, the population is assumed to have similar baseline transmission, case detection and reporting, risk factors and medical practices in both periods.

These principles are frequently violated in post-licensure studies. Potential confounding and biases must be minimized in study design and analyses, or taken into account during result interpretation. It is also essential to define which exposure is evaluated (individual vaccination or vaccination programme) and which effect is measured. This may help decision-makers clarify which type of study is needed and how to interpret the results.

Introduction

Before licensure, a vaccine must demonstrate its immunogenicity and/or protective efficacy in clinical trials, as well as its quality and safety [1], [2]. Once the vaccine is authorized by regulatory authorities and used in the population at large, post-licensure studies are required to assess vaccine safety and effectiveness, as well as impact of vaccination programmes [3]. This monitoring of the benefit–risk balance is currently an integral part of the life cycle of vaccines [4].

Vaccine efficacy, and the study designs used to measure it, are clearly defined [3], [5], [6]. But the distinction from vaccine effectiveness is often ignored [3], [5], [6]. The term impact of a vaccination programme is widely used in the literature and by international agencies but what it exactly means and how to measure it is usually not described and certainly not standardized [1], [2], [7], [8], [9]. Overall, post-licensure vaccine studies use diverging terms to describe different types of effect. This confusion in terminology implies that the type of post-licensure effects that is expected from a vaccine – or a vaccination programme – is rarely clarified at the level of decision-makers.

Post-licensure studies are mainly observational because the real-life effects of a vaccine administered in a public health programme are difficult to measure in an experimental design. Observational designs are based on a number of assumptions that are rarely met, and their findings are thus prone to biases. Decision-makers in the vaccine world need to know which effect is measured and which are the methodological limitations to be able to interpret and use the findings of post-licensure studies.

Our objective is to review and delineate, among the various evaluations of vaccine intervention, what applies to the effectiveness of vaccines and to the impact of vaccination programmes. We propose a bridge between the effects of vaccine, as defined in previous work [10], [11], and epidemiological measures of public health impact. We describe relevant methods to measure these effects and discuss the assumptions and potential biases that are involved.

Section snippets

Definitions

Vaccine efficacy is commonly defined as the direct effect of a vaccine measured in pre-licensure randomized clinical trials, where vaccination is allocated under optimal conditions (Table 1) [3], [10]. It is estimated by comparing disease occurrence between vaccinated and non-vaccinated individuals in a population. When vaccination is randomly administered and blinding is ensured, it is assumed that any differences in disease occurrence can be attributed to the direct effect of the vaccine [3],

Study designs

Post-licensure vaccine-related effects are measured in observational studies. These may involve confounding and biases that random allocation and blinding ideally control in pre-licensure randomized trials [3]. Table 3 describes the study designs to measure vaccine effectiveness and vaccination programme impact as well as the population groups that are compared.

Assumptions and requirements

Several assumptions have to be made in vaccine studies. The main types of bias and confounding factors are described in Table 4.

Greenwood and Yules stated three conditions for valid inference in vaccine studies [60]: (1) the persons must be alike in all relevant material aspects, including equal susceptibility to the disease; (2) the probability to be exposed to infection must be identical in the vaccinated and unvaccinated groups; and (3) the probability of being vaccinated must be independent

Conclusions

Authorization of new vaccines is increasingly based on surrogates of protection and information on the clinical benefit is often lacking when national authorities must decide on vaccine policies. As placebo-controlled clinical trials are no longer considered ethical when the benefit of an intervention is proven, effectiveness and impact measures provided by post-licensure observational studies are becoming crucial for decision-making.

Observational studies measure different effects of the

Acknowledgments

We thank Pierre Van Damme and Koen Van Herck for their useful input on a previous version of this manuscript, as well as the organizers and participants to the workshop “Current challenges and new methodological approaches to assess vaccine effectiveness and vaccination impact” held in Veyrier-du-Lac in September 2009.

References (67)

  • G. de Serres et al.

    Use of surveillance data to estimate the effectiveness of the 7-valent conjugate pneumococcal vaccine in children less than 5 years of age over a 9 year period

    Vaccine

    (2012)
  • M. Valenciano et al.

    Study designs for timely estimation of influenza vaccine effectiveness using European sentinel practitioner networks

    Vaccine

    (2010)
  • J. Puig-Barbera et al.

    Effectiveness of the 2010–2011 seasonal influenza vaccine in preventing confirmed influenza hospitalizations in adults: a case–case comparison, case–control study

    Vaccine

    (2012)
  • A. Gagneur et al.

    Impact of rotavirus vaccination on hospitalizations for rotavirus diarrhea: the IVANHOE study

    Vaccine

    (2011)
  • M.M. Patel et al.

    The impact of indirect benefits of vaccination on postlicensure vaccine effectiveness estimates: a scenario analysis

    Vaccine

    (2010)
  • G. Hanquet et al.

    Surveillance of invasive pneumococcal disease in 30EU countries: towards a European system?

    Vaccine

    (2010)
  • A. Hviid

    Effectiveness of two pertussis vaccines in preterm Danish children

    Vaccine

    (2009)
  • M.I. Khan et al.

    Effectiveness of Vi capsular polysaccharide typhoid vaccine among children: a cluster randomized trial in Karachi, Pakistan

    Vaccine

    (2012)
  • (2005 May)
  • Guidelines on clinical evaluation of vaccines: regulatory expectations

    (2004)
  • R.T. Chen et al.

    Epidemiologic methods in immunization programs

    Epidemiol Rev

    (1996)
  • Periodic benefit-risk evaluation report (PBRER)

    (2012)
  • M. Haber et al.

    Measures of the effects of vaccination in a randomly mixing population

    Int J Epidemiol

    (1991)
  • Vaccines and Biologicals Training manual on the critical regulatory function for vaccines: evaluation of clinical performance through authorized clinical trials. Prepared for national regulatory authorities of vaccine-procuring and vaccine-producing countries

    (2003)
  • From the centers for disease control and prevention. Impact of vaccines universally recommended for children – United States, 1900–1998

    JAMA

    (1999)
  • Centers for Disease Control and Prevention (CDC)

    Impact of vaccines universally recommended for children – United States, 1990–1998

    MMWR Morb Mortal Wkly Rep

    (1999)
  • World Health Organization (WHO). Generic protocol for monitoring impact of rotavirus vaccination on gastroenteritis...
  • M.E. Halloran et al.

    Direct and indirect effects in vaccine efficacy and effectiveness

    Am J Epidemiol

    (1991)
  • M.E. Halloran et al.

    Study designs for dependent happenings

    Epidemiology

    (1991)
  • Centers for Disease Control and Prevention (CDC). Manual for the surveillance of vaccine-preventable diseases. 4th ed....
  • J. Clemens et al.

    Evaluating new vaccines for developing countries. Efficacy or effectiveness?

    JAMA

    (1996)
  • J.E. Tate et al.

    Uptake, impact, and effectiveness of rotavirus vaccination in the United States: review of the first 3 years of postlicensure data

    Pediatr Infect Dis J

    (2011)
  • J.E. Tate et al.

    Decline and change in seasonality of us rotavirus activity after the introduction of rotavirus vaccine

    Pediatrics

    (2009)
  • Cited by (105)

    View all citing articles on Scopus
    View full text