Original Article
New strategies are needed to improve the accuracy of influenza vaccine effectiveness estimates among seniors

https://doi.org/10.1016/j.jclinepi.2008.06.014Get rights and content

Abstract

Objective

The magnitude of the benefit of influenza vaccine among elderly individuals has been recently debated. Existing vaccine effectiveness estimates derive primarily from observational studies, which may be biased. In this paper, we provide a methodological examination of the potential sources of bias in observational studies of influenza vaccine effectiveness in seniors and propose design and analysis strategies to reduce bias in future studies.

Study Design and Setting

We draw parallels to bias documented in observational studies of therapies in other areas of medical research including pharmacoepidemiology, discuss reasons why existing adjustment methods in influenza studies may not adequately control for the bias, and evaluate statistical approaches that may yield more accurate estimation of influenza vaccine effectiveness.

Results

There is strong evidence for the presence of bias in existing observational estimates of influenza vaccine effectiveness in the elderly and the failure of current adjustment methods to reduce bias.

Conclusion

Promising approaches for reducing bias include obtaining more accurate information on confounders, such as functional status, avoiding all-cause death in favor of outcomes, such as pneumonia or influenza-related pneumonia, and evaluating the extent to which bias is reduced by these and other methods using the ‘control’ period before influenza season.

Introduction

In the United States, inactivated trivalent influenza vaccine is recommended for all persons aged 65 years and older to reduce influenza-related hospitalizations and deaths [1]. Accurately estimating the public health benefit of this strategy is important for pandemic planning and to judge the need for alternate prevention approaches during nonpandemic periods, such as universal childhood immunization to produce herd immunity in seniors [2], [3], [4], [5]. The need for alternative strategies depends on the magnitude of the benefit of the current strategy, which has been recently debated [6], [7], [8], [9], [10].

One problem is that there have been few randomized trials of the efficacy of influenza immunization in older adults, and the available trial evidence is based primarily on healthy, younger seniors. For example, although the largest trial of influenza vaccine efficacy in the elderly, which was restricted to healthy persons, observed a reduction in risk of serologically confirmed uncomplicated influenza infection in participants 60–69 years of age, the study lacked power to draw firm conclusions about the magnitude of vaccine effectiveness among those 70 years and older [11]. Furthermore, the efficacy findings in younger, healthy seniors may not apply to older and frail seniors because advanced age and the presence of serious medical conditions are associated with important changes in the function of the immune system, including a diminished immune response to influenza vaccine [12], [13], [14], [15], [16], [17], [18].

Without data from randomized controlled trials, existing estimates of influenza vaccine effectiveness in seniors derive from observational studies, which may be biased. Numerous observational studies have compared the risk of pneumonia hospitalization and all-cause mortality in vaccinated and unvaccinated seniors during influenza season and have consistently reported substantial reductions in risk for vaccinated seniors, including reductions on the order of 50% for all-cause mortality [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36]. These results have been interpreted by some as evidence that influenza vaccination substantially reduces the risk of death and hospitalization in the elderly [1], [37], [38], [39], [40], [41], [42] and have been incorporated into economic analyses reporting that the senior vaccination program is both cost-saving [20], [21], [43], [44] and among the most effective of all clinical preventive services [45].

However some studies, including a recent review of the literature [6], suggest that bias due to differences between vaccinated and unvaccinated seniors may account for some or all of previously observed risk differences. In the present article, we briefly review the evidence for bias in published observational studies of influenza vaccine effectiveness and then provide a methodological examination of the potential sources of this bias. We draw parallels to bias documented in observational studies of therapies in other areas of medical research, including pharmacoepidemiologic database studies of elderly persons, and we discuss reasons why existing adjustment methods may inadequately control for the bias. Last, we propose and evaluate study design and analytic strategies that may yield more accurate estimation of the public health benefit of influenza vaccine in future studies.

Section snippets

Evidence of bias in existing studies

Existing observational studies of influenza vaccine effectiveness typically use data from research databases, such as the General Practitioners Research database, or health care utilization data systems, such as those maintained by some health maintenance organizations in the United States. Schneeweiss and Avorn [46] have discussed general methodological issues that arise when using such databases in health research including data inaccuracies and residual confounding, but the methodological

Possible sources of bias

A credible explanation recently posed [6], [9] for the source of the healthy vaccinee bias in existing observational studies is selective underuse by frail seniors. Although universally recommended for seniors, receipt of influenza vaccine is voluntary and thus may be preferentially sought by motivated and healthier individuals. This healthy adherer effect has been well documented in evaluations of persons randomized to placebo in randomized trials [51], [52], [53], [54], [55]. For example, in

Problems with existing confounder adjustment methods

Healthy vaccinee bias can be overcome if relevant confounders are measurable and measured well. Most previous observational studies relied solely on information available in research and health care utilization databases to attempt to characterize confounders, often adjusting for the number of outpatient visits and for chronic illness variables defined by groupings of International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnostic codes assigned to patient

Strategies to reduce bias

Given the evidence for bias in existing observational estimates and the inability of previous adjustment methods to control for it, we recommend that new strategies be considered to reduce bias in future studies. First, a wider array of potential predictors of vaccination should be studied to more accurately characterize confounders, particularly given the potency of confounders, such as functional status that are not reliably classified by the information available in research and health care

Conclusion

Assessing the public health benefit of influenza vaccination among elderly individuals is an important problem and one fraught with considerable methodological challenges. There is strong evidence for the presence of bias in existing observational estimates of influenza vaccine effectiveness in the elderly and that current adjustment methods do not adequately control for it. Promising approaches for reducing this bias include obtaining more accurate information on confounders, such as

Acknowledgment

Internal funds supported this publication.

References (76)

  • S.A. Harper et al.

    Prevention and control of influenza. Recommendations of the Advisory Committee on Immunization Practices (ACIP)

    MMWR Recomm Rep

    (2005)
  • W.P. Glezen et al.

    Commentary: benefits of influenza vaccine in US elderly—new studies raise questions

    Int J Epidemiol

    (2006)
  • M.L. Pearson et al.

    Influenza vaccination of health-care personnel: recommendations of the Healthcare Infection Control Practices Advisory Committee (HICPAC) and the Advisory Committee on Immunization Practices (ACIP)

    MMWR Recomm Rep

    (2006)
  • P.A. Piedra et al.

    Herd immunity in adults against influenza-related illnesses with use of the trivalent-live attenuated influenza vaccine (CAIV-T) in children

    Vaccine

    (2005)
  • D. Weycker et al.

    Population-wide benefits of routine vaccination of children against influenza

    Vaccine

    (2005)
  • L. Simonsen et al.

    Mortality benefits of influenza vaccination in elderly people: an ongoing controversy

    Lancet Infect Dis

    (2007)
  • L. Simonsen et al.

    Impact of influenza vaccination on seasonal mortality in the US elderly population

    Arch Intern Med

    (2005)
  • L.A. Jackson et al.

    Evidence of bias in estimates of influenza vaccine effectiveness in seniors

    Int J Epidemiol

    (2006)
  • L.A. Jackson et al.

    Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors

    Int J Epidemiol

    (2006)
  • T. Jefferson

    Influenza vaccination: policy versus evidence

    BMJ

    (2006)
  • T.M. Govaert et al.

    The efficacy of influenza vaccination in elderly individuals. A randomized double-blind placebo-controlled trial

    JAMA

    (1994)
  • K. Goodwin et al.

    Antibody response to influenza vaccination in the elderly: a quantitative review

    Vaccine

    (2006)
  • W.E. Beyer et al.

    Antibody induction by influenza vaccines in the elderly: a review of the literature

    Vaccine

    (1989)
  • R.J. Simons et al.

    Altered immune status in the elderly

    Semin Respir Infect

    (1990)
  • B.R. Ruf et al.

    Open, randomized study to compare the immunogenicity and reactogenicity of an influenza split vaccine with an MF59-adjuvanted subunit vaccine and a virosome-based subunit vaccine in elderly

    Infection

    (2004)
  • I. Kang et al.

    Age-associated change in the frequency of memory CD4+ T cells impairs long term CD4+ T cell responses to influenza vaccine

    J Immunol

    (2004)
  • Y. Deng et al.

    Age-related impaired type 1 T cell responses to influenza: reduced activation ex vivo, decreased expansion in CTL culture in vitro, and blunted response to influenza vaccination in vivo in the elderly

    J Immunol

    (2004)
  • W.B. Ershler et al.

    Influenza and aging: age-related changes and the effects of thymosin on the antibody response to influenza vaccine

    J Clin Immunol

    (1984)
  • D.A. Foster et al.

    Influenza vaccine effectiveness in preventing hospitalization for pneumonia in the elderly

    Am J Epidemiol

    (1992)
  • K.L. Nichol et al.

    The efficacy and cost effectiveness of vaccination against influenza among elderly persons living in the community

    N Engl J Med

    (1994)
  • J.P. Mullooly et al.

    Influenza vaccination programs for elderly persons: cost-effectiveness in a health maintenance organization

    Ann Intern Med

    (1994)
  • S.E. Ohmit et al.

    Influenza vaccine effectiveness in preventing hospitalization among the elderly during influenza type A and type B seasons

    Int J Epidemiol

    (1995)
  • D.M. Fleming et al.

    Study of the effectiveness of influenza vaccination in the elderly in the epidemic of 1989–90 using a general practice database

    Epidemiol Infect

    (1995)
  • J. Puig-Barbera et al.

    Reduction in hospital admissions for pneumonia in non-institutionalised elderly people as a result of influenza vaccination: a case-control study in Spain

    J Epidemiol Community Health

    (1997)
  • K.L. Nichol et al.

    Benefits of influenza vaccination for low-, intermediate-, and high-risk senior citizens

    Arch Intern Med

    (1998)
  • K.L. Nichol et al.

    Relation between influenza vaccination and outpatient visits, hospitalization, and mortality in elderly persons with chronic lung disease

    Ann Intern Med

    (1999)
  • C.P. Schade et al.

    Influenza immunization and mortality among diabetic Medicare beneficiaries in West Virginia

    W V Med J

    (2000)
  • J. Nordin et al.

    Influenza vaccine effectiveness in preventing hospitalizations and deaths in persons 65 years or older in Minnesota, New York, and Oregon: data from 3 health plans

    J Infect Dis

    (2001)
  • K.L. Nichol et al.

    Influenza vaccination and reduction in hospitalizations for cardiac disease and stroke among the elderly

    N Engl J Med

    (2003)
  • K.L. Nichol et al.

    Effectiveness of influenza vaccine in the community-dwelling elderly

    N Engl J Med

    (2007)
  • B.C. Voordouw et al.

    Influenza vaccination in community-dwelling elderly: impact on mortality and influenza-associated morbidity

    Arch Intern Med

    (2003)
  • D.S. Fedson et al.

    Clinical effectiveness of influenza vaccination in Manitoba

    JAMA

    (1993)
  • J. Puig-Barbera et al.

    Effectiveness of the MF59-adjuvanted influenza vaccine in preventing emergency admissions for pneumonia in the elderly over 64 years of age

    Vaccine

    (2004)
  • P. Mangtani et al.

    A cohort study of the effectiveness of influenza vaccine in older people, performed using the United Kingdom general practice research database

    J Infect Dis

    (2004)
  • S.T. Wang et al.

    Economic evaluation of vaccination against influenza in the elderly: an experience from a population-based influenza vaccination program in Taiwan

    Vaccine

    (2005)
  • E. Hak et al.

    Clinical effectiveness of influenza vaccination in persons younger than 65 years with high-risk medical conditions: the PRISMA Study

    Arch Intern Med

    (2005)
  • G.A. Poland

    If you could halve the mortality rate, would you do it?

    Clin Infect Dis

    (2002)
  • T. Jefferson et al.

    Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review

    Lancet

    (2005)
  • Cited by (79)

    • Guidance for design and analysis of observational studies of fetal and newborn outcomes following COVID-19 vaccination during pregnancy

      2021, Vaccine
      Citation Excerpt :

      Depending on the characteristics of a confounding factor (e.g., prevalence in the vaccine-exposed and vaccine-unexposed groups and strength/direction of association with the outcome), estimates can be biased toward or away from the null value, with the potential to dilute, obscure, or reverse true associations. Confounding bias has been well-documented in observational studies of influenza vaccination; healthier older adults are more likely to be vaccinated, which exaggerates beneficial effects of vaccination and creates a “healthy vaccinee bias” [15]. In the obstetrical population, the influence of confounding factors is less predictable and may depend on local vaccine recommendations/practices or other factors that affect uptake.

    • Effectiveness of Influenza Vaccination Among Older Adults Across Kidney Function: Pooled Analysis of 2005-2006 Through 2014-2015 Influenza Seasons

      2020, American Journal of Kidney Diseases
      Citation Excerpt :

      To obtain an unbiased estimate when either propensity score analysis or logistic regression analysis was correctly modeled, we used the double robust estimation with the adjustment for age, sex, eGFR, diabetes, COPD/asthma, coronary heart disease, cancer, cerebrovascular disease, rheumatologic disease, history of pneumonia/influenza, dementia, heart failure, liver cirrhosis, number of outpatient visits and hospitalizations, and influenza seasons.23 For sensitivity analysis, we estimated ORs during the off-influenza season (ie, June 1 through August 31) to account for the possibility of healthy vaccinee bias, for which individuals receiving an influenza vaccination may have a better prognosis unrelated to the vaccination effects because they are healthy enough to come into the clinic and/or they have a high awareness about the importance of vaccination.24 Additionally, a prespecified subgroup analysis was performed for age (65-74 vs ≥ 75 years), sex (male vs female), diabetes (no vs yes), and history of cardiac disease (coronary heart disease or heart failure; no vs yes).

    View all citing articles on Scopus
    View full text