Vaccination type comparisons

Three studies including two vaccine types.23–25 Adjuvanted vaccination, while associated with a higher cost of vaccination, was cost-effective when compared with standard vaccination as higher efficacy rates increased health benefits.23 25 Although the two studies focusing on adjuvanted vaccination identified it as being cost-effective, the incremental cost-effectiveness ratio (ICER) results were very different (dominant vs a cost per QALY gain). It is difficult to assess where the difference in the result arose from, due to a lack of reporting incremental health benefits and net costs. However, there were some key differences in the studies that could have affected this, including modelling characteristics (decision tree vs Bayesian network model), the population risk level, sources of effectiveness evidence and in particular the influenza attack rate. The latter in particular stands out as being very different between studies; Piercy et al (2004) assumed a rate of 5% (author’s assumptions), whereas Baio et al 23 used estimates from the literature, with a mean of 16%.23 25

Quadrivalent vaccination was cost-effective when compared with trivalent vaccination in the base case scenario modelled by Meier et al. 24 However, this study does not make it clear whether the data used to inform efficacy of the vaccination for the base case results were specific to the elderly. The study also considered two scenarios (a best-matching season and a worst-matching season) but did not report the results separately for the elderly population.

Two studies which compared vaccination types included probabilistic sensitivity analysis. Baio et al 23 estimated that adjuvanted vaccination was over 90% likely to be cost-effective compared with standard vaccination.23 Meier et al 24 included probabilistic sensitivity analysis.24 The authors found that quadrivalent vaccination was estimated to be cost-effective in between 68% and 87% of scenarios compared with trivalent across the total modelled population. However, this was not restricted to the elderly population subgroup; thus, it is impossible to draw conclusions from this study about the uncertainty around estimates that are specific to the elderly population.

Vaccination strategy comparisons

Vaccination intervention (ie, strategies that increase the level of vaccination within a population) appears to be cost-effective throughout the studies for both the primary and sensitivity analyses conducted, with the exception of the study of Allsup et al 26 which reported a significantly higher ICER. As previously noted, studies included different levels of vaccination in their intervention/comparator arms; however, this does not appear to cause any systematic differences in results.

The study of Allsup et al 26 is noted to have significantly different results to the other studies. This study focused on the low-risk population only. In addition, it applied lower reductions in the risk of hospitalisation from vaccination and lower costs of hospitalisations. It also applied a cost of vaccination promotion where the other studies did not, increasing the validity of costs as this is likely to be needed to reach high vaccination levels. This was also the only study completed alongside a randomised controlled trial (RCT); the remainder were economic models using secondary data sources. Studies conducted within RCT have high internal validity, and this study appeared to be robust and was overall well reported. However, Jit et al 27 noted that there are some issues when relying on one single trial for an economic evaluation has problems, in particular, a lack of external validity as the vaccine efficacy changes each season; it is possible that the study was conducted in a season in which vaccine matching was poor.27

The susceptibility of the population has important implications for the rate of complications; if more people are at risk for complications, vaccination is likely to produce larger health gains. One study compared results between the low- and high-risk population.28 As expected, the vaccination of high-risk individuals was demonstrated to be more cost-effective than vaccinating low-risk individuals, as this population is more susceptible to complications, which are costly and negatively impact quality of life.28 This trend seems to be reflected in the other studies identified, for example, with Brydak et al 29 and Lugner et al 30; as the latter have a greater proportion of high-risk individuals, they have a more favourable ICER per QALY gained (in the same countries).29 30

A passive vaccination strategy was found to be more cost-effective compared with no intervention than a comprehensive/targeted strategy.31 Comprehensive strategies are associated with greater health benefits, but the passive strategy has reduced costs as they avoid the additional consultation costs, only vaccinating when people present at the general practitioner (GP) for other reasons.

One model included transmission rates and captured externalities arising from herd immunity.30 When these indirect effects were included, results became more favourable (from an ICER that was judged to be below cost-effective thresholds to dominant), as it demonstrated cost savings. The inclusion of herd immunity has important implications for the vaccination coverage in the intervention and comparator arm. Herd immunity means that the impact of increasing vaccination levels is not linear, for example, an equal change in the coverage rate between studies could have very different results depending on what the comparator/usual care coverage rate is, as the scope for benefits from herd immunity will be different. While this does not affect this review because only one study included herd immunity, it is an important point for future researchers looking to compare study results as more studies including herd immunity become available in the future.

It would be expected that different countries have comparatively different cost-effectiveness results, due to differences in healthcare service design and population differences that may affect the attack rate and complication rates. However, there does not seem to be a clear pattern demonstrated across the studies included. For example, three studies included the UK, but results are very different; one found vaccination to be cost saving and more effective (dominant), another found it to be cost increasing but cost-effective and another found it to be cost-increasing and not cost-effective.26 30 31 Some of this difference can be explained by assumptions made about the vaccination cost and other study differences (eg, susceptibility of the population). However, one key issue when interpreting results across countries is a lack of evidence specific to each population (discussed further in the critical appraisal section).

One study which compared vaccination to no intervention included probabilistic sensitivity analysis and determined that vaccination was 79.93% likely to be cost-effective (below the threshold of 3 GDP per capita).29 Across all studies (irrespective of intervention and comparator), scenario and one-way sensitivity analysis demonstrated key variables to be the incidence of influenza, vaccine efficacy, discount rates and complication rates (mortality and hospitalisation). In general, studies showed that despite parameter uncertainty, the majority of tested scenarios still demonstrated vaccination to be cost-effective. Further detail on sensitivity analysis conducted is provided in an overview table in the online supplementary material.

Intervention and comparator

Three studies directly compared two vaccine types and found that the cost-effectiveness of vaccination varied by type of vaccine.23–25 Six studies included a comparison of a strategy to vaccinate older people to no vaccination strategy.23 24 26 28–31 Of these studies, the comparator arm included a proportion of participants who would be vaccinated which ranged from 0% to 27% in low- or mixed-risk populations and 68% in a high-risk population. The reported coverage rates for the countries included in the studies ranged from 12% to 76%. The data indicate that practice varies between countries and that the decision to implement or change a vaccination strategy for older people needs to take the underlying coverage rate into account. Two studies considered a vaccination level very similar to reality, improving the validity of these studies.24 29 One study included antiviral drugs as a comparator but did not directly compare them to a vaccination intervention arm and so results are not presented.31

Four studies, which synthesised evidence from multiple sources within an economic model, did not report vaccination type (eg, adjuvant, quadrivalent, etc).24 28 30 31 As there are different vaccination types available, it would be more robust if evaluations focused on a specific type; in these cases, it is not known whether data may have been synthesised from studies for different vaccination types, reducing validity.

Effectiveness data

One study was conducted alongside an RCT.26 This was single blinded only as double blinding was not feasible due to the intervention type (injection), which may introduce bias. One study identified the efficacy inputs from prospective cohort studies which were not randomised or blinded to intervention, increasing the risk of bias.23 The authors did not report detailed information about the cohort study design or participants, so it is not possible to assess the validity and robustness of the data and analyses used in the economic model. The remaining studies synthesised data about vaccine efficacy/effectiveness from several sources. None of these economic evaluations reported the methods used to identify and select studies to estimate effectiveness data inputs. Most of the authors reported that they needed to supplement data on vaccine efficacy with evidence from other countries or apply the same efficacy values across different countries in the same study.24 25 28–31 This means that it is not possible to assess the validity and robustness of the vaccine efficacy/effectiveness data used in the economic models. For example, Scuffham and West31 used the same efficacy source for three countries, and many of the papers use the same network meta-analysis for effectiveness of the influenza vaccination. This reduces the validity of study results and suggests that the studies may be over-representing a limited evidence base.

Measure of health benefit

Four studies undertook a cost-effectiveness analysis alone, with the measure of health benefit provided in terms of life-years gained or deaths averted,23 25 28 31 two studies performed a cost-utility analysis with quality-adjusted life-years (QALYs) as the measure of health benefit24 30 and two studies included both cost-effectiveness and cost-utility analyses.26 29 All studies assessed life-years gained as a result of avoiding death related to influenza complications. In the four studies reporting cost-utility results, a single study accounted for morbidity by applying lost utility related to mortality to estimate QALYs.26 The other studies also accounted for short-term changes in utility as a result of having influenza.24 29 30

The robustness of QALY estimates is questionable due to the limited relevant evidence and small study samples of utility studies. A review concluded that no utility studies using robust methods were available for influenza, which was reflected in the cost-utility evaluations identified.35 Lugner et al 30 took utility decrements relating to influenza from an English postal survey of 288 patients with confirmed influenza, using the European Quality of Life 5-Dimension Health Questionnaire.30 36 The generalisability of this study to the elderly Dutch population is difficult to assess as patient characteristics were not reported. Meier et al 24 and Brydak et al 29 applied utility values taken from a survey of 15 working age healthcare workers in the USA with utilities scored using the health utilities index.24 29 37 This is arguably not relevant to the elderly population. The WHO guidelines recommend that economic evaluations are cost-utility analyses; however, without robust utility data, cost-utility evaluations will inevitably have limited validity.32

ILI was used in all studies as a proxy for influenza. The severity and subsequent cost of influenza can vary substantially; therefore, there are concerns that this outcome is too broad.38 However, this is the primary end point of clinical trials, and subsequently, studies are restricted by this.38 Longer term outcomes associated with influenza were not considered, likely due to a lack of robust clinical data and the impact of confounding factors in the long term.11 The likely direction of bias caused by excluding long-term outcomes is unclear; it may understate the value of vaccination if it increases a person’s health over time or may overstate the value of vaccination if side effects occur in the long term.

Costs

Seven out of eight studies used a 1-year time horizon for costs, consistent with an influenza season and the short-term/immediate associated costs; therefore, discounting costs were irrelevant.23 25 26 28–31 The remaining study considered repeat vaccinations and costs each influenza season over a lifetime.24 Where relevant, studies applied country-specific guidelines for discounting outcomes, ranging from 1.5% to 5% annually.

The majority of studies considered direct costs only. This approach is justified as indirect costs are less important in a retired population. Societal costs (eg, over-the-counter drugs) are likely to be minimal due to the nature of the illness. Appropriately, the only study that included productivity losses did so because they considered herd immunity effects in the working age population; the study also included a scenario in which only direct costs were considered.30 All studies included costs associated with purchasing vaccines, influenza-related treatments, primary care visits and hospitalisations. Six studies included administration costs.25 26 28–31 The total cost of vaccination varied from €8.50 to €35.95 per person, and differences between studies mainly occurred due to variation in administration costs rather than variation in vaccine cost; lower costs were associated with opportunistic vaccination. Transportation costs, over-the-counter medication costs and private nursing home costs were considered in the societal perspective of one study.24 A single study included vaccination side effect costs; in the short term, these are rare and usually require only minimal medical attention.31 The inclusion of side effect costs did not have a significant effect on total costs or overall ICER results in this study (supported by a sensitivity analysis).31 Longer term evidence is lacking, with the majority of studies in the elderly population having short time horizons; hence, data on longer term side effects would not be identified and could not have been included by studies; in the future if studies expand their time horizons, this may have implications on results.1 11 Only one study applied costs to promote vaccination.26 No studies considered the cost of service redesign to increase vaccination (eg, opening additional clinics) which may be relevant for healthcare systems with restraints on resources available (eg, in the case of an ageing population). Thus, the true cost of vaccination is likely to be underestimated across studies depending on the healthcare setting.

Model design

On the whole, modelling approaches were relatively simple (eg, static decision tree models), reflecting gaps in the evidence base and the short time horizon captured in effectiveness studies. As the main outcomes of the vaccination occur over a single influenza season and occur once only during this time (ie, influenza infection or no infection), decision trees are a clear and logical structure for the model and are likely to be sufficient to capture the majority of health outcomes and costs. One study used a dynamic transmission model and was therefore able to capture the impact of transmission and herd immunity.30 This characterises interactions across the population and is therefore more reflective of the spread of disease and the indirect benefit for those not vaccinated. Overall, this model was reported well and seemed to be robust, but it did require more data than the simpler model approaches. The authors in this case had access to data on population interactions, but this may not be available in every country. Models that exclude herd immunity are likely to underestimate the cost-effectiveness of influenza vaccination; however, WHO guidelines recommend that a static model is acceptable as a conservative approach.32

More complex methodologies may capture the nature of the disease more accurately; however, given the noted lack of clinical evidence, it is unlikely that the current evidence base is sufficient to inform these.11 For example, patient-level simulations can also be more precise when modelling population interactions.39 Additionally, stochastic models, with events occurring randomly, can more realistically model epidemics, as these occur by chance of transmission among population interactions.20

Methods to address uncertainty

Most scenarios tested in sensitivity analysis still resulted in favourable cost-effectiveness results for influenza vaccination. However, there were limitations to the range of sensitivity analyses reported, given that many of the studies recognised that there are issues with the evidence base. Methods used to analyse uncertainty were varied, such that results are not comparable. The WHO recommends that sensitivity analyses for economic evaluations of vaccines should vary the following five parameters as a minimum: discount rates, vaccine efficacy, influenza incidence, influenza complication rates and vaccine price.32 None of the studies identified varied all of the recommended parameters, and because of this, we cannot fully assess the robustness of the results. One of the studies performed a comprehensive one-way sensitivity analysis, presenting a resulting tornado diagram.29 One study did not perform any one-way/scenario analysis.23 The remaining studies chose to vary a limited selection of key variables but did not explain the rationale for these.24–26 28 30 31 Three studies provided and justified ranges used in sensitivity analyses.24 25 29 The remainder detailed the ranges but did not provide justification, which means that validity cannot be assessed.26 28 30 31 Three studies included probabilistic sensitivity analysis, but one did not report the results restricted to the elderly population alone.23 24 29 This lack of thorough investigation and transparency restricts our ability to assess the robustness of the results or to explore the transferability of study results to alternative countries, settings and influenza seasons.