Article Text

Original research
Effectiveness of mRNA COVID-19 vaccines against Omicron and Delta variants in a matched test-negative case–control study among US veterans
  1. Yinong Young-Xu1,
  2. Gabrielle M Zwain1,
  3. Hector S Izurieta2,
  4. Caroline Korves1,
  5. Ethan I Powell1,
  6. Jeremy Smith1,
  7. Abirami Balajee1,
  8. Mark Holodniy3,4,5,
  9. David O Beenhouwer6,7,
  10. Maria C Rodriguez-Barradas8,9,
  11. Sheldon T Brown10,11,
  12. Vincent C Marconi12,13,14
  1. 1White River Junction VA Medical Center, US Department of Veterans Affairs, White River Junction, Vermont, USA
  2. 2Food and Drug Administration, Silver Springs, Maryland, USA
  3. 3Veterans Affairs Palo Alto Health Care System, Palo Alto, California, USA
  4. 4Public Health Program Office, Department of Veterans Affairs, Washington, District of Columbia, USA
  5. 5Department of Medicine, Stanford University, Palo Alto, California, USA
  6. 6Veterans Affairs Greater Los Angeles Healthcare System, Los Angeles, California, USA
  7. 7Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, California, USA
  8. 8Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
  9. 9Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
  10. 10Department of Medicine, Icahn School of Medicine at Mount Sinai, New York City, New York, USA
  11. 11James J. Peters Veterans Affairs Medical Center, New York City, New York, USA
  12. 12Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, USA
  13. 13Atlanta Veterans Affairs Medical Center, Decatur, Georgia, USA
  14. 14Department of Global Health, Rollins School of Public Health, Atlanta, Georgia, USA
  1. Correspondence to Dr Yinong Young-Xu; Yinong.Young-Xu{at}va.gov

Abstract

Objective To estimate the effectiveness of messenger RNA (mRNA) booster doses during the period of Delta and Omicron variant dominance.

Design We conducted a matched test-negative case–control study to estimate the vaccine effectiveness (VE) of three and two doses of mRNA vaccines against infection (regardless of symptoms) and against COVID-19-related hospitalisation and death.

Setting Veterans Health Administration.

Participants We used electronic health record data from 114 640 veterans who had a SARS-CoV-2 test during November 2021–January 2022. Patients were largely 65 years or older (52%), male (88%) and non-Hispanic white (59%).

Main outcome measures First positive result for a SARS-CoV-2 PCR or antigen test.

Results Against infection, booster doses had higher estimated VE (64%, 95% CI 63 to 65) than two-dose vaccination (12%, 95% CI 10 to 15) during the Omicron period. For the Delta period, the VE against infection was 90% (95% CI 88 to 92) among boosted vaccinees, higher than the VE among two-dose vaccinees (54%, 95% CI 50 to 57). Against hospitalisation, booster dose VE was 89% (95% CI 88 to 91) during Omicron and 94% (95% CI 90 to 96) during Delta; two-dose VE was 63% (95% CI 58 to 67) during Omicron and 75% (95% CI 69 to 80) during Delta. Against death, the VE with a booster dose was 94% (95% CI 90 to 96) during Omicron and 96% (95% CI 87 to 99) during Delta.

Conclusions Among an older, mostly male, population with comorbidities, we found that an mRNA vaccine booster was highly effective against infection, hospitalisation and death. Although the effectiveness of booster vaccination against infection was moderately higher against Delta than against the Omicron SARS-CoV-2 variant, effectiveness against severe disease and death was similarly high against both variants.

  • public health
  • epidemiology
  • immunology

Data availability statement

Data may be obtained from a third party and are not publicly available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This study included a large, diverse population of US veterans and near real-time access to medical records.

  • We used a test-negative case–control approach to implicitly account for differences in health-seeking behaviours between cases and non-cases.

  • By restricting participation to individuals who had used the Department of Veterans Affairs health system in the past 2 years and were also tested for COVID-19 in the VA system, we increased our confidence that participants were active users of the VA healthcare system for vaccination, medical office visits and hospitalisations; nonetheless, some misclassification of vaccination status and missing hospitalisation records (for veterans seeking care outside of the Veterans Health Administration) were still possible.

  • Symptoms and reasons for SARS-CoV-2 testing were infrequently coded.

  • Sequencing data were not yet available for individual SARS-CoV-2 laboratory tests so we defined dominant circulation periods for each SARS-CoV-2 variant of interest based on the Centers for Disease Control and Prevention estimates of variant circulation in the USA.

Introduction

In March 2020, the WHO declared COVID-19, the disease caused by SARS-CoV-2, a pandemic.1 By December 2020, the US Food and Drug Administration (FDA) issued emergency use authorisations for two messenger RNA (mRNA) vaccines, based on the spike glycoprotein of the Wuhan strain of SARS-CoV-2, for prevention of COVID-19.2 3

Despite early reports on the effectiveness of these vaccines4–9 and the persistence of anti-SARS-CoV-2 antibodies following vaccination,10 frequent mutations have increasingly challenged the effectiveness of these vaccines.11 12 In July 2021, a new, more infectious variant, B.1.617.2 (Delta), became the most dominant variant in the USA.13 In November 2021, another variant, B.1.1.529 (Omicron), was identified in the USA and became the dominant variant by mid-December. Although the Omicron variant is more infectious than Delta, it is considered less deadly.14 However, the number of mutations in Omicron’s spike protein, compared with the Wuhan strain, is larger than Delta, raising questions regarding the effectiveness of the FDA-authorised vaccines against Omicron.15

We previously reported9 11 the effectiveness of COVID-19 mRNA vaccines against infection, hospitalisation and mortality among Veterans Health Administration (VHA)-enrolled veterans through September 2021, when the major circulating variants in the USA included Alpha, Beta and Delta. We found the estimated vaccine effectiveness (VE) to remain high in the first 3 months after vaccination and to decrease significantly after 5 months.11 With the rise of the Omicron variant in December 2021–January 2022 and the lengthening time from vaccination with the two-dose primary series, we updated our estimates of the VE to measure continued effectiveness of both mRNA vaccines in a population including the entire USA and included VE estimates for the booster (three-dose) vaccination.

Starting in September 2021, the Centers for Disease Control and Prevention (CDC) Advisory Committee on Immunization Practices recommended a booster dose of COVID-19 vaccine to increase protection.16 17 As of early March 2022, the VHA had over 4 million fully vaccinated veterans (two doses of mRNA vaccine or one dose of Johnson & Johnson’s Janssen COVID-19 vaccine) and over 1.6 million veterans had received an mRNA booster.18 As of March 2022, the VHA reported more than 613 000 COVID-19 cases and 21 309 confirmed deaths.18

Early reports19–22 showed lower VE against infection for the fully vaccinated (two-dose series) and even for those who received a booster (three-dose) during the Omicron period, but the VE against hospitalisation23 remained high during both periods. These early reports on the Omicron variant were regional in the USA, focused on one of the mRNA vaccines not both or were from outside the USA. We sought to estimate the VE against infection, hospitalisation and mortality for mRNA vaccines authorised in the USA (Pfizer-BioNTech and Moderna) for the fully vaccinated (two-dose) and those who received a booster (three-dose) of mRNA vaccines during the Delta and Omicron dominant periods13 in the US veteran population, which includes individuals across the USA with underlying health conditions and diverse socioeconomic backgrounds.

Methods

We followed the Strengthening the Reporting of Observational Studies in Epidemiology reporting guideline.

Patient and public involvement statement

Neither patients nor the public were involved in this study.

Data source

We used electronic medical records and COVID-19 laboratory test data from the VHA Corporate Data Warehouse (CDW) to define infection irrespective of symptoms. The VHA CDW holds electronic medical records for about nine million US veterans who use the VHA’s network of 1293 healthcare facilities, including hospitals and outpatient clinics.24 Using the CDC variant tracking data,13 we presumed positive tests during January 2022 were Omicron SARS-CoV-2 infections and those positive during November 2021 were Delta. Because December 2021 represented an overlap of both variants, we excluded positive tests from that month. As Medicare claims were available only up to August 2021, we used them to find COVID-19 diagnoses, vaccinations and hospitalisations prior to the study period to exclude patients with history of COVID-19 and to augment our vaccination records. Since this population of veterans was largely over 65 years old, we supplemented CDW data with available Medicare data because even frequent users of VHA may use Medicare for additional healthcare services. We used VHA CDW records to determine date of death and hospitalisations with admission diagnosis related to COVID-19.

Study design

To estimate the VE against infection, we conducted a matched test-negative case–control study.

The inclusion criteria were the following: (1) tested at the VHA for SARS-CoV-2 via antigen or PCR; (2) enrolled in VHA benefits for 2+ years and a veteran; (3) met the VHA use criteria of one inpatient or two outpatient visits during the 2 years prior to the study period; (4) resident of US state or Washington, DC; (5) exposed in the community (excluding those hospitalised >1 day prior to test); (6) 18 years old or older at the time of the test, with valid demographic data (excluding those with missing sex or date of birth); (7) COVID-19-naïve (excluding those with a positive test or diagnosis code in VHA or Medicare prior to the study period); (8) veterans with positive tests during the study period were included as cases; (9) veterans with negative tests during the study period were included as controls; (10) the test occurred during the study period (November 2021 (Delta predominance) or January 2022 (Omicron predominance)); and (11) classifiable vaccination status: 0–3 doses of mRNA vaccination. Exclusions were (1) negative tests taken within 10 days following a positive test and (2) veterans vaccinated with Janssen vaccine.

We matched each case (positive tests) with up to four controls (negative tests) based on the Health and Human Services geographical region and SARS-CoV-2 laboratory tests within 3 weeks of the case specimen collection date as both are measures of local disease burden. Demographic characteristics of the matched population were described by reporting the frequency and proportion for categorical variables and the mean (SD) for continuous variables. Standardised mean differences25 were used to describe differences in characteristics between cases and controls. For estimated VE against hospitalisation and death, we used a matched case–control design. Depending on the analysis, cases were those hospitalised or were those who died within 30 days of the positive COVID-19 test. Controls were those not hospitalised or who did not die within 30 days of their SARS-CoV-2 tests, irrespective of test results. Any patient with dual Medicare-VHA enrolment who had a claim for COVID-19-related hospitalisation prior to the study period was already excluded, thus ineligible to be a control. Hospitalisations were defined as a hospital admission diagnosis related to COVID-19 after a positive SARS-CoV-2 test.

To better inform our interpretation of VE against hospitalisation and mortality in the two time periods, we conducted a subanalysis where we assessed the VE against COVID-19-related hospitalisation and death among those already infected, restricting our hospitalisation and mortality analysis to patients with a positive test. Cases were defined as before, and controls were those who were not hospitalised or who did not die within 30 days of a positive test. We also compared the average length of stay (LOS) and rates of intensive care unit (ICU) admission among those hospitalised in the Delta and Omicron periods.

Exposure

We defined two-dose vaccination as receipt of two doses of an mRNA COVID-19 vaccine and booster vaccination as receipt of a third dose of an mRNA vaccine (about 95% of mRNA third doses occurred approximately 6 months after the second dose). We designated exposure to the two-dose regimen or a booster from 14 days following vaccination, excluding events on days 0–13. We designated patients as unvaccinated if they had no record of vaccination either in VHA CDW or in Medicare.

Statistical analysis

We used conditional logistic regression to calculate the odds ratio (OR) with 95% CI for the association between positive SARS-CoV-2 testing and two or three doses of mRNA COVID-19 vaccine during the Delta and Omicron periods. In the model, we included the number of mRNA vaccine dose (two or three) as the primary explanatory variable, an indicator variable for SARS-CoV-2 presumed variant (Delta or Omicron) based on test date, and their interaction terms. We estimated the association between SARS-CoV-2 infections and the number of mRNA vaccine dose by the OR, adjusting for age (continuous), race, rurality, VHA benefits priority and comorbid conditions (cancer, congestive heart failure, hypertension, immunocompromising conditions, obesity and diabetes). Confounders were determined based on a prior study9 11 and known factors associated with SARS-CoV-2.26 We also used conditional logistic regression to analyse the VE against hospitalisations and death. Due to sample size and the scope of the study, we did not analyse one dose of mRNA vaccine.

Severity of disease was determined by the average LOS in days between admission date and discharge date for overall COVID-19-related hospitalisations and the average LOS in an ICU during a hospitalisation related to COVID-19.

All tests were two-tailed and we chose 0.05 as the level of statistical significance. We performed data analysis using SAS V.9.4.

Results

Study population

In November 2021 (Delta period) there were 4134 positive SARS-CoV-2 tests (cases) and 170 316 negative tests (controls) (figure 1). After matching, 4134 cases and 16 536 controls remained (see table 1 for baseline characteristics); 2300 (56%) cases and 3951 (24%) controls were unvaccinated at the time of testing. In January 2022 (Omicron period) there were 32 983 positive tests (cases) and 263 780 negative tests (controls). After matching, 32 983 cases and 126 363 controls remained. At the time of testing, 13 153 (40%) cases and 29 110 (23%) controls remained unvaccinated. During both periods, cases tended to have a lower Charlson Comorbidity Index score and to be younger than controls. Patients included in this study were largely 65 years or older (94 447, 52%), male (158 395, 88%) and non-Hispanic white (105 437, 59%). The cases in both the Delta and Omicron periods had similar proportions of comorbid conditions at baseline, while the controls during the Delta period had a slightly higher proportion of comorbid conditions (table 1). The reasons for testing and the presence of symptoms from a random sample of individuals who presented for testing are displayed in online supplemental table S1.

Table 1

Baseline characteristics of matched study subjects

Figure 1

Attrition. mRNA, messenger RNA; VHA, Veterans Health Administration; DC, Washington, DC; VA, Department of Veterans Affairs.

VE against infection

For those with two-dose mRNA vaccine, the VE was 54% (95% CI 50 to 57) for Delta and 12% (95% CI 10 to 15) for Omicron. The VE for those who received an mRNA vaccine booster was 90% (95% CI 88 to 92) in the Delta period and 64% (95% CI 63 to 65) in the Omicron period (table 2). For the Delta period, the median time from the second dose to the SARS-CoV-2 test date was 241 days and 35 days from the third dose. As is expected this median time from the second dose of vaccination increased for the Omicron period to 289 days and 72 days from the third dose.

Table 2

Estimated vaccine effectiveness against laboratory-confirmed SARS-CoV-2 infection by dose and variant

Hospitalisation and death

The VE against COVID-19-related hospitalisation for those who received two-dose mRNA vaccine was 75% (95% CI 69 to 80) in the Delta period and 63% (95% CI 58 to 67) in the Omicron period (table 3). The VE against COVID-19-related hospitalisation for those who received an mRNA vaccine booster was 94% (95% CI 90 to 96) in the Delta period, higher than the two-dose VE, and 89% (95% CI 88 to 91) in the Omicron period (table 3). In a subanalysis restricted to those with a positive COVID-19 test during the study period, we evaluated the VE against progression to hospitalisation among the infected only and we found that the VE for booster against COVID-19-related hospitalisation was 53% (95% CI 12 to 75) in the Delta period, not statistically different from the two-dose VE, and 78% (95% CI 74 to 81) in the Omicron period, almost twice higher than the two-dose VE (online supplemental table S2). Among Omicron-associated hospitalisations (regardless of vaccination status), the LOS averaged 6.5 days (95% CI 6.2 to 6.7 days), compared with 7.2 days (95% CI 6.6 to 7.8) among Delta-associated hospitalisations. Of the hospitalisations during the Delta period, 11.3% (95% CI 8.8% to 13.8%) resulted in ICU admission; in contrast, 8.3% (95% CI 7.2% to 9.5%) of COVID-19-related hospitalisations during the Omicron period resulted in ICU admission (figure 2).

Table 3

Estimated vaccine effectiveness against COVID-19-related hospitalisation and death by dose and variant

Figure 2

Average hospital length of stay and intensive care unit (ICU) use during the Delta and Omicron periods. Length of stay is the number of days between admission and discharge dates.

For those who received two doses of mRNA vaccines, the VE against death was 92% (95% CI 85 to 97) during the Delta period and 77% (95% CI 67 to 83) during the Omicron period. The VE against death for those who received an mRNA vaccine booster was 96% (95% CI 87 to 99) and not different from the two-dose VE during the Delta period, and was 94% (95% CI 90 to 96), higher than the two-dose VE, during the Omicron period.

Discussion

Our study shows that veterans who received an mRNA vaccine booster were highly protected against COVID-19-related outcomes, particularly hospitalisation and death, during both the Delta and Omicron periods. These findings largely align with other VE studies of Omicron.19–23 27 Also, among veterans tested positive for COVID-19, irrespective of vaccination, those infected during the Omicron period were less likely than those infected during the Delta period to be hospitalised and, if hospitalised, had shorter LOS and lower likelihood of ICU admission.

We found that the booster dose significantly increased protection against infection, regardless of symptoms, providing higher protection than the two-dose regimen during both variant-predominant periods. However, the VE of the booster dose against Delta was consistently higher than that against Omicron. These results were generally aligned with published findings. Early results from Southern California20 showed that booster doses improved the VE against infection when compared with two doses during both the Delta and Omicron periods. Similarly, a study from Israel focusing specifically on the Pfizer mRNA vaccine during the Delta period estimated that a booster vaccination reduced the rate of confirmed infection compared with those who did not receive a booster, bringing the VE up to that reported against the Alpha variant.28

Although we found moderate VE against hospitalisation after two-dose vaccination, the booster dose significantly improved the VE against hospitalisation during the Delta and Omicron periods. The estimated VE of the booster dose against mortality was high during both the Delta and Omicron periods. These estimates of high VE against mortality resembled the reduced risk of mortality reported elsewhere in the USA.29

A study from South Africa that focused on two doses of the Pfizer vaccine found an estimated VE against hospitalisation of 93% (95% CI 90 to 94) during the Delta period and 70% (95% CI 62 to 76) during the Omicron period.19 While we found a similar drop in effectiveness from the Delta to the Omicron period, our VE estimates were lower for both periods. Waning protection over time and differences in populations may contribute to the discrepancy in our findings.

Our supplementary cases-only analysis showed that booster vaccination was still highly effective against hospitalisation among vaccine breakthrough cases in both the Omicron and Delta periods, with a higher point estimate for the Omicron period (online supplemental table S2). This is consistent with a study from the UK showing reduced risk of hospitalisation among those infected with the Omicron variant compared with the Delta variant.30

Our results highlight the value of booster doses for protection against infection and more importantly against hospitalisation and death. They also revealed distinctive features of the two variants and lower severity of illness for the Omicron variant as compared with the Delta variant.

Strengths and limitations

The strengths of this study include a large, diverse population and near real-time access to medical records. The VHA population offers insight into populations often under-represented in US studies: racial minorities with a wide range of socioeconomic backgrounds. Also, the VHA veteran population is older (47% above 65),31 sicker32 and predominantly male (90% male).31

Although data on symptomatic versus asymptomatic infections and reasons for SARS-CoV-2 testing were infrequently recorded, in a random sample of veterans who tested positive and had a detailed record, about a quarter reported no symptoms. Possible misclassification of vaccination status and missing hospitalisation records are of concerns because veterans could go elsewhere for vaccination and hospitalisation. For veterans enrolled in Medicare, we checked records through August 2021 (latest available Medicare data) for COVID-19 vaccination, diagnoses and related hospitalisations. We also limited the study population to veterans who routinely sought care at the VHA to minimise misclassification. We assessed patient vaccination status as of November 2021 and January 2022. Other studies, including one of our own, have examined waning mRNA VE against infection,11 33 34 so we did not labour further on this topic. Nevertheless, veterans with underlying conditions and health-seeking behaviours may have been vaccinated earlier, thus increased their likelihood of having reduced immunity.

Sequencing data were not available for individual SARS-CoV-2 laboratory tests so we defined variant periods based on the estimated variant proportions in the USA by the CDC. To reduce misclassification, we excluded December 2021, when Delta and Omicron variants shared dominance, analysing November 2021 (when Delta was almost 100% dominant) and January 2022 (when Omicron was 90%–100% dominant) separately.13 In a future study we plan to use individual sequencing data after they become available in the VHA.

Conclusion

In November 2021, the CDC expanded the booster recommendation to everyone 18 years and older 6 months after their second mRNA dose or 2 months after receiving Johnson & Johnson’s Janssen vaccine.7 At that time, the two-dose mRNA VE among our study population of VHA veterans was 54%, 75% and 92% against COVID-19 infection, hospitalisation and death due to the Delta variant, respectively. Our findings indicate that mRNA vaccine boosters were very effective against severe COVID-19-related outcomes during both the Delta and Omicron periods among a predominantly male population. Moreover, the booster vaccination reduced the likelihood of hospitalisation among infected patients. Overall, the mRNA vaccine boosters provided a level of protection against Omicron like that of two-dose vaccine against Delta, with VE at 64%, 89% and 94% against infection, hospitalisation and death, respectively. We can witness the impact of the mRNA vaccine boosters as case numbers continue to drop and the surge of the Omicron variant abates across the USA.

Data availability statement

Data may be obtained from a third party and are not publicly available.

Ethics statements

Patient consent for publication

Ethics approval

The study was approved by the Institutional Review Board (IRB) of the Department of Veterans Affairs Medical Center in White River Junction, Vermont (reference number: 1593089), which waived the requirement for informed consent due to impracticality and low risk of the study.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors YY-X, HSI, CK and VM conceptualised and designed the study. YY-X, EIP, GMZ, JS, AB and CK conducted the data analysis. YY-X, GMZ, HSI, CK, EIP, JS, AB, MH, DOB, MR-B, STB and VM reviewed the titles, abstract and full-text papers for eligibility. YY-X and GMZ drafted the paper, and YY-X, GMZ, HSI, CK, EIP, JS, AB, MH, DOB, MR-B, STB and VM reviewed the drafts and approved the final version. YY-X is the guarantor of the paper. YY-X had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

  • Funding This project was funded by both the US Food and Drug Administration and the US Department of Veterans Affairs Office of Rural Health.

  • Disclaimer This article represents the authors’ best judgement and should not bind or obligate the VA, the FDA or any other institution.

  • Competing interests YY-X, EIP, GMZ, JS, AB and CK acknowledge having received funding from Pfizer for other research projects other than this one.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.