Article Text
PDF
Abstract
Objectives This study investigated the association between vaccine stockout and immunisation coverage, and infant/under 5 mortality rates.
Design A retrospective cohort study.
Setting Low-income and middle-income countries.
Participants A cohort of 131 low-income and middle-income countries from 2004 to 2019.
Primary outcome measures Main outcomes included immunisation coverages of (1) diphtheria-tetanus-pertussis containing vaccine (DTP), (2) measles containing vaccine (MCV), (3) BCG and (4) oral polio vaccine (OPV). We also included infant and under 5 mortality rates as secondary outcomes.
Results The countries that experienced national-level stockouts of DTP and MCV had 3.7 and 4 percentage points lower coverage rates of DTP3 and MCV1, respectively, compared with the countries without the stockout events (p<0.01). Moreover, the statistically significant differences in the immunisation coverage rates across the countries with and without the stockout events are 2.4 percentage points and 2.6 percentage points for BCG and OPV, respectively (p<0.01).
Conclusion Our findings show that the incidence of vaccine stockout events is associated with the decreased immunisation coverages for children in low-income and middle-income countries. However, we did not observe a statistically significant association between the increasing frequency of vaccine stockout and infant and under 5 mortality rates.
- health services accessibility
- community child health
- public health
Data availability statement
Data are available in a public, open access repository.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
This is the first study that evaluates the association between national-level vaccine stockouts and immunisation coverages among the low-income and middle-income countries, allowing direct comparisons of the estimates across the different regions.
The study employed both country and year fixed effects into a panel regression model to account for different baseline characteristics across countries as well as secular time trends shared within the countries.
To address multiple hypothesis testing of various outcomes across the regions, the study applied the Holm correction to determine statistical significance.
Due to potential time-variant unobserved heterogeneity in the countries, caution should be exercised when deriving causal implication from the results since our model estimates are subject to omitted variable bias.
Introduction
Infant vaccination is one of the most significant and cost-effective public health intervention programmes available.1 2 A remarkable number of vaccine-preventable deaths have been averted in low-income and middle-income countries (LMICs) through vaccination programmes. According to a recent modelling study, the lives of 36 million children under 5 in LMICs were saved due to vaccination over the last 20 years.3 However, the disease burden caused by vaccine-preventable illnesses still remains substantial in LMICs, partly because of their low infant immunisation coverage rates.4
Reliable supply and availability of potent vaccines is a vital component of improving infant vaccination coverage in LMICs. From their comprehensive systematic review of related literature, Phillips et al. identified three principal determinants of vaccination coverage—individual’s intent to vaccinate, community access and health facility preparedness.5 Particularly, the study highlighted the significance of a stable vaccine supply as a source of improving facility readiness and sustainable management of national routine immunisation programmes. Furthermore, there is a growing body of evidence implying negative impacts of country-level supply disruption (e.g., vaccine stockout) on immunisation coverage rates in LMICs.6–8 As such, a shortage of vaccines caused by stockout events is an important challenge for LMICs because it would hamper their continuing efforts to protect children from deadly preventable diseases.9
To mitigate the incidence of vaccine stockout, global societies have been gathering resources and efforts for improvement of supply chain systems, recognising potential impacts of the stockouts on child and infant mortality.10 11 Specifically, the Global Vaccine Action Plan (GVAP) 2011–2020, a comprehensive framework to promote universal access to immunisation in all communities, introduced the global vaccine availability indicator as one of key evaluation measures.11 12 Moreover, the Immunisation Agenda 2030, which succeeded to most of the principles of the GVAP, prioritised ‘supply and sustainability’ as a strategic goal, underscoring the importance of sustainable supply chain systems for equal access to quality assured vaccines across the communities and full stock availability at health facility level.13
Despite the global efforts to decrease the incidence of vaccine stockout via supply chain strengthening, there is a lack of evidence available that would allow to determine the potential effectiveness of implemented programmes on global immunisation coverages. Since most of previous studies on this topic relied on the data from a single country, limitations exist for application of their findings to different settings of LMICs (i.e., a lack of external validity).6–8 Thus, it is imperative to provide more complete evidence of the relationship between vaccine stockout and immunisation coverage to policy-makers in both LMICs and international organisations. Using the 2004–2019 WHO/UNICEF Joint Reporting Form on Immunisation (JRF) data, we investigate the association between stockout and vaccine coverage, as well as infant and under 5 mortality rates.
Methods
Data
We used multiple sources of country-level data from 2004 to 2019: (1) WHO/UNICEF JRF, (2) WHO/UNICEF Estimates of national immunisation coverage (WUENIC), (3) UNICEF Data Warehouse and (4) World Bank World Development Indicators (WB-WDI).14–16 The JRF is a standardised tool for monitoring and evaluating the performance of immunisation systems among the WHO member states. It collects a wide range of national administrative data on vaccines and immunisation, such as expenditure, supply and commodities, and safety, all of which are updated annually. From the JRF, we used vaccine supply and logistics indicators, such as national-level and district-level vaccine stockout events for diphtheria-tetanus-pertussis (DTP), measles containing vaccine (MCV), BCG and oral polio vaccine (OPV).14 For the immunisation coverage of countries, we selected WHO/UNICEF estimates rather than administrative or official coverage estimates available in the dataset.14 The infant and under 5 mortality were obtained from the UNICEF Data Warehouse.15 Lastly, the WB-WDI provides the information on country’s population density, land area (km2) and mean years of schooling for female.16
Study population
Using the World Bank income-level country classification system, we excluded high income countries from the analysis. For subgroup analysis, we categorised the countries by the WHO regions: the African Region (AFR), the Region of the Americas (AMR), the Eastern Mediterranean Region (EMR), the European Region (EUR), the South-East Asia Region (SEAR) and the Western Pacific Region (WPR). A panel dataset was established consisting of LMICs over 16 years. In total, 131 countries were included in the dataset and the sample size by the WHO regions is as follows: 44 (AFR), 25 (AMR), 16 (EMR), 18 (EUR), 11 (SEAR) and 17 (WPR).
Patient and public involvement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Measures
The primary outcomes of interest were the immunisation coverage of (1) DTP containing vaccine, (2) MCV, (3) BCG and (4) OPV in LMICs. In addition, infant and under 5 mortality rates were examined as a secondary analysis to identify the potential influence of vaccine stockouts on the one of most significant health outcomes (i.e., mortality) via immunisation coverage.
Two different types of vaccine stockouts (i.e., national and district level) were established as independent variables in this study. First, the incidence of vaccine stockouts was determined using two variables illustrating the occurrence of vaccine stockouts over the year (i.e., yes/no) and duration in month from the vaccine supply and logistics indicators in the JRF.11 A national-level stockout is reported in the JRF when the stock levels of a specific vaccine become exhausted or fall below the recommended 3 months buffer stock. This may indicate a potential disruption to the capacity of the national immunisation programme. For a district-level stockout, the recommended buffer stock reduces to 1 month in the vaccine supply chain. For example, the duration of the event for respective vaccines needs to be at least 1 month together with ‘yes’ in the stockout indicator variable to be counted as a complete vaccine stockout in the analysis. This operational definition of vaccine stockouts applied same to both national-level and district-level stockout events across the different vaccine types.
Second, we created a variable indicating the number of vaccine stockout events based on the count of stockout incidence among the four selected vaccines over the year. For instance, if a country experienced two separate national-level stockouts in DTP and MCV, it would be quantified as two national-level stockout events in a given year for the country. This variable was used as a predictor for a regression analysis investigating the association with infant and under 5 mortality rates. With this analysis, we will be able to determine the levels of association on the intensive margin (i.e., an accumulated number of vaccine stockouts). The stockouts of a vaccine would imply that a remaining duration of safety stocks for the vaccine has reduced to less than 3 months and a stable supply of the vaccine through the national immunisation campaign could be at risk.11
Statistical analysis
To assess the overall performance of immunisation programmes from a vaccine supply perspective, we visualised the annual trends of DTP3 immunisation coverage and the proportion of stockout events (i.e., national, district or either) from any vaccines by the WHO regions. We selected the coverage of DTP3 as a principal indicator of national immunisation programme performance, considering its universal recommendation across countries and availability to monitor the programme performance over time.17 Moreover, we graphed the distributions of national-level stockout events by vaccine types and frequency of the events from the selected vaccines across the WHO regions.
To evaluate the association between vaccine stockouts and immunisation coverage across different types of vaccines, we included both country and year fixed effects into a panel regression model.18 In particular, absorbing time-invariant unobserved heterogeneity, the country fixed effects address bias from different baseline characteristics across countries. Also, the year fixed effects eliminate secular time trends shared among the countries. To improve the accuracy of the estimated effects of stockout events, we incorporated additional covariates in the fixed effects model. The list of covariates is logged population density, logged land area and logged average years of schooling for female, all of which are associated with either or both the independent and outcome variables.19–21 We employed cluster-robust standard errors to account for the heteroskedasticity and a potential serial correlation within the panel. We applied the same modelling approach for subgroup analysis by the WHO regions to document heterogeneity of the effect across the regions. To address multiple hypothesis testing of various outcomes across the WHO regions (i.e., 24 (4 outcomes by 6 regions) comparisons), we applied the Holm correction to determine statistical significance.22 Statistical analyses were conducted using R Software (V.4.2.0; The R Project).
Results
Sample characteristics
On average, 29% of LMICs experienced at least a single national-level stockout event from the four selected vaccines each year (table 1). There is no single vaccine type mainly contributable to stockout events—there was an exception during the period between 2016 and 2019 where 44% of stockout events were attributable to OPV. About 40% of the countries experienced stockouts caused by multiple types of vaccines in a year. By the WHO regions, 40% and 25% of stockout events occurred in AFR and AMR, respectively. Moreover, approximately 20% of stockout events were reported in low-income countries, which represents the lowest proportion compared with both upper-income and lower-middle-income countries.
National-level stockouts of low-income and middle-income countries by vaccine type, region and income
Trends and distribution of vaccine stockouts
AFR and SEAR showed an improvement in their coverages of DTP3 since 2004 (figure 1). Specifically, the coverage in SEAR increased consistently from approximately 80% in 2014 to 95% in 2019. AFR also experienced the progress in the first 5 years of the study period; however, the coverage indicator was remained stable at around 80% as of 2010. Regarding the national-level stockout events, the performances of EMR, SEAR and WPR were superior to those of AFR, AMR and EUR, considering that most of their stockout event estimates were remained below 25% with a few exceptions (e.g., SEAR).
Trends of DTP3 coverage and stockout events by the WHO regions from 2004 to 2019. AFR, African Region; AMR, African Region; EMR, Eastern Mediterranean Region; EUR, European Region; SEAR, South-East Asia Region; WPR, Western Pacific Region.
Regarding the distribution by vaccine types, in AFR, the proportion of national stockouts of DTP reduced over time since 2010, whereas that of national stockout caused by OPV increased in the same period (online supplemental figure S1). In AMR, DTP and OPV are primary factors contributing to national stockout events. In EMR, national stockouts caused by BCG are more frequent than other WHO regions. In online supplemental figure S2, which describes the distribution of the number of stockout events from the four selected vaccines, SEAR maintained the lowest rates throughout the study period.
Supplemental material
Association between stockout and immunisation coverage
Our findings demonstrate national-level stockouts of the vaccines are associated with lower immunisation coverage rates among LMICs (table 2). Specifically, from the pooled analysis, the countries that experienced national-level stockouts of DTP and MCV had 3.7 and 4 percentage points lower coverage rates of DTP3 and MCV1, respectively, compared with the countries without the stockout events (p<0.01). Moreover, the statistically significant differences in the immunisation coverage rates across the countries with and without the stockout events are 2.4 percentage points and 2.6 percentage points for BCG and OPV, respectively (p<0.01). From the subgroup analysis, we found that countries in SEAR showed the stockouts of DTP are associated with the decreased coverage of DTP3 (−4.1 percentage points; p<0.05). Additionally, stockouts of OPV in EUR are associated with the lower OPV coverage (−4.9 percentage points; p<0.01).
Association between national-level stockout and immunisation coverages across the regions
Association between stockout and infant and under 5 mortality
In our secondary analysis, the results show that there are no statistically significant associations between the number of stockout events and infant and under 5 mortality rates (table 3).
Association between national-level stockout frequency and infant and under 5 mortality across the regions (unit: per 100 000 live births)
Discussion
Through the global-level analysis of vaccine stockout and immunisation coverage, we attempted to document comprehensive evidence beyond a single country on the association of those two variables. From the visual exercise that illustrated the DTP3 coverages and the proportions of different levels of stockout events across the WHO regions (figure 1), we found that there is no coherent evidence demonstrating higher rates of DTP3 coverage are attributable to lower occurrence of stockout events. For instance, although countries in EMR were less likely to experience stockout events than those in AMR, average coverage rates of DTP3 in EMR were lower than their counterparts. When comparing AFR against SEAR, however, we identified a negative relationship between vaccine stockouts and the DTP3 coverage. These mixed findings may point out that analysis of stockout events by each type of vaccines is necessary to better understand their direct impacts on immunisation coverages.
Regarding the distribution of stockout events based on the World Bank income level (table 1), we observed that the low-income countries experienced the least frequency of stockout events compared with the upper-income and lower-middle-income countries. This finding is noteworthy, given that a majority of low-income countries have extensively benefited from the immunisation supply chain strengthening programmes provided by the Global Alliance for Vaccines and Immunisation (GAVI). This evidence may partly reflect the effectiveness of GAVI’s efforts in strengthening immunisation supply chain systems among their eligible countries.23 We also noticed that the distribution of stockout events varied across regions, with approximately 70% of the stockouts occurring in AFR and AMR. According to Lydon et al., causes of the stockout events primarily stemmed from domestic factors, such as government funding delays and inadequate forecasting and stock management practices.11 These findings highlight the importance of not only providing financial support but also offering technical assistance to enhance the skills and capabilities of personnel, ultimately mitigating the occurrence of stockout events in these regions.
The outcomes from regression models provide a clear link between national-level stockout events and immunisation coverages of corresponding vaccines. Particularly, the evidence from the pooled analysis including the four vaccine types, DTP, MCV, BCG and OPV, suggests that countries that experienced national-level stockouts had approximately 2–4 percentage points lower immunisation coverage rates. Please note that the null findings from the subgroup analysis by the WHO regions may be attributable to insufficient statistical power. However, considering the variations in the degree of associations across the regions, we recommend that future studies conduct additional analyses to further understand the extent of this variation.
From the secondary regression analysis for mortality outcomes, we observed that the higher frequency of stockout events from any selected vaccines is not reflected to infant and under 5 mortalities in a short term. This finding may support the prior evidence that almost half of infants who lost an initial chance of vaccination due to stockouts would catch up the missed doses after vaccines become available.7
To the best of our knowledge, this is the first study that evaluates the association between vaccine stockouts and immunisation coverages among the LMICs, allowing direct comparisons of the estimates across the WHO regions. Our results confirm the prior evidence from a single country that suggests a negative association between vaccine stockouts and immunisation coverage among infants and children in LMICs. For example, in their study using administrative data from the Nigerian government, Gooding et al. showed that the occurrence of district-level stockout events is associated with the decreased number of vaccinated children in the district from all the seven different types of vaccines (e.g., Penta, MCV, BCG).7 They also observed that the impact remains mostly up to 2–3 months for some of the vaccines after the event. Additionally, the study by Sato that examined individual’s vaccine uptake in Nigeria using the same administrative data as Gooding et al. showed that the odds of receiving vaccines (e.g., DTP3, BCG) among children in the districts where stockout events occurred was 17% lower than that of those in the districts without the events.8 In their survey conducted in a tertiary hospital in South Africa, Burnett et al. also emphasised that vaccine stockouts were the primary barrier to the timely administration of vaccines among eligible children. As noted by Gooding et al., there could be three possible pathways that explain the negative association between vaccine stockouts and immunisation coverage: (1) direct stockout impact, (2) learning effect and (3) regimen interruption.7 Therefore, our results could be understood as a collection of the impact from these three mechanisms.
Our findings provide significant implications to policymakers in LMICs and many stakeholders in global health, emphasising the value of well-functioning supply chain system for success of routine immunisation programmes. Particularly, the evidence from our analysis would support their ongoing efforts to mitigate the incidence of stockout events and help prioritise funding allocation for transformation of national supply chain system.7 It is worth mentioning that processes of the transformation should incorporate development of the central data collection and reporting system for supply chain, given its advantage of producing high-quality data for robust analysis of various immunisation activities within a country.7 8
This study has two main limitations. First, the country-level fixed effects model cannot fully address the omitted variable bias problem (i.e., endogeneity). Thus, caution should be exercised when deriving causal implication from the results since the model estimates are subject to bias from time-variant unobserved heterogeneity among the countries.18 Second, since our dataset does not provide the detailed information on temporality of vaccine stockouts and immunisation coverage geographics, we could not distinguish the degree and magnitude of effects by different levels of stockout events (i.e., national, district or both).
Conclusions
Our findings demonstrate that the incidence of vaccine stockout events is associated with the decreased immunisation coverages for children in LMICs. Routine immunisation programmes would receive significant benefits from supply chain system strengthening.
Data availability statement
Data are available in a public, open access repository.
Ethics statements
Patient consent for publication
Acknowledgments
We are grateful to Dr. Ryoko Sato and Dr. Sunil Mehra for their helpful comments on earlier version of the manuscript.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors DL, MCL and TTK contributed to conceptualisation, interpretation of findings, writing and editing. DL led data cleaning and statistical analysis. KL and MS contributed to editing and supervised the study. All authors have read and approved the final manuscript. DL is responsible for the overall content as the guarantor and accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.