Article Text

Download PDFPDF

Original research
Vaccination assessments using the Demographic and Health Survey, 2005–2018: a scoping review
  1. Luke M Shenton1,
  2. Abram L Wagner1,
  3. Mengdi Ji1,
  4. Bradley F Carlson1,
  5. Matthew L Boulton1,2
  1. 1Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor, Michigan, USA
  2. 2Department of Internal Medicine, Division of Infectious Disease, University of Michigan Medical School, Ann Arbor, Michigan, USA
  1. Correspondence to Dr Abram L Wagner; awag{at}umich.edu

Abstract

Objective To characterise studies which have used Demographic and Health Survey (DHS) datasets to evaluate vaccination status.

Design Scoping review.

Data sources Electronic databases including PubMed, EBSCOhost and POPLINE, from 2005 to 2018.

Study selection All English studies with vaccination status as the outcome and the use of DHS data.

Data extraction Studies were selected using a predetermined list of eligibility criteria and data were extracted independently by two authors. Data related to the study population, the outcome of interest (vaccination) and commonly seen predictors were extracted.

Results A total of 125 articles were identified for inclusion in the review. The number of countries covered by individual studies varied widely (1–86), with the most published papers using data from India, Nigeria, Pakistan and Ethiopia. Many different definitions of full vaccination were used although the majority used a traditional schedule recommended in the WHO’s Expanded Programme on Immunisation. We found studies analysed a wide variety of predictors, but the most common were maternal education, wealth, urbanicity and child’s sex. Most commonly reported predictors had consistent relationships with the vaccination outcome, outside of sibling composition.

Conclusions Researchers make frequent use of the DHS dataset to describe vaccination patterns within one or more countries. A clearer idea of past use of DHS can inform the development of more rigorous studies in the future. Researchers should carefully consider whether a variable needs to be included in the multivariable model, or if there are mediating relationships across predictor variables.

  • paediatric infectious disease & immunisation
  • public health
  • international health services
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Strengths and limitations of this study

  • The Demographic and Health Surveys (DHSs) are some of the most used sources of national-level vaccination data.

  • Most DHS studies find consistent relationships between sociodemographic variables and vaccination outcomes.

  • There are large variations in how often a country’s DHS dataset is used.

  • A limitation is the use only of English language material.

  • Studies using other national-level vaccination surveys were not included.

Introduction

Vaccinations have been a cost-effective method to control and achieve elimination and eradication of common and sometimes deadly infectious diseases.1 The introduction of routine vaccinations in the USA, for example, has led to a >90% decline in cases of diphtheria, measles, mumps, pertussis, polio, rubella, smallpox and tetanus since the prevaccine era.2 Nevertheless, every year, more than 2.7 million individuals die from acute illnesses caused by common vaccine-preventable diseases.3 The overwhelming majority of vaccine-preventable deaths among children <5 years occur in low-income and middle-income countries.4

Based on the prevalence and severity of disease and on the availability of a safe and effective vaccine, WHO recommends that countries include nine vaccines on their publicly funded vaccine schedule for young children.5 Referred to as the Expanded Programme on Immunisation (EPI), the schedule initially recommended vaccination with BCG, diphtheria–tetanus–pertussis vaccine (DTP), polio vaccine and a measles-containing vaccine (MCV). Since 2004, five additional paediatric vaccines have been added to the WHO EPI: hepatitis B vaccine (HepB), Haemophilus influenzae type b vaccine (Hib), rubella vaccine, pneumococcal conjugate vaccine (PCV) and rotavirus vaccine. Individual countries decide which vaccines to publicly fund and also to make available on the private market resulting in wide variation globally in the adoption of these vaccines. For example, in 2015, 194 countries included three doses of DTP and polio in their immunisation schedule whereas only 84 included rotavirus.6 Many countries now use a pentavalent vaccine, which includes DTP, HepB and Hib vaccines in one vial. Substantial efforts on the part of Gavi The Vaccine Alliance and other international agencies are devoted to logistically and financially supporting the introduction of new and underused vaccines.7 These efforts are particularly important because a discouragingly high number of children consistently do not receive some or all of the vaccines that were first recommended by the WHO. According to WHO, 19.4 million children have not received three doses of DTP, with a majority (11.7 million) living in just 10 countries: Nigeria, India, Pakistan, Indonesia, Ethiopia, Philippines, the Democratic Republic of the Congo, Brazil, Angola and Vietnam.8 With the exception of Brazil, all of these countries have vaccination coverage regularly assessed as part of the Demographic and Health Survey (DHS) programme.

Nationally representative surveys, like those of the DHS programme, have been essential to evaluating country-specific and region-specific vaccination programmes over time. DHS programmes are funded and facilitated by the US Agency for International Development (USAID). The DHS programme was launched in 1984 with a goal of advancing global understanding of health and population trends in low-income and middle-income countries (LMICs). Since its inception it has provided technical assistance for over 300 surveys in 93 developing countries across the globe. Today, the programme is known for collecting and disseminating accurate, nationally representative data on a variety of topics including fertility, family planning, maternal and child health, gender, HIV/AIDS, malaria and nutrition. Host countries have ownership of data collection, analysis, presentation and use and the data are designed to ultimately be used in policy formation, programme planning and monitoring and evaluation.9

A large number of prior studies have amalgamated data from several different DHS datasets, or have included data from many countries, but none has systematically evaluated how these past studies have actually used the vaccination data provided by DHS.10–12 Given that DHS has had widespread use over several decades in evaluating vaccination programmes through identification of undervaccinated groups, and characterising systematic barriers to vaccination, a clearer idea of past use of DHS can inform the development of more rigorous studies in the future. The purpose of this scoping review was to characterise studies which have used DHS datasets to evaluate childhood vaccination status. Specifically, we report on the global distribution of studies, list the predictors used in multivariable regression models, and examine the different definitions of ‘full vaccination’ across studies and how these relate to the WHO EPI recommendations.

Methods

This scoping review was completed by following the steps outlined by the Preferred Reporting Items of Systematic Reviews and Meta-Analyses Extension for Scoping Reviews.13

Search strategies

Searches were performed in three different electronic databases: PubMed/MEDLINE, PopLine and EBSCOhost’s Africa-Wide Information, Global Health, Global Health Archives and Health Policy Reference Center databases. The search terms used were: “Vaccine” (and its variations such as vaccination and vaccinate), “Immunization” (and its variations such as immunize), “demographic and health surveys”, “demographic and health survey”, “DHS”, “National Family Health Survey”, and “NFHS”. Within PubMed the exact search was the following:

(“demographic and health surveys” OR “demographic and health survey” OR “DHS” OR “National Family Health Survey” OR “NFHS”) AND (immuniz* OR Vaccin*) AND (“2000/01/01”[PDAT]: “3000/12/31”[PDAT]).

In addition, the searches were limited to only return papers published between 1 January 2005 and 31 December 2018. References from articles found to be relevant were searched in order to identify additional articles.

Eligibility criteria

The titles of all papers returned through use of the search terms were initially screened for relevance. The abstracts of all remaining papers were then accessed with specific inclusion and exclusion criteria in mind. Abstracts and manuscripts were included if they met all inclusion criteria: (1) studies were conducted using DHS data from LMICs; (2) studies looked at routine vaccination coverage as the primary outcome; (3) studies were cross-sectional in design; (4) studies used either the DHS or the National Family Health Survey (NFHS), a similar study conducted only in India; (5) studies looked specifically at the vaccination outcome of children (usually aged between 0 and 60 months). A set of exclusion criteria was also created: (1) studies published before 2005 or after 2018 (though studies with an online publication in 2018 but print publication in 2019 were included); (2) studies that looked only at the vaccination outcome of adults; (3) studies that looked at population in high income countries; (4) studies that used modelling or projections instead of just analysing the data provided or (5) systematic reviews.

Study selection

LS removed all duplicates and assessed all titles for relevance. Then three reviewers (LS/BFC/AW) independently assessed all abstracts and full-text publications for eligibility using the eligibility criteria laid out. All disagreements were resolved by discussion between reviewers.

Data extraction

In addition to assessment for relevance, data were also extracted independently by three reviewers (LS/BFC/AW). A data extraction form was designed using Google Sheets and was piloted before beginning data extraction. Data from three main categories were gathered during data extraction. The first area was the study population, including the countries of interest, the subpopulation of children being examined, years of the survey administration and whether any surveys besides DHS or NFHS were used. The second category was the outcome of interests: which individual vaccines were assessed, whether full or under vaccination was examined, and if full or under vaccination was examined how were they defined. Lastly, data on vaccination predictors were gathered. We tabulated whether a given study included the most common predictors found in a previous systematic review of vaccination timeliness14: maternal education, wealth index, urbanicity, sex of child, age of mother, birth order, birth delivery location, number of antenatal care (ANC) visits, media exposure and paternal education.

Study methodological quality evaluation

We modified the Downs and Black checklist15 for assessing biases in systematic reviews because all eligible studies used a similar data source. The checklist included the following criteria:

Introduction/study population

  1. Is the hypothesis/aim/objective of the study clearly described? (1=yes, 0=no).

  2. Are the main outcomes (including defining full vaccination, if applicable) to be measured clearly described in the introduction or methods? (1=yes, 0=no).

  3. Are the characteristics of study population eligibility criteria (including age range) clearly described? (1=yes, 0=no).

Descriptive statistics

  1. Does the paper use weighting and clustering? (1=yes, 0=no).

  2. Does the paper provide estimates of random variability (eg, 95% CI of weighted estimates or SE) for the main outcomes? (1=yes, 0=no).

Analytical statistics

  1. Does the paper use do a multivariable analysis? (1=yes, 0=no).

  2. Does the paper show distribution of confounders/covariates? (1=yes, 0=no).

  3. Does the paper describe how the researchers arrived at the final list of confounders? (2=a priori knowledge or used directed acyclic graph (DAG), 1=used p values from crude analysis or used stepwise technique, 0=did not describe or did not use multivariable analysis).

  4. Does the paper write out p values under 0.05? (1=yes, or provided 95% CIs, 0=no).

The quality score could range from 0 to 10, and we describe the average values with a mean and median quality score among all studies.

Synthesis of study findings

Given the heterogeneity of outcomes, predictors and study populations of the included studies it was not possible to combine the results into a meta-analysis. Instead, we present a narrative summary of the data. We describe the distribution of studies by population, what predictor variables are used (and what direction of association they have with outcome), and how full vaccination is defined. In the discussion, we provide recommendations for future analyses of DHS data.

A choropleth map was created using freely available shapefiles from Natural Earth16 in QGIS V.3.6 (QGIS Development Team). The map shows how many studies using data from only one country were published by country. We also show if a country’s data was part of a multicountry study, and we identify countries which had a standard DHS dataset administered between 2003 and 2016 but which did not have a published study. The years 2003–2016 were chosen as a lag time of 2 years compared with the scoping review inclusion criteria to account for delays in publishing the data and writing up a manuscript.

Patient and public involvement

This research was done without public involvement. Members of the public were not invited to comment on the study design and were not consulted, nor were they invited to contribute to this document to improve accessibility.

Results

Our search terms initially yielded 938 papers; 318 from PubMed, 323 from EBSCOhost and 211 from POPLINE. An additional 86 papers were identified through searching the references of selected papers. After removing duplicates, 551 papers remained. These papers’ abstracts were screened using the inclusion and exclusion criteria to narrow down the study pool to 143 papers. However, during full-text screen and data extraction another 18 studies were removed, which left 125 (figure 1).

Figure 1

Diagram of studies’ selection into a scoping review of vaccination studies using the Demographic and Health Surveys.

The quality sum score (possible range from 0 to 10) was on average 6.48 with a median of 7. The most commonly missed items contributing to a lower quality sum score were absence of exact p values or CIs (64% did not), not including estimates of random variability for the outcome (52%), and failure to account for appropriate use of clustering and weights (44%).

DHS has operated in a total of 92 countries since its inception, and between 2003 and 2016, has conducted surveys in 71 different countries.

Overall, 23 (18%) studies used DHS datasets from multiple countries, ranging from 217–19 to 86 countries.11 Seven studies used data from multiple African countries,20–26 4 from just Asian countries,17 18 27 28 1 from the Americas19 and the remainder (11) used data from multiple continents.10–12 29–36 For one study, we were unable to determine what exact countries were included in the analysis.36

Figure 2 is a choropleth map showing which countries’ DHS dataset have been used for vaccination studies. The most frequently represented country is India (26 studies, 21%), followed by Nigeria (17, 14%), Ethiopia and Pakistan (seven each, 6%), and Bangladesh (6, 5%). Notably, there are many countries (44) in the Americas, Europe and Africa, which had one or more DHS conducted between 2003 and 2016 yet for which there are no corresponding single-country papers published using DHS data in this scoping review. However, most of these countries were a part of multicountry studies. Only five countries’ DHS datasets were not part of any (single country or multicountry) DHS study: Cabo Verde, Maldives, Morocco, Sri Lanka and Ukraine.

Figure 2

Map of countries by the number of published studies using Demographic and Health Survey (DHS) datasets. Shading corresponds to number of studies using DHS data from only one country; hash marks indicate a study using multiple countries.

Characteristics of the papers are shown in table 1. About half (51%) of studies included children 12–23 or 24 months of age, and the two next most common age ranges were 12–59 or 60 months of age (11%) and 0–59 months of age (8%).

Table 1

List of papers included in a scoping review of studies assessing vaccination status using the Demographic and Health Survey (DHS)

Full vaccination was assessed in three-fourths (94, 75%) of papers; otherwise, the four most common vaccines assessed one at a time were MCV (39, 31%), DTP (36, 29%), polio (33, 26%) and BCG (27, 22%). There were at least 12 different definitions of full vaccination used in the papers including in this scoping review. Of the 94 papers which evaluated full vaccination coverage, most (66, 70%) used a traditional schedule based off of the four vaccines first recommended for the WHO’s EPI in 1974: one dose BCG, three doses polio, three doses DTP and one dose MCV. Five (5%) papers modified this traditional definition to include a birth dose of polio, and 11 others used a pentavalent vaccine instead of DTP (of these, three had a four-dose polio schedule, and eight had a three-dose polio schedule). Other papers modified the traditional definition in order to include yellow fever (in a total of 4 four papers), measles–mumps–rubella vaccine (in one paper), or to exclude certain vaccine series, like measles, polio or BCG. Some measure of DTP was included in all definitions of full vaccination. No papers included information about PCV or rotavirus vaccine as an outcome in a multivariable regression model, although one used rotavirus vaccine as a predictor variable.19

Four variables were used in a majority of studies. The top 10 variables used in a study (with their relationship shown in a model) are maternal education (in 94, or 75% of studies), wealth index (88, 70%), urbanicity (79, 63%), child’s sex (73, 58%), mother’s age (60, 48%), birth order (51, 41%), delivery location (42, 34%), ANC visits (34, 27%), media exposure (33, 26%) and paternal education (32, 26%).

The relationship between the most commonly used predictor and vaccination outcomes is shown in figure 3. For most predictors, there is a relatively clear relationship to vaccination outcome. For a majority of studies, greater vaccination coverage (across any vaccination outcome considered) was related to maternal education (in 84% of studies that considered the variable), higher wealth index (83%), more ANC visits (76%), greater media exposure (76%), an institutional birth (69%) and more paternal education (56%). For several predictors, a large proportion of studies found no significant relationship. This was especially true for child’s sex (66% of studies), more paternal education (44%) and urbanicity (43%). Sibling composition was one variable for which there was no clear relationship with the outcome: in 41% of studies, having more older siblings was associated with lower vaccination coverage, in 8% it was associated with higher vaccination coverage, and for the rest of studies, there was no significant relationship (35%) or there was a significant, non-monotonic relationship (12%).

Figure 3

Commonly reported predictors of vaccination status used in studies using the Demographic and Health Survey.

Discussion

Vaccination programmes enjoy wide support from many international health organisations and national governments. Vaccination has achieved the sole instance of human disease eradication—smallpox, while polio, measles and rubella have been eliminated in some regions of the world.1 37 Global vaccination coverage has increased in recent years but 12.8 million children in 2015 still had not yet received DTP dose 1,6 a common marker of routine immunisation initiation. Regularly conducted studies on vaccination uptake are necessary to assessing population-level susceptibility and immunisation programme reach while also ensuring that countries are on track with international guidelines for maintaining high vaccination coverage and the control or elimination of certain vaccine-preventable diseases. The DHS datasets tend to be very large, both in number of variables looked at and number of participants surveyed. This allows the examination of many possible associations with sufficient statistical power and the ability to control for a number of possible confounders.

DHS is not conducted in all LMICs, only in certain countries with a USAID presence, and it is conducted at irregular intervals. However, it is one of the most widely available surveys for assessing vaccinations globally. This systematic review found wide variation in how full vaccination was defined across 125 studies using DHS data between 2005 and 2018. However, the majority of studies did look at full vaccination and defined it according to the WHO’s EPI schedule; one dose BCG, three doses polio, three doses DTP and one dose MCV. Additionally, studies looked at similar subpopulations (children <5) and very similar predictors, with the most common being maternal education, wealth, urbanicity and child’s sex.

The vaccines commonly evaluated reflect priorities of international efforts. For example, polio was targeted for elimination by 2018.38 Measles is also subject to an international elimination effort,39 40 and all six WHO regional offices have established target dates for elimination.41 BCG was one of the first vaccines ideally administered shortly after birth (joined more recently in certain locations with HepB and polio birth doses). And DTP dose three has long been used as a proxy for adherence to repeat visits to immunisation appointments.42 43 As more vaccines are added to the vaccine schedule, not only does it become more complicated, but it likely introduces the potential for greater diversity among countries in their respective EPI schedules. Over the past few decades, DHS has operated in 92 countries. However, a significant number of papers came from a relatively small number of countries. We note the most commonly used countries (India, Nigeria, Ethiopia, Pakistan and Bangladesh) are among the 12 most populous countries in the world, and, with the exception of Bangladesh, are among the five countries with the most number of unvaccinated children.8 Given that countries have control over their own vaccine policies and use a wide variety of socioeconomic variables across individual countries, more country-specific analyses of DHS vaccination data is important.

Recommendations for future analyses

This study identified the variables commonly used as explanatory variables in multivariable regression models. Many studies appeared to use the DHS datasets to test the significance and estimate the strength of association for many explanatory variables concomitantly. Since DHS is a cross-sectional study, it cannot be used to investigate the effect of an exposure which could vary across time, such as education or urbanicity. However, a strength of DHS is its ability to be used as a hypothesis generating device. Associations can subsequently be examined in other types of studies, such as cohort studies.

However, given consistent relationships between commonly used predictors and outcomes, it is worth revisiting the use of DHS datasets in multivariable analyses. First, given this consistency, it is more important than ever to consider the plausible causal relationships across all variables used in a model. An approach widely used in epidemiology is to chart the directionality of relationships among variables through DAGs.44 Online software, like dagitty.net, can be used to build these models and assess which variables should be included in the final multivariable model. A potential problem is inclusion of so many variables in one model can obscure the mediating effects of certain variables.45 For example, researchers examining the relationship between media exposure and vaccination status may include maternal age as a confounder. However, the parameter estimate for maternal age in this multivariable model includes the mediator media exposure. Theoretically, a model with age as the main predictor and with media exposure as a main predictor would have different sets of covariates. Although the potential impact of inappropriately controlling for mediation is context-specific, one study suggests parameter estimates may change up to 10%–25%.46

Evolving immunisation schedules mean that future studies will likely take local programmatic considerations into account. However, to make cross-country comparisons, studies could still provide an estimate of full vaccination using the traditional BCG, three doses polio, three doses DTP and one-dose MCV schedule.

Timeliness has also emerged as an important dimension of vaccination uptake within the past two decades.47 48 Measures of timeliness require vaccination dates,14 information missing from many individuals in the DHS datasets. For example, in the 2006–2007 Pakistan DHS EPI immunisation cards, and thus data on vaccination dates, were available for just 10% of cases.49

Finally, researchers analysing DHS data should be aware of its structure and limitations. Most DHS samples are stratified and based on clusters. Studies should use survey procedures and weights to ensure that estimates are representative of the national population and that standard errors are honest reflections of the sampling structure. Additionally, because DHS includes so many individuals with unknown vaccination age, any study should account for this substantial left censoring, through Turnbull estimation methods50 or accelerated failure time models. A substantial minority of studies examined did not specify the age range of the study population. This has implications for timeliness but should be presented in studies calculating more traditional measures of vaccine uptake that do not incorporate timing or age.

The DHS provides national estimates from politically neutral sources over time, in countries where USAID operates. Its continued existence ensures that reliable, comparable and nationally representative data sources are publicly available. Other surveys, like the District Level Household Survey and the Annual Health Survey in India and the Multiple Indicators Cluster Survey (MICS) in over 100 countries, are developed in close collaboration with DHS.51 52

Limitations

There are several limitations to this study. Because the study populations, use of explanatory variables and definitions of outcomes differed among studies, we were unable to conduct a meta-analysis to compare the association of various explanatory variables on outcomes. We did not examine the grey literature or non-English language papers as part of this review, nor did we review reports which may have listed vaccination coverage, but did not include some statistical analysis. Inclusion of these types of articles could have included data from more countries. Vaccination data from the DHS is limited in that it partially comes from information contained on vaccination cards,53 and partially from parental recall—with its obvious potential for errors. However, some countries, such as Ethiopia, have attempted to combat this problem in recent years through the introduction of a Health Facility Questionnaire. This questionnaire is used to record vaccination information for all children, who were discovered to not have a vaccination card during administration of the Woman’s Questionnaire.54 In addition, since the DHS is a standardised questionnaire there is limited opportunity to modify the survey to be locally relevant and take predictors into account that may only be relevant in parts of the country. However, overall the DHS programmes are widely available surveys providing researchers, policy-makers and the public with nationally representative data. These data provide a basis for evaluation of immunisation programmes that would either not exist or not be as robust in their absence.

Conclusions

This scoping review of papers about vaccination published using DHS data found diversity in analyses and qualities of studies. Although certain countries—like India, Nigeria, Pakistan and Ethiopia—have had ≥7 vaccination studies published using DHS data, there are dozens of countries whose vaccination data have not yet been published within single-country studies. Studies find consistent relationships between greater vaccination uptake and more maternal education, higher wealth index, more ANC visits, greater media exposure, and institutional delivery. The relationship between birth order and vaccination status is more varied across countries. Researchers using the DHS datasets should understand the limitations of using recorded vaccination dates, and should clarify the interpretation of estimates from multivariable analyses given the potential for mediation.

Acknowledgments

We are grateful to the DHS personnel who diligently carry out their work and to USAID for sponsoring the DHS projects.

References

Footnotes

  • Twitter @abramwagner

  • LMS and ALW contributed equally.

  • Contributors MB conceived of the study design, helped interpret the data and revised the manuscript critically for important intellectual content. LS and BFC downloaded manuscripts, assessed their fit for this systematic review, abstracted data from the manuscripts, completed qualitative synthesis and helped revise the manuscript critically for important intellectual content. MJ abstracted data from the manuscripts and helped revise the manuscript critically for important intellectual content. AW helped interpret the data, and drafted the article. All authors gave final approval of the manuscript to be published.

  • Funding AW’s salary was supported by the National Institute Of Allergy And Infectious Diseases of the National Institutes of Health under Award Number K01AI137123.

  • Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

  • Map disclaimer The depiction of boundaries on this map does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. This map is provided without any warranty of any kind, either express or implied.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available in a public, open access repository. The data abstracted from these studies are publicly available: https://doi.org/10.6084/m9.figshare.12177135.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.