Objective To evaluate interventions to improve routine vaccination coverage and caregiver knowledge in China's remote west, where routine immunisation is relatively weak.
Design Prospective pre–post (2006–2010) evaluation in project counties; retrospective comparison based on 2004 administrative data at baseline and surveyed post-intervention (2010) data in selected non-project counties.
Setting Four project counties and one non-project county in each of four provinces.
Participants 3390 children in project counties at baseline, and 3299 in project and 830 in non-project counties post-intervention; and 3279 caregivers at baseline, and 3389 in project and 830 in non-project counties post-intervention.
Intervention Multicomponent inexpensive knowledge-strengthening and service-strengthening and innovative, multisectoral engagement.
Data collection Standard 30-cluster household surveys of vaccine coverage and caregiver interviews pre-intervention and post-intervention in each project county. Similar surveys in one non-project county selected by local authorities in each province post-intervention. Administrative data on vaccination coverage in non-project counties at baseline.
Primary outcome measures Changes in vaccine coverage between baseline and project completion (2010); comparative caregiver knowledge in all counties in 2010.
Analysis Crude (χ2) analysis of changes and differences in vaccination coverage and related knowledge. Multiple logistic regression to assess associations with timely coverage.
Results Timely coverage of four routine vaccines increased by 21% (p<0.001) and hepatitis B (HepB) birth dose by 35% (p<0.001) over baseline in project counties. Comparison with non-project counties revealed secular improvement in most provinces, except new vaccine coverage was mostly higher in project counties. Ethnicity, province, birthplace, vaccination site, dual-parental out-migration and parental knowledge had significant associations with coverage. Knowledge increased for all variables but one in project counties (highest p<0.05) and was substantially higher than in non-project counties (p<0.01).
Conclusions Comprehensive but inexpensive strategies improved vaccination coverage and caretaker knowledge in western China. Establishing multisectoral leadership, involving the education sector and including immunisation in public-sector performance standards, are affordable and effective interventions.
Statistics from Altmetric.com
Strengths and limitations of this study
The study evaluated routine service strengthening and innovative multisectoral engagement in 16 counties in remote western China.
Standard household survey methods were used to evaluate vaccine coverage and caregiver knowledge.
An attempt to assess secular changes in vaccination coverage was included, but interpretation is limited by lack of survey data and use of administrative data for non-project sites at baseline.
By contrast, confidence in the observed impact of the interventions remains high for the coverage of vaccines introduced after the start of the project, and for the householder knowledge variables assessed.
The global Expanded Program on Immunization (EPI) was launched in 1974,1 and has significantly reduced the incidence of measles, polio and newborn tetanus.2–4 China's EPI started in 1978 and currently includes BCG, oral polio (OPV), diphtheria/tetanus/pertussis (DTP), hepatitis B (HepB), Japanese encephalitis (JEV), hepatitis A (HAV), measles (MV) or measles/mumps/rubella (MMR) and meningococcal (MenV) vaccines.5 ,6
However, financial and social sector decentralisation7 and variable geographic and socioeconomic conditions in China result in uneven access to health services, including vaccination.8 For example, in 2004, administration of the HepB birth dose was less often timely in western (49.5%) than central (72.7%) and eastern (81.9%) regions9; we previously described variable coverage with this vaccine according to several related variables.10 Ongoing measles transmission and the 2011 outbreak of imported poliovirus in western China11 indicate that children there remain vulnerable to vaccine-preventable diseases, due to relatively low access to and knowledge of the EPI.
To improve routine vaccination coverage in western areas similar to those periodically affected by vaccine-preventable disease outbreaks, from mid-2006 to mid-2010, UNICEF supported China's Ministry of Health (through China's Centre for Disease Control (CDC) and local counterparts) to implement an EPI-strengthening project in four western provinces (Guangxi, Guizhou, Tibet and Shaanxi), housing 8.7% of China's population.12 The objective was to strengthen the routine EPI in these provinces by increasing its prioritisation for local funding and policy support, enhancing related service capacity and increasing caregivers’ demand for EPI services. The goal was to narrow the gap in coverage between China's eastern and western regions, reducing the risk of disease outbreaks and improving equity. While there are reports on a health-communication approach to improving EPI coverage in China,13 this project involved multiple sectors, predicting the growing focus on a systems approach to health service strengthening, as opposed to simply supporting services with supplies, personnel and training.14 Only one similar and much smaller project has been described in China.15
To independently evaluate EPI coverage and related knowledge and associations in project and non-project sites, to assess the project's impact on EPI access and uptake, and to provide guidance for other counties in China's west, baseline and final evaluations of the project were conducted. We describe the project's impact on vaccine coverage and related caregiver knowledge, associations with families not utilising vaccination services, and explain how the project inputs and strategy influenced the observed outputs.
Project location and strategy
Guangxi, Guizhou, Tibet and Shaanxi provinces have 109, 88, 76 and 80 counties, respectively. Project funding permitted implementation in only four counties in each province, with a total population of 4.04 million.10 The criteria for county selection were developed by China CDC and included average Gross Domestic Product per capita (predominantly very low), the rate of timely birth dosing with HepB and of full coverage with the four traditional EPI vaccines (BCG, OPV, DTP and MV) as defined by the WHO and UNICEF,16 ,17 measles incidence and the hospital delivery rate in 2004 (table 1). The 16 counties were selected by the four province offices of the CDC, whose staff also used local knowledge of EPI implementation, prioritisation and quality. Staff members were instructed to purposively select counties considered representative of average EPI performance based on these criteria.
The project components collectively comprised a comprehensive approach to EPI strengthening: (1) advocacy to increase awareness and funding of the EPI among local leaders, planners and finance officers; (2) EPI standard-setting according to national guidelines and local conditions; (3) training of EPI and health staff at county, township and village levels, using nationally developed materials; (4) flexible communication strategies adapted to different target populations including women of reproductive age and pregnant women (PW), to increase demand; (5) cold chain, computer and public communication equipment and (6) activities to improve cooperation between local health, education and other sectors and among communities. These components were based on global recommendations17 ,18; more details are provided in web-annex 1. UNICEF-supported project implementation costs averaged approximately $14 000 per county per year, apart from the surveys reported here; local government covered the balance of project expenses.
The project promoted hospital delivery according to China's national Safe Motherhood Program (SMP), and the HepB birth dose in all project county maternity units. China's SMP was phased in over 2002–2010, and provides subsidised (usually free) hospital delivery along with related information to rural women.19 Village doctors are paid a small incentive (around US$1.5) to identify and inform PW of this programme. In project counties where nationally funded subsidies and incentives had not yet started, local government provided the same benefits. Village doctors provided PW with information on immunisation, and maternal and child health, and were also responsible for procuring and administering the HepB birth dose to newborns delivered at home.
Baseline and final household surveys were conducted in project counties in 2006 and 2010. In 2006, local EPI services were also assessed through health facility surveys, facilitating project design. Similar surveys were conducted in the non-project counties, in 2010.
To assess the project's impact over time, we surveyed selected indicators in project counties at baseline in 2006 and at project completion in 2010. Each survey assessed vaccine coverage, related knowledge, attitudes and practices (KAP) of caregivers, immunisation services, socioeconomic status, demographic variables, mothers’ education and parental out-migration. To assess secular changes, we requested each province to select one reference project county for comparison with one non-project county from the same prefecture, having broadly similar EPI performance, management and local government support, and similar population size and composition, geography and socioeconomic status, in 2004 (table 1). It was not possible to directly compare changes in EPI coverage in non-project counties, as we could not survey those counties in 2006; however, 2004 administrative vaccine coverage data were available for three of the four non-project counties at baseline. Vaccine coverage and householder KAP in both, the reference project county and selected non-project counties, were compared in 2010, using identical survey methods in each.
The baseline household and health facility surveys were implemented from September to November 2006, by China CDC and UNICEF. The follow-up household survey was conducted from March to November 2010, by Peking University's Institute of Child and Adolescent Health (ICAH), supported by China CDC and UNICEF, using a questionnaire in all surveyed counties. Developed by China CDC, the questionnaire included the same content in 2006 and 2010. EPI coverage and KAP were assessed during interviews with the selected child's mother or primary caregiver.
Ethical approval to conduct the project and surveys was provided by an independent panel at China CDC and by local authorities.
Location and sampling
For each survey, the WHO standard EPI cluster survey methodology was followed.20 In each county, two-stage cluster sampling was used to first select 30 clusters (villages), with a probability of selection according to each village's population size. In the second stage, seven children born anytime in 2004 (for the 2006 survey), and between 1 June 2007 and 31 May 2008 (for the 2010 survey), were selected in series, beginning by randomly selecting a dwelling from a numbered list of households, finding the nearest age-appropriate child, followed by the next closest, and continuing for up to seven from each cluster. This yielded 210 children per county and a coverage precision of ±10%.20 In Tibet, where the population is very sparse, 10 townships were selected randomly in each county, and all children born during the same dates were investigated. The investigation in each county ended when samples reached 210.
Vaccination records were reviewed at households, using certificates (kept by mothers/caregivers) and cards (brought by EPI providers from health facilities to surveyed households). Surveyors relied on the certificate if both records were available; if neither was available, the child was considered unvaccinated. BCG scars were assessed on site. Small gifts were provided to caregivers; there were no refusals. Surveyors applied the following criteria for age-appropriate vaccination: birth and inoculation dates recorded correctly; first dose administered at the appropriate age; dose interval ≥28 days for OPV and DTP; BCG, OPV3 (the third dose of OPV) and DTP3, and MV1 received before age 12 months, and the certificate or card fully completed. The WHO definition for fully immunised children (FIC) was used.18 ‘Timely’ HepB1 is within 24 h of delivery, and HepB3 by age 12 months16; booster (fourth) dose of DTP and second dose of measles-containing vaccine (MV or MMR) administered between 18–24 months were considered age appropriate.
ICAH and national EPI experts developed the 2010 survey implementation protocol, trained investigators and analysed the data. Pilots were conducted and the process was adjusted before the start. Trained staff from the ICAH, and from provincial and prefecture offices of the CDC, undertook the survey and questionnaire.
All data were double-entered into EpiData3.1 (USCDC, Atlanta, Georgia, USA), and checked for consistency. Cleaned data were analysed using SPSS V.13.0 (Chicago, Illinois, USA); crude comparisons between reference project and selected non-project counties were made using the χ2 test. Multiple logistic regression was used to assess associations with timely coverage of BCG, OPV, DTP and MV. Potential influences on EPI coverage were assessed; those significantly associated (p≤0.10) during univariate backwards regression using the Wald test were retained. ORs adjusted for the influence of other variables were calculated in a multivariate regression model. Statistical significance was set at p≤0.05, and was two-tailed.
In total, 3390 child/caregiver pairs were surveyed in project counties at baseline, and 4129 in project and non-project counties, in 2010. During the same year, 70 pairs were excluded as the children were the wrong age; there had been no such exclusions in 2006. Among those remaining in 2010, 3299 (79.9%) lived in project counties, and 830 (20.1%) in non-project counties. The majority of children in Guangxi, Guizhou and Tibet belonged to one of China's ethnic minorities; virtually all in Shaanxi province were Han ethnicity. The majority (91.1%) lived in rural areas; most children (98.1%) were local residents (table 2). The male:female ratio was 1.29:1. These characteristics were similar to those collected at baseline (data not shown). Project reference and non-project counties were poorly matched on some parameters (table 1), but local CDCs considered their respective EPIs qualitatively similar. The hospital delivery rate in project counties was 50% in 2004 and 90.2% in 2010; in non-project counties it was 42% in 2004 and 88% in 2010.
Vaccine coverage in 2010 and changes from baseline in project counties, calculated using recommended methods,20 are presented for some vaccines in table 3, and for all vaccines in web-annex 2. Surveyed 2006 baseline coverage in most counties was close to the 2004 administrative data provided (table 1), encouraging cautious use of 2004 administrative data for non-project counties. Compared to baseline, the coverage of timely HepB1 in project counties increased the most (by 35.2% overall, and 58.6% in one county). Full, timely coverage with the four traditional vaccines increased by 21.1%. Coverage of HepB3, DTP3, OPV3, BCG and MV all increased significantly, and the proportion of children with a BCG scar rose 12.9% (table 3; web-annex 2). Large increases in FIC were particularly evident in Guizhou and Shaanxi. However, FIC decreased by 16.3% in one remote Tibetan county where EPI access remains very infrequent.
Vaccine coverage in 2010 was usually higher in the reference project counties than in the selected non-project county, in many cases significantly so (table 3). This was particularly true for timely coverage of HepB1 and the newly integrated EPI vaccines (DTP and measles boosters and MenV in most provinces; and JEV and HAV in two provinces each) (table 3 and web-annex 2). The BCG scar rate was also significantly higher in project than in non-project counties in Tibet and Shaanxi. The observed differences in coverage of the four traditional vaccines were mostly not present at baseline (table 1).
Changes in KAP
There were major and significant changes in the proportion of project county mothers/caregivers with basic EPI knowledge (table 4 and web-annex 3). Knowledge increased by ≥22% for each item, with 56.7% more knowing that vaccination status is checked at kindergarten- and school-entry. The knowledge of mothers/caregivers in Tibet increased most, despite high baseline EPI coverage in Tibetan project counties. Significantly more caregivers in project than in non-project counties had EPI-related knowledge, especially related to vaccines required at birth and observation time after vaccination (table 4).
Influences on FIC rate
Influences on FIC rates were assessed using multivariate analysis that first took account of ethnicity; place of birth, residence and vaccination; maternal education and age; parents’ migration status and EPI-related knowledge (table 5). Maternal education and age did not proceed into the final regression model. In fact, only 45% of the primary caregivers interviewed were the child's mother and most had only completed 6 years of elementary school. However, the analysis results did not change when the education variable was expanded to include all caregivers. Compared with children in Shaanxi, children in Guangxi and Guizhou were less likely to have received the four traditional EPI vaccines on time. Ethnicity also conferred a disadvantage on some groups, as did birth at home. Place of residence was not a significant influence, but there was a trend favouring children living in larger settlements over those in villages (p<0.07). Children whose parents had both migrated out before 12 months were less likely to have been fully vaccinated on time, as were children vaccinated at home compared to those vaccinated at village clinics or at township health facilities (p=0.08). Children whose mothers/caregivers knew to report their health status to EPI staff before vaccination in 2010 were more likely to be FIC (table 5).
Just as China's public health situation has improved dramatically since the 2003 SARS crisis, China's EPI is also developing rapidly. In 2007, China introduced several new EPI vaccines to prevent meningococcal meningitis, Japanese encephalitis, hepatitis A, rubella and mumps, and long-term protection against DTP and measles using booster doses was formalised. HepB vaccine user fees were cancelled. According to China's 2012 annual report of administrative EPI data, coverage of FIC and the recently introduced HAV was >95% and >85%, respectively. Coverage in western China, however, remains relatively low, as it was during the 2004 national EPI review,9 particularly for the new vaccines.21–23
Using standard methods, we found that vaccine coverage in project counties increased significantly for all antigens during the project period. It is notable that both BCG scarring and MV coverage increased in most counties, and that, in two provinces, coverage of most routine vaccines and BCG scarring was significantly higher in project reference counties than in selected non-project counties located in the same prefecture. These findings suggest that the combination of activities undertaken was successful in improving EPI access, uptake, implementation and prioritisation within project areas, and more than in non-project counties in at least two provinces (albeit with less certainty due to the absence of a formal baseline survey in non-project sites). In the two provinces where the project did not appear to influence routine vaccine coverage, the EPI had also benefited from substantive non-project input during the same period; indeed, many of the same strategies used in this project were also introduced throughout Guizhou.15 ,24 ,25 Campaign-related activities also explain the widely varying baseline measles incidence in Guizhou and Shaanxi. However, in all provinces, a large project benefit was observed on coverage with HepB1 and the three new vaccines assessed, in 2010. Higher coverage with these new vaccines in the 16 project counties is most likely related to project activities, as they were only introduced in 2008.
In addition, the knowledge of caregivers increased significantly over the course of the project, and was much higher in project reference counties than in non-project counties in 2010. Given the relatively low rates of knowledge for many of the knowledge variables at baseline, and at follow-up in non-project counties, we confidently attributed the observed differences to project inputs. Indeed, knowledge in non-project counties in 2010 was rather weak, even related to the HepB birth dose that has been available in China since 2002. However, although higher than at baseline, knowledge also remained low for many variables in some project counties. Although a variety of locally adapted EPI promotion activities were implemented using various media, face-to-face communication and active participation by community members, they seem to have improved EPI-related participation and behaviour more than knowledge.
Our two surveys provide rare, semi-independent subnational data on EPI coverage in China. In spite of China's great achievements in disease control and generally high EPI coverage, it is apparent that many children are not fully protected against vaccine-preventable diseases, especially in remote areas. This is also true among migrant children in urban areas.26 Unvaccinated children carry a higher risk of disease and transmission of disease to others, as experienced when imported poliovirus reappeared in Xinjiang in 2011.11 According to unpublished UNICEF assessments, in that outbreak, low household awareness of and participation in the EPI resulted in low OPV coverage. There remains, therefore, an ongoing need to identify affordable and feasible strategies to ensure timely EPI coverage throughout China. From our evaluation, we conclude that this is particularly important when new antigens are introduced.
Our findings provide a foundation for recommending this project's approach in poor rural areas of China and elsewhere. In addition to promotion of hospital delivery, which was successful in project and non-project counties and is known to improve timely HepB1,10 ,27–30 the project focused on improving knowledge among both, the community and EPI workers, and improving service standards and equipment. Moreover, it promoted multisectoral engagement in the EPI through involvement of several non-health sectors. Given the project's greater influence on HepB1 coverage (despite similar hospital delivery rates) and on new vaccine coverage than on traditional vaccine coverage, this may have been a key factor in increasing the prioritisation of the EPI in project counties. Multisectoral engagement included establishment of a leadership group comprising representatives of the local governor and health, education and migration management offices, widening stakeholder participation in promoting, planning, managing and implementing the EPI. This has been reported elsewhere in Guizhou.15 In our project provinces, the local government and education bureaus issued related documents and a multisector working mechanism was developed; EPI implementation was incorporated into government performance appraisal indicators. The observed improvement in mothers’/caregivers’ knowledge of kindergarten-entry and school-entry EPI certificate checking verifies the participation of the education system. Given China's virtually universal school enrolment, annual school-entry checking, if implemented, should prove effective in reducing the proportion of children and adolescents unprotected against vaccine-preventable diseases, a major cause of ongoing measles transmission.
The findings also enable recommendation of vaccination at a scheduled EPI service as opposed to at home (again verifying earlier findings10 ,15), and ensuring the EPI-related knowledge of caregivers of children left behind by out-migrating parents, as also found in Qinghai province.31
Some limitations and other issues pertaining to the study merit discussion. First, we compared only one project and one non-project county in each province. EPI performance in all counties in 2004 was only based on administrative data; although 2004 administrative coverage in project counties was similar to surveyed coverage in 2006, administrative coverage can be unreliable in poor rural areas, so selection of a non-project county relied on provincial authorities’ overall perspectives on county EPI staff performance, as well as more objective population and economic data. This was the best alternative to including more non-project counties or surveying non-project counties at baseline, neither of which was possible. We fully acknowledge that the project's impact on routine vaccine coverage is epidemiologically tenuous due to the lack of equivalent survey data from non-project areas at baseline, but on balance feel the evidence allows confidence in our interpretation. Second, in addition to parallel, EPI-focused initiatives in Guangxi and Guizhou,15 ,24 ,25 non-project counties would have benefited from secular health system strengthening in China, which was likely to have indirectly improved the EPI,32 ,33 so the evaluation did not assess project impact in isolation. Although it is difficult to generalise from one reference and one non-project county, given that these secular improvements benefited all counties, we conclude that the observed better performance on traditional vaccine coverage in Tibet and Shaanxi, and on HepB1 and most new vaccine coverage and also on caregiver knowledge in all four provinces, is almost certainly a direct result of the project. Finally, we note that, while in most cases the non-project county EPI coverage was similar to or above that found among the four project counties in each province, by the end of the project it was lower, again suggesting the interventions’ influence.
Third, some children's parents were away during the survey, which was then answered by less knowledgeable grandparents or other caregivers. This would have biased improvements in project counties down, as rural–urban migration in China has increased since baseline and more children are now left behind with elderly caregivers.
Finally, as with any project, the observed improvements may not be sustained and it has been almost 5 years since the project was completed. However, vaccination has been a high priority in China for decades, and continues to be supported at high levels of government; evaluations such as this one provide good material for advocacy and publicity. The additional US$14 000 provided per county per year (excluding funds provided by local government) was a small investment in counties averaging a population of 250 000 and with various sources of EPI funding. These include central and local government funding, ad hoc project funding (for campaigns, etc), separate funding for infrastructure, manpower, coverage monitoring and disease surveillance, vaccine and syringe procurement, cold chain maintenance, etc. In this context, and given possible inputs from the other sectors engaged, project inputs were inexpensive and should be replicable. The EPI target cohort in each province is small, at approximately 1.4% of the total population in provinces with a high population of ethnic minorities (such as Tibet, Guizhou and Guangxi), and only 1.2% in Han-dominant provinces (such as Shaanxi), again limiting the related cost of the programme.
We conclude that some counties in western China perform relatively poorly on the EPI, but vaccination coverage in these areas can be improved with a relatively simple and inexpensive set of activities. In addition to communications, EPI-specific training and service strengthening, a key element of the successful approach implemented may have been engaging with related government departments and elevating the priority given to the programme by local leaders.
The authors gratefully acknowledge support for the surveys provided by staff in the four project provinces and 20 surveyed counties, and Dr Lance Rodewald (WHO Beijing) for helpful comments on earlier versions of this report.
YZ and YX are joint first authors.
Contributors YZ, YX, XZ, XL and DH designed the study. YZ, YX, XZ and CY conducted the field work and surveys. DH, YX and YZ wrote the paper. All the authors agreed to submission of the manuscript for publication.
Funding This evaluation was funded by UNICEF China.
Competing interests None declared.
Ethics approval China CDC and county authorities at the project sites.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.