Introduction Worldwide, suboptimal immunisation coverage causes the deaths of more than one million children under five from vaccine-preventable diseases every year. Reasons for suboptimal coverage are multifactorial, and a combination of interventions is needed to improve compliance with immunisation schedules. One intervention relies on reminders, where the health system prompts caregivers to attend immunisation appointments on time or re-engages caregivers who have defaulted on scheduled appointments. We undertake this systematic review to investigate the potential of reminders using emails, phone calls, social media, letters or postcards to improve immunisation coverage in children under five.
Methods and analysis We will search for published and unpublished randomised controlled trials and non-randomised controlled trials in PubMed, Scopus, CINAHL, CENTRAL, Science Citation Index, WHOLIS, Clinicaltrials.gov and the WHO International Clinical Trials Platform. We will conduct screening of search results, study selection, data extraction and risk-of-bias assessment in duplicate, resolving disagreements by consensus. In addition, we will pool data from clinically homogeneous studies using random-effects meta-analysis; assess heterogeneity of effects using the χ2 test of homogeneity; and quantify any observed heterogeneity using the I2 statistic.
Ethics and dissemination This protocol does not need approval by an ethics committee because we will use publicly available data, without directly involving human participants. The results will provide updated evidence on the effects of electronic and postal reminders on immunisation coverage, and we will discuss the applicability of the findings to low and middle-income countries. We plan to disseminate review findings through publication in a peer-reviewed journal and presentation at relevant conferences. In addition, we will prepare a policymaker-friendly summary using a validated format (eg, SUPPORT Summary) and disseminate this through social media and email discussion groups.
Review registration number PROSPERO registration number CRD42014012888.
- Childhood immunisation
- Low and middle-income countries
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This protocol complies with the PRISMA-P guidelines.
We will employ the Grading of Recommendations Assessment, Development and Evaluation approach to establish the certainty of the evidence on the effectiveness of reminders in improving childhood immunisation coverage.
Non-randomised controlled trials are eligible for inclusion in this review. This study design tends to have a high risk of selection bias, but we plan to alleviate this risk by conducting sensitivity analyses to evaluate the robustness of the evidence to risk of bias by excluding studies with a high risk of selection bias.
We anticipate a wide clinical variation across studies relating to participants, study settings and country income levels. We will mitigate the impact of such heterogeneity on the validity of our findings by pooling data only from clinically homogeneous studies.
Immunisation is a healthcare process in which an individual gains immunity or resistance to a disease through administration of a vaccine.1 Immunisation is important in all stages of an individual's life as it helps children and adolescents fight infectious diseases and restores waning immunity in adults.2 Besides benefits to the individual who receives the vaccine, immunisation helps protect his or her family members, friends and the community at large.3 The WHO estimates that immunisation currently averts 2–3 million deaths every year in all age groups. These are deaths that would have been caused by diphtheria, tetanus, whooping cough (pertussis), measles and other vaccine-preventable diseases.4
The percentage of people who receive one or more vaccines of interest in relation to the target population is referred to as vaccination coverage. The vaccination coverage by 1 year of age for the third dose of a vaccine containing diphtheria, tetanus and pertussis (DTP3) is regarded as a proxy for childhood immunisation coverage worldwide.5 Low childhood immunisation coverage is a significant public health issue, with great implication on children's health.6 The WHO estimates that DTP3 coverage increased from 76% in 1990 to 86% in 2014 worldwide.4 However, the increase in immunisation coverage is uneven; in 2014, DTP3 coverage was 96% in Europe and the Western Pacific and only 77% in Africa. The estimated number of children under 1 year of age who did not receive DTP3 was 18.7 million worldwide in 2014, with more than three-fifths of them residing in 10 low and middle-income countries: the Democratic Republic of Congo, Ethiopia, India, Indonesia, Iraq, Nigeria, Pakistan, the Philippines, Uganda and South Africa.4 As a consequence of low immunisation coverage, 1.5 million children continue to die from vaccine-preventable diseases worldwide yearly.7
Immunisation coverage is affected by factors related to the health system, healthcare workers and caregivers (ie, parents or other persons assuming the parental role).8–10 Caregiver factors that influence childhood immunisation coverage include (but are not limited to) low socioeconomic status, low parental education, younger maternal age, lack of knowledge about the importance of immunisation, negative attitudes towards immunisation, fear of side effects and forgetting vaccination schedules and appointments.9 ,10
In line with the optimisation of childhood immunisation coverage, policymakers and healthcare workers need to put in place purposeful, structured, repeatable and adaptable approaches to sustain high immunisation coverage during childhood, irrespective of who the children are or where they live.11 A potentially important strategy for achieving and sustaining high childhood immunisation coverage involves communicating the dates of scheduled vaccination visits and details of the vaccines on schedule to caregivers as well as re-engaging those caregivers who have missed vaccination appointments to bring their children to receive the vaccines due.12–15
Various systematic reviews have assessed the effectiveness of such reminder methods in improving childhood immunisation coverage, and we summarise the most relevant ones below.13–16 Williams et al13 conducted a systematic search and found 46 eligible studies published between 1980 and 2009 on strategies to optimise immunisation coverage among children under five in developed countries. Twenty-two studies focused on reminders to advise parents of upcoming vaccinations that are due and remind parents of those children who are overdue. The reminders ranged from automated telephone calls and generic postcards to personalised letters and home visits. There was heterogeneity of effects between types of reminders, with an overall effect being a median point change in coverage of +11% (range −11% to +24%). However, these data were collected six or more years ago and may not be applicable to low and middle-income countries where most unimmunised children live.
In another review, Oyo-Ita and coauthors assessed the effectiveness of intervention strategies to boost and sustain high-childhood immunisation coverage in low and middle-income countries.14 The authors included six studies published between 1996 and 2009, but excluded studies focusing on reminder and recall methods because they were included in an existing review.15 The latter assessed the effectiveness of patient reminder and recall systems in improving immunisation coverage, and compared the effects of various types of reminders in different patient populations. The authors included 47 studies available by May 2007 involving participants of all ages, and found that caregiver reminders improved childhood immunisation coverage: OR 1.47, 95% CIs 1.28 to 1.68). The review focused on high-income countries where (potential) recipients of care have primary healthcare providers whom they visit regularly.15 Thus, the findings of the review would not be directly applicable to low and middle-income countries.
More recently, Harvey et al16 conducted a systematic review and meta-analysis of parental interventions to improve early childhood (0–5 years) vaccine uptake. The authors conducted six pooled analyses involving 28 studies, and report that all interventions were somehow effective, although the effects were inconsistent, with a combination of postal and telephone reminders being the most effective reminder method: risk difference 0.1132, 95% CI 0.033 to 0.193. However, the authors excluded studies that did not provide outcome data in terms of the number of children completely immunised or up-to-date for their age from meta-analyses. In addition, interventions for which only one study was found were excluded from pooled analyses. Overall, included studies of reminder interventions in the analyses were published in 2011 or earlier.16 The review therefore provides very low-certainty evidence on the effects of newer methods of reminders on childhood immunisation coverage. Recognising this limitation, the authors indicate that strategies used in most included studies may not be relevant to caregivers of today.
It would therefore be prudent to conduct a new systematic review that includes the newer methods of reminders such as text messages, twitter and other forms of social media. In addition, Harvey et al did not use Grading of Recommendations Assessment, Development and Evaluation (GRADE)17 or a similar approach to assess the certainty of the evidence on the effectiveness of reminder methods in improving childhood immunisation coverage. We propose to fill this evidence gap by conducting a systematic review that will include all methods of sending reminders, and use the GRADE system to establish the certainty of the evidence and present the data in ‘Summary of Findings’ tables.18 In addition, we will use a validated format to summarise the review findings for non-research end users, including (but not limited to) healthcare workers, programme managers and policymakers
Our objective is to assess the effects of caregiver reminders on the uptake of the WHO-recommended vaccines in children less than 5 years of age.
Methods and analysis
Criteria for considering studies for this review
Types of studies
We will include randomised controlled trials (RCTs, defined as studies in which participants were allocated to interventions at random) and non-randomised controlled trials (Non-RCTs, defined as trials that allocated participants to treatment arms by a non-random method such as alternation between groups and the use of birthdates or weekdays). We will include RCT and non-RCT trials with allocation at both individual and cluster levels. For trials with allocation at the cluster level, we will only include those with at least two intervention and two control clusters.
Types of participants
The participants of interest for this review will be caregivers (ie, parents or other persons fulfilling the parental role) of children in need of the WHO-recommended vaccines.
Types of interventions
Eligible interventions will be electronic mail (emails, text messages, twitter, other forms of social media, telephone calls) or postal mail (letters or postcards) delivered to caregivers to remind them of scheduled vaccination visits for their children or recall those who have missed vaccination visits. We will include multifaceted interventions involving any of the eligible interventions and conduct a subgroup analysis by nature of interventions (single or multifaceted). Eligible comparisons include no intervention, standard immunisation practices in the given setting, other interventions, or the same interventions delivered at a different level of intensity. Interventions meant only to inform or educate caregivers about the importance of vaccination are outside the scope of this review and will be excluded. In addition, we will exclude studies focusing on reminders for providers (rather than recipients) of care.
Types of outcome measures
The primary outcome for this review is vaccination coverage in children under five (as defined by the authors). Childhood vaccination coverage can be reported in a variety of ways, including coverage with individual vaccines, uptake of a combination of vaccines, DTP3 coverage, the proportion of fully immunised children and the percentage of children up-to-date with recommended vaccines.6 ,14–16 We will include studies with all these outcomes. The secondary outcomes will be the cost of the intervention, the incidence of vaccine-preventable diseases, the acceptability of the intervention, adverse events following immunisation and undesirable effects of the interventions.
Search methods for identification of studies
We have developed a comprehensive search strategy for peer-reviewed literature (see online supplementary appendix). Searches will be conducted in PubMed, Scopus, Cochrane Central Register of Controlled Trials (CENTRAL), ISI Web of Science (Science Citation Index), Cumulative Index of Nursing and Allied Health (CINAHL) and PDQ Evidence. We will also check reference lists of relevant reviews and full-text articles assessed for eligibility in this review. In addition, we will search the WHO International Clinical Trials Registry Platform and Clinicaltrials.gov for ongoing trials.
Data collection and analysis
Selection of studies
Two review authors (MC and VM) will independently screen the search outputs for potentially eligible studies. We will obtain the full text of studies deemed potentially eligible by at least one of the two authors, and the two authors will independently assess them for eligibility against the study inclusion criteria (ie, types of studies, participants, interventions and outcomes). We will endeavour to obtain English translations for eligible articles published in languages other than English.
Data extraction and management
We will use the Cochrane Collaboration's Review Manager (RevMan: http://ims.cochrane.org/RevMan) for data management and statistical analyses. The two authors will use a predesigned pilot-tested data collection form to independently extract data on study design and methods, country setting (including income level as defined by the World Bank),19 participant characteristics, intervention characteristics, study outcomes and study funding sources.
Assessment of risk of bias in included studies
Risk of bias in included studies will be assessed independently by MC and VM, using the Cochrane Risk of Bias tool.20 The following domains will be assessed: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessors, completeness of outcome data, completeness of outcome reporting and other potential sources of bias (eg, baseline imbalance and conflicts of interest). Judgements will be made for each domain as low risk of bias, unclear risk of bias or high risk of bias according to the criteria in the Cochrane handbook. For cluster-randomised controlled trials, we will include additional domains for assessment such as ‘loss of clusters’. We will categorise each included study into one of three levels of bias: low, moderate and high risk of bias. Studies classified as low risk of bias for all domains will be considered to be at low risk of bias. Studies with a high risk of selection, detection or attrition bias will be categorised as having high risk of bias. All other studies will be considered to have moderate risk of bias.
Resolving discrepancies following duplicate assessments
The two authors (MC and VM) conducting independent screening of search outputs, assessment of study eligibility, extraction of data from included studies and assessment of risk of bias in eligible studies will compare their results after each stage and resolve any differences by discussion and consensus. Should there be no consensus between the two, a third author (FM or CSW) will arbitrate.
Measures of effect
We will express study results for binary data as risk ratios and 95% CIs. For cost of interventions, we will report the costs in the currencies provided by the trial authors, with the current US$ equivalents. Where the data are reported as a small number of ordinal categories, these data will be converted to binary data. For example, in the event that the acceptability of interventions is categorised as acceptable, moderately acceptable and not acceptable, the groups will be converted into two groups: acceptable (acceptable or moderately acceptable) versus not acceptable. For continuous data, such as the incidence of vaccine-preventable diseases, we will calculate unadjusted mean differences (MD) with their SDs.
We will use both fixed-effect and random-effects methods to pool data from clinically homogeneous studies, and compare the results to assess the impact of statistical heterogeneity. We will then present the results from the random-effects method, unless it is contraindicated (eg, if there is funnel plot asymmetry). In the presence of funnel plot asymmetry, we will present the estimates of effects from both methods of meta-analyses, with the assumption that funnel plot asymmetry indicates that neither method is more appropriate. If both show a presence (or absence) of effect, we will be reassured; if they do not agree, we will report this. We will include data from eligible cluster RCTs in relevant meta-analyses after controlling for the design effect, using the intracluster correlation coefficient (ICC).
We will calculate overall intervention effects using generic inverse variance20 and use the GRADE approach to categorise the certainty of the evidence for the effect of each intervention on each outcome as high, moderate, low or very low.17
Unit of analysis issues
We expect that controlled trials which allocated participants at cluster levels will appropriately control for clustering. However, if such data are instead reported as if allocation to interventions was performed at the level of the individual, we will request individual participant data to use in estimating the ICC. If individual participant data are not available, we will obtain external ICC estimates from external sources and use them to re-analyse the data.20
Dealing with missing data
We will contact the authors of included studies to supply relevant unreported data on our primary outcomes. For studies reporting outcomes only for participants who completed the trial or followed the protocol, we will request the additional information that would permit us to conduct intention-to-treat analyses. Should we not receive any requested missing data, we will describe the missing data and discuss the extent to which the missing data could alter our results. We will conduct sensitivity analyses to assess the impacts of missing data on estimates of effect.
Assessment of heterogeneity
We will assess clinical variation across studies by comparing the distribution of important characteristics relating to participants (eg, age), study settings (eg, single or sample of medical practice or clinic settings) and country income level according to the World (high, middle or low). We will pool data from clinically homogeneous studies, examine statistical heterogeneity between study results using the χ2 test of homogeneity (with significance defined at the α level of 10%), and describe any observed statistical heterogeneity between study results using the I2 statistic.20
Assessment of reporting biases
Should we include 10 or more studies in a meta-analysis, we will use funnel plots to assess the possibility of publication bias because when there are fewer studies, funnel plot asymmetry tests are not a reliable means of differentiating true asymmetry from the play of chance.21 Funnel plot asymmetry can emanate from real inconsistency of effects of interventions; publication and other biases (eg, selective outcome reporting), poor methodological quality (poor methodological design, inadequate analysis, fraud) or chance.22
We will carry out subgroup analyses for the primary outcome (vaccination coverage), with subgroups defined by intervention subtype (emails, telephone calls, text messaging, twitter, letters and postcards), nature of intervention (single or multifaceted intervention), country income level as defined by the World Bank (high-income, middle-income and low-income countries)19 and study design (RCTs and non-RCTs). Where appropriate, the intention-to-treat principle and the available case principle will be used to analyse the data.
We will conduct sensitivity analyses to investigate the robustness of the results to risk of bias (low vs high risk of bias). For sensitivity analyses, we will define ‘high risk of bias’ based on each of three criteria, namely inadequate concealment of intervention allocation, inadequate blinding of outcome assessment and substantial loss to follow-up and/or withdrawals from allocated interventions. If we include cluster-randomised trials and insufficient information is available to adjust for clustering, we will use individuals as the unit of analysis and perform sensitivity analyses to assess the potential bias that may have occurred as a result of inadequately controlling for the effects of clustering. We will also perform sensitivity analyses if ICCs were obtained from external sources.
Ethics and dissemination
The review protocol does not need approval by an ethics committee because we will use publicly available data without directly involving human participants. We published an outline of the protocol26 in the PROSPERO International Prospective Register of Systematic Reviews in 2014, registration number: CRD42014012888. The results will provide updated evidence on the effects of electronic and postal reminders on immunisation coverage, and we will discuss the applicability of the findings to low and middle-income countries. We plan to disseminate review findings through publication in a peer-reviewed journal and present at relevant scientific conferences. In addition, we will prepare a policymaker-friendly summary using a validated format (eg, SUPPORT Summary)27 and disseminate this through social media and email discussion groups.
The authors did not receive any external funding for this manuscript. They wrote this protocol during their routine work in their respective institutions, but the views expressed therein are those of the authors and not those of their institutions. Neither the authors’ institutions nor any funder or sponsor played a role in developing the protocol. The authors acknowledge the Editor and the referees for critical and constructive comments on an earlier version of this manuscript.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Twitter Follow Charles Wiysonge at @CharlesShey
Contributors MC led the development of the protocol, wrote the first draft, coordinated and integrated comments from coauthors, and approved the final version for publication. FM and VM critically revised successive drafts of the manuscript and approved the final version for publication. CSW conceived the study, provided supervision and mentorship to MC, critically revised successive drafts of the manuscript, approved the final version for publication and is the guarantor of the manuscript.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.