Conceptual model

The conceptual model for the intervention is based on a hybrid of the theory of planned behaviour (TPB) and the value–attitude–behaviour hierarchy model (figure 1).19 20 It also incorporates strategies derived from motivational interviewing and self-affirmation.21 22 According to the TPB, behaviour (in this case, following the recommended vaccination schedule) is influenced by intentions (in this case, vaccine hesitancy), which are a result of attitudes towards the behaviour, perceived behavioural control and norms. This intervention primarily focuses on strategies for influencing attitudes—that is, tailored messages addressing individual behavioural beliefs (eg, beliefs that immunity is best achieved through exposure to a pathogen or ‘natural immunity’) framed according to personal values (eg, emphasising the benefits of vaccination for preventing spread of illness among the young and elderly for those who value protecting one’s community). By affirming individual patient values and identity, using non-judgmental and empathetic language, emphasising autonomy (ie, adding tenets from motivational interviewing and self-determination theory)21 23 24 and constructing controlling tones of messages, this can minimise reactance and counterarguments. Individually tailored messages, in general, are known to have greater effects on attitude change than are universal (untailored) messages.20 25 26 According to the value–attitude–behaviour hierarchy model,20 values influence attitudes and behaviour across cultures and domains, including recycling, consumer behaviour and alcohol consumption.26 27 This hybrid approach to establishing the conceptual model allows us to focus the intervention strategies on addressing a select set of known determinants of vaccine hesitancy and behaviour, rather than incorporating the universe of behaviour change techniques into our intervention.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Model of parental vaccine values, vaccine attitudes, hesitancy and behaviour.  

Study design and registration

This is a three-armed, individually randomised clinical trial with longitudinal follow-up. Study arms include (1) the VAYB (tailored) intervention, (2) an untailored version of the intervention and (3) usual care. The study is registered with ClinicalTrials.gov (see table 1 for details).

Study overview and setting

In the trial, participants are active in the study from the time of enrolment (from late pregnancy or the first 2 months of their infant’s life) until their infant reaches 15 months of age (489 days). The primary vaccination outcome to be assessed is a dichotomous outcome of vaccination status (up to date versus not) that is based on the average number of days undervaccinated for all vaccinations in the recommended vaccine schedule. Assessment of vaccination for the 2-month and 4-month vaccines occurs when the infant is 200 days old (to provide additional time beyond the exact date the vaccine was due). Secondary vaccination outcomes and outcomes related to attitudes and hesitancy are assessed at age 489 days. This time period was chosen to encompass three critical decision-making points associated with the vaccination process: (1) during pregnancy when many vaccination decisions and attitudes are being formed28 29; (2) during the time period that corresponds to the ages when the initial infant series is recommended (generally at ages 2, 4 and 6 months) and (3) during the second stage of the infant vaccination series at age 12–15 months when vaccines different from those offered at the initial stage are introduced. The primary outcome (vaccination behaviour) is assessed using data from the electronic medical record (EMR), augmented with data from the Colorado immunisation registry, Colorado Immunization Information System (CIIS).

The study takes place via the Internet. Participants in the VAYB and untailored arms view educational materials on a web-enabled device or computer of their own and are prompted to view this information again at specific time points during the study (described below). Participants enrolled in the usual care arm receive by mail Vaccine Information Statements (VISs) for all recommended vaccines in the child’s first year of life; VISs are not provided by mail to participants in the VAYB or untailored arms. Participants in all arms complete surveys at baseline, and three additional time points (table 2). Participants are reminded to take the survey at these intervals via a series of emails. Following survey completion, participants are taken automatically to the website that contains either tailored or untailored information, depending on the study arm. The infants of participants in all arms receive care at participating clinics (an eligibility criterion, see below) where VISs are provided to all study groups as part of routine care.

Study population and inclusion/exclusion criteria

Women in the third trimester of pregnancy enrolled at Kaiser Permanente Colorado (KPCO) between April 2016 and October 2017 are recruited for the trial. KPCO is a non-profit, managed care organisation serving ~667 000 individuals. Each year ~5000 pregnant women and 140 000 children receive healthcare at KPCO clinics. Study participants can enrol from the first recruitment outreach that occurs in the last trimester of pregnancy to when their infant is ≤2 months of age. The infant must be enrolled in the KPCO health plan to continue participation in the study.

A combination of EMR data and study screening questions are used to determine study eligibility. First, the EMR is used to identify English-speaking women, currently enrolled at KPCO, and ≥18 years of age in the last trimester of pregnancy, based on clinically determined expected delivery date. All identified women with a diagnosis International Statisical Classification of Disease, or (ICD10) code in the past 8 months indicating potential abortion, miscarriage, adoption, fetal anomalies, or genetic disorders in the pregnancy, or a high-risk maternal condition (ie, cancer) are flagged for potential exclusion. Medical chart reviews are conducted on these women and they are definitively excluded as potential participants if the EMR indicates that their fetus has a high-risk condition (eg, fatal heart condition, trisomy 18 and anencephaly), or they have a spontaneous or elective abortion, social issues (such as domestic violence) or serious health concerns. Screening questions are delivered online before consent to ensure participants plan to use KPCO medical care for their child, are ≥18 years of age, and are currently pregnant or have a child less than 2 months of age. During the course of the study, participants are removed if they have a fetal demise, infant death, if the infant loses KPCO insurance coverage for greater than 90 days, if they request to be removed from the study or if they die. These data are obtained from a monthly data extraction from the EMR and patient report.

Consent and recruitment

Recruitment occurs via a multistep process. After the EMR is used to screen for initial eligibility, a series of two letters, three emails and one phone call are sent to potential participants 1–2 weeks apart to direct patients to the KPCO study registration website created specifically for this study. On this registration website, identity and eligibility are confirmed, and the participant is consented by signing an online form.

After consent, participants are directed to the study website where they set up login information and are provided with a preintervention questionnaire that assesses their baseline intention to vaccinate, vaccination values, logistical barriers to vaccination, vaccine hesitancy (used for randomisation) and demographics, and reconfirms eligibility. Previously developed and validated measures are used to assess these items.30–32 On completion of this questionnaire, participants are considered to be ‘enrolled’ in the study and are randomised (described below). The screening, consent and enrolment process is repeated monthly until the target sample size is reached.

Assignment of interventions

Participants are randomised on a 1:1:1 basis between the VAYB, untailored and usual care arms. The allocation assignment is generated by back-end software embedded in the study website. Randomisation occurs immediately following enrolment into the study (ie, after completion of the preintervention questionnaire) and remains in place throughout the study. Stratified randomisation along with a permuted block technique is used such that participants are first stratified into either a hesitant or non-hesitant group, based on responses to the preintervention questionnaire. Hesitancy status is assessed using a five-item validated measure developed by Opel (personal communication) and participants are categorised based on the measure’s suggested (but unpublished) cut-offs. Participants from each group are then added to their own set of blocks that each contains six slots. There are two slots available for each of the three study arms. These slots are randomly ordered when the block is created. When all six slots are filled, a new block with six randomly ordered slots is added.

Blinding

Participants are not informed about which study arm they are assigned to, but descriptions of the three potential arms for assignment are provided in the study consent documents. Thus, although they are not told specifically which arm they are in, they are not blinded to their study assignment. The project manager for the study will convert study data to unlabelled arms (ie, arm 1, 2 or 3) allowing for the rest of the study team to be blinded to study arm assignment during the analysis and data interpretation phases of the project. Unblinding will occur when data analysis is complete for the primary study outcome. Clinics, where participants receive care, are not aware of the individuals participating in the study unless brought up by the patient during a clinical encounter.

Sample size calculation

We considered, based on prior studies,3 33 an OR between 2.0 and 3.0 for up-to-date vaccination status between the intervention study arms and usual care to be clinically meaningful. For this, we estimate a needed sample size of 477 (OR=3.0) to 1002 (OR=2.0) participants. This sample size is based on an assumption of 15% of the recruited population being vaccine-hesitant (as has been the case in prior studies in this population)33 and therefore not up to date in their infant’s vaccination, a 1:1:1 randomisation allocation ratio, two-sided tests of statistical significance, 80% statistical power and a 5% type I error rate. Accounting for an attrition of 15%, we need to enrol 561–1179 participants.

Interventions

Tailored intervention

In the VAYB arm, messages were tailored for multiple constructs including intention to vaccinate, personal attitudes about vaccines, vaccination values (table 2), vaccination beliefs and concerns, logistical barriers to vaccination, and child’s name, sex and birthday. Data to inform this initial tailoring come from the preintervention questionnaire. Interim questionnaires are used to refresh the tailored information at three times during the study period. Tailoring occurs based on an embedded algorithm that is part of the VAYB website. An in-depth description of the process used to develop the VAYB intervention, and the resulting content, is described in detail elsewhere but examples are provided in table 3.31 32

View this table:

• View inline
• View popup

Table 3

Examples of VAYB website content for two topics, showing tailoring based on three different values

On completion of the preintervention questionnaire where initial tailoring information is obtained, participants are automatically directed to the VAYB website, which is individually customised based on their responses. The most highly tailored content is in three ‘Just for You’ tiles that are displayed prominently on the page (figure 2A). These tiles reflect the top three vaccine topics of concern that each participant indicates they want more information about and are further customised to highlight the vaccination values the participant most endorses, and to reflect their most recently reported intention to vaccinate. The remaining content is lightly tailored to reflect participant’s attitudes, concerns, hesitancy and demographics, but is not tailored based on vaccination values. Highlighted text on the home page (figure 2A) is used to further identify additional information that is most relevant to the participant based on their survey answers. The website is refreshed three times during the course of the study based on interim assessments of participants’ attitudes, beliefs, concerns, values and vaccine hesitancy. Specifically, when the infant is 4–6 months of age, participants reanswer all questions excluding the value items and questions used to assess vaccine hesitancy, and the content is refreshed accordingly. Values are reassessed again in a third survey when the infant is 10–12 months of age and the website is refreshed to reflect any new content. Vaccine hesitancy level is reassessed at a fourth survey and the content is again refreshed. Participants receive a gift card after each survey is completed. For all time points, vaccination intention is assessed immediately before and within the hour after viewing the website content (VAYB and untailored arms). A reminder for this vaccination intention assessment is sent to non-responders after 1 day.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

(A) ‘Landing page’ of the VAYB website annotated to highlight various types of message tailoring. The right arrow denotes the ‘Just for You’ tiles that represent the most highly tailored content on the VAYB website. The left arrow denotes additional text reflecting lightly tailored material that is particularly salient to the participant based on their survey responses. (B) Landing page of corresponding untailored website that lacks message tailoring. VAYB, Vaccines and Your Baby. 

Outcomes

The primary outcome of the study is a dichotomous categorisation of vaccination status (up to date vs not up to date) that is defined based on a continuous measure of days undervaccinated. This outcome is assessed at age 200 days to cover vaccines in the initial infant vaccination series and to minimise the loss to follow-up. The following six vaccines recommended by the Advisory Committee on Immunization Practices will be assessed: hepatitis B, rotavirus, diphtheria-tetanus-acellular pertussis, Haemophilus influenzae type b, pneumococcal conjugate vaccine and polio. All vaccination data are obtained from KPCO’s EMR and CIIS.

To categorise vaccination status, we will first assess the number of days undervaccinated for the 2-month and 4-month vaccines (combined), by calculating the difference between when a vaccine dose was actually administered and when a vaccine dose should have been administered according to the vaccination schedule recommended by the Advisory Committee on Immunization Practices,35 plus an additional 30-day ‘leeway’ to account for vaccination that did not occur at exactly the minimal interval between doses. For example, the first dose of rotavirus vaccine is due at age 2 months (61 days) but is not considered late until age 92 days. Days undervaccinated for this dose begin accruing on day 93. The number of days undervaccinated is then summed across all doses and vaccines to calculate a total number of days undervaccinated for each infant and can range from 0 to 648 days. Infants with 0 total days undervaccinated (assessed specifically for the 2 and 4 months vaccines) at 200 days will be considered up to date on their vaccination status; those with ≥1 day undervaccinated (representing at least a 30-day delay for at least one vaccine) will be considered not up to date.

A secondary vaccination metric that is assessed is up-to-date status for measles–mumps–rubella and varicella vaccine at 489 days, when delay for the first dose of these vaccines begins. This metric is useful because it incorporates outcomes related to parents’ decision-making about these two vaccines recommended at age 12–15 months that are not offered previously.

The interventions’ impact over time on a variety of additional secondary outcomes that are based on the constructs of our conceptual behavioural model (figure 1) and assessed as part of the baseline and interim questionnaires will also be assessed. These include changes over time in vaccination attitudes and hesitancy, and how these relate to study arm, vaccination values and vaccination status. Vaccination attitudes are assessed using measures previously developed by our team and others,30 values are assessed using a novel vaccination values framework we have developed (manuscript in preparation) and vaccine hesitancy is assessed using a five-item validated measure developed by Opel (personal communication). A variety of covariates and potential moderators will be assessed as part of this analysis including patient age, gender and insurance (some patients have Medicaid KPCO coverage), and mother’s age, race and ethnicity. Also included will be metrics measuring website engagement (VAYB and untailored arms only) including time spent on the website, number of times viewing the website, number and order of pages viewed, and match between stated concerns and website material viewed (VAYB arm only).

Participant retention

To assist with retention, participants receive a US$20 gift card incentive for each survey they complete. However, even with this incentive, we expect some drop off in survey participation. Because our primary outcome is vaccination status, mothers who do not participate in all the study surveys are still able to have the primary study outcome assessed, so long as their child maintains coverage and continues to seek care within the KPCO health system. On the basis of past studies, we expect that the proportion of mothers who discontinue KPCO coverage after the birth of their infant to be ~15%, and our study is powered with this attrition in mind.33

Data security and storage

To ensure that the data are protected, several methods are used. Personal identifying data collected on study websites are limited to a participant generated username, email address and child birthdate. The only other data collected on the study websites are vaccine attitudes, beliefs, values and demographics. The study websites use virtualised servers housed at redundant data centres and access is password protected. Virtual servers are backed up automatically onto encrypted tape for recovery and security. Data provided to researchers from the website are encrypted if they are transmitted across the Internet. Data use agreements are in place across all study team member sites.

All medical record data are collected and stored at KPCO behind the firewall in secure password protected files. This data set is linked to a study ID. A limited data set devoid of personal identifying information will be used for data analyses. Data will be shared with study team members through a secure file transfer. Only members of KPCO research project team have access to the personal identifiers linking the study IDs to specific study participants.

Analytic framework

We will use a modified intention-to-treat framework for the analysis of vaccination outcomes. This analytic cohort will include infants of all randomised mothers who maintained KPCO health coverage for the allotted amount of time (200 days for the primary outcome and 489 days for the secondary outcome) with no more than 90 days of no coverage, and thus have vaccination data available for assessment. For survey outcomes, we will use a modified intention-to-treat analysis that includes all participants with data from at least one non-baseline questionnaire.

Missing data

As described above, nearly all vaccines provided to KPCO patients are documented in the EMR, and doses provided outside KPCO are documented in CIIS. Therefore, we expect there to be minimal missing data for vaccination outcomes. To ensure the most complete record, CIIS will be cross-checked for all participants to identify any vaccine doses given to infants outside the KPCO system that are missing from the KPCO EMR. Participants who do not have vaccination data present in either system will be assumed to have not gotten a vaccine dose elsewhere.

For survey data, due to our recruitment strategy, we anticipate no missing data at baseline, as completion of the baseline survey is a criterion for entry into the study. However, there may be missing data for subsequent surveys as these are not required to remain in the study. For missing data in surveys beyond baseline, multiple imputation models will be developed for analyses involving multiple survey points where greater than 10% of subjects would be lost due to missing values.

Subgroup analyses

The main subgroup analysis planned is examining the efficacy of our intervention by vaccine hesitancy status (dichotomous variable), as defined by the five-item Opel measure described above.

Assessment of harms and adverse events

Study participants are provided with contact information for the research team and encouraged to contact the team if they experience any adverse events (AEs) related to their participation in the study (eg, being contacted after an infant death). AEs are expected to be very unlikely given the nature of the study and our monitoring procedures. However, should any significant AEs occur, they will be reported to the appropriate institutional authorities.

Informed consent

All mothers in the study are informed about the study, the risks and benefits, and provide written informed consent via an online registration process prior to participating in the study. As part of the consent process, participants are informed that they may withdraw from the study at any time without impacting their clinical treatment.

Access to data

The data will be accessed only by authorised persons directly involved in the study from the University of Colorado Denver, KPCO and the University of Michigan. Access to a de-identified, aggregated version of the data set and analysis code will be available on request and approval of the study team.

Dissemination plans

Results of the study will be presented at national and international research conferences and through peer-reviewed publications. Any changes to the study protocol will be clearly communicated to journals publishing the study results in a manner that aligns with the journal’s policies for reporting clinical trials. Consolidated Standards of Reporting Trials (CONSORT) guidelines36 will be followed when reporting study outcomes. Study materials, such as questionnaires and screenshots of the intervention websites, will be available to researchers on request from the study principal investigators. If the VAYB intervention proves to be efficacious in reducing delays in the timeliness of infant vaccination, the study team will work with web-developers and community organisations to explore options to make the website available to the general public.

Patient and public involvement

Patients were first involved in this research when designing the intervention, which is informed by the literature, and by the research teams prior to clinical and research experience. The bulk of patient involvement is as research participants. They will not be involved in recruitment or conduct of the study, data analysis or dissemination.