Introduction Emerging evidence suggests community health workers (CHWs) delivering preventive maternal and child health (MCH) interventions through home visiting improve several important health outcomes, including initiation of prenatal care, healthy birth weight and uptake of childhood immunisations.
Methods and analysis The Arizona Health Start Program is a behavioral-based home visiting intervention, which uses CHWs to improve MCH outcomes through health education, referral support, and advocacy services for at-risk pregnant and postpartum women with children up to 2 years of age. We aim to test our central hypothesis that mothers and children exposed to this intervention will experience positive health outcomes in the areas of (1) newborn health; (2) maternal health and healthcare utilisation; and (3) child health and development. This paper outlines our protocol to retrospectively evaluate Health Start Program administrative data from 2006 to 2015, equaling 15 576 enrollees. We will use propensity score matching to generate a statistically similar control group. Our analytic sample size is sufficient to detect meaningful programme effects from low-frequency events, including preterm births, low and very low birth weights, maternal morbidity, and differences in immunisation and hospitalisation rates.
Ethics and dissemination This work is supported through an inter-agency contract from the Arizona Department of Health Services and is approved by the University of Arizona Research Institutional Review Board (Protocol 1701128802, approved 25 January 2017). Evaluation of the three proposed outcome areas will be completed by June 2020.
- community health worker
- maternal and child health
- home visiting
- propensity score matching
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
A 10-year retrospective observational study of a community health worker home visiting intervention using propensity score matching.
Size and diversity of mothers in the intervention group (9665) will be matched to one or more characteristically similar mothers in the comparison group.
Less than 1% of intervention participants were involved in other home visiting programmes.
Analysis may have limited external validity for populations who differ along socioeconomic status, race and ethnicity.
Over the last decade, the community health worker (CHW) workforce has been recognised by the WHO and several US entities as an evidence-based approach to address health disparities.1–3 In the USA, the CHW workforce has gained recognition and visibility, as evidenced by the creation of a US Department of Labor Standard Occupational Classification (21-094) in 2010, to include CHWs as a health profession in the Patient Protection and Affordable Care Act.4 According to the American Public Health Association (APHA), a CHW is a frontline public health worker who is a trusted member of and/or has an unusually close understanding of the community served. This trusting relationship enables the worker to serve as a liaison/link/intermediary between health/social services and the community to facilitate access to services and improve the quality and cultural competence of service delivery.5
Emerging evidence suggests CHWs delivering preventive maternal and child health (MCH) interventions through home visiting improve several important maternal and child outcomes.6 7 Globally, CHW home visiting interventions are associated with several primary prevention efforts that promote the initiation of any, early, and adequate prenatal care,8 9 initiation of any and exclusive breastfeeding,7 10–13 reduction of maternal morbidity and perinatal mortality,14 and the uptake and completion of childhood immunisations.7 15 In the USA, CHW home visiting interventions are associated with decreased incidence of preterm birth9 16–18 and low birth weight,9 16–22 and increases in up-to-date immunisations among newborns and toddlers.23 CHWs share the language, socioeconomic status (SES) and life experiences of their clients, making them a fundamental asset to reducing health inequalities among disenfranchised groups.24 Moreover, CHWs are recognised as integral contributors in collaborative health-based and community-based teams by improving comprehensive care and addressing the social determinants of health that contribute to health improvements and cost savings.25 26
Arizona launched the first iteration of the Health Start Program (HSP) in 1984, when Arizona ranked among the lowest five states for the number of women receiving any or adequate prenatal care.27 HSP is a statewide programme that employs CHWs to engage at-risk, low income, and racially and ethnically diverse mothers and improve maternal and child outcomes. HSP has been managed by the Arizona Department of Health Services (ADHS), Bureau of Women’s and Children’s Health since 1992.28 In 1994, the Arizona state legislature passed the Arizona Children and Families Stability Act, Arizona Revised Statutes (ARS) § 36–697, which formalised and expanded HSP and articulated the purpose, requirements and administration of the programme. HSP is a community-based outreach programme that identifies, screens, and enrols pregnant women early in their pregnancies and assists them with obtaining early and consistent prenatal care. The programme also provides prenatal and postpartum education, information and referral services, client advocacy, and emphasises timely immunisations and developmental assessments for their children. Since its inception, Arizona HSP’s mission has been to educate, support and advocate for families at risk by promoting optimal use of community-based family health care services and education services through the use of community health workers (CHWs) who live in and reflect the ethnic, cultural and socioeconomic characteristics of the community they serve. 28
Arizona is the sixth largest state in the nation, with a population of 6.8 million people. The state shares an international border with Mexico and is home to 21 federally recognised American Indian Tribes and Nations, making it uniquely racially and ethnically diverse. Arizona has a higher proportion of Latino (30.9%) and American Indian (5%) residents compared with the country (17.8% and 1%, respectively) and a comparatively smaller proportion of African American residents (5% compared with 13% nationally).29
In 2015, nearly a quarter of the population lived in rural areas, where the poverty rate reached 30%, almost double that of the national poverty rate.29 Approximately 20% of Arizona families with children live below the federal poverty line, compared with 18% nationally. Poverty disparately affects Arizona’s Latino (36%) and American Indian (46%) families and children.29 Arizona ranks as the fifth highest US state for adult female poverty rate in the country, with more than one quarter of Arizona families headed by single-mother households.29 The initial framework for the HSP was developed in the 1980s and 1990s to address the social determinants associated with the steady decrease in the rate of women receiving prenatal care. In the most recent Arizona Title V Maternal and Child Health Needs Assessment (2017), approximately 74% of pregnant women initiated prenatal care in the first trimester (compared with 61% in 2015 and 81% in 2013), and 7.9% had no prenatal care.29 There were disparities among mothers by race/ethnicity who received prenatal care, notably American Indian mothers having the highest rates of ‘inadequate’ prenatal care (25%) compared with all women in Arizona (15%).29
It is widely recognised that late prenatal care is associated with preterm birth, low birth weight and infant mortality. In 2014, 9% of babies born in Arizona were premature and 7.2% were low birth weight.29 Historically, low-income mothers have experienced higher rates of premature birth and low birth weight in Arizona30 and nationally.31 There are also apparent racial disparities for birth outcomes in Arizona. Preterm birth rates are highest among Black (12.2%), American Indian (9.4%) and Latino (9.2%) compared with all preterm births (9.1%) in the state. Preterm births increase the risk of low birth weight; similar trends persist with the highest rates of low birth weight among Black residents (10.32%) compared with White residents (5.36%) and the total Arizona population (7.2%).29 Preterm and low birthweight baby delivery costs have been shown to be 25 times more than uncomplicated newborn deliveries.32 Although prenatal care and birth outcomes in Arizona have improved over the years, many under-resourced women continue to experience significant challenges and barriers to obtaining healthcare services.
Our goal is to describe the research protocol for a retrospective comparative evaluation to assess the impact of Arizona’s HSP, a CHW home visiting perinatal support programme, on multiple maternal, infant and child health outcomes. Broadly, the goal for the study is to meet the federal Home Visiting Evidence of Effectiveness (HomVEE) standard for evidence-based effectiveness. We will use a matched comparison group design that meets the published standard for HomVEE’s ‘moderate’ rating, defined by HomVEE as: ‘(1) baseline equivalence established on tested outcomes and demographic characteristics and controls for baseline measures of tested outcomes, if applicable; and (2) no confounding factors; must have at least two participants in each study arm and no systematic differences in data collection methods’ (note: a ‘high’ rating is reserved for randomised controlled trials).33
We plan to objectively test our central hypothesis that mothers and children exposed to HSP during the study period of 2006–2015 will experience positive health outcomes in the areas of newborn, maternal and child health (table 1). Specifically, our aims include:
Aim 1: assess the impact of HSP on newborn health.
Aim 2: assess the impact of HSP on maternal health and care utilisation.
Aim 3: assess the impact of HSP on early child health and development.
Methods: intervention, participants and outcomes
HSP is significant in that it is one of the longest-standing programmes in Arizona and employs CHWs in 14 distinct Arizona counties to engage at-risk, low-income mothers in order to improve birth outcomes (figure 1). CHWs serve as the primary interventionist for the programme. In 2016, HSP CHWs provided services to 2534 unduplicated clients, conducted 16 698 home visits and facilitated 461 classes.28 Women are eligible to enrol in HSP if they (1) live in the targeted service area, (2) are pregnant or post partum with a child under the age of 2 years and (3) have one or more social or medical risk factors. Social risks can include but are not limited to: single-parent status, underserved racial or ethnic group, education equal to or less than high school level, annual income less than $40 000, and Medicaid or no insurance. Medical risks are broad and can include previous preterm birth, low birth weight, chronic disease, high maternal body mass index (BMI) and substance use. Women can be of any age and there are no income requirements to participate.
CHWs connect clients to prenatal care and increase client’s continuity of care during and after pregnancy. CHWs identify, screen and enrol eligible women; provide prenatal and postpartum education; provide referral and advocacy services; and emphasise timely immunisations and developmental assessments for children. Although not an exhaustive list, table 1 outlines the primary intervention activities conducted by the CHW. HSP CHW home visits are guided by an asset-based approach and two primary theories of behaviour change, the Trans Theoretical Model and the Social Cognitive Theory. Identifying assets acknowledges and supports the existing strengths and capabilities of individuals and resources to promote community-driven development and positive change.34 The Trans Theoretical Model assumes that behaviour modification in individuals is a multistage process in which people move through stages of readiness for change,35 and Social Cognitive Theory states that stages occur in the context of reciprocal relationships between the person’s environment, their behaviour and their cognition.36 CHWs are a community asset and well positioned to support HSP clients; they share both lived experiences and cultural knowledge of the community they serve. The home visiting sessions promote behaviour change through assessment, goal planning, referral, advocacy and follow-up activities, coupled with education through meaningful adult learning models. These interactions are designed to encourage personal agency of adult learners to integrate new knowledge and create a cognitive structure that makes sense of their own surroundings and situations.37 Through behaviour change theories and adult learning models, the HSP CHWs privilege the co-construction of knowledge among all participants, assume all are co-learners, and encourage critical thinking about self-sufficiency, empowerment, and personal agency related to the five HSP goals (table 1).
HSP CHW core competencies, roles and training
According to the HSP policy and procedure manual, CHWs must (1) live and work in the service area, (2) reflect the ethnic, cultural and socioeconomic characteristics of the communities they serve, (3) be able to read and write in English, (4) have a high school diploma or General Educational Development and (5) pass a criminal history background check within the Department of Public Safety records to be eligible to work for the state-funded programme. It is highly recommended (though not required) that CHWs have posthigh school training and education in MCH, early childhood development education, family studies, social work, nursing or a closely related field.28 Before a CHW can initiate unsupervised outreach or home visits, they must complete 40 hours of training in both the 10 CHW Core Competencies set forth by the CHW Core Consensus Project,38 which are recognised by the Arizona state legislature HB 2324 Voluntary CHW Certification,39 and the HSP Core Training.28 An additional 8 hours of home visit shadowing with a senior CHW are required.
Nationally recognised, the 10 CHW Core Competencies include: (1) Cultural and Systems Mediation; (2) Culturally Appropriate Health Education; (3) Care Coordination and Case Management; (4) Coaching and Social Support; (5) Advocacy; (6) Capacity Building; (7) Direct Service; (8) Individual and Community Assessments; (9) Outreach and (10) Research and Evaluation.38 HSP Core Training covers: (1) Essential Health Start Information (HSP basics, visits and community outreach); (2) Communication and Emotional Support; (3) Nutrition and Physical Activity (family nutrition and physical activity, infant nutrition and physical activity); (4) Health Education (healthy pregnancy, prenatal care, discomforts during pregnancy, labour and delivery, postpartum care and family planning, early childhood development and parenting skills, infant health and child health) and (5) Safety (home safety for infants and children, child abuse and domestic violence).28 CHWs are required to complete 12 hours of continuing education per year.
HSP administrative data from 2006 to 2015 is the primary source for identifying the retrospective intervention group. All Health Start clients enrolled during the 10-year observation period will be included in this study if their records are identified and linked from the HSP database to the Vital Records Birth Database (VRBD). Records will be linked based on the mother’s date of birth and first name. In order to be a candidate for the HSP study cohort, the mother’s date of birth must be an exact match while her first name must be at least 95% similar, using Jaro-Winkler similarity.40 Mother’s last name may change due to marriage; therefore, this criterion is not required to identify the intervention cohort. We will obtain the following information for each HSP study cohort mother: a unique identifier (ID), first name similarity percentage, last name similarity percentage, HSP enrollment date, programme closure information (ie, programme completion, reason for closure) and the child's birthdate. Using the process described above, 15 576 HSP records were linked to the VRBD.
Intervention cohort sample size
Our evaluation intervention group will include all HSP participants enrolled within 24 months of the date of birth of the child during 2006–2015. Of the initial 15 576 records identified through the HSP-to-VRBD data link, 5911 fall outside of the 24-month (either before or after) HSP enrollment window and will be excluded from all subsequent analyses. The resulting 9665 HSP-associated births constitute the basis of this study (figure 2). Because HSP participants can enrol before or after birth, we will limit the analysis for Aims 1 and 2 (newborn and maternal health outcomes) to those births for which the mother was enrolled during pregnancy. This final criterion results in 6493 HSP-attributed births for the evaluation of Aims 1 and 2. Aim 3 (child health outcomes) will be evaluated using the larger set of 9665 HSP-associated births.
Synthetic comparison group
A comparison group of women not exposed to the HSP (non-HSP) will be created from all births occurring in Arizona during 2006–2015 (derived from VRBD). After identifying our study population we will use propensity score matching (PSM) to generate a statistically-similar synthetic control group that has, on average, the same observable preprogramme characteristics as the HSP mothers.41 The pool of potential comparators will come from all Arizona births that occurred over the study period (2006–2015). This process will be guided by HomVEE standards requiring that the covariates used to balance the treatment and control groups be associated with both treatment status and the outcomes of interest.42 Because the HSP eligibility criteria focus on social and medical risks, we will prioritise these types of measures in the PSM model, in addition to characteristics that have been shown to have strong associations with our outcomes of interest in previous empirical and theoretical work.
We will employ radius matching to identify comparison group mothers across the common support region.43 We will use the following measures in the PSM model: mother’s birth year, mother’s age at birth, county of residence. Additional indicator variables include: child’s birth order, maternal educational attainment, health insurance payer (Medicaid being a proxy for low-income status), race, ethnicity, availability of information for the father on the birth certificate, maternal country of birth, previous history of hypertension and median household income by zip code of residence. In addition to these demographic and socioeconomic characteristics, we will restrict potential comparators to mothers within the same fiscal years in order to account for economic conditions and any potentially shifting programme parameters. Imposing within-year matches will allow us to analyse the programme’s efficacy over time by cohort.
Comparison mothers may match to more than one HSP mother, based on the propensity score. Preliminary efforts to identify matches resulted in a potential synthetic comparison group of nearly 23 000 non-HSP mothers. Due to the respective sizes of the intervention and comparison populations, lack of statistical power is not a significant issue for this project.
Patients and the public were not involved in the design or planning of the study.
HSP is a primary prevention intervention to improve MCH outcomes among at-risk, racially and ethnically diverse, rural and urban mothers and children of Arizona. We will use four ADHS administrative datasets to evaluate Aims 1–3 including HSP Database, VRBD, Hospital Discharge Database (HDD) and Arizona State Immunization Information System (ASIIS) Database. Aim 1 (HSP impact on newborn health) will be measured by preterm birth, birth weight, and newborn hospital length of stay and associated charges. Aim 2 (HSP impact on maternal health) will be measured by prenatal care initiation and frequency, method of delivery, maternal morbidities and inter-pregnancy intervals. Aim 3 (HSP impact on child health) will be measured by uptake of age-appropriate immunisations, and emergency room and inpatient encounters and charges (table 2).
Methods: data management, monitoring and statistical analysis
The four datasets that will be accessed for this study will be securely stored and protected through an honest broker. We designated the Center for Biomedical Informatics and Biostatistics’ Biomedical Informatics Services at the University of Arizona as the honest broker to facilitate the de-identification, transfer, and management of data, as well as maintain protected health information anonymisation and Health Insurance Portability and Accountability Act (HIPAA) compliance. In this role, the honest broker can identify individuals overlapping between relevant databases, and assign de-identified study codes that would enable cross-linking individuals between the systems.
The honest brokers will link the HSP database to the VRBD to generate a comparison group. They will match both the HSP and non-HSP groups to HDD and ASIIS databases using personally identifiable information (eg, name, date of birth, social security number). The honest brokers will create a separate de-identified ‘limited data set’ for our analyses to compare the mean outcomes of HSP mothers to the comparison group mothers.
The motivation for using PSM to create a synthetic comparison group is to be able to observe the ‘counterfactual’ to HSP participation, that is, what would have happened in the absence of the programme. We will explore this by comparing outcomes between HSP mothers and those matched to them by the propensity score. More specifically, the average treatment effect generated by PSM will estimate the impact of the programme on the population of both HSP mothers and those who ‘look like’ HSP mothers by taking the difference in outcomes between HSP mothers and their matches, and vice-versa.
Our analytic population is of sufficient size to detect meaningful programme effects from low-frequency events, including preterm births, low and very low birth weights, maternal morbidity, and differences in immunisation and hospitalisation rates over a relatively long period. This is also true for specific subgroups served by HSP (eg, Hispanics, American Indian, economically disadvantaged).
Once we establish proper covariate balance between the intervention and matched-control groups, point estimates of the treatment effects will be estimated by comparing outcomes using Stata V.14 software and specifically the teffects command.44 Following Abadie and Imbens,45 46 this command considers the fact that propensity scores (ie, the parameter that determines the comparison population) are estimated when calculating the standard errors, and thus generates confidence intervals. The propensity scores will not be used as a covariate in traditional regression analysis because it is less effective in forcing baseline equivalence and assumes the relationship between the score and the outcome is linear.41
Both the HSP enrollment information and VRBD are administrative data sources, established and maintained for public health monitoring purposes. As such, we do not anticipate missing data to be a significant issue. We assume that such instances (as we find them) are very likely to be the result of human error and not any systematic issues with the data collection and/or reporting processes. Where missing-ness does occur in the variables that make up the propensity score model, we will control for these using dummy variables in place of the missing observations. In the case of missing outcome variables, we will restrict the analytic sample to the non-missing observations, and inspect control variables to verify that there are no systematic differences.
Our evaluation will build on a previous evaluation of HSP conducted by Hussaini et al, which found that HSP participation was associated with a reduction in the likelihood of a low birthweight outcome.21
The Hussaini study used data from 2007 and compared 484 HSP enrollees to almost 5000 non-HSP women; our study compares 9665 HSP enrollees to approximately 23 000 non-HSP women spanning 10 years of service. Based on observed covariates, the Hussaini study matching process did not result in baseline equivalence across the two groups. For example, the comparison group was on average 4 years older (28.2 vs 24.3) than the HSP mothers. Additionally, Hussaini et al matched to comparison mothers on ex post medical risks, which likely created a bias in favour of finding a positive HSP effect. Our PSM model will generate a comparison group that achieves baseline equivalence of observed covariates. Additionally, we explicitly match on SES variables as required by the HomVEE-published standard for matched comparison group design studies.33 Specifically, we match on two individual measures of SES: maternal education and indicators for primary payer for the birth procedure. While these variables satisfy HomVEE’s documented standard for measuring SES for group design studies with a ‘moderate’ rating, we also use the maternal zip code of residence to include a measure of mean household income. Finally, we will build on the scope of the original study in two significant ways: (1) by expanding the number of the outcomes considered, including maternal and child outcomes over time, and (2) by performing a number of subgroup analyses that investigate programme impacts based on when in the course of the pregnancy the HSP intervention began, mother’s country of origin and maternal age (ie, teen mothers).
The primary limitation is the identifying assumption that selection into the HSP is driven by observable characteristics. This is a limitation common to most PSM analyses. Attenuation bias is a possibility if HSP mothers are incorrectly identified and linked to state birth certificate data. However, the effect of this would be to underestimate (in absolute value) the magnitude of the resulting coefficients, meaning the true effect is likely to be larger (ceteris paribus). In addition, the analysis may have limited external validity for populations who differ along SES, race and ethnicity.
Ethics and dissemination
Data will be collected by the Arizona Department of Health Services for surveillance and monitoring. Protocol complies with the University of Arizona Biomedical Informatics Service group information security policies including, Information Security Policy (IS-100), Computer and Network Access Agreement (IS-700), Acceptable Use of Computers Policy (IS-701), Electronic Privacy Statement Policy (IS-1000) and Data Classification and Handling Standard (IS-2321).47
On completion of the study, we will initiate major dissemination strategies, including (1) peer-reviewed publications in targeted journals; (2) scholarly presentations at scientific conferences and public health governance meetings; (3) interactive web-based promotional and training materials and (4) strategic informational and planning meetings. In collaboration with ADHS, we aim to submit published journal articles to Mathematica Policy Research for consideration of the HSP as a HomVEE evidence-based practice home visiting model. We will identify local and national forums for dissemination of preliminary results. Findings will be shared with ADHS leadership, Arizona Health Care Cost Containment System (Arizona Medicaid), Arizona Public Health Association, American Public Health Association, MCH-specific conferences and professional forums, the Arizona Association of Federally Qualified Community Health Centers, Association of Health Plans, CHW workforce coalitions, and Maternal, Infant, and Early Childhood Home Visiting.
We would like to acknowledge the Arizona Department of Health Services (ADHS) and the several ADHS Offices who have supported this partnership, including ADHS Office of Women and Children’s Heath and the Health Start Program, ADHS Immunisation Program, and ADHS Vital Statistics. We deeply appreciate their commitment to participating in this study designed to improve the health of women and children of Arizona. We also acknowledge the members of the University of Arizona and Northern Arizona University—ADHS Steering Committee for continued guidance in data sharing and data protection and security. We acknowledge the several Health Start Programs and hundreds of the Health Start Program CHW workforce who have worked tirelessly to improve maternal and child health equity among thousands of mothers, babies and children.
Contributors All authors contributed equally. SS, MB and SR conceptualised the original study protocol. PW and VP conducted data analysis and contributed greatly to the preparation of the analytic methods and data management sections. SS, KM and SR led all protocol writing and editing. MC provided ongoing edits to early and late stage drafts. ADHS will oversee the study and (1) provide input to study design, conduct, data analysis and interpretation of results and review of draft manuscripts; (2) conduct monthly meetings and conference calls to discuss the evaluation impact study, challenges, and barriers and (3) provide Heath Start Program data, documents and data results, programme training and policies manuals as needed.
Funding The Arizona Department of Health Services (ADHS) funds the study, from 1 July 2017 through 30 June 2022. Health Resources Services Administration (HRSA) Maternal, Infant, Early Childhood Home Visiting (MIECHV) provided 17 months of additional federal funding through ADHS during the study period.
Map disclaimer The depiction of boundaries on this map does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. This map is provided without any warranty of any kind, either express or implied.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Approved by the University of Arizona Research Institutional Review Board (Protocol # 1701128802), dated
25 January 2017.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.