Introduction Economic evaluations of complex interventions in early child development are required to guide policy and programme development, but a few are yet available.
Methods and analysis Although significant gains have been made in maternal and child health in resource-constrained environments, this has mainly been concentrated on improving physical health. The Learning Clubs programme addresses both physical and mental child and maternal health. This study is an economic evaluation of a cluster randomised controlled trial of the impact of the Learning Clubs programme in Vietnam. It will be conducted from a societal perspective and aims to identify the cost-effectiveness and the economic and social returns of the intervention. A total of 1008 pregnant women recruited from 84 communes in a rural province in Vietnam will be included in the evaluation. Health and cost data will be gathered at three stages of the trial and used to calculate incremental cost-effectiveness ratios per percentage point improvement of infant’s development, infant’s health and maternal common mental disorders expressed in quality-adjusted life years gained. The return on investment will be calculated based on improvements in productivity, the results being expressed as benefit–cost ratios.
Ethics and dissemination The trial was approved by Monash University Human Research Ethics Committee (Certificate Number 2016–0683), Australia, and approval was extended to include the economic evaluation (Amendment Review Number 2018-0683-23806); and the Institutional Review Board of the Hanoi School of Public Health (Certificate Number 017-377IDD- YTCC), Vietnam. Results will be disseminated through academic journals and conference presentations.
Trial registration number ACTRN12617000442303.
- cost-effectiveness analysis
- cost – utility analysis
- return on investment
- early childhood development
- learning clubs intervention
- infant’s health
- infant’s development
- perinatal common mental disorders
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- cost-effectiveness analysis
- cost – utility analysis
- return on investment
- early childhood development
- learning clubs intervention
- infant’s health
- infant’s development
- perinatal common mental disorders
Strengths and limitations of this study
This study will contribute one of the first economic evaluations of a complex intervention to improve early child development in a resource-constrained setting.
The study will use a societal perspective to collect the costs of health service use and to productivity of participation in the Learning Clubs programme.
Cost consequences will be adopted to obtain secondary outcomes, including infant’s health and development, women’s health, family environment, and community empowerment and awareness.
Learning Clubs programme is multicomponent intervention; hence, it is difficult to capture the health and social benefits of each component
Cost estimation of health service use may be constrained by low literacy among study participants.
There is growing awareness globally of the importance of the first 1000 days from conception for early childhood development (ECD). Approximately 200 million children aged under 5 years living in low-income and middle-income countries (LMICs) are exposed to risk factors1–10 that prevent them from fulfilling their potential development11 12 and reduce functional capacity through adulthood.13–17 Recognising this, funders in all countries are now considering ECD interventions as investments generating significant returns.18
Pregnant women in rural Vietnam experience food insecurity, body mass index less than 18.5, iodine deficiency, iron deficiency anaemia and intimate partner violence at higher rates than in well-resourced settings. Around 30% of women have clinically significant symptoms of common mental disorders during the perinatal period.19–22 In addition, maternal services are restricted to providing physical, rather than mental healthcare. These risks have led to poor fetal and infant health and development.23–25
While the importance of ECD has been acknowledged widely in health and social terms, less attention has been given to its economic benefits. In 2015, a systematic review of early childhood interventions in LMICs by the Independent Evaluation Group of the World Bank included 55 studies, but found that only two reported cost data.26 There is a serious lack of data about health service use associated with interventions for ECD, especially from LMICs.27
The Learning Clubs intervention and trial
Learning Clubs for Women’s Health and Infant Health and Development is an innovative multicomponent psychoeducational intervention designed to improve the physical and mental health of women and the health and development of their infants in a resource-constrained setting, by addressing multiple risks at the same time. Its impact is being established in a two-arm parallel group cluster randomised controlled trial being conducted in 84 communes in a rural province in Vietnam in which 42 communes are assigned randomly to receive the intervention in 2018. All women in the trial receive pregnancy and child healthcare services from commune health stations including free pregnancy checks, giving birth in commune health stations (if the hospital is too far and the commune health station is qualified for birth assistance) or hospitals nearby, National Growth Monitoring and Expanded Immunisation Programmes (hereafter referred to as the usual standard of care). Women in the intervention group are invited to participate in Learning Clubs, which comprise facilitated small groups which meet in the commune at fortnightly intervals and proved perinatal stage-specific information and learning activities from pregnancy to 1 year post partum. The intervention draws on content from interventions demonstrated to be effective in addressing at least one risk in a resource-constrained setting and includes five main components: macronutrition and micronutrition for women and infants, gender empowerment, strengthening the intimate partner relationship, women’s mental health and providing sensitive, responsive care for infants. Learning Clubs are designed to be facilitated at the primary care level by members of the Women’s Union with support from the local health and education sectors. The intervention is to be offered in eight sessions during the pregnancy, and a combination of 11 sessions and 1 home visit in the first year after giving birth. The Learning Clubs programme is described in detail elsewhere.28
Economic evaluations in health contexts are undertaken to guide decision-makers how to allocate resources to improve health outcomes in the most effective ways.29 This economic evaluation will measure the cost to stakeholders of the Learning Clubs programme and compare it to the health, economic and social benefits arising from the initiative. The economic analysis will be presented in terms of the cost-effectiveness and cost consequences of achieving improved health and development outcomes compared with the usual standard of care, cost–utility analysis (CUA) where all health outcomes are expressed in a common metric (quality-adjusted life years, QALYs) and cost–benefit analysis where outcomes are expressed in economic terms and results are presented as investment metrics.
Aim and objectives
This study aims to provide a comprehensive economic evaluation of the Learning Clubs programme. The specific objectives are:
To calculate the direct and indirect costs of implementing the Learning Clubs programme compared with the cost of the usual standard of care.
To estimate the benefits of the intervention in terms of improved child health and development and maternal mental health.
To calculate the incremental cost-effectiveness ratio (ICER) between the Learning Clubs programme and standard care approaches.
To calculate the economic and social return on investment (ROI) from the Learning Clubs programme and its wider adoption.
To recommend the best possible strategies for improving ECD and the perinatal mental health of women in Vietnam.
The protocol was developed using the Consolidated Health Economic Evaluation Reporting Standards guideline.30 The economic evaluation will include cost-effectiveness, cost–consequence and CUAs from the health provider and societal perspectives as well as an analysis of the economic and social ROI (see figure 1).
The trial and this evaluation are being conducted in a Northern rural area of Vietnam (Hanam Province). The administrative health system consists of three levels: commune, district and provincial. Provincial level provides both prevention and treatment services for maternal and child health through the provincial general hospitals and the Centre for Preventive Medicine. Some districts may have a district general hospital. Each commune has a commune health station, which provides the standard of care for pregnant women and children. District health centres are in charge of providing professional support to commune health stations. In addition, the private sector provides a small percentage of health services (such as ultrasound pregnancy checks, and pharmaceutical treatment of common child health problems), and is growing significantly. The universal social health insurance covered nearly 60% of the population in 2010 and was expected to cover 70% by 2015 in Vietnam.31
Study population and sample size
The economic evaluation will collect information from 1008 participants recruited in 84 selected communes (504 women in each arm) of the Learning Clubs programme. The sample size is calculated to detect a difference in the infant’s development (8% in the intervention group and 15% in the control group) with 80% statistical power and a significance level of 0.05, and intracluster correlation coefficient=0.03. All pregnant women less than 20 gestational weeks who live in the selected communes are invited to participate by commune health staff and village health workers. Women having cognitive or serious physical disability are not recruited in the study. Women in the control group receive the usual standard of care, and women in the intervention group receive the free of charge Learning Clubs programme plus the usual standard of care.28 The control group will be the comparator in the economic evaluation.
Data are collected in both study arms in a baseline survey (early pregnancy), and programme outcomes will be measured at follow-up 1 (after 32 weeks of gestation), follow-up 2 (when the infant is 1 year old) and follow-up 3 (when the infant is 2 years old).
Independent data collectors from the Hanam provincial Center for Disease Control will be trained by Hanoi Research and Training Center for Community Development (RTCCD). They will collect data at baseline and three follow-up surveys. Data collectors are blinded to intervention allocation.
Patient public involvement
This protocol was developed in consultation with with local experts and members of the community.
Health and development outcomes
The Learning Clubs programme is a multicomponent intervention, hence there are various benefits of the programme. The outcomes are categorised in four main groups: infant’s health and development, women’s health, family environment, and community empowerment and awareness. The outcomes measured in the programme are shown in table 1.
The primary outcome being measured is the child’s development in terms of cognitive, language, motor and social-emotional domains at the age of 2 years, which will be assessed by the Bayley Scales of Infant and Toddler Development (BSID), third edition.32 The BSID has not been validated against a gold-standard comparator in Vietnam or any other LMICs. However, it has been translated into Vietnamese, culturally verified through community consultations and back-translated into English by our bilingual research team. It has been found to be sensitive to variations in ECD and acceptable to children and families in this setting.24 33
Infant’s health will be measured by several outcomes including wasting, the use of antibiotics, exclusive breast feeding, common illness symptoms, inpatient and use of outpatient health services. Infant wasting is assessed by data collectors to collect the weight and height of the children. The weight for height ratio will be based on the child’s age in months and sex.34 The remaining infant health outcomes will be assessed using the study-specific questions.
Women’s health outcomes are quality of life, maternal common mental disorders, productivity change of mothers and absenteeism form work due to child illness. The health-related quality of life of mothers will be assessed by EQ-5D-5L which is a common questionnaire which was validated in Vietnam to measure generic health status among specific and general populations.35–37 The EQ-5D-5L value set for Vietnam is not available, so we will use the cross-walk value for Thailand, a neighbouring country that is both geographically and culturally close to Vietnam.38 The questionnaire has two parts: part 1 consists of five questions about mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Each dimension has five levels ranging from no problems to severe problems. Part 2 is an analogue scale, which asks participants to report their health status. The scale ranges from ‘best imaginable health state’ to ‘worst imaginable state’.39
The prevalence of common mental disorders among mothers will be measured by the 21-item Depression, Anxiety and Stress Scale, which was validated in Vietnam with a sensitivity of 79.1%, a specificity of 77.0% and a cut-off score greater than 33 to indicate clinically significant symptoms.40 Productivity will be measured using the Work Productivity and Activity Impairment Questionnaire for general health problems. The tool will be used to capture both absenteeism and presenteeism over the past 7 days with six questions in both paid and unpaid work and regular activities. The outcomes are the per cent of work time missed, per cent impairment while working, per cent overall work impairment, per cent activity impairment due to general health problems.41 The absenteeism from work due to child illness will be measured in days by the study-specific questions.
Outcomes of family environment consist of home environment for infant development, intimate partner relationship, domestic violence against women and gender empowerment indicators (the engagement of the husband in household tasks, child caregiving activities and women’s mental health). The home environment for infant development will be measured by the Infant/Toddler Home Observation for Measurement of the Environment Inventory. This instrument has been translated and adapted by our team for use in rural Vietnam and found to be acceptable and sensitive.42 Intimate partner relationship will be assessed by Intimate Bond Measure, which has been validated among pregnant women and new mothers in rural Vietnam.43 Domestic violence against women will be measured by WHO Multi-Country Study on Women’s Health and Domestic Violence Questionnaire. This questionnaire has been adapted for use in Vietnam.44 The remaining outcomes of family environment are assessed using the study-specific questions developed by the research team.
Finally, the programme will benefit the community, therefore, outcomes of community empowerment and awareness will be considered such as the involvement of mass social organisations in delivering messages about the care of very young children in their routine meetings, the change in knowledge of the importance of ECD of the head of the communal People’s Committee and the number of loudspeaker messages delivered in each commune to advocate for ECD. This information will be collected using self-administered questionnaires sent to each commune.
A societal perspective will be adopted, but it will be limited to include only the costs of productivity losses, provision of unpaid informal care and health service costs. The Learning Clubs programme is a community-based health promotion intervention, therefore, productivity costs and informal care costs are the key components.
The length of the intervention is 18 months. After recruitment and completion of the baseline survey in 2018, the Learning Clubs programme is introduced to women in the intervention communes, commencing from around 22 weeks gestation until 1 year after birth in addition to the usual standard of care. There will be no intervention when the child is aged from 1 to 2 years. Women in the control group will receive the usual standard of government provided care for the same interval. The time horizon for the economic evaluation is the trial length, which is 30 months (excluding the programme preparation period).
Identification, measurement and valuation of costs
The study will collect all costs relevant to running the Learning Clubs programme (programme-related costs) and costs born by the study participants (participant’s costs) (table 2).
Program related costs
Programme-related costs include all costs related to whole-of-program activities and can be categorised into the phases of trial implementation (start-up, ongoing central management activities and ongoing programme activities). The cost of the start-up phase is the money spent on designing, piloting and conducting research evaluations of the programme. The ongoing central management activities consist of costs related to the central management of the programme, which are not associated with any specific programmes. Ongoing programme activities include all Learning Club activities for participants. For this study, all costs associated with start-up activities and ongoing central management activities will be excluded because this intervention will be compared with the usual standard of care that is operating with government support. The ongoing programme activities costs cover items in five categories (personnel, materials and supplies, equipment, media and policy advocacy, and training and transportation), which will be collected at national, provincial and commune levels27 45 46 (table 2).
Program-related costs will be recorded every 3 months by working with programme officers and accountants. The 3-month interval is a suitable time for minimising recall bias and the burden for programme staff. The costs will be based on financial and administrative records. All costs will be collected by the research team using the CostIt Software V.4.5 developed by WHO,46 which has been designed to record and analyse costs in health interventions.47 This study will use the programme template with a minor change to record the programme-related costs.
Unit costs for program-related costs are different for each category and based on the programme budget line. For example, in terms of personnel, unit costs are salary per month for part-time and full-time staff, and per day for consultants. Materials and supplies costs are measured on a monthly basis. Unit costs for policy advocacy and training are costs per meeting/workshop/training course.
Participants and their household costs
There two types of costs born by the Learning Clubs programme participants include healthcare seeking costs and the opportunity cost of participating the programme. These costs will be collected from follow-up 1 (when the participants are in late pregnancy) to follow-up 3 (when the children are 2 years old). A structured interview with data entered on a hand-held device will be used to collect all related costs.
Learning Clubs participants may use health services at hospitals, private clinics and commune health stations, and provided by the traditional healers. This evaluation will include both inpatient and outpatient health service use for participants: women who are pregnant and become mothers of index children and the index children. The costs related to health service use consist of direct medical costs, direct non-medical care costs and indirect resource use. For each type of service use, information will be collected on the number of health visits, the name of the health facility, health insurance use, travelling costs, out-pocket payment, travelling time, treatment time, waiting time and the number of caregivers who are adults. Informal care is a substantial part of the total costs and it illustrates the burden on the family rather than health or social services.48 The cost of informal care will be calculated by multiplying the total time spent by caregivers away from income-generating work and the market wage rate of the informal caregiver using the opportunity cost method.49 Further, the total healthcare expense will be calculated by taking into account the reimbursement of social/health insurance.50 The reimbursement of social or private health insurance will be included using the list of health services covered by social health insurance51 and the specific guidelines for private insurance.
In addition to the health service use, the costs related to participating in the Learning Clubs programme and productivity loss will be also considered. All intervention participants will receive relevant information and attend the meetings free of charge, so only indirect costs will be considered such as travelling method, distance from the house to the meeting venue, travelling time, number of adults attending the meetings, meeting duration (in hours). The friction cost approach is employed to estimate the productivity. This approach aims to calculate the productivity changes by adjusting the cost with reference to the friction periods.48 There is no postal code in Vietnam, therefore, using postal code for estimating travel expense is not feasible. In addition, the distance from participants’ house to the meeting venue is not far, hence the travelling cost will be calculated by self-reports.
The economic evaluation within the trial consists of a cost-effectiveness, a cost–consequence and a CUAs. In the long term, an economic ROI analysis will be conducted.
Cost-effectiveness and cost–consequence analyses
Cost-effectiveness analysis is one of the most common approaches to economic evaluation in health research programme. It calculates the cost of the change in health outcomes due to the intervention compared with standard care, for example, the cost per unit of health outcome or effect. The analysis calculates the ICERs, which is the difference in costs and health outcomes of two or more options.29 In this study, ICER will be calculated for the primary outcome—improvement of infant’s development in terms of cognitive, language, motor and social-emotional scales.
Ci and Ei are the costs and effectiveness measure of alternative i.
Moreover, due to this complex public health intervention, cost–consequence analysis is recommended to capture the full societal impact of the interventions. Cost–consequence analysis is a method, which includes a wide range of outcome measures of the intervention in health and social aspects.52 ICERs will be calculated with all statistically significant outcomes in terms of infant’s health, women’s health, family environment and community empowerment.
The CUA measures the incremental cost of achieving the health outcomes when they are measured using a common metric such as QALYs.
Using EQ-5D-5L, respondents make choices for each domain and their health status in a 1-digit number. Then these numbers will be combined in a 5-digit number, which can be converted into a utility weight. Each of the health outcomes will be expressed in QALYs so that the total QALYs can be calculated for both the intervention and control groups. These total QALYs can then be divided by the costs for each group and the difference in QALYs per unit of cost can be compared.29 In this protocol, the incremental cost will be caculated with the difference in QALYs of study women.
Ci and QALYi are the costs and QALY measure of alternative i.
Return on investment analysis
The investment case for the intervention will be assessed by calculating the ROI using a validated, peer-reviewed modelling approach, which has been applied in studies on reproductive, maternal, newborn and child health48, stillbirths49, mental health50, adolescent health and well-being51 and cardiovascular disease.53–57
This analysis calculates the economic benefits of the intervention arising from mortality and morbidity averted due to the Learning Clubs programme. These benefits occur over the working lives of the participants due to increase workforce participation and improved productivity. Savings from health expenditure averted are also included as benefits. These benefits are compared with the cost of the intervention incurred by the government and participants. The social ROI can also be calculated using standard measures of the value of a statistical life year such as the ‘full-income’ approach for LMICs.58
Currency, price date and conversion
All benefits, costs and ICERs will be calculated in Vietnamese dong and converted to US dollars, using purchasing power parities published by International Monetary Fund (IMF) for meaningful comparisons.48
Where applicable, benefits and cost will be discounted at the standard World Bank and WHO-CHOICE rate of 3%.59 Sensitivity analysis will be undertaken with alternative rates.
All data for this economic evaluation will be collected using hand-held electronic devices. The data will be uploaded everyday to a secure cloud-based storage system. The RTCCD data manager will check for missing values and clean the data daily. At the end of each survey, the data will be uploaded and stored on the secure Monash University server. A code number will be used to identify each study participants. No name and identifiable information will be entered. Only authorised researchers can access the data. Data will be analysed when the last follow-up survey (follow-up 3) is completed.
All study participants who attend the Learning Clubs meetings and complete surveys will be included in analyses using the intention-to-treat principle. The participant-related costs will be calculated for each phase. Missing data of costs and outcomes will be imputed using multiple imputation according to the multivariate imputation by chained equations algorithm with three steps (imputation, analysis and combination).60 This approach is considered as the most effective method to deal with missing data. Costs of each phase (pregnancy, first and second year postnatal) will be calculated based on the costs estimated at 1 month in each survey.
A multilevel model approach will be used for data analyses to address missing data, cost skewness and the difference of costs among clusters.61 Baseline variables will be included in the regression models of costs and outcomes to adjust the difference between control and intervention groups, and other sociodemographic characteristics such as economic status, educational level, occupation. For non-normal distribution of continuous costs and outcomes (such as quality of life among mothers and the BSID score of children), a generalised linear regression will be employed in STATA V.13 (StataCorp LP). Logit model will be used for binary outcomes (having common mental disorders and infant wasting). Parameter uncertainty will be addressed by deterministic and probabilistic sensitivity analysis.48
Cost price for the intervention will use a bottom–up approach. The cost-effectiveness ratios (ICERs) will be calculated by comparing the intervention group to the control group using cost per percentage point improvement of infant’s development, infant’s health and maternal common mental disorders. Non-parametric bootstrapping will be used for the cost-effectiveness acceptability curves. CUA will be presented by cost per QALY gained. ROI will be expressed in common metrics such as the benefit cost ratio, and internal rates of return.
It is challenging to conduct an economic evaluation of a complex public health intervention due to its diverse potential benefits to health and social sectors. Typically, a microlevel approach has been used and promoted among economic evaluations which mainly focused on calculating the cost-effectiveness of a particular health or clinical outcome such as mortality rate. This method is not helpful for decision-makers taking into account the overall context.52 62 The Learning Clubs programme is a complex public health intervention, which aims to address five main areas including macro and micronutrition; gender empowerment; men’s engagement in household work, women’s mental health; and child health and development.28 The programme has complex causal pathway because it may have positive impacts at individual, family and community levels such as infant’s health and development; women’s health; family environment; and community empowerment and awareness. Therefore, a macrolevel method is adopted by adding a cost–consequence analysis to the CEA and CUA in this protocol to capture all health or non-health outcomes. In the long term, the ROI analysis provides policy-makers in resource-constrained environments with an easily understood metric to prioritise health interventions in framing health budgets.
This protocol will contribute to the gap in evidence about cost data in the ECD area by providing evidence of both within-trial and long-term impacts. This is also the first economic evidence of a complex intervention for ECD in LMICs.
The Learning Clubs programme is being implemented among women living in a rural area who have limited health and economic literacy. The economic evaluation data will be collected in baseline and three follow-ups, which may create recall bias in terms of collecting the participants’ programme-related costs. A diary method is usual recommended to use to address this bias, however, it is not suitable for the participants in this study due to low literacy among some and lack of familiarity with routine recording among most. Hence, the costs will be extrapolated based on participants’ estimation in 1-month or 6-month periods.
Despite the potential limitation in the estimates of Learning Clubs participants’ costs and the complexity of the Learning Clubs programme, the economic evaluation is designed with careful consideration of these factors. It is expected that the economic evaluation will provide evidence of the cost and benefits of a first-ever comprehensive intervention for ECD in a LMICs. In addition, the evaluation will inform policy-makers about the relative value for money of the Learning Clubs programme at the provincial level and the likely indicators for scaling nationwide.
TN is supported by a Monash International Postgraduate Research Scholarship. JF is supported by the Finkel Professorial Fellowship, which is funded by the Finkel Family Foundation in a donation to Monash University. TT and SH are supported by National Health and Medical Research Council Early Career Fellowships. SL is supported by a National Health and Medical Research Council Career Development Fellowship.
Contributors TN, KS and JF were involved in developing the conceptual framework for the economic evaluation protocol and wrote the paper. TT contributed on data analysis method. SL, DBH, SH, TT, HT and B-AB contributed to the secondary outcomes of the protocol and reviewed the paper.
Funding The economic evaluation is funded by Grand Challenges Canada under the Saving Brain Initiatives (grant number TTS – 1803-22331).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.