Introduction Improving complementary feeding in Ethiopia requires special focus on dietary diversity. The Sustainable Undernutrition Reduction in Ethiopia (SURE) programme is a government-led multisectoral intervention that aims to integrate the work of the health and agriculture sectors to deliver a complex multicomponent intervention to improve child feeding and reduce stunting. The Federal Ministries of Health and Agriculture and Natural Resources implement the intervention. The evaluation aims to assess a range of processes, outcomes and impacts.
Methods and analysis The SURE evaluation study is a theory-based, mixed methods study comprising impact and process evaluations. We hypothesise that the package of SURE interventions, including integrated health and agriculture behaviour change communication for nutrition, systems strengthening and multisectoral coordination, will result in detectable differences in minimum acceptable diet in children 6–23 months and stunting in children 24–47 months between intervention and comparison groups. Repeated cross-sectional household surveys will be conducted at baseline and endline to assess impact. The process will be assessed using observations, key informant interviews and focus group discussions to investigate the fidelity and dose of programme implementation, behavioural pathways of impact and contextual factors interacting with the intervention. Pathways of impact will also be explored through statistical analyses.
Ethics and dissemination The study has received ethics approval from the scientific and ethical review committees at the Ethiopian Public Health Institute and the London School of Hygiene and Tropical Medicine. The findings will be disseminated collaboratively with stakeholders at specified time points and through peer-reviewed publications and presentations.
- child feeding
- dietary diversity
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The mixed methods evaluation comprises impact and process evaluation to support course correction and interpretation of primary outcome results.
The study will also provide evidence on drivers of diet, agriculture and child feeding-related behaviours in the context of a large-scale behaviour change intervention.
A quasi-experimental study design is used and Ethiopia has a very active nutrition landscape in which the roll out of similar interventions into comparison areas may occur over the study period.
Linear growth failure or stunting in children under 5 years of age reflects suboptimal nutrition or health conditions in a population and negatively impacts cognitive development.1 Growth failure is most pronounced between 6 and 23 months of age, during which period poor infant and young child feeding (IYCF) practices and disease drive stunting in low-income settings.2 Though optimal child feeding practices are essential to child survival, growth and development, globally 36% of children 6–8 months have not yet begun to eat complementary foods and 85% of children aged 6–23 months are not fed the minimum acceptable diet (MAD) (comprising minimum dietary diversity and number of meals per day).3
WHO recommends that behaviour change communication is made central to strategies to improve IYCF.4 These interventions seek to influence the knowledge, attitudes, beliefs and social norms that underpin dietary practices.5 Child feeding education targeted to caregivers of children under 24 months increases height gain and, in food insecure populations, decreases stunting6 7. Consequently, the promotion of breastfeeding and complementary feeding have been identified by the Lancet Maternal and Child Nutrition Series (2013) as key actions to reduce undernutrition.7 Behaviour change interventions may also provide opportunities to link nutrition education to other actions within a broader framework to address stunting.8
The intersection between low dietary diversity and poor complementary feeding practices forms an important nexus for multisectoral programming in subsistence agriculture settings. Diet quality is an essential component of complementary feeding9 and agriculture has an important role to play to ensure the availability, access and use of diverse foods.10 The promotion of nutrition-sensitive agriculture practices can also enhance food security and women’s empowerment to accelerate progress in IYCF.11
An estimated 37% of Ethiopian children under 5 years have stunted growth.12 IYCF practices are poor and dietary diversity is among the lowest in the world.13 Nationally, 7% of children 6–23 months old consume the MAD and initiation of complementary feeding is delayed in 51% of children.12
Ethiopia builds on more than a decade of experience adapting tools and developing message delivery platforms to promote breastfeeding and, increasingly, complementary feeding.14 15 Prior research has identified that barriers to improved complementary feeding include poor diversity of consumption at household level, social norms related to animal source foods, insufficient paternal engagement and low women’s empowerment.16 17 The government has recognised that leveraging contributions from both health and agriculture sectors is necessary to accelerate progress in child feeding and stunting reduction.
Ethiopia benefits from professionalised health and agriculture sector platforms that deliver government programming to every district and kebele (administrative subdistrict) in the country. Health extension workers (HEWs), women deployed at a ratio of approximately 1:250 households, oversee routine health post and community-based health and nutrition services. Agriculture extension workers (AEWs), largely men and provided for at a ratio near 1:167 households, deliver farmer training and demonstration gardens.
Nutrition policy takes a broad multisectoral framework.18 The National Nutrition Programme maps key actions identified across multiple sectors to improve the nutritional status of target groups and to reduce stunting.19 Federal guidelines have also been formulated to support nutrition governance through the formation of multisectoral coordination committees at each level of government.20
The Sustainable Undernutrition Reduction in Ethiopia (SURE) is a government-led multisectoral programme to integrate health and agriculture sector service delivery for nutrition outcomes. It is a complex intervention that both incorporates and enhances the former community-based nutrition (CBN) programme by expanding nutrition services and integrating selected agricultural services. Service delivery is underpinned by actions to strengthen systems and to facilitate multisectoral coordination.
CBN has been an important national health initiative since 200821. The main components include: (i) monthly growth monitoring and promotion for children under 24 months; (ii) vitamin A supplementation and deworming and (iii) quarterly screening for acute malnutrition. SURE aims to enhance CBN by expanding services to improve complementary feeding and dietary diversity. Using the socioecological theory of behaviour change, the SURE package includes components targeted to individual, social, organisational, community/societal and policy spheres22 (see table 1).
Enhanced nutrition services include:
Counselling of mother–father pairs delivered jointly by teams of one HEW and one AEW. By jointly targeting mothers and fathers, the intervention focuses first on negotiating improved nutrition and agricultural behaviours and second on building awareness of gender-based roles and interactions between men and women within the household to facilitate improved child feeding practices.
Women’s and men’s group dialogues in community networks of 30 households led by a HEW or AEW, respectively, to negotiate community actions to improve dietary diversity and child feeding. To influence community norms, all women and men resident in the network are invited to participate.
Health and agriculture development army leaders (a woman or man, respectively) selected from one of each six households to support government policy conduct routine meetings with the remaining five women or men in the network . Following an abbreviated local SURE training, the leaders support dissemination of IYCF and agricultural messages during these meetings.
Cooking demonstrations on how to prepare complementary foods are delivered jointly by HEWs and AEWs to all members of a 30-household network.
AEWs grow demonstration gardens in communities with emphasis on dietary diversity and nutrition-sensitive agriculture practices, and they train skills at farmer training centres.
A one-time input of improved seeds or poultry is provided to the lowest socioeconomic decile among households with children under 24 months at the start of programme implementation. AEWs advise each recipient on care of inputs and use of the resulting foods produced.
Media messages to promote IYCF practices and dietary diversity are broadcast nationally by radio to districts and regions.
The enhanced nutrition services use a range of social and behaviour change communication tools. The release of global recommendations for IYCF practices by WHO has enabled the development of locally adaptable evidence-based counselling packages for behaviour change interventions in low and middle income countries around the world.9 The standard Unicef training package focuses on provision of knowledge and skills to community healthcare workers to negotiate evidence-based child feeding.23 The SURE training packaged used the IYCF counselling process (3As—Assess, Analyse, Act) developed by Unicef and further adapted the same techniques for application to nutrition-sensitive agriculture advising.
Unicef’s global IYCF counselling package also supplies tested, locally adaptable pictorial tools to support message delivery.23 These tools were previously adapted and subsequently refined for use in Ethiopia by the Alive & Thrive project and the Federal Ministry of Health (FMOH) and adopted for use by SURE.24 The SURE programme then developed an original set of pictorial tools to support agricultural messages, including diversity of crop selection, land and water management, livestock and poultry, hygiene and women’s empowerment. In addition, a locally adapted seasonal food production calendar and a food group poster encouraging daily consumption of four of six food groups were developed to assist household and community-based programme activities.
Systems strengthening and multisectoral coordination components include:
The 5-day integrated SURE training provided at the outset of programme roll-out to all HEWs and AEWs in each district. The training includes an overview of nutrition, dietary diversity, IYCF practices, nutrition-sensitive agriculture practices and facilitation and negotiation skills. For many AEWs, SURE is the first nutrition training received.
An integrated monitoring system designed to feed upward into reports compiled by the multisectoral coordination committees and to facilitate joint health and agriculture sector work planning and supportive supervision at kebele level.
Financial support and oversight of the routine function of kebele and district multisectoral coordination committees, including regular meetings to review SURE and other nutrition activities comprising the National Nutrition Programme. Members are orientated to committee roles and responsibilities under federal guidelines at the outset of the programme.
SURE is focused on selected districts in Ethiopia’s four agrarian regions (Amhara, Oromia, Tigray and SNNP). There are three phases of programme implementation planned. The first phase rolled out the SURE programme to 63 districts between February and July 2017 and the programme presently reaches an estimated 800 000 children under 24 months. The second and third phases of programme implementation will include an additional 87 districts in total and are planned for 2018 and 2019, respectively. This study protocol pertains only to districts included in the first phase of programme implementation.
The baseline survey was completed in July 2016. The first round of process evaluation data collection was conducted in June 2017 and the second round in December 2017. A third round of process study data collection is planned in 2018, to be followed by the endline survey in 2019. This paper provides the protocol for the overall programme impact evaluation and for the process evaluation studies.
Aim and objectives
The SURE evaluation aims to demonstrate the extent to which complementary feeding and dietary diversity for young children are increased and stunting reduced among SURE intervention districts versus comparison districts. The findings of the evaluation study will be used to inform the Government of Ethiopia of the effectiveness of the programme and to support learning and course correction.
The objectives are:
To evaluate impact by gathering, analysing and synthesising evidence to determine whether SURE results in decreased stunting and increased dietary diversity and MAD among infants and young children.
To evaluate the quality of programme implementation and to inform refinements to intervention design and delivery.
To understand the mechanisms of change by which SURE activities impact key outcomes and nutritional status of children.
To develop a contextual record over time of factors that may have affected implementation, mechanisms of change and outcomes.
Methods and analysis
The programme theory, summarised in figure 1, was developed using the Unicef conceptual framework for undernutrition and agriculture–nutrition impact pathways developed by the International Food Policy Research Institute.25 Programme impact pathways map activities to explain how the intervention is expected to contribute to a chain of results that produce the intended or actual impacts. We theorise that the package of interventions improves child growth and health status by increasing household acquisition of diverse foods and improving child feeding practices, in addition to reducing exposure to pathogens through improved human and animal hygiene and increasing access to child healthcare services. These changes to nutrition and agriculture behaviours may be supported by increased paternal contribution to child diets and care practices. The variety and frequency of programme contacts, including interpersonal counselling, men’s or women’s group dialogues, community events such as cooking demonstrations and media messages, are expected to impact the range of outcomes.
The SURE study is a quasi-experimental theory-based evaluation using mixed methods. Repeated cross-sectional household surveys conducted at the same time of year at baseline and endline from intervention and comparison districts will be used to determine impact on stunting and children’s diets. Quantitative and qualitative methods will be used to examine processes—including the quality of implementation, mechanisms of change and contextual data—to help interpret findings of the impact evaluation.
Study site and population
FMOH selected 150 districts in the four agrarian regions (Amhara, Oromia, Tigray and SNNPR) to receive the SURE intervention. Of those, 63 were selected by FMOH to receive the intervention in the first of three roll-out phases. The evaluation study area was selected from first-phase districts to provide both the longest duration of intervention and the largest potential difference between intervention and comparison groups. The intervention targets primarily households with children under 24 months. Community participants and health and agriculture staff are also targeted.
Main hypotheses and outcome indicators
Primary outcomes include stunting and MAD. The main study hypothesis is that complete and sustained programme implementation over a minimum period of between 24 and 28 months will lead to at least:
A 0.21 increase in mean height-for-age z score in children aged 24–47 months.
A 10% reduction in stunting in children aged 24–47 months.
An increase of 0.17 in the number of food groups consumed by children 6–23 months.
A 6% increase in children 6–23 months consuming the MAD.
Children in the 24–47 months age range will be assessed for stunting outcomes. Growth faltering is most pronounced from 0 to 23 months of age.26 Assessment of stunting in children after this period will allow for the longest duration of exposure to the intervention (fully 24 months).
A range of other behavioural outcomes and programme coverage indicators will also be measured to assess programme impact pathways (see table 2).
Sample size and selection
The sample size calculations for the household survey are based on detecting a change at endline in nutritional status and key IYCF practices that can be attributed to the intervention. Detectable differences in key IYCF indicators for intracluster correlation coefficients of 0.03 and 0.08 with 80% power with a significance level of 5% are shown in table 3.
Sample sizes at baseline and endline are presented in table 4. Sample sizes are based on a fixed number of kebeles per district (four at baseline and six at endline—the smaller baseline sample size was due to a shortened data collection window) and a fixed number of households (15) per kebele. Based on data summarised from the baseline survey completed in July 2016, we estimate that 15% of households will have two eligible children 0–47 months. The number of children we expect to sample at endline is shown in table 4.
Prior to the baseline survey, 36 intervention districts were randomly selected within four regional strata with the number per strata determined in roughly equal proportions to the 50 first-phase intervention districts. Accordingly, 18 districts were selected from Oromia, 8 from Amhara, 6 from SNNP and 4 from Tigray regions. Random selection was done in Stata 14. Comparison districts were selected in equal number per the four regional strata to those selected among the intervention districts (see figure 2). Districts were roughly matched based on stunting prevalence tertiles (low, medium or high based on district-level small area estimates provided by the World Bank27) and the presence of the Productive Safety Net Programme as a proxy for food insecurity. Districts were not individually matched but selected to provide for a reasonable balance of characteristics between groups. The same intervention and endline districts selected at baseline will be sampled again at endline.
Kebeles selected for both baseline (4) and endline (6) were selected at study outset using probability proportional to size sampling from lists and population data provided by district officials. Selections were completed in Stata 14. Each was completed independently, producing different kebele samples for baseline and endline. At baseline, gotes (sub-kebeles) were next selected by simple random sampling conducted by team supervisors on arrival to the kebele during data collection (using the paper in hat method). Within the gote catchment area, a complete listing of all households with children under 24 months was then conducted by data collectors and 15 were selected using systematic random sampling. At endline, the same gote selection procedures will be used.
At baseline, all resident children 0–47 months within a selected household and their respective mothers were listed in the following age groups: 0–5 months, 6–23 months and 24–47 months. Where only one child for any/all age categories was present, all eligible children were selected (up to three children). Where multiple children from a single age category were present, one child was randomly selected per category by the computer-assisted personal interview survey programme. The mother of each selected child was also selected for interview. The same procedures will be used at endline.
Data collection tools
The baseline and endline household questionnaires comprise eight modules (see online supplementary file 1). Household-level modules comprise basic characteristics, household food security and dietary diversity, agriculture practices including food production and consumption and women’s empowerment. A module on IYCF knowledge, behaviours, child care practices and intervention services is directed to the mother of the child 0–23 months. A module on primary health and agriculture services targeted to households with children 6–47 months is directed to the mother of the child 6–47 months. The final module comprises anthropometry and finger prick blood samples for estimation of haemoglobin concentration of the child(ren) 6–47 months.
The household survey questionnaires were developed to conduct face-to-face interviews with household members. Questionnaires were translated into Amharic, Oromifa and Tigrigna, back-translated into English and field-tested prior to baseline survey deployment.
Impact on stunting and MAD in addition to other secondary outcomes will be estimated through mixed effects models with a random effect for cluster (kebele) to account for correlations among women and children in the same cluster. Fixed effects will be area (intervention vs comparison), time (follow-up vs baseline) and the interaction between area and time. The key effect of interest is the interaction, which estimates the change from baseline to follow-up in the intervention area relative to change in the control area (analogous to a difference in differences approach). The primary statistical analysis will be carried out at the individual level and will account for clustering.
Following the baseline survey, the following analysis procedures were used: indicators (and other factors) were tabulated by area (intervention/comparison). No significance tests were performed to test for differences. Descriptive statistics for continuous variables included the mean, SD, median, range and the number of observations. Categorical variables were presented as numbers and percentages. Missing data did not exceed that allowed for as data were monitored for completeness during the survey and followed up at the time of collection.
At future follow-up, the results and differences from baseline data will be tabulated by area and presented with the estimated effect of the SURE programme and 95% CI. A confirmatory cluster level analysis will also be carried out. All analyses will adjust for confounders, chosen from among contextual factors that change over time, by assessing whether the adjustment changes the effect estimate by 15% or more. Missing data will be identified by a data manager and followed up at the time of data collection. Unless there is considerably more missing data than that allowed for, no formal statistical methods will be used to account for missing data in the analysis.
Causal mediation analyses will be explored as part of the process evaluation to identify mechanisms of change. Structural equation modelling and/or a non-parametric approach based on the potential outcomes framework will be used to explore potential prehypothesised mediators of any observed intervention effects.
Mixed methods will be used to evaluate programme processes. Outcome indicators from the endline survey will be used to assess programme reach and behavioural mechanisms of change. Qualitative studies will contribute data on the fidelity, dose and effectiveness of programme activities in intervention areas and drivers of behaviour change in intervention and comparison areas. Overall process evaluation outcomes, research questions and methods are summarised in tables 5 and 6.
Four qualitative studies will contribute data in response to questions on fidelity, dose and effectiveness:
Fidelity and dose of the SURE programme training delivered to HEWs and AEWs.
Fidelity and dose of the SURE programme delivered to orientate multisectoral coordination teams at district and kebele levels.
Function and effectiveness of local multisectoral nutrition coordination teams to support delivery of SURE programme activities.
Fidelity and dose of the SURE programme delivered to mothers and fathers of children under 2 years.
Three qualitative studies will assist to evaluate behavioural changes and mechanisms of programme impact:
Barriers and facilitators affecting adoption of recommended IYCF practices and diverse food consumption.
Barriers and facilitators affecting adoption of nutrition-sensitive agricultural practices and diverse food production.
Role of gender and social norms in the delivery and practice of IYCF and nutrition-sensitive agriculture messages.
Sample size and selection
Samples will be drawn purposively from preselected endline districts and kebeles from all four regions of programme implementation. To ensure that a broad range of implementation areas are sampled, the number of districts selected for each qualitative study will comprise: Oromia (n=3), Amhara (n=2), Tigray (n=2) and SNNP (n=2). For the three process studies investigating behavioural change and mechanisms of impact, comparison districts will also be sampled to capture contextual factors—for example similar nutrition interventions—as follows: Oromia (n=1), Amhara (n=1), Tigray (n=1) and SNNP (n=1). The number of kebele sites and the total number of observations, interviews and focus group discussions conducted for each study will depend on reaching the theoretical saturation point among participants with a particular set of characteristics. To ensure heterogeneity, a diverse set of eligible participants will be recruited for each study based on known or hypothesised cultural, socioeconomic characteristics or other factors.
Data collection tools and analysis
Appropriate topic guides and tools for each study and method will be developed and tested prior to field deployment. Framework analysis will be used to analyse qualitative data and Nvivo V.11 will be used to code and construct thematic areas from the collected data. Further analyses will be carried out to generate conceptual models or theories that might potentially explain change or absence of change in key behavioural indicators.
Patient and public involvement
Patients were not involved in designing the study or setting the primary research outcomes. Patients will be invited to contribute to the evaluation dissemination strategy.
Ethics and dissemination
The baseline protocol was approved by the scientific and ethical review committee (SERC) at Ethiopian Public Health Institute (Ref number: SERO-54-3-2016) and by the London School of Hygiene and Tropical Medicine (LSHTM) ethics committee (Ref number: 10937) prior to data collection. Process evaluation studies have been approved by SERC (Ref number: SERO-047-01-2017) and study approval processes are ongoing. The full protocol has been approved by the LSHTM ethics committee (Ref number: 12268). By approval of both institutional review boards, informed oral consent is given by caregivers of young children and witnessed by the local HEW as a result of the known reticence of illiterate persons in Ethiopia to sign or thumbprint unfamiliar documents. Written informed consent is obtained from all other study participants.
The findings of the implementation process evaluation will be disseminated to partners, including the FMOH and the Federal Ministry of Agriculture and Natural Resources and regional representatives, through a consultative workshop to facilitate programme course correction. A final dissemination meeting will also be held to present final results of the evaluation study. Peer-reviewed publications and conference presentations will disseminate results to the broader research community.
The potential synergies between agriculture and nutrition are well recognised,10 28 and evidence of multisectoral programme impact is beginning to emerge.29 30 Yet evidence of the ‘how’ and ‘why’ underlying the effectiveness of complex nutrition interventions remains sparse.10 This paper describes the design for impact and process evaluation of a multisectoral nutrition education intervention linking nutrition-sensitive agricultural practices to improved complementary feeding. The results of the evaluation will also provide evidence on drivers of diet, agriculture and child feeding-related behaviours in the context of government-led behaviour change intervention at scale.
We gratefully acknowledge the support of Professor Simon Cousens of the London School of Hygiene and Tropical Medicine in the design of this evaluation. We acknowledge and thank the Government of Ethiopia for their valuable contributions and feedback, especially the Federal Ministry of Health and the Federal Ministry of Agriculture and Natural Resources. We are grateful to the larger team at the Food Science and Nutrition Research Directorate, Ethiopian Public Health Institute for their commitment to training and oversight of data collection throughout the study. We also thank the many participants in the study who have and will participate in the study, including child caregivers, community participants, development army leaders, health and agriculture extension workers, district-level staff and members of the multisectoral coordination committees.
Contributors CM, THB, MMS, SE, DK, GA, AA, EA and ADD conceived the study; CM, THB, MMS, JS, EA and ADD developed the study protocol; CM led the writing of this paper with contributions from THB, MMS, JS and EA. SE leads the Ethiopian Public Health Institute team (TB led up to March 2017). ADD leads the London School ofHygiene and Tropical Medicine team. All authors read and approved the final version of the manuscript.
Funding This study was supported by the Children’s Investment Fund Foundation. The funders did not participate in the study design, data collection or analysis, review of the manuscript or decision to publish. The grant number is G-1610-01691.
Competing interests None declared.
Patient consent Not required.
Ethics approval Scientific and ethical review committee (SERC) at Ethiopian Public Health Institute and the London School of Hygiene and Tropical Medicine ethics committee.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement This is a study protocol and no new data were created.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.