Article Text
PDF
Abstract
Background Between 1999 and 2007, the Ministry of Health and Population in Egypt scaled up the Integrated Management of Childhood Illness (IMCI) strategy in 84% of public health facilities.
Objectives This retrospective analysis, using routinely available data from vital registration, aimed to assess the impact of IMCI implementation between 2000 and 2006 on child mortality. It also presents a systematic and comprehensive approach to scaling-up IMCI interventions and information on quality of child health services, using programme data from supervision and surveys.
Methods We compared annual levels of under-five mortality in districts before and after they had started implementing IMCI. Mortality data were obtained from the National Bureau for Statistics for 254 districts for the years 2000–2006, 41 districts of which were excluded. For assessment of programme activities, we used information from the central IMCI data base, annual progress reports, follow-up after training visits and four studies on quality of child care in public health facilities.
Results Across 213 districts retained in the analysis, the estimated average annual rate of decline in under-five mortality was 3.3% before compared with 6.3% after IMCI implementation (p=0.0001). In 127 districts which started implementing IMCI between 2002 and 2005, the average annual rate of decline of under-five mortality was 2.6% (95% CI 1.1% to 4.1%) before compared with 7.3% (95% CI 5.8% to 8.7%) after IMCI implementation (p<0.0001). IMCI implementation also led to marked improvements in the quality of child health services.
Interpretation IMCI implementation was associated with a doubling in the annual rate of under-five mortality reduction (3.3% vs 6.3%). This mortality impact is plausible, since substantial improvements occurred in quality of care provided to sick children in health facilities implementing IMCI.
- Primary Care
- Public Health
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/3.0/ and http://creativecommons.org/licenses/by-nc/3.0/legalcode
Statistics from Altmetric.com
Article summary
Article focus
-
Assesses the impact of the strategy on Integrated Management of Childhood Illness (IMCI) on child mortality, through a retrospective analysis, using routinely available data from vital registration and surveys.
-
Describes the effect of IMCI on quality of child health services, using programme data from supervision and surveys.
-
Presents a systematic and comprehensive approach to scaling-up IMCI interventions under routine programme conditions.
Key messages
-
IMCI implementation was associated with a doubling in the annual rate of under-five mortality reduction (3.3% vs 6.3%) compared with levels before districts started implementing IMCI.
-
Substantial improvements occurred in the quality of care provided to sick children in health facilities implementing IMCI.
Strengths and limitations of this study
-
The evaluation covered IMCI implementation at national scale in a country with a large population. Therefore, it is of a different magnitude to previous IMCI evaluations.
-
The evaluation was based on data that were routinely collected through the national health information system that were corroborated with findings from independent surveys. This is a unique strength currently called for in the recommendations of the UN Commission on Information and Accountability, which aims to accelerate progress towards the MDGs.
-
The paper describes a systematic model of scaling-up the IMCI strategy that can be replicated and is relevant for other settings.
-
The analysis was based on an observational design and therefore is open to selection and measurement biases and confounding.
-
Health service utilisation in relation to IMCI implementation and changes in population-based coverage with other interventions could not be assessed.
-
The mortality impact is plausible, since substantial improvements occurred in all four components of the causal pathway, from health facility supports to appropriate management of child illness, caregiver knowledge about home care and caregiver satisfaction with the services received.
Introduction
In 1995, The WHO and UNICEF launched the Integrated Management of Childhood Illness (IMCI) strategy to improve child survival.1 The strategy aims to reduce child mortality through appropriate management of major causes of child death including pneumonia, diarrhoea, malaria and malnutrition, improved caregiver knowledge and home care practices during illness, and prevention of illness.
The Government of Egypt adopted the IMCI strategy in 1997 with the aim to accelerate reduction in under-five mortality. In the year 2000, the under-five death rate was 47/1000 live-births2 and most deaths were caused by neonatal conditions (44%), pneumonia (15%) and diarrhoea (13%).3 A national IMCI programme was established and a national plan for scaling-up IMCI activities was adopted in 1999, with the target that all primary healthcare (PHC) units in the country should provide care in accordance with IMCI by 2010.
In 2007, the proportion of PHC facilities implementing IMCI reached 84% and there was an internal demand to assess the impact. Previous evaluations of the IMCI strategy in other countries had been few and had limitations, either because they were restricted to trials in small geographical areas,4 ,5 or because the IMCI strategy was not fully implemented.6–8 We therefore conducted this retrospective evaluation with two objectives: to assess the impact of IMCI implementation in Egypt and to document the scaling-up approach followed to implement the IMCI strategy fully in order to identify best practices. We used a causal pathway to link programme activities with intermediate outcomes and impact (figure 1). In this paper, we report on the impact of IMCI implementation on under-five mortality in Egypt in the period 2000–2006. We also present selected data on intermediate outcomes, in particular on quality of care provided to sick children in PHC facilities as a critical element in the causal pathway from programme activities to the mortality impact.
Casual pathway.
Methods and data sources
Population of interest
Egypt is a middle-income country. Administratively, at the time to which this analysis refers, the country was composed of 27 governorates and 256 districts. In 2000, the gross national income per capita was $1495 (PPP international dollars).9 As per Egypt 2006 census, adult literacy was 70.3%”, 96% of the households had access to improved drinking water source and 86% to improved sanitation facilities.10 The country has a well-developed PHC system that provides free maternal and child care services to 97% of the population. However, inequalities in access exist and populations in rural areas and in Upper Egypt have poorer health indicators than those in urban areas and in Lower Egypt.11 Our evaluation covered the entire population of children under-five years of age in the country. For the mortality analysis, the unit of analysis was the district. For the quality of care analysis, the unit of analysis was the health facility.
Intervention under evaluation
In Egypt, IMCI implementation in the period 2000–2006 focused on the improvement of quality of care provided to sick children in government-run PHC facilities, in accordance with national IMCI guidelines. These guidelines covered the identification and management (or timely referral) of children with pneumonia, diarrhoea, malnutrition, neonatal conditions, sore throat or wheezing. They also promoted feeding counselling for all children less than 2 years of age, identification of those needing vitamin A supplementation and identification of those needing immunisations for referral to scheduled sessions.
The national IMCI programme, composed of a central IMCI unit and governorate and district coordinators, led the implementation under auspices of a director general. The IMCI strategy was included in the Basic Benefit Package of the Health Sector Reform in 1999, and a government budget line was allocated to it. The programme developed a systematic approach for scaling-up activities, with well-defined sequential steps and quality criteria for each step (figure 2). All steps are associated with 2 activities; timely documentation and advocacy.
Key steps of the systematic approach to scaling-up of the Integrated Management of Childhood Illness strategy in Egypt.
Since 1999, the IMCI strategy has been introduced in 3–5 new governorates every year. The initial focus was on districts with highest under-five mortality. The principle was to introduce IMCI at a fast rate of implementation in all health facilities in a district before entering a new district. In each health facility, an adequate number of staff were trained to manage all sick children according to specific IMCI guidelines.
The expansion of IMCI activities was assisted by a high level of political support, the presence of a national IMCI programme, the early involvement of academia and professional organisations and strong partnerships.
Key activities in each of the three components of the IMCI strategy were:
Strengthening health system supports: The programme distributed case management tasks between physicians and nurses based on their job description. It developed a flow chart to guide the organisation of work and patient flow in health facilities, as well as tools for improving drug management, recording and reporting. Supervisors were trained in supportive supervision including observation of case management using standard checklists. The programme set up an IMCI monitoring system with a central data base to which health facilities with trained staff provided monthly reports, through the governorate and district management teams. The central unit gave feedback on the data at least quarterly and developed a detailed annual IMCI report.
Strengthening health providers skills: The IMCI unit tailored training to the job descriptions of doctors and nurses and developed two types of courses, with durations of 7 and 4 days, respectively. During a course, emphasis was placed on adherence to quality criteria, including a criterion that at least 35% of time was spent on hands-on, clinical practice. Newly trained healthcare providers received a follow-up visit by a qualified professional 4–6 weeks after training. Results were recorded in a central data base and corrective actions taken as needed. From the beginning, the central IMCI unit engaged with academia and professional organisations who participated in planning and adaptation. Besides in-service training, a comprehensive package of WHO regional tools for IMCI preservice training facilitated the introduction of IMCI guidelines in medical schools.
Strengthening family and community practices: The strategy for improving family and community practices focused on counselling caregivers on key family practices during sick and well child visits. In addition, activities to promote good care practices were deployed at community level in several districts.
Comparison
To assess the impact of IMCI implementation, we compared the annual levels of under-five mortality in districts after they had started implementing IMCI with levels before they were implementing IMCI. For each year between 2000 and 2006, we classified districts as either implementing IMCI or not, according to the date that staff were initially trained in IMCI in the district. We arbitrarily chose a cut-off point for training of August each year in order for a district to be considered to have implemented IMCI in a given year. All districts were part of the study population in 2000 and those districts not implementing IMCI in the beginning gradually shifted to the IMCI group over time.
Outcomes
In order to explore the plausibility of a causal effect of the IMCI activities on under-five mortality, we developed a causal pathway to set out our assumptions (figure 1). We defined four levels of outcomes. At the first level we assessed whether programme activities had been implemented. We then assessed whether activities led to improved health facility supports, appropriate treatment, improved caregiver knowledge and caregiver satisfaction with the services. We also examined data on utilisation of health services. Finally, we examined district-level trends in under-five child mortality to assess mortality impact.
Programme activities and intermediate outcomes
For the assessment of programme activities, we used information from the central IMCI data base and annual progress reports. On average, monthly reports were received from about 80% of IMCI-implementing health facilities since 2000. To assess intermediate outcomes, we used data from four sources: (1) an assessment conducted by the IMCI programme with direct observation of case management and review of facility supports, based on the methodology recommended by WHO,12 before and after (4 weeks and 6 months) IMCI implementation, covering 97 health facilities in 13 districts in four governorates, targeted for IMCI introduction between 2001 and 200213; (2) a cross-sectional health facility cluster survey of outpatient child care services with direct observation of case management and review of facility supports, based on the methodology recommended by WHO, carried out by the IMCI programme and WHO in 50 health facilities randomly selected out of 294 health facilities implementing IMCI (HFS-2002)14; (3) a comparison of quality of child care in 48 health facilities implementing IMCI in three governorates versus 48 health facilities implementing ARI and CDD programmes in three governorates without IMCI, conducted by an independent expert between 2002 and 200315 based on the methodology recommended by WHO for health facility surveys16; and (4) 6083 follow-up visits with direct observation of case management and review of facility supports, based on the methodology recommended by WHO,12 conducted by the IMCI programme between 1999 and 2007, 4–6 weeks after healthcare providers were trained in IMCI.17
Utilisation of health services
We reviewed monthly health service utilisation data reported in the IMCI routine monitoring system for the 217 districts between 2000 and 2006. However, the data were incomplete, showing important gaps and inconsistencies and were not suitable for the purpose of our analysis.
Mortality impact
We obtained under-five child death rates by district from the National Bureau for Statistics for the years 2000–2006. Data were available for 254 districts, 41 districts of which were excluded. The reasons for exclusion were as follows. In Egypt, deaths are recorded by the district in which they occur. This results in extremely high under-five child death rates being recorded in districts with large paediatric referral hospitals that do not reflect the death rate among children living within the district. We therefore excluded such districts (n=11) from our analysis. During the period 2000–2006 a number of new districts were created; we excluded 13 districts from our analyses because of such changes. Since we only obtained death rate estimates from the National Bureau of Statistics but not numbers of actual deaths, we also excluded 13 districts with small estimated populations of under-five children (population less than 2000), to reduce ‘noise’ in the data resulting from small absolute numbers of deaths in these districts. We examined the remaining 217 districts for evidence of extreme variation in death rates likely to be the result of small numbers or data errors and excluded a further four districts for this reason. We therefore used mortality data from 213 districts in our analysis. The under-five population of these 213 districts was 6 620 706, representing 93% of the total under-five population in the country.
The data were analysed using Stata V.11 (StataCorp, College Station, Texas, USA: http://www.stata.com). We first fitted random effects linear regression models, using restricted maximum likelihood, with log (under-five death rate) as the outcome, ignoring the presence or absence of IMCI. To estimate the average annual reduction in under-five mortality across the 213 districts retained in the analysis, we fitted various random effects models of log (under-five death rate) against year to the data from these districts. The best-fitting model was that which included random intercepts and random slopes at both the governorate and district level. To assess the effect of IMCI implementation on under-five mortality, we identified 127 districts that started IMCI implementation between 2002 and 2005 and therefore had under-five mortality data before and after IMCI implementation. We used a linear spline approach, which allowed the rate of under-five mortality decline to change following IMCI implementation. We included both random intercepts and random slopes at both the district and governorate level in the model.
Results
IMCI training
In 2000, IMCI had been introduced in 3% of all government PHC facilities; this figure rose to 76% in 2006 by the end of which 12 729 health providers in 3524 health facilities had been trained in IMCI. Training reports indicate that quality criteria were usually met, such as 35% of time was dedicated to clinical practice. Of the newly trained staff, 97% received a follow-up visit from a skilled supervisor 4–6 weeks after training.
Health system improvements in health facilities
Data from IMCI implementing health facilities, including follow-up visits and reports on the children seen and services provided, submitted monthly, showed that the introduction of IMCI in a district and health facility led to substantive improvements in the availability of medicines necessary to treat sick children as well as supplies and equipment (table 1). For example, the availability of all 14 essential medicines for case management of childhood illness increased from 1% before IMCI introduction to 57% 6 months after IMCI introduction. These improvements were sustained over time as illustrated by the results from follow-up covering 6083 observations.
Appropriate management of sick children reflecting improved skills of health providers
The quality of care provided to sick children improved dramatically after healthcare providers were trained in IMCI. Integrated assessment of the sick child did not exist before the introduction of IMCI but, after IMCI training, over 90% of healthcare providers correctly assessed a child for five main symptoms, at 4–6 weeks after training, and at 6 months after training (table 2). The proportion of children whose immunisation status was assessed during a sick child visit increased from 19% to over 90%, while the proportion of children less than 2 years of age who were assessed for feeding increased from 1% to over 90%. Strikingly only 10% of children needing an antibiotic were prescribed an oral antibiotic correctly before the introduction of IMCI, compared with over 80% after healthcare providers had been trained in IMCI.
Improved caregiver knowledge and caregiver satisfaction
The proportion of caregivers of a child prescribed an antibiotic who knew how to give the treatment rose from 7% before IMCI introduction to 67% or above when the child was seen by an IMCI-trained healthcare provider (table 3). Similarly, caregivers advised by an IMCI trained provider had much better knowledge on how to prepare and give ORS and how to care for the sick child at home than those advised by a non-IMCI-trained provider. Results from the preassessment and postassessment show that caregivers noted the difference in quality of care. The proportion of caregivers who reported satisfaction with the services received rose from 56% before IMCI introduction to 99% 6 months after IMCI was implemented.
Mortality results
The mean death rate across the 213 districts retained in the analysis (27.6 in 2000) was lower than that across all districts (32.3 in 2000), reflecting the exclusion of districts with paediatric referral hospitals with very high reported death rates. The average annual reduction in under-five mortality across the 213 districts (ignoring IMCI status) was estimated to be 5% (95% CI 4.2% to 5.8%; p<0.001). The decline was similar in the 41 districts excluded from the analysis.
There is strong evidence that the rate of mortality decline was faster after IMCI implementation than before (table 4). Across all the 213 districts included in the analysis, the estimated average annual rate of decline was 3.3% in the period before IMCI implementation compared with 6.3% after IMCI implementation (p=0.0001). For the 127 districts which implemented IMCI between 2002 and 2005 and for which pre-IMCI and post-IMCI implementation annual mortality reduction rates could be individually estimated, the average annual rate of decline pre-IMCI implementation was 2.6% (95% CI 1.1% to 4.1%) compared with 7.3% (95% CI 5.8% to 8.7%) after implementation (p<0.0001; table 4). Districts that initiated IMCI implementation in any of the 4 years between 2002 and 2005 showed a consistent pattern of slower average mortality declines pre-IMCI implementation than postimplementation (table 4 and figure 3).
Analysis of average annual rates of mortality decline pre- and post-IMCI implementation by year of implementation
Graphs showing acceleration of reduction in under-five child mortality following IMCI implementation compared with the preimplementation period.
Discussion
Our analysis suggests that the IMCI strategy in Egypt was associated with a doubling in the annual rate of under-five mortality reduction in districts implementing IMCI (3.3% vs 6.3%). This mortality impact is plausible, since substantial improvements occurred in all four components of our causal pathway. Health facility supports improved substantially after IMCI introduction, as illustrated by the data on availability of essential medicines and follow-up after training. The proportion of sick children who received appropriate management of illness increased and caregivers received counselling on home care and feeding. Caregivers were better able to recall key messages on home care and when to return to the healthcare provider. An increased proportion of caregivers reported to be satisfied with the services received.
We could not assess health service utilisation in relation to IMCI implementation in our analysis. Nor were we able to ascertain changes in population-based intervention coverage because we were unable to disaggregate available data from national surveys according to the geographical and temporal expansion of the IMCI strategy in Egypt. Results from Demographic and Health Surveys conducted in Egypt between 2000 and 2008 suggest an increase in rates of antenatal care and skilled care at birth at the national level, while coverage of ORT and antibiotic treatment for diarrhoea did not change substantially. However, we cannot comment on differences in coverage of any of these interventions between districts implementing or not implementing IMCI. Investments in safe motherhood were substantive during the period covering our evaluation and they were focused on 75 districts in Upper Egypt. While increases in the coverage of antenatal care and skilled care at birth were reported during the study period, they are unlikely to have confounded our under-five mortality results as the district level scale-up of these activities was not related to that of the IMCI strategy.
Our findings demonstrate that implementation of the IMCI strategy in health facilities, if done systematically, leads to improved treatment of childhood illness and improved caregiver knowledge, and these improvements appear to have resulted in reduced child mortality. The strategy can only be effective for those children who have contact with health services providing IMCI. In Egypt the proportion of children with suspected pneumonia seeking care from an appropriate health provider ranged from 66% in 2000 to 73% in 2008. This is comparable to findings in an IMCI evaluation study in Tanzania where utilisation of health facilities was high.5 While the private sector plays an important role in the provision of child health services in Egypt, a fair proportion of medical personnel in private practice also hold positions in the public health service and hence may have been exposed to IMCI.
Our analysis was based on an observational design and therefore is open to selection and measurement biases and confounding. However, the great majority of districts in the country were included in the evaluation and the average annual reduction in under-five mortality was similar in districts included and excluded from the analysis. Restricting the analysis to 127 districts for which trends both before and after IMCI implementation could be estimated, hence in which each district acts as its own control, did not alter our findings. While the possibility of residual confounding cannot be excluded, for confounding to explain the observed effect would require that other major changes affecting child survival occurred in districts at approximately the same time as IMCI was introduced. We are not aware of any major programme initiatives that could explain our results. Another possible explanation for our finding of more rapid mortality decline following IMCI implementation could be that there was already an accelerating trend of mortality decline throughout the country. Then any random choice of cut-off point in the time series followed by an analysis of trends before and after the cut-off would observe a more rapid decline after the cut-off point. To investigate this possibility, we performed 1000 simulations in which we randomly identified a cut-off for each district and then looked at the difference in the gradients of the slopes before and after the random cut-off points. While almost 99% of simulations showed a greater decline after than before the random cut-off points, only 12 of 1000 simulations (1.2%) yielded differences as large as, or larger than, the difference we observed before and after IMCI implementation. We interpret this to indicate some of the difference we detected comparing the period before IMCI implementation with that after implementation may be attributable to an underlying accelerating trend, but that it is unlikely that such an accelerating trend can entirely explain the difference we observed. We excluded from our analyses districts with large paediatric referral hospitals (with very high death rates which do not reflect the mortality experience of children living in those districts) and districts with very small populations or unstable death rates as well as districts whose boundaries substantially changed during the period. Rerunning our analysis including small/unstable districts and districts with referral hospitals resulted in a smaller estimated change in the annual rate of mortality decline after the introduction of IMCI (4.1% pre-IMCI vs 6.4% post-IMCI: p=0.02). Given that the inclusion of these districts is expected to introduce noise into the analysis the pattern of these results is unsurprising. Previous studies that have evaluated the mortality impact of IMCI include an observational study in Brazil, an ecological study in Peru, a controlled trial in four districts in Tanzania and a cluster randomised trial in Bangladesh.4–6 ,8 In Brazil and Peru, where IMCI implementation in health facilities and the community was weak, there was no evidence of an impact on mortality.18 ,19 In Tanzania, where the strategy focused on improved quality of care in health facilities, under-five mortality was reduced 13% more in the two IMCI implementing districts than in the comparison districts. In Bangladesh, similar reductions in under-five mortality were observed in intervention and control clusters. However, this study had low statistical power because of a sharp reduction in under-five mortality in the control group which had not been anticipated in the original design of the study.
Our analysis is unique in several ways. The evaluation covered IMCI implementation at national scale in a country with a large population. Therefore, it is of a different magnitude to previous IMCI evaluations. The evaluation was based on data that were routinely collected through the national health information system and by the national IMCI programme, as well as independent surveys to assess the quality of care. While incompleteness of data are a limitation—WHO's estimate of vital registration completeness in Egypt in 2006 was estimated at 90%20—the acceleration in average annual mortality reduction found in our analysis is supported by comparable findings reported by Countdown to 2015: tracking progress in maternal newborn and child survival.21 In addition, the four sources of data that we used to assess the quality of care in Egypt showed remarkable consistency.
Many lessons can be learned from the approach that Government of Egypt has taken to expand the provision of child health services according to IMCI quality standards in primary care facilities. First, the step-wise introduction of the IMCI strategy at governorate and district levels (as reflected in figure 1) directed efforts to improve essential health system elements. Training was only one of the components of the programme to be scheduled once there was readiness in health facilities and hence, IMCI was implemented as a strategy rather than a ‘training programme’. Second, the development of quality criteria for each step, and tools to monitor adherence, provided IMCI coordinators and district health managers with clear guidance on what and how to implement and how to assess quality. Third, the presence of a strong programme structure, with staff specifically dedicated to IMCI implementation at national, governorate and district levels, enabled implementation of activities according to plans. The high proportion of health providers who received a follow-up visit after training is testimony to programme strength. The establishment of a central data base for continuous monitoring of programme implementation and service provision facilitated regular and timely feedback at all levels. IMCI also enjoyed a high level of political commitment and strong partnerships with international partners, coordinated by WHO. It was included in the Basic Package of Benefits of the Health Sector Reform resulting in allocation of national budget for programme implementation and service delivery. The early involvement of academia in adaptation and planning of the IMCI strategy also led to a strong commitment that was translated into strengthened curricula and inclusion of IMCI in teaching of paediatrics and community health.
In conclusion, IMCI is one of the effective strategies for improving child survival and achieving the Millennium Development Goal of child mortality reduction. To be fully beneficial, IMCI should be implemented as a comprehensive strategy rather than a training programme, with special emphasis to systematic planning, adherence to quality criteria of programme implementation, and continuous monitoring of progress and timely feedback. In Egypt, a focus on care in primary health facilities was relevant because access and care-seeking was high. In other settings where access to and use of health facilities is low, more emphasis on community-based interventions may be needed in order to increase access to and coverage of essential health services. Because IMCI encompasses multiple preventive and curative interventions, it not only is useful to tackle childhood illness but effectively serves as a key strategy for primary child care.
Acknowledgments
The authors would like to thank José Martines and Cesar Victora who reviewed earlier drafts of the paper. The authors alone are responsible for the views expressed in this publication and they do not necessarily represent the decisions or the stated policy of the WHO.
Footnotes
-
Contributors All authors participated in the conception, writing and discussion of the paper. SC conducted the mortality analysis while A-NMA and SP compiled the data on programme implementation and quality of care.
-
Funding This study was funded by WHO. Funds were limited to the costs of the consultant in charge of statistical analysis and was provided by WHO/HQ.
-
Competing interests MAR was directly responsible for scaling-up of the IMCI strategy in Egypt. A-NMA, SF and SP provided technical support. BD provides technical support to selected activities. SC was funded by WHO to conduct the mortality analysis. All authors declare no conflict of interest.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement Data on a health facility survey on outpatient child care (IMCI) services, Egypt, March 2002, by the Ministry of Health and Population and the WHO, can be made available through correspondence with the author.