Article Text
PDF
Abstract
Objective To assess whether pulse oximetry improves health workers’ performance in diagnosing severe childhood pneumonia at health centres in Southern Ethiopia.
Design Parallel cluster-randomised trial.
Setting Government primary health centres.
Participants Twenty-four health centres that treat at least one pneumonia case per day in Southern Ethiopia. Children aged between 2 months and 59 months who present at health facilities with cough or difficulty breathing were recruited in the study from September 2018 to April 2019.
Intervention arm Use of the Integrated Management of Childhood Illness (IMCI) algorithm and pulse oximeter.
Control arm Use of the IMCI algorithm only.
Primary and secondary outcome measures The primary outcome was the proportion of children diagnosed with severe pneumonia. Secondary outcomes included referred cases of severe pneumonia and treatment failure on day 14 after enrolment.
Result Twenty-four health centres were randomised into intervention (928 children) and control arms (876 children). The proportion of children with severe pneumonia was 15.9% (148 of 928 children) in the intervention arm and 3.9% (34 of 876 children) in the control arm. After adjusting for differences in baseline variables children in the intervention arm were more likely to be diagnosed as severe pneumonia cases as compared with those in the control arm (adjusted OR: 5.4, 95% CI 2.0 to 14.3, p=0.001).
Conclusion The combined use of IMCI and pulse oximetry in health centres increased the number of diagnosed severe childhood pneumonia.
Trial registration number PACTR201807164196402.
- community child health
- public health
- respiratory infections
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Random allocation of health centres to intervention and control arms.
Participating health centres were typical of such institutions in rural communities in Ethiopia.
Robust training on how to use the Integrated Management of Childhood Illness algorithm, and how to measure oxygen saturation.
Due to the nature of the intervention, inability to mask the health workers and the study participants of the intervention.
Unequal number of children seeking healthcare between the two comparison arms.
Introduction
Pneumonia killed approximately 920, 000 children aged less than 5 years in 2015.1 The mortality rate is especially high in Ethiopia, and 59 deaths per 1000 live births occurred in 2017 in children aged less than 5 years.2 Ethiopia ranks sixth among countries with the highest number of deaths from pneumonia in children aged less than 5 years.1
The WHO Integrated Management of Childhood Illness (IMCI) improves the quality of child care for common illnesses,3 4 but there is poor diagnostic precision for childhood pneumonia based on clinical features.5 The ability of healthcare providers to count breaths and classify respiratory rate in children using the IMCI algorithm is a challenge.6
Clinical signs of pneumonia, such as tachypnoea, inability to drink or breast feed and head-nodding, used in the IMCI algorithm, are not able to identify hypoxic children with severe pneumonia as precisely as pulse oximetry.7 Consequently, many children with severe pneumonia are dying because hypoxaemia is not adequately recognised and/or oxygen therapy is unavailable.1
This study is an extension of a study in which we first assessed the health system support of IMCI, where we found that the basic supplies for effective management of pneumonia were inadequate. Indeed, in a large proportion of the surveyed health facilities, essential drugs, vaccines, job aids and equipment were lacking. None of the health centres and health posts had a pulse oximeter. We also determined that many health facilities had an insufficient number of IMCI-trained health workers, and were deficient in requisite supportive supervision. In addition, health workers’ knowledge of managing severe pneumonia was low.8
For the above-mentioned reasons, an urgent need exists for interventions that assist health workers to improve the diagnosis of severe childhood pneumonia. The objective of the study was to assess whether pulse oximetry improves health workers’ performance in diagnosing severe childhood pneumonia at health centres in Southern Ethiopia.
Methods
This study adheres to the Consolidated Standards of Reporting Trials (CONSORT) statement extension for cluster randomised trials,9 available as supplementary files (CONSORT checklist). The full protocol of this trial was published at protocols.io (DOI: http://dx.doi.org/10.17504/protocols.io.88mhzu6).
Trial design
The study constitutes a parallel cluster-randomised controlled trial conducted at 24 health centres. An equal number of health centres were allocated into intervention and control arms, that is, the combined use of IMCI algorithm,10 and pulse oximetry (intervention arm), and the IMCI algorithm only (control arm). No changes were made to the design of the study after its commencement.
Study setting and participants
The study was conducted in the Gedeo Zone of Southern Ethiopia. The zone’s population is more than 1 million people, of which approximately 170 000 are children younger than 5 years.11 There are 146 health posts (operational unit for health extension workers), 38 primary health centres and 1 teaching and referral hospital. At these institutions, pneumonia is among the top-10 causes of outpatient treatment, and currently health professionals in study settings use the WHO IMCI algorithm to manage pneumonia and other common childhood illnesses.10
The study comprises 24 primary health centres in the rural Gedeo Zone, each of which treats at least one case of pneumonia per day. Children who were aged 2 months to 59 months with cough or had difficulty breathing for less than 14 days were included and followed for treatment outcome. Children aged 2 months to 59 months with cough or difficulty breathing for more than 14 days, or whose diagnoses were for other conditions, such as pulmonary tuberculosis, were excluded. In both arms, the assessment for inclusion and exclusion of children were identical. Pulse oximetry was not used to screen patients for inclusion in the study. Since there was no pulse oximeter in the study settings, no pulse oximeters were removed from their usual clinical practice for the purpose of the study. The recruitment of children started on September 2018 and lasted until April 2019.
Trial interventions
Health workers from the intervention arm used the WHO IMCI algorithm,10 and a paediatric fingertip pulse oximeter (ADC Adimals 2150) to diagnose pneumonia. Oxygen saturation was measured two times, 5 min apart. The measurement was taken when the child was calm, and recorded when the pulse oximetry accurately reflected consistent, high-amplitude plethysmographic waveforms associated with stable oxygen saturation for 1 min. Hypoxaemia (oxygen saturation less than 90%) diagnosis was based on the average of the two measures. Health workers from the control arm used the same WHO IMCI algorithm, but without measuring oxygen saturation.
Training was given for health workers on IMCI algorithm and how to use pulse oximetry. An IMCI-trained paediatrician offered the training, which was supported by a video-based exercise and practical session that was developed by the WHO and adopted by the Ethiopian Federal Ministry of Health.12 Prior to implementing the trial, a pilot study was conducted to estimate the intraobserver and interobserver reliability of the pulse oximetry. Detail about training, data collection and pilot results were given in trial protocol.
Study outcomes
The primary study outcome was severe pneumonia diagnosed using the IMCI algorithm in both arms.10 The IMCI criteria used to diagnose severe pneumonia or very severe disease is presented in table 1. In addition to the IMCI criteria, average oxygen saturation less than 90%,13 was used in the intervention arm.
WHO IMCI criteria used to diagnose severe pneumonia or very severe disease
The secondary study outcomes were: treatment failure on day 14 after enrolment,14 and severe pneumonia cases referred to the hospital. We considered treatment failure at day 14 if any of the following signs were present: development or persistence of general danger signs (eg, inability to drink or breast feed, vomits everything, convulsions, lethargy or unconsciousness), persistence of fever (axillary temperature ≥37.5°C), persistence of tachypnoea (respiratory rate ≥50 breaths/min in children aged 2 months to 11 months and ≥40 breaths/min in children aged 12 months to 5 years), chest wall in-drawings, presence of persistent cough, recurrence of fever, withdrawal from the trial or death. Treatment failure on days 2, 5 and 14 was initially planned in this study. But for the following reasons only treatment failure on day 14 was included in the final analysis: in Ethiopia, children with pneumonia will be managed at home and severe pneumonia is expected to be treated at a hospital. If the child had cough or difficulty breathing after 14 days of treatment, children would have been assessed for other diseases, for example, tuberculosis.
Follow-up visits
Children in both the intervention and control arms were followed for a total of 14 days. Visits were scheduled on day 2, day 5 and day 14 after enrolment. For children who missed a scheduled follow-up visit, health workers contacted the families at home on the following day. A child who could not be located was considered as lost to follow-up. In such cases, we contacted the participants’ family and neighbours by phone to collect information about deaths, relocation or hospitalisations.
Study size
The sample size was calculated based on a difference in effect size of 10%, power of 90%, 95% significance level, intraclass correlation coefficient of 0.025 and a minimum of 25 children with cough per cluster. Based on previous research, we expected health workers using the IMCI-alone algorithm to identify 4% of children with severe pneumonia.15 With this assumption, the estimated number of clusters was 11 in each arm. Drop-out of the entire clusters is uncommon; however, according to a recommendation by Rutterford et al,16 we incorporated one extra cluster per treatment group. Therefore, the total size of the cluster was 12 per treatment group.
Randomisation
Randomisation was done after we obtained consent from the district and health facilities head. The randomisation units were health centres. From 38 health centres in the study area, 31 health centres with at least one pneumonia case treated per day were included in the sampling frame. Accordingly, 24 of 31 health centres were randomly selected using the simple random sampling method. Of those 24, 12 were randomly selected for the intervention group and 12 for the control group. This random selection was performed at the University of Bergen using a list generated by SPSS version 20 (IBM Corp., Armonk, N.Y., USA) software. Due to the nature of the intervention, it was impossible to blind the data collectors and the study participants. Those doing the analysis were not blinded to the intervention allocation.
Analyses
We used Stata V.15 (Stata Corp LLC) for data analysis. An independent sample t-test for normally distributed continuous variables, Mann-Whitney U test for skewed continuous variables and χ2 test for categorical variables were used to compare baseline data in the two arms.
Mixed effect logistic regression with random intercept to account for clustering by health centres was used to estimate the effect of the intervention on primary and secondary outcomes. The effect estimate for primary outcome was expressed in OR with 95% CI, and p value less than 0.05 was considered as statistically significant. To control for potential confounding factors, some of the unbalanced baseline variables, such as child’s vaccination, parents’ wealth and educational status were considered during the analysis. Individual level variables such as child’s age, stunting and sex constituted other potential confounders, and were also adjusted for the regression analysis. Intraclass correlation coefficient for the primary outcome was estimated from the random effect model output.
Where values for baseline variables were missing, they were treated as missing and the missed values for few variables were less than 5%. Analyses were by intention-to-treat (ITT) principle. Those who died or withdraw from the trial were classified as treatment failure and included in the ITT analysis.
Patient and public involvement
There is no tradition in Ethiopia to invite patient organisations to take part in the planning, design and follow-up of trials. In this study, and before the implementation of the intervention, meetings were held with representatives from the Gedeo Zone Health Department, district health offices and head of health facilities. The meeting was about the burden of childhood pneumonia in the study area and at national level, and the purpose of the study. After having the authorities consent to carry out the study, written permission was obtained from zone and district health offices. All patients were informed about the purpose of the study, and caregivers were told that involvement is voluntary and that they could withdraw at any time regardless of reason. Caregivers were briefed about the oximetry procedure and routine pneumonia diagnosis. The caregivers were informed that the instrument would not harm the child. They were also assured that refusal to participate in the study would not affect their medical care in health centres. The findings of this study will be communicated to each of the health institutions and through the institutions to the patients.
Results
Twenty-four health centres were randomly and equally assigned into intervention and control arms. Nine hundred and twenty-eight children in the intervention arm and 876 children in the control arm were recruited and diagnosed for severe pneumonia and followed from September 2018 to April 2019. All of the 24 health centres and all recruited children were included for the primary and secondary outcome analysis on the basis of the original assignment (figure 1 for details on participants’ flow and recruitment).
Trial profile. HC, health centre; IMCI, Integrated Management of Childhood Illness.
Baseline characteristics
The baseline characteristics of the study group were comparable, except for pneumococcal and Haemophilus influenzae type b vaccines, and educational and wealth status of parents (table 2).
Baseline comparison between groups at individual and cluster level
Hypoxaemia
A total of 1804 children were enrolled in the study, of which 928 children were enrolled into the intervention and 876 into the control arm. Of the 928 children in the intervention arm, 135/928 (14.5%) had oxygen saturation of less than 90%. A total of 148 severe pneumonia cases were diagnosed in the intervention arm and, of these, 65 cases (43.9%) met the IMCI algorithm and had oxygen saturation <90%, 70 (47.3%) did not meet the IMCI algorithm, but had oxygen saturation <90% and 13 (8.8%) met the IMCI algorithm, but had oxygen saturation >90%. The overall median oxygen saturation was 94% (IQR=91%–96%). The median oxygen saturation among children diagnosed with severe pneumonia was 82%, (IQR=72%–86%), while among non-severe pneumonia cases the median was 94% (IQR=92%–96%).
In addition, we estimated the sensitivity and specificity of the two clinical signs used in WHO IMCI algorithm to diagnose pneumonia (fast breathing and chest in-drawing). Fast breathing had the highest sensitivity (94%), but specificity was low (25%) as compared with chest in-drawing (81%). Furthermore, using a combination of both symptoms did not improve their predictive ability (see online supplementary table 1).
Supplemental material
Primary outcome
The cluster adjusted proportion of diagnosed severe pneumonia was 148/928 (15.9%, 95% CI 4.7 to 27.2) for the intervention arm and 34/876 (3.9%, 95% CI 1.2 to 6.6) for the control arm, and p<0.001. The crude OR (COR) of being diagnosed with severe pneumonia for children in the intervention arm was 4.7 (95% CI 1.9 to 11.8; p<0.001) as compared with the control arm. The effect of the intervention remained the same after adjusting for each of the baseline variables (table 3). In addition, children who were boys, aged 2 months to 11 months, height for age z-scores less than −2, and not fully vaccinated for pneumococcal and H. influenzae type b were more likely to be diagnosed with severe pneumonia than their counterparts (table 3).
OR from the multilevel logistic regression model comparing the proportion of diagnosed severe pneumonia between the arms
We carried out effect modification analysis to see whether the effect of pulse oximeter in identifying severe pneumonia was modified by the type of health profession (bachelor degree vs diploma degree nurses). The impact of pulse oximeter is not modified by the health professionals’ medical background knowledge for diagnosing severe childhood pneumonia (p value=0.828). Therefore, we have removed the interaction term and present the model without interaction and present the effect of pulse oximetry adjusted for the baseline confounders.
Secondary outcomes
After examining the clustering effect, the proportion of children with severe pneumonia referred to the hospital was 116/148 (78.4%, 95% CI 67.6 to 89.2) in the intervention arm and 15/34 (44.1%, 95% CI 6.9 to 81.3) in the control arm, with p=0.496. Among these, 62/116 (53.4%) in the intervention arm and 11/15 (73.3%) in the control arm, reached the hospital and received the standard treatment.
Table 4 shows the total treatment failure, and treatment failure by specific causes, between the arms. The proportion of treatment failure at day 14 was 132/928 (14.2%, 95% CI 6.0 to 22.4) in the intervention arm and 93/876 (10.6%, 95% CI 5.2 to 16.1) in the control arm (p=0.622). There were two deaths in each of the intervention and control arms. Nine children from the intervention and three children from the control arms had persistent cough at day 14. The COR of treatment failure for children with oxygen saturation <90% was 3.3 (95% CI 1.87 to 5.80) as compared with children with oxygen saturation greater or equal to 90%.
Treatment failures by specific causes at day 14
Sensitivity analysis
The cut-off point we used in the trial proposal to define hypoxaemia was an issue raised by the Regional Committees for Medical Research Ethics, South East Norway that approved our study. The committee commented that the cut-off points of oxygen saturation under 90% to define hypoxaemia is too low as it would contradict the Norwegian guidelines. Therefore, oxygen saturation cut-off point <92% was used for the sensitivity analysis. If we use an oxygen saturation cut-off <92% to define hypoxaemia, the proportion of children with hypoxaemia would be 298/921 (32.4%, 95% CI 16.2 to 48.5). Moreover, the cluster adjusted proportion of severe pneumonia would be 304/928 (32.8%, 95% CI 18.2 to 47.3) in the intervention arm, and COR 13.3 (95% CI 5.0 to 35.3) as compared with control arm.
One of the health centres in intervention arm is found at high altitude (2993 m above sea level) and we did a sensitivity analysis adjusting oxygen saturation level at <87%.13 The proportion of children with severe pneumonia in intervention arm was 124/928 (13.4%; 95% CI 8.0 to 21.4) and 34/876 (3.9%, 95% CI 1.2 to 6.6) in the control arm. The COR of being diagnosed with severe pneumonia for children in the intervention arm was 4.2 (95% CI 1.8 to 9.5) as compared with the control arm.
As it was the first time that mid-level health workers in this rural part of Ethiopia used pulse oximeters, this could result non-valid pulse oximeter readings. Assuming that 85% of the readings would be valid, we randomly select 85% of cases with pulse oximeter measurements and did sensitivity analysis. The proportion of children with severe pneumonia in intervention arm was 126 of 789 children (15.9%; 95% CI 10.0 to 21.9) and 34/876 (3.9%, 95% CI 0.8 to 6.9) in the control arm. The adjusted OR of being diagnosed with severe pneumonia for children in the intervention arm was 5.3 (95% CI 1.9 to 14.4; p=0.001) as compared with the control arm.
Discussion
Introducing pulse oximetry into the WHO IMCI algorithm significantly increased the diagnosis of severe childhood pneumonia in health centres.
The IMCI algorithms are based on clinical symptoms, and do not involve any objective diagnostic test to identify children with severe pneumonia.17 Health workers often misclassify pneumonia from severe pneumonia cases because of difficulty with interpretation of danger signs.18 In our trial, we attempted to show how pulse oximetry assisted health workers in identifying severe pneumonia cases through detecting hypoxaemia. The combined use of pulse oximetry with the WHO IMCI algorithm achieves better performance than the IMCI algorithm alone in identifying children with hypoxaemia requiring oxygen therapy.7 19
Chest in-drawings and fast breathing are keys to enable health workers to identify and provide treatment for childhood pneumonia.20 However, IMCI-based respiratory rate and chest in-drawing increase the misclassification of pneumonia cases.21 In subgroup analysis for the intervention arm of 135 children with hypoxaemia, 56 of them did not have chest in-drawing. This means that, without pulse oximetry, 56/135 (42%) children would have been missed and inappropriately treated. Pulse oximetry identified 67% of children without chest in-drawing or danger signs.22
Children aged 2 months to 11 months, and partially immunised children, were more likely to be diagnosed with severe pneumonia (approximately twofold as compared with those fully immunised). Our findings are consistent with other results, in which partial immunisation constitutes a risk factor for childhood pneumonia.23 A finding from another study also demonstrated that older children were less likely to develop childhood pneumonia.24
There was no difference in severe pneumonia cases referred to a hospital between the arms. However, the total number of severe pneumonia cases referred to a hospital in the study area increased as compared with the number of cases referred to a hospital in the study area prior to the implementation of the intervention. In our previous survey of 66 severe pneumonia cases, only 18 (27%) were referred to a hospital.8 In other observational research, the utilisation of pulse oximetry improved the decision-making of health workers in referring children with severe pneumonia.19 22
There was also no difference in treatment failure between the trial arms. A large number of children from the intervention arm did not go to hospital. Limited access to transport was the main reason for low compliance with referral in the study settings.8 This implies that only providing pulse oximetry may be insufficient to improve treatment outcome, as both pulse oximetry and adequate management, including oxygen therapy, are critical.25 However, our study suggests that providing adequate pneumonia treatment at peripheral hospitals and health centres needs to be considered and should be future research area. Such strategies that decentralise treatment have, for example, been shown to reduce maternal mortality.26
One of the main strengths of this study is that it was based on randomly selected health centres, which are typical of rural communities in Ethiopia. Moreover, we measured oxygen saturation after checking the reliability of pulse oximetry in the study area. The intrarater and inter-rater reliability estimates ranged from good to excellent.27 In this trial, except for educational and wealth status of parents and vaccination status of children, the baseline characteristics of the study arms were balanced. To control for bias due to possible confounding factors, we used estimates adjusted for those potential confounders.
The study also possesses certain limitations that are worth noting. First, the number of children who attended the facilities was larger in the intervention arm than in the control group. This might be due to the following two reasons: (1) the base population in the intervention arm was large (63 285) as compared with the control base population (50 629). Accordingly, proportionally more children 2600/63 285 (4.1%) from the intervention arm attended the study facilities than the control arm 1755/50 629 (3.5%), (2) during the study period, approximately 1 million people were displaced due to inter-communal violence in the study area.28 From six refugee camps for the displaced people, four camps were found in the intervention areas.29 This might have drawn more children to seek medical care in the intervention centres.
Second, the trial also did not confirm the diagnosis of severe pneumonia using radiological examination. Hypoxaemia can also occur in diseases other than childhood pneumonia, including sepsis, meningitis and severe malaria.30 Our trial could include children with acute bronchitis or acute bronchiolitis and lobar pneumonia. We used the WHO IMCI algorithm, which labelled these diseases clinically as pneumonia, and our findings should be interpreted within this context.
Third, it is well documented that obtaining accurate saturations readings in sick children is challenging and that our documented pulse oximetry readings may not always be accurate for three main reasons: (1) the pulse oximeter may have given an inaccurate reading, for example, when there was motion artefacts, poor perfusion or irregular rhythms31; (2) the requirement to record an oxygen saturation that had been stable for 60 s may have forced health workers to document a reading, even when this was not always possible; (3) it was the first time that health workers at these institutions used pulse oximeter; therefore, health workers may document an inaccurate pulse oximeter reading. However, the sensitivity analysis based on 85% of anticipated valid pulse oximetry readings showed that our conclusion is similar.
The fourth limitation of the study was that there were missing values for few of the baseline variables. However, the missing values were less than 5% and there was no significant difference between the two trial groups. Another limitation was that we used oxygen saturation level <90%, which is high for children living at high altitude. This may increase the false positive cases of severe pneumonia. However, our sensitivity analysis shows that the result remains statistically significant after adjusting oxygen saturation level of <87% for children living at high altitude.13
Finally, the CI for the effect estimate of the primary outcome in the intervention arm is wide. It is well known that estimates accounting for clustering can result in wide CI. Therefore, we recommend future research to be conducted using a larger sample size.
Implications
Our results could be beneficially applied to health centres with mid-level health workers, where the management of childhood pneumonia is based on WHO IMCI algorithm. Therefore, the application of pulse oximeter to supplement the existing WHO IMCI algorithm to health centres in rural Ethiopia could assist health workers to find more cases of severe pneumonia.
Conclusion
The addition of pulse oximetry to the WHO IMCI algorithm significantly increased the number of diagnosed severe pneumonia cases in health centres, and could help to substantially reduce childhood mortality from hypoxaemia. Specific interventions that improve compliance with referrals and decentralise treatment to health centres for the management of severe childhood pneumonia are urgently needed.
Acknowledgments
We would like to thank the Norwegian Programme for Capacity Development in Higher Education and Research for Development (NORHED) for funding this study. We also sincerely acknowledge the contributions of the Gedeo Zone Health Department and district health offices in helping to successfully launch the implementation of this study. We are grateful for the health workers and health facilities where the study was conducted. We would also like to sincerely thank the study participants involved in this study.
References
Footnotes
Contributorship SHT conceptualised the idea, designed the study, wrote the protocol, coordinated the data collection, analysed and interpreted the data and drafted this manuscript. BL conceptualised the idea, guided the study design, wrote the protocol, analysed and interpreted the data, took part in the proposal writing and writing of this manuscript. EL provided methodological advice, supported the analysis and revised the draft paper. KAJ provided methodological advice and revised the draft paper. YG took part in the training of staff at the health centres, monitored data collection and revised the draft paper. All authors read and approved the submitted version of the manuscript.
Funding This work was supported by the Norwegian Programme for Capacity Development in Higher Education and Research for Development (NORHED).
Disclaimer The funder had no role in the design, data collection, analysis, interpretation, writing of the manuscript or decision to submit the paper for publication.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The study was approved by the institutional review board of the College of Medicine and Health Sciences at Hawassa University (ref: IRB/009//2017) and the Regional Committees for Medical Research Ethics, South East Norway (ref: 2017/2473/REK sør-øst). Children were included in the study after giving written informed consent by parents.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement The data for this trial are available at https://osf.io, https://osf.io/ahdtx/