Article Text

Download PDFPDF

Original research
Process evaluation of an implementation strategy to support uptake of a tuberculosis treatment adherence intervention to improve TB care and outcomes in Malawi
  1. Lisa M Puchalski Ritchie1,2,3,4,
  2. Esther C Kip5,
  3. Hayley Mundeva2,
  4. Monique van Lettow5,6,
  5. Austine Makwakwa7,
  6. Sharon E Straus1,2,
  7. Jemila S Hamid8,
  8. Merrick Zwarenstein9,
  9. Michael J Schull1,10,
  10. Adrienne K Chan1,10,
  11. Alexandra Martiniuk11,12,
  12. Vanessa van Schoor13
  1. 1Department of Medicine, University of Toronto, Toronto, Ontario, Canada
  2. 2Knowledge Translation Program, St Michael's Hospital Li Ka Shing Knowledge Institute, Toronto, Ontario, Canada
  3. 3Emergency Medicine, University Health Netowrk, Toronto, Ontario, Canada
  4. 4Institute of Health policy, management, and evaluation, university of toronto, toronto, ontario, canada
  5. 5Dignitas International, Zomba, Malawi
  6. 6University of Toronto Dalla Lana School of Public Health, Toronto, Ontario, Canada
  7. 7Malawi National TB control Program, Lilongwe, Malawi
  8. 8Department of Mathematics and Statistics, University of Ottawa, Ottawa, Ontario, Canada
  9. 9Family Medicine, Schulich School of Medicine and Dentistry Department of Family Medicine, London, Ontario, Canada
  10. 10Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
  11. 11The George Institute for Global Health, Newtown, New South Wales, Australia
  12. 12The University of Sydney, Sydney, New South Wales, Australia
  13. 13Dignitas International, Toronto, Ontario, Canada
  1. Correspondence to Dr Lisa M Puchalski Ritchie; lisa.puchalskiritchie{at}utoronto.ca

Abstract

Objective To assess implementation and to identify barriers and facilitators to implementation, sustainability and scalability of an implementation strategy to provide lay health workers (LHWs) with the knowledge, skills and tools needed to implement an intervention to support patient tuberculosis (TB) treatment adherence.

Design Mixed-methods design including a cluster randomised controlled trial and process evaluation informed by the RE-AIM framework.

Setting Forty-five health centres (HCs) in four districts in the south east zone of Malawi, who had an opportunity to receive cascade training.

Participants Forty-five peer-trainers (PTs), 23 patients and 20 LHWs.

Intervention Implementation strategy employing peer-led educational outreach, a clinical support tool and peer support network to implement a TB treatment adherence intervention.

Outcome measures Process data were collected from study initiation to the end-of-study PT meeting, and included: LHW and patient interviews, quarterly PT meeting notes, training logs and study team observations and meeting notes. Data sources were first analysed in isolation, followed by method, data source and analyst triangulation. Analyses were conducted independently by two study team members, and themes revised through discussion and involvement of additional study team members as needed.

Results Forty-one HCs (91%) trained at least one LHW. Of 256 LHWs eligible to participate at study start 152 (59%) completed training, with the proportion trained per HC ranging from 0% to 100% at the end of initial cascade training. Lack of training incentives was the primary barrier to implementation, with intrinsic motivation to improve knowledge and skills, and to improve patient care and outcomes the primary facilitators of participation.

Conclusion We identified important challenges to and potential facilitators of implementation, scalability and sustainability, of the TB treatment adherence intervention. Findings provide guidance to scale-up, and use of the implementation strategies employed, to address LHW training and supervision in other areas.

Trial registration number NCT02533089.

  • tuberculosis
  • public health
  • health services administration & management

Data availability statement

No data are available. Data from this study will not be shared in keeping with the guarantee during the consent process that transcript data would be accessible only to the study team.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Strengths and limitations of this study

  • This process evaluation provided an in-depth understanding of barriers and facilitators to implementation, scalability and sustainability of the tuberculosis adherence intervention.

  • The main limitation of this study is reliance on self-report data which is subject to a number of sources of bias including recall and social desirability bias.

  • Use of a mixed-methods approach with data collection through multiple methods and sources, allowed for triangulation across data sources, helping to reduce the impact of self-report bias on our findings.

  • As only lay health workers participated in the present study findings may not be generalisable to other healthcare worker cadres.

Background

Tuberculosis (TB) remains an important cause of morbidity and mortality globally, with 10 million new cases and 1.5 million deaths attributed to TB in 2018.1 Despite steady improvement since the global TB incidence peak in 2004,2 continued effort is needed to accelerate the annual rate of decline to achieve targets toward ending the global TB epidemic by 2030.3 Although one of many factors, incomplete treatment continues to contribute to the high global TB burden, with treatment success rates for 2017 of 85% of new cases and 56% of multi-drug resistant TB globally.4

Similar to the global pattern, TB incidence peaked in Malawi in 2003 at 403 cases per 100 000 population,5 and has gradually declined to 181/100 000 in 2018.5 TB treatment completion rates for this period have similarly improved, but vary substantially across districts (66%–90%) and remain below the 90% target set out in the 2021–2025 national TB and leprosy control strategic plan.6

In response to the global shortage of skilled health workers, currently estimated at 7.2 million and increasing with an estimated shortage of 12.9 million by 2035,7 outpatient TB care is commonly task-shifted to lay health workers (LHWs) in many low-income and middle-income countries (LMICs) where the shortage is greatest. While systematic reviews have found LHWs effective in improving TB treatment success rates compared with usual care, the effect size is generally small.8 9 Despite the recognised need to optimise the impacts of LHW programmes, a recent systematic review found relatively few studies examining strategies to improve health provider practices in LMICs in general and LHW practices in particular.10While adequate training and supervision are recognised as essential to optimising the effectiveness of LHW programmes, evidence of effectiveness for approaches to training and supportive supervision for LHW programmes are lacking,8 11 with training alone found to have small effects on LHW practice.10

With 28 nurses and two physicians per 100 000 population,12 Malawi is among the countries considered to be in crisis, with respect to human resources for health. LHWs, known as health surveillance assistants in Malawi, are a paid cadre of health workers totalling more than 9000 in 2017, which is insufficient to meet the ministry of health target of 1 LHW per 10 000 population.13 LHWs in Malawi provide a number of preventative and curative tasks, and are the principal providers of outpatient TB care.14 Given the crucial role of LHWs in providing TB care in Malawi and LMICs in general, effective and sustainable options to address LHW training, and supervision are needed.

In collaboration with local stakeholders, we conducted formative work to identify training and supervision needs among LHWs providing TB care in Malawi,15 and developed and pilot tested a TB treatment adherence intervention to address the identified knowledge and skill gap.16 17 The TB adherence intervention was designed to support TB treatment adherence by addressing two common barriers to adherence, identified through an extensive search of both the international and Malawi specific TB and general adherence literature.18 Specifically the intervention targeted lack of patient understanding of TB and its treatment, and poor patient-provider interaction, both known to negatively impact treatment adherence. The TB adherence intervention is reported elsewhere in detail17 19 but in short required LHWs to ask about adherence at each patient interaction, assess factors related to or risk factors for non-adherence, and to support adherence through provision of education and counselling appropriate for the patients stage of treatment or to seek additional support for patients as needed (such as referral for assessment of unexpected or severe side effects). To enable LHWs to implement the intervention, an implementation plan employing three strategies was developed, including: educational outreach to address knowledge gaps in LHWs understanding of TB (transmission, importance of adherence, consequences of non-adherence), and provide skills in patient counselling and adherence support; a clinical support tool to act as a clinical support to address adherence during patient interactions and providing a guide to adherence assessment and counselling: and a peer support network to allow for lessons learnt during training and supervision to be shared among peer trainers, to support peer trainers in their role.

The pilot trial, conducted in a single district employing educational outreach and a clinical support tool to support implementation, found the intervention to be feasible and acceptable to participants, and while not statistically significant given the small sample size, showed improvement in TB treatment completion rates.16 17 Given these findings and stakeholder interest in exploring potential scale-up, we refined the intervention and implementation strategy based on feedback and our implementation experience in the pilot study and evaluated the refined intervention using a mixed-methods design. Effectiveness of the intervention in improving TB treatment completion rates was evaluated in a cluster randomised controlled trial, reported in detail elsewhere,20 with findings of the process evaluation reported here.

Methods

Study aim

Our objective was to assess implementation and to identify barriers and facilitators to implementation, sustainability and scalability of an implementation strategy to provide LHWs with the knowledge, skills and tools needed to implement an intervention to support patient TB treatment adherence: to inform scale up and to assess the potential for this approach to be used to address training and supervision needs in other areas of care provided by LHWs.

Intervention

A detailed description of the development process and the original intervention has previously been published.15 16 Based on feedback and our implementation experience in the pilot study, the intervention was refined for the present study (see table 1 for detailed description of the intervention and implementation strategy). In brief, our strategy employed onsite peer-led educational outreach, a clinical support tool and peer support network to implement a TB treatment adherence intervention. The implementation strategy was designed to address LHW training and supervision needs to support implementation of an evidence-based approach to providing TB adherence support. The educational outreach component utilised a train-the-trainer approach, with TB focus LHWs trained as peer-trainers (PTs), who then provided cascade training (on-site, peer led, training) to LHWs providing TB care at intervention health centres (HCs). The clinical support tool was designed as a guide to patient education and counselling, and as a clinical support and guide to patient assessment and adherence counselling. In addition, to support development of a peer support network, a small phone stipend was provided quarterly to facilitate communication between PTs outside formal quarterly group meetings. Limited additional support was available to PTs by contacting the study team by phone, quarterly PT meetings and occasional filed visits from the study team and/or Dignitas International mentors during routine site visits to provide support and mentorship to front-line staff.

Table 1

Description of the intervention and implementation strategy

PTs were trained off-site over 1 week. Training was provided by LMPR, who also conducted the PT training in the pilot study. Training was provided in English, with support from a sociolinguistic level translator, and a second study team member in attendance to support training in one large district. PTs received certificates from the study team at completion of PT training, as well as training materials (training manual in Chichewa, stationary, training log book) and a supply of clinical support tools for their site, also in Chichewa. PTs were asked to provide cascade training onsite during regular work hours at their base HC, with all LHWs routinely providing TB care at the HC encouraged but not required to participate in training. LHWs who completed the cascade training were provided with certificates by the study team. Training stipends were not provided.

Study design

The study protocol for the complete mixed-methods study including both the pragmatic cluster randomised controlled trial and process evaluation components was previously published and is presented briefly here.19 Effectiveness of the intervention in improving TB treatment success rates was assessed within the context of a pragmatic cluster randomised trial in four districts in the south east zone of Malawi, with 51 HCs randomised to the intervention arm and is reported in detail elsewhere.20 The process evaluation employed a mixed-methods design informed by the Reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework,21 22 and our experience with the pilot study. In keeping with our pragmatic design, data sources for the process evaluation were selected to limit contact with participants in order to reduce the potential for process evaluation data collection to act as a booster to implementation. Thereby optimising our ability to assess intervention effectiveness as close to real world conditions as possible. Process evaluation data sources included: interviews with LHWs and patients at intervention sites, and a document review of training logs, quarterly PT meeting notes, mentor field visit reports, and study team meeting notes.

We defined reach as the number and proportion of LHWs who completed cascade training. Effectiveness included benefits to patients including impact on TB treatment completion rates and providers as a result of the intervention, challenges encountered, areas where further improvement was needed to achieve effectiveness goals, and any negative effects attributed to implementation of the intervention. Adoption was defined as the number and proportion of HCs with at least one trained LHW providing care in addition to the sites PT. Implementation included fidelity of the cascade training, provision of supportive supervision and use of the clinical support tool during patient care. Maintenance was defined as ongoing use of the intervention beyond the initial implementation period to the end of the 1-year trial period, including efforts to ensure new staff were appropriately trained and able to participate in implementation. Barriers and facilitators to implementation, scalability and sustainability, were defined to include both perceived and experienced barriers and facilitators. Suggestions for programme improvement included both efforts trailed by participants to improve implementation during the course of the study, as well as suggestions, based on participants experience during the study period.

Participants

The study was conducted in four districts in the South East zone of Malawi, with 51 of 103 HCs routinely providing TB care randomised to the intervention arm. All HCs randomised to the intervention arm that had an opportunity to participate in cascade training were eligible for inclusion in the process evaluation.

Interview participants included LHWs at intervention sites and patients/guardians for patients less than 18 years of age who began TB treatment on or after the trial start date (1 October 2016) and who were followed at a participating HC. Two to four participants from each group (LHWs and patients) were selected in each data collection period from each district and a maximum of two from any one HC.

LHWs were selected for interviews using mixed purposeful sampling to represent the range of LHW (age, gender, years of experience) and HC characteristics (rural/urban), with three LHWs chosen to be interviewed at both study onset and conclusion. Convenience sampling was used to select patients/guardians for interviews. Patient/guardians were selected to represent the range of characteristics in terms of age, gender, and TB characteristics (new/recurrent, pulmonary/non-pulmonary).

Informed written consent was obtained from all interview participants. Consent was obtained from guardians and assent obtained for children under 18 years of age.

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting or dissemination plans of our research.

Data collection

Process evaluation data were collected throughout the implementation period beginning with PT training which took place between 9 May and 3 June 2016, through to the final end of study quarterly PT meetings in October of 2017. See figure 1 for timing of data collection for each data source.

Figure 1

Process evaluation data sources and Timeline. LHW, lay health worker.

Quarterly meetings

PTs were brought together at the end of the cascade training period and then quarterly for the remainder of the implementation period. Meetings focused on sharing ideas and experiences, providing updates on the progress of training and implementation, and posing questions to the study team. Meetings were held in English with in-line translation provided as needed by the research assistant (RA) and study coordinator (SC). Notes were taken independently by two study team members. An initial meeting report was compiled by the RA from the hand written notes, and was circulated to the other study team members for review and revised as necessary.

Quarterly meetings were also held with members of the health management teams in study districts, typically immediately following PT meetings and regular contact maintained with AM who is based in the National TB control programme. Meetings focused on providing updates to leadership on implementation progress and to receive updates on any TB policy and/or programming changes and/or system challenges with potential to impact implementation. Brief notes were taken and discussed by the team following meetings to ensure accuracy.

Training logs, study team and mentorship data

PTs were provided with a log book and asked to use the book to document the details of and their experiences during cascade training and ongoing implementation. Items of interest to be documented in the log book included: number of LHWs trained; changes/additions made to cascade training; challenges to training or implementation; questions or concerns for discussion with the study team or PT group. Log books were reviewed by the study team at quarterly meetings and verbal reports provided when logs books were not available.

Notes were taken during regular study team meetings, to document study progress, challenges encountered and to document feedback from mentors, input from district and national ministry of health staff, and experiences of the study team during field visits during collection of interview data. One mentor prepared and submitted a more formal report, however, this mentor was based in one of the two districts that Dignitas International was withdrawn from and therefore only a first quarter report was provided.

Interviews

Interviews were conducted with patients and LHWs at intervention sites at two time points during the implementation period. Interviews were conducted at a time convenient to participants, in a private location at or near the participants’ HC. The first round of interviews was conducted in the first quarter of implementation (November and December 2016) and the second round conducted in the final quarter of the implementation period (August and September 2017). LHW interviews were conducted by a trained RA and patient interviews conducted by the SC. Both the RAs and SC were native Malawians, and fluent in English and Chichewa. Interviews were conducted face to face, in Chichewa, using a semistructured interview guide. LHW interviews began with collection of basic demographic data and details of their participation in the cascade training. This was followed by open ended questions asking about their experience with the training, use of the clinical support tool, supervision received and suggestions for improvement. Patient interviews similarly started with collection of demographic data, including current and past TB diagnosis if any. Patient interviews also began with open-ended questions with probing as necessary to ensure topics of interest were addressed. Topics of interest in patient interviews included: understanding of TB and its treatment, experience in receiving TB care at their base HC including experience with the clinical support tool, and suggestions for how provision of TB care could be improved.

Interviews were digitally audio recorded, transcribed verbatim and translated to English. To ensure accuracy and conceptual equivalence all interviews were verified by the SC, who operates at the level of a socio-linguistic translator.23

Analysis

Document review

Meeting and study team notes, including reports from mentors, quarterly PT meetings, and meetings with district and national Ministry of Health staff, were reviewed independently and discussed (ECK and LMPR) on a regular basis and notes made of ongoing and/or emerging themes throughout the implementation period.

Interviews

LHW interviews were analysed using directed content analysis,24 with interviews as the unit of analysis. NVivo V.10 (QSR International, Southport, UK) was used to organise and code the data. An initial coding framework was developed based on findings from the pilot study. Analysis occurred in two rounds. First, two study team members (ECK and HM) read and coded the transcripts independently. The coding framework was then revised through discussion and consensus, and input from a third study team member (LMPR) as needed. The revised coding framework was then applied independently by the same two study team members with discrepancies again resolved through consensus. Themes were sought across individuals with consideration of gender, age, years of experience providing TB care, district, and time of interview (first and last quarter of 1-year implementation period).

Patient interviews were analysed using conventional content analysis,24 again with interviews as the unit of analysis and NVivo used to organise and code the data. Transcripts were coded independently by two study team members (ECK and HM). Based on this initial round of independent coding, an initial coding framework was developed by the two coders through discussion, with involvement of a third study team member (LMPR) as needed. The initial coding framework was then applied independently by the same two study team members, in two further rounds with minor refinement of the framework between rounds.

Triangulation

Methods, data source and analyst triangulation25 were employed with interviews, quarterly meeting and study team meeting notes. Convergence and divergence in themes and subthemes was sought first across data collection methods and then across analysts. Findings from all sources were considered together to provide a comprehensive assessment of implementation, including potential inaccuracies in self-report data contributing to assessment of RE-AIM dimensions, and a comprehensive understanding of barriers and facilitators to implementation, sustainability and scalability.

Results

Characteristics of participants

Forty-five of 51 HCs randomised received the intervention. Six HCs did not have an opportunity to receive cascade training and were excluded from the process evaluation. Reasons for HCs not receiving cascade training included: four TB focus LHWs who were on leave or at other trainings did not attend PT training, one PT died immediately following PT training and before beginning cascade training at his base HC and one participant reported at the end of training that their HC no longer provided TB care. PT attendance at quarterly meetings varied from 80% to 89%. Seven PTs did not attend the final meeting and could not be reached to confirm final numbers of LHWs trained and remaining at the HC.

Twenty-three patients participated in interviews. Patients ranged from 11 to 59 years of age, 13 (57%) were female, the majority of patients had been diagnosed with a first episode of pulmonary TB, with only 2 (9%) diagnosed with non-pulmonary TB and 5 (22%) diagnosed with a second TB episode.

Twenty LHWs participated in interviews, with three LHWs interviewed in both the first and last quarter of implementation. LHWs ranged from 30 to 57 years of age, from 4 to 20 years of experience working as a LHW and from 1 to 20 years providing TB care. Ten (50%) of LHWs interviewed were female.

Implementation process outcomes

Example quotes for RE-AIM categories where applicable are provided in table 2.

Table 2

Example quotes for RE-AIM, barrier and facilitator results categories

Reach

Of the 256 LHWs eligible for training at the start of cascade training, 152 (59%) LHWs completed the training by the end of the initial training period. The proportion of LHWs receiving cascade training varied across sites, ranging from 0% to 100%. An additional 17 LHWs, who had initially declined to participate or were transferred into implementation sites after the initial training period, completed cascade training by the end of the study, for a total of 169.

Effectiveness

There was no significant effect of the intervention on TB treatment success, adjusted OR 1.35 (95% CI 0.93 to 1.98), with high variation in implementation quality a potential contributing factor.20

LHW interview participants and quarterly PT meeting notes, revealed a variety of benefits of the intervention to LHWs including: increased knowledge, improved skills in patient–provider interactions and counselling, better collaboration among providers, and improved patient care. While some PTs reported use of the phone stipend to discuss challenges or problem solve with their peers during initial cascade training and reported this support as essential to their success, few reported continued use of the peer support network beyond the first quarter of implementation. While some frustration with challenges in recruiting eligible LHWs to participate in training was reported by some PTs, no negative outcomes as a result of implementation were reported and many PTs indicated they would be interested in continuing in their role as PT.

Patient interview results were somewhat less consistent, with mixed effects in some areas. All patients reported and valued receiving supportive care from the LHW providers. In addition, the majority of participants noted LHWs to probe for and provide an opportunity to discuss and address challenges faced during the course of treatment, with reports of LHWs failing to assess challenges encountered predominantly occurring in the initial implementation period. The majority of patients reported LHWs to be their primary source of TB information, and showed a good understanding of TB and its treatment. Patients found the clinical support tool as part of their education and counselling beneficial, noting it improved their understanding of TB treatment and the importance of adherence to successful treatment. Understanding of TB transmission however was mixed, and did not seem to improve substantially over time. In addition, patient understanding of their TB type, was mixed, with approximately half of patients interviewed reporting they were not told what type of TB they have. This finding also did not change substantially overtime. No negative outcomes were reported by patients with respect to their current TB treatment.

Adoption

Of the 45 HCs that had an opportunity to receive cascade training, 41 (91%) reported at least 1 LHW had completed training. No clear pattern was evident for HC setting (rural vs urban) or HC funding type (Ministry of Health or non-ministry funded) with respect to willingness of LHWs to participate in implementation.

Implementation

Tailoring of cascade training was permitted at the discretion of the PT provided all content was covered and opportunities for practice and discussion were available. Similarly, approaches to supportive supervision were discussed and practiced during PT training, but PTs were free to select the techniques most suitable to their style and team needs. The initial cascade training period was extended by 2–3 weeks to accommodate staff absences.

While all PTs reported providing complete cascade training, both PTs and LHWs reported variability in number and duration of sessions including: providing fewer than the recommended eight sessions by extending the length of individual sessions, providing replacement sessions for staff as needed and training new staff transferred into the HC after the initial training period. However, interviews with LHWs revealed a few instances where training was incomplete or not offered despite PT reports. Similarly, although provision of supportive supervision was variable with respect to amount and approach to supervision provided, particularly beyond the initial period of cascade training, LHWs generally reported supportive supervision to be improved compared with preimplementation levels and the availability of local support and mentorship from their PT valued. Use of the clinical support tool in patient education and counselling changed over the course of the 1-year trial period, starting with relatively low levels of use with 7/11 (64%) of patients interviewed reporting never seeing the tool and 3/11 (27%) reporting see the tool 2 or more times in the first quarter, to high levels use with 1/12 (8%) of patients reporting never seeing the tool and 11/12 (92%) reporting seeing the tool on 2 or more TB clinic visits by the final quarter of the study.

Maintenance

All sites who participated in cascade training reported ongoing use of their training and clinical support tool in provision of care at 1-year postimplementation. As a result of transfers to other HCs, two HCs no longer had any trained LHWs on site, however PTs at these HCs remained onsite and were engaged in direct provision of TB care. In addition, several PTs reported plans to provide cascade training to LHWs who had previously declined to participate and to new staff as needed to support continued implementation, with one site reporting completion of 4 of 8 modules at the time of the end of study meeting.

Barriers to and facilitators of implementation, sustainability and scalability

Example quotes for barriers and facilitators are provided in table 2.

Lack of incentives was the primary barrier identified, and was the most common reason given by LHWs declining to participate. In addition although not specifically stated, it appeared that lack of stipends may have played a role in the few reports of PTs refusing to provide training or providing a substantially reduced version of the training only. A second barrier noted was that the PT and LHWs were busy, however explanations suggested that in some cases, lack of incentives may have played a role. As one participant noted, ‘they say they are busy but to me it is really just about incentives’. A third barrier noted was that more than one PT was needed due to the demands of the TB focus person role. Additionally, although not described as such PT attitude, lack of interest and perhaps lack of confidence, appeared to play a role in some cases, where interview participants reported PTs not willing to provide training or providing only a condensed version of training, despite LHWs requests to participate.

Facilitators commonly noted as suggestions to address noted barriers, included to provide incentives, and train more than one PT per site. Intrinsic motivation was a commonly noted facilitator, with a desire to improve knowledge and skills, noted to over-ride the desire for incentives among some participants. A desire to improve patient care and outcomes were also commonly reported reasons for participation. Although PTs were free to contact the study team by phone with questions or concerns as needed, this was rarely exercised with PTs typically waiting for quarterly meetings to raise questions to the study team. Despite this, participants felt that regular monitoring and check-ins from the study team to provide support and motivation by providing opportunities for questions and discussion, particularly early in implementation, would facilitate implementation.

Discussion

The TB treatment adherence intervention was designed based on formative work to address an identified gap in knowledge and skills among LHWs providing TB care in Malawi, and employed evidence-based implementation strategies to support its use. This process evaluation highlighted important barriers to and facilitators of implementation, scalability and sustainability of the intervention. As well as challenges not previously identified in the pilot evaluation of the intervention, which in contrast to the present study, achieved high levels of implementation and found a non-significant improvement in TB treatment completion rates.16 17

Although many sites achieved high levels of implementation, substantial variability was found both within and across districts, particularly with respect to reach, which ranged from 0% to 100%. This finding is in contrast to those of the pilot study16 17 where only a single LHW initially declined to participate and later requested and completed make up training sessions to catch up to peers and to allow their participation. While several factors may have contributed, lack of financial incentives was the principal reason given by LHWs who chose not to participate. Although standard practice at the time, incentives were also not provided in the pilot study, where, while suggested as a potential facilitator, lack of incentives did not deter participation. At the time of the current study, a policy change no longer permitting training stipends had been recently implemented, with reports of training refusals occurring among several healthcare workers cadres in the study districts as a result.

A second noted barrier to implementation noted by LHWs was being too ‘busy’, however, comments associated with this reason suggest that lack of incentives was related to this response in some cases. Additionally, recognition that implementation was occurring in the context of a formal study, appeared to further exacerbate the incentive issues, with some participants noting ‘incentives’ common practice in studies conducted in their area. Given this it is possible that lack of incentives may be less of a deterrent to participation once the policy has become standard and under routine programmatic conditions.

Although impacts are somewhat mixed, incentives are a frequently noted barrier to and facilitator of implementation in LMICs. Systematic reviews have found modest to moderate effects of financial incentives on healthcare worker practice,10 and monetary incentives linked to motivation and performance, and attrition in community health worker programmes,11 with several studies finding lack of financial incentives, an important barrier to implementation.26–28

Two systematic reviews, however, have found the impact of incentives less clear, noting the potential for important negative effects. While both reviews found that both financial and non-financial incentives can enhance performance, performance based financial incentives could lead to neglect of unincentivised tasks29 and noted concerns of potential negative impacts from incentives particularly to unpaid volunteer LHWs that may undermine the high moral status placed on volunteer LHWs in some settings.30

PTs work load was the third common barrier to implementation noted by LHW participants, who suggested that more than one PT be trained per site to ensure one PT could always be available to provide support and supervision. Additionally, PT lack of interest or confidence was also noted as an important barrier by some LHW participants. Lack of confidence was also noted initially by some PTs in our pilot study, but resolved with encouragement from the study team and their peers, with no notable impact on implementation. Similar to the stipend issue, it is unclear why lack of interest or confidence was more persistent over time among PTs in the current study. While it is possible this barrier may be of less importance under routine programmatic conditions, based on findings in our companion study31 addition of leadership training for PTs may be considered to support PTs and improve their effectiveness in this role.

Several studies have found workload to be an important barrier to implementation in both community health worker and LHW programmes. Work overload was noted to negatively impact participation of community health workers in implementation of a micronutrient dissemination programme.26 In a systematic review of determinants of performance in Malaria prevention and control programmes, Chipukuma et al32 found poor performance among LHWs as a result of large population coverage and multiple tasks. They noted however that this effect could be mitigated by appropriate training, supervision and adequate resources. Finally, Glenton et al’s30 systematic review of barriers and facilitators to implementation of LHW programmes in maternal and child health, found unrealistic LHW workloads including large coverage areas to negatively impact LHW performance.

Among LHWs who elected to participate, the primary reasons given were intrinsic motivations, including wanting to improve personal knowledge and skills, and to provide better care and through this to improve patient outcomes. Intrinsic motivation and positive effects of participation in the intervention was also noted in the pilot study17 31 and companion study,31 and suggest that emphasis on these outcomes when introducing the programme may help to motivate participation.

Two studies have previously identified intrinsic motivations as an important facilitator of implementation in LHW programmes. Intrinsic motivators including altruism, social recognition, knowledge gain and career development, were identified as facilitators to maternal and child health LHW programmes, in Glenton et al30 systematic review. Grant et al33 found a team based goals and incentives programme reinforcing intrinsic motivation to improve teamwork, motivation and performance among health teams which included LHWs. Future work to evaluate this and other approaches to optimising intrinsic motivators are needed, given the prohibitive cost of the stipend approach to facilitating implementation, particularly in large LHW programmes.

Based on observations in the pilot study where PTs maintained contact by phone to provide support to each other outside the formal quarterly meetings, we provided a phone credit stipend to PTs trained together in the current study to support development of a peer support network. In contrast to the pilot study, PT contact appeared to be limited outside the formal quarterly meetings, particularly outside the initial period of cascade. However, a few PTs noted support from peers during this initial period as instrumental to their success. The reason for this difference is unclear, and given the mixed reports of value placed on availability of peer support, warrants further evaluation. Similar to our experience in the pilot study, Sodhi et al27 found peer support important to implementation of an intervention for midlevel health workers in Malawi employing peer-led educational outreach. Given these findings, further research to assess options for and impacts of peer networks to support implementation efforts in Malawi and LHW programmes in general is needed.

While the importance of supportive supervision to the success of LHW programmes is widely recognised11 32 as noted by Kot et al29 evidence to direct selection of approaches for and implementation of supportive supervision are lacking. Additionally challenges to provision of adequate supportive supervision have been noted due to staff shortages and resource issues leading to logistical challenges.34 Based on its recognised importance we elected to explore the impact of including training in supportive supervision as a component of PT training, as a feasible and sustainable option to providing supportive supervision at the local level. However, the impact of this addition to the PT training is unclear. Many LHW participants in the process evaluation interviews, as well as, participants in a companion study on PT leadership style,31 noted receiving and appreciating supportive supervision from their PTs. This suggests that support at this level is feasible and well received, however negative reports with respect to PT supervision suggest that more work is needed to optimise supportive supervision provided by PTs to support implementation. PTs also suggested that supportive supervision through regular monitoring and ‘check in’s’ from the study team would be appreciated as opportunities to bolster motivation and address questions that arise during implementation. As a potential facilitator inclusion of training in and provision of supportive supervision from master trainers is an important area for future implementation and evaluation.

Limitations

The primary limitation of this study is reliance on PT self-report data, which is subject to a number of sources of bias including recall bias and social desirability bias. Indeed, at least one PT initially gave a false report of the number of LHWs they had trained, only revealing the true number at the final quarterly meeting, which were much less that initially reported. Additionally several PTs did not attend the final meeting and could not be reached to confirm the final numbers of trainees. As a result final numbers of LHWs trained and remaining at intervention sites may be under and over estimates, respectively, although the difference is likely to be small as relatively few were trained beyond the initial training period and few transferred or left their posts over the 1-year trial period. Use of a mixed-methods approach with data collection through multiple methods and sources, helped to mitigate the effects of self-report bias to some degree by creating opportunities from more neutral sources to provide additional information, allowing for analysis of concordance and discordance across data sources. In addition, as the process evaluation data was predominantly qualitative in nature, scores could not be calculated for RE-AIM dimensions. Although participating sites were reflective of the distribution of HCs with respect to rural/urban location and ministry/non-ministry funding, due to the relatively small number of urban and non-ministry funded sites, it was not possible to assess for factors impacting implementation that may be unique to these sites. As only LHWs participated in the current study, findings may not be generalisable to other healthcare worker cadres.

Conclusions

This process evaluation identified important challenges to and potential facilitators of implementation, scalability and sustainability, of the TB treatment adherence intervention and use of the implementation strategies employed that were not previously identified in the pilot study. The primary barrier identified was lack of stipends, with intrinsic motivation and increased support for and from PTs important potential facilitators. As provision of training stipends is not feasible for wide spread implementation given the large LHW workforce in Malawi. Solutions to addressing identified barriers are essential to scale-up of the current intervention and to use of the implementation strategies and in particular, the training approach employed, to address LHW training needs in other areas. Suggestions identified in the current study include a focus on intrinsic motivators and ensuring sufficient number and adequate training of PTs both in programme content and approaches to supportive supervision. These may be insufficient alone to address the lack of incentives in the current climate. Required rather than optional participation under regular programmatic conditions may further support implementation and it is hoped that resistance will reduce as the non-incentive policy becomes standard.

Given the lack of a significant effect of the intervention on patient TB treatment outcomes and substantial challenges to implementation encountered in this large scale implementation study, further research to assess effectiveness of the intervention in the context of high levels of implementation quality and fidelity is needed, before wide scale implementation can be considered. In addition, further work to develop implementation strategies to address barriers to implementation are needed before wider scale implementation of the TB adherence intervention or use of the strategies employed to support other implementation activities can be considered.

Data availability statement

No data are available. Data from this study will not be shared in keeping with the guarantee during the consent process that transcript data would be accessible only to the study team.

Ethics statements

Ethics approval

This study has been approved by the St. Michael's Hospital Research Ethics Board (protocol #15-282) and the Malawi National Health Sciences Research Committee (protocol # 15/9/1479). Written consent was obtained from interview participants.

Acknowledgments

Jan Barnsley for her consultation into the qualitative approach used in the pilot and current studies. We also wish to thank the LHW participants in the study.

References

Footnotes

  • Contributors All authors contributed to the study design (LMPR, ECK, AM, MvL, AM, SES, JSH, MZ, MS, AC, ALCM and VvS). LMPR led all aspects of the study and was responsible for the first draft of the manuscript. LMPR, ECK, AM, HK and SES contributed to the revisions of the educational outreach programme, training manual and clinical support tool. ECK conducted patient interviews, and verified all transcripts. ECK, HM and LMPR conducted the analysis. All authors (LMPR, ECK, AM, MvL, AM, SES, JSH, MZ, MS, ALCC, AM and VvS) participated in critical revisions of the manuscript, read and approved the final manuscript.

  • Funding This work was supported by the Canadian Institutes of Health Research KAL-139700. SES is funded by a Tier 1 Canada Research Chair and the Squires Chalmers Chair in Medicine. A Martiniuk was funded by a National Health and Medical Research Foundation (NHMRC) Translating Research into Practice (TRIP) Fellowship from 2016 to 2019.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.