Objectives The aim of this review is to map out the use of process evaluation (PE) in complex interventions that address non-communicable diseases (NCDs) and neglected tropical diseases (NTDs) to identify gaps in the design and conduct, as well as strengths, limitations and implications, of this type of research in low- and middle-income countries (LMICs).
Design Scoping review of PE studies of complex interventions implemented in LMICs. Six databases were searched focused on studies published since 2008.
Data sources Embase, PubMed, EbscoHost, Web of Science (WOS), Virtual Health Library (VHL) Regional Portal and Global Index Medicus: Regional Indexes AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), WPRIM (WPRO) Global Index Regional Indexes, MEDLINE, SciELO.
Eligibility criteria Studies conducted in LMICs on PEs of randomised controlled trials (RCTs) and non-RCTs published between January 2008 and January 2020. Other criteria were studies of interventions for people at risk or having physical and mental NCDs, and/or NTDs, and/or their healthcare providers and/or others related to achieve better health for these two disease groups. Studies were excluded if they were not reported in English or Spanish or Portuguese or French, not peer-reviewed articles, not empirical research and not human research.
Data extraction and synthesis Data extracted to be evaluated were: available evidence in the utilisation of PE in the areas of NCDs and NTDs, including frameworks and theories used; methods applied to conduct PEs; and in a subsample, the barriers and facilitators to implement complex interventions identified through the PE. Variables were extracted and categorised. The information was synthesised through quantitative analysis by reporting frequencies and percentages. Qualitative analysis was also performed to understand facilitators and barriers presented in these studies. The implications for PEs, and how the information from the PE was used by researchers or other stakeholders were also assessed in this approach.
Results 303 studies were identified, 79% were for NCDs, 12% used the label ‘PE’, 27% described a theory or framework for the PE, and 42% used mixed methods to analyse their findings. Acceptability, barriers and facilitators to implement the interventions, experiences and perceptions, and feasibility were the outcomes most frequently evaluated as part of the PEs. Barriers and facilitators themes identified were contextual factors, health system factors, human resources, attitudes and policy factors.
Conclusions PEs in NCDs and NTDs are used in LMICs with a wide variety of methods. This review identified many PEs that were not labelled by the authors as such, as well as a limited application of PE-related theories and frameworks, and heterogeneous reporting of this type of study.
- diabetes & endocrinology
- coronary heart disease
- infectious diseases
- public health
Data availability statement
Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
Six databases focused on process evaluation of complex interventions implemented in low- and middle-income countries in the area of non-communicable diseases and neglected tropical diseases were searched.
The search strategy was designed with an experienced librarian and it was built based on the systematic review of Liu et al.
All languages were not included due to limited human resources but we expect that the four major selected languages covered the majority of relevant literature.
Data extraction was not duplicated because of the high number of studies; this was mitigated by previous specific training of the two reviewers and frequent meetings to clear doubts.
Only a subset (n=50) of studies were included in the subsample analysis of barriers and facilitators, which may have limited the number of identified barriers or facilitators.
In low- and middle-income countries (LMICs), the double burden of chronic long-term non-communicable (NCDs) and neglected tropical diseases (NTDs) represents a major challenge to vulnerable populations and their health systems.1 2 NCDs kill 41 million people each year and one-third of them are between 30 and 69 years old.3 This group of diseases is having an increasing impact in LMICs with variability in terms of burden and mortality within and between these countries.4–6 NTDs affect more than 1 billion people worldwide and they are the most vulnerable populations with poor access to healthcare services.7 8 NCDs and NTDs impact individuals,9 10 communities and health system responses.4 11 12 Complex interventions are needed to address these complex and multifaceted health challenges13 14 in order to tackle global health problems,15 16 and therefore achieve the Sustainable Development Goal 3 to ‘Ensure healthy lives and promote well-being for all at all ages’.17 However, interventions with multiple components face diverse challenges when these interventions are implemented on the ground,18 especially in LMICs, where the context and health system barriers add additional complexity.19 Given this, process evaluation (PE) is a type of study that can be used to understand how complex interventions are implemented, clarify causal mechanisms and identify contextual factors associated with variation in outcomes rather than simply measuring effectiveness outcomes.20 PEs are also useful to close translation gaps and reduce research waste,21 because they can inform future interventions, policy and practice.20 22
Two recently published systematic reviews explored the use of PEs in the literature. One study focused on primary healthcare interventions addressing NCDs in randomised controlled trials (RCTs) worldwide.23 24 Another systematic review was conducted on PE studies of knowledge translation interventions without restriction of health conditions.25 These reviews found that PEs used both quantitative and qualitative methods of data collection, and that they can be conducted at different moments during a project such as feasibility testing phases, alongside evaluations of effectiveness or alongside scale-up studies. These reviews were restricted to publications in English language only, and found limited LMIC representation. Liu et al23 24 in their study that included PEs conducted in primary care settings found only three RCTs for NCDs in LMICs (one study in India, Malaysia, and Zambia, respectively). Scott et al25 did not report the regions of the implementation of the studies included in their review. Neither of these reviews explored the main barriers and facilitators of implementing complex interventions on NCDs and NTDs in LMICs.
This scoping review aims to identify gaps in the design and conduct of PE of complex interventions for NCDs and NTDs in LMICs. These findings expand the knowledge of previous systematic reviews by focusing on LMICs, where barriers to implement complex interventions are frequent and challenging. The objectives of this scoping review are: (1) determining the proportion of studies labelled as PE and identifying other terms used instead of PE in the literature; (2) identifying theories and frameworks used in PEs of complex interventions for NCDs and NTDs in LMICs; (3) examining the design and conduct of PE of complex interventions for NCDs and NTDs in LMICs; and (4) summarising barriers and facilitators identified in a subsample of studies of PEs of complex interventions for NCDs and NTDs in LMICs.
The full protocol for this scoping review was published elsewhere.26 Results are reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews.27
Studies on PEs of RCTs and non-RCTs (feasibility studies, observational studies, quasi-experimental studies) were included. Given that findings in previous systematic reviews23–25 showed that PEs are often not explicitly labelled as such, we included studies with data that: tried to understand how complex interventions were implemented in the field; measured PE outcomes; clarified causal mechanisms; and/or identified contextual factors beyond measuring effectiveness outcomes. Studies conducted in LMICs according to the World Bank definition in 201928 and published between January 2008 and January 2020 were included. This date was chosen as it was when the UK’s Medical Research Council (MRC) updated its guidance for developing and evaluating complex interventions.29 ,30 If a study was conducted in high-income countries and LMICs, only the data from the LMICs were included.
Studies considered for this scoping review had interventions targeted at people at risk or having physical and mental NCDs (type 2 diabetes mellitus, type 1 diabetes mellitus, cardiovascular disease, depression, chronic obstructive pulmonary disease and chronic kidney disease),31 and/or NTDs (online supplemental material 1),32 and/or healthcare providers (physicians, nurses, technicians) and/or others related to achieve better health (reforms in universal coverage, public policy, service delivery and leadership) for these two disease groups.
The types of interventions included were complex interventions according to the MRC guideline that considered ‘intervention with several interacting components’ but also could include other dimensions of complexity such as: ‘number and difficulty of behaviours required by those delivering or receiving the intervention’; ‘number of groups or organisational levels targeted by the intervention’; and ‘degree of flexibility or tailoring of the intervention permitted’.29 30 In addition, we included studies implemented in diverse settings such as community, health system or at policy level. The comparators were treatment as usual, active control or placebo control, as well as absent control condition (eg, pre/post-evaluations). In addition, studies with multiple active interventions were included.
Studies were excluded if they were not reported in English or Spanish or Portuguese or French, not peer-reviewed articles, not empirical research and not human research.
Embase, PubMed, EbscoHost, Web of Science (WOS), Virtual Health Library (VHL) Regional Portal and Global Index Medicus: Regional Indexes AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), WPRIM (WPRO) Global Index Regional Indexes, MEDLINE, SciELO.
A different search strategy (online supplemental material 2) was designed for each database and with an experienced librarian.
Covidence (Veritas Health Innovation, Melbourne, Australia)33 was used in the screening process. Titles and abstracts were screened after the importation of citations from the databases, and manuscripts were selected to proceed to data extraction.
Five reviewers (ML-P, HL, MO, XY, AM) in pairs considered the potential eligibility of all abstracts and titles identified from executing the search strategy. Disagreements were solved by a third reviewer. Agreement on the screening selection among reviewers was between 90% and 92%. Also, teams of two independent and blind reviewers evaluated full texts of all potential eligible studies. First, the reviewers calibrated their judgements using a smaller set of articles to ensure the same understanding of inclusion and exclusion criteria. Subsequently, they reviewed the complete set of articles. Any disagreements were resolved by group discussion or arbitration until reaching consensus. Agreement on the screening selection among reviewers was between 70% and 81%. Additionally, we included studies: (1) if the search strategy identified protocols with a published PE article and (2) if a PE study was identified during the full-text screening through further reference checking. Also, these potential eligible articles were reviewed by duplicated too.
Data collection process and data items
Data extraction included:
Full description of the study and complex intervention.
Details about the PE such as clear labelling as a PE, purpose, prespecified protocol, stage when the PE was applied, specific framework and/or theory of the PE18 20 34–38 and how the framework and/or theory of the PE was used (informed by, applied theory or tested theory),39 methods used, leading team of the evaluation and type of analysis (quantitative, qualitative, mixed or multimethod), as well as PE outcomes defined by the literature that were measured (online supplemental material 3).40
Lessons learnt about the PE: barriers and facilitators for implementation of complex interventions and implications for PE in LMICs (how the information of the PE was used). Data extraction on lessons learnt about the PE was literally extracted. The tools and detailed information for the data collection can be found in the protocol.26
Reporting of results
Variables were extracted and categorised (eg, data related to the methods of the PE and details of the intervention). The information was synthesised through quantitative analysis by reporting frequencies and percentages.
Qualitative analysis was also performed to understand facilitators and barriers to implement the complex interventions in the field as well as the implications for PEs in LMICs (how the information from the PE was used by researchers or other stakeholders). Fifty studies were randomly chosen from the total sample of included articles for the qualitative analysis using the web platform Research Randomizer (https://www.randomizer.org) to guarantee saturation. Based on the information extracted from these studies, codes were generated and used to synthesise and analyse the data.
Results are presented in the following four sections:
General characteristics of the interventions.
Available evidence in PE in the areas of NCDs and NTDs, including frameworks and theories used.
Methods applied to conduct PE.
Findings from the PEs related to barriers and facilitators of implementation of the complex interventions, as well as findings about implications of the PE.
Patient and public involvement
There was no patient nor public involvement in this study.
Our search strategy identified 11 297 articles, 9977 were screened after removing 1320 duplicates, and 285 studies were included. The reasons for exclusions are available in figure 1 and the main reason was that they were not PE studies. Additional 18 articles were included, 11 studies that the search strategy identified as protocols but with published PE results and 7 articles which were found during the full-text screening through further reference checking (figure 1). Thus, a total of 303 studies were included in this review (online supplemental material 4) and 18 of them were articles published in Spanish or Portuguese.
General characteristics of the interventions
Disease and region
Of the 303 studies, 239 of 303 (79%) focused on NCDs, 62 of 303 (20%) on NTDs and 2 of 303 (1%) included both groups of diseases or had a health system perspective. Among the most common NCDs, studies focusing on cardiovascular diseases were 86 of 239 (36%) and type 2 diabetes with 50 of 239 (21%) studies. Regarding NTDs, dengue was the most common condition with 19 of 62 (31%) studies.
In regard to the geographical regions where the studies were conducted, 83 of 303 (27%) were from sub-Saharan Africa, 65 of 303 (21.5%) from Latin America and 16 of 303 (5%) from countries in the Middle East and North Africa. More than 50% of the studies were conducted in upper middle-income countries (table 1).
Study design and context
Seventy-five studies (75 of 303, 25%) were RCTs or cluster RCTs (cRCTs), 19 of 303 (6%) were qualitative studies and 11 of 303 (3%) were mixed/multimethod studies (table 1). Furthermore, 139 of 303 (48%) studies were implemented in primary healthcare setting, and 85 of 303 (28%) studies were conducted in urban areas (31% NCDs and 18% NTDs).
Characteristics of the evaluated interventions and study results
Most of the interventions aimed at management (139 of 298, 47%) or prevention (99 of 298, 33%) of specific conditions, with few studies focusing on rehabilitation (13 of 298, 4%) or including more than two types of interventions (eg, prevention and management, 14%) (figure 2). Additionally, the most common interventions were: health education (92 of 300, 31%); clinical practice guidelines (56 of 300, 18%); task-shifting (55 of 300, 17%); self-management (50 of 300, 17%) and use of technology (42 of 300, 14%). Health education was frequently reported with other interventions, such as self-management (21 of 300), task-shifting (20 of 300) and training of health personnel (17 of 300). Conversely, the least common interventions were peer support, vector control, screening and epidemiological surveillance. In terms of target population, 62 of 300 (21%) of the studies’ interventions focused on health workers and patients; and other groups were intervened such as community members or lay people. Most of the studies reported positive results of the efficacy of their studies (180 of 303, 59%), whereas only 22 of 303 (7%) reported negative results. More information is shown in table 1.
Frameworks and theories
Only 35 of 303 (12%) studies defined their study design as ‘PE’ (13% for NCDs and 6% for NTDs). Other terms used by researchers included ‘mixed methods study’, ‘qualitative study’, ‘feasibility study’, ‘pilot study’ or terms related to PE outcomes such as ‘fidelity’, ‘acceptability’, ‘evaluation of interventions/programmes’ or ‘implementation/integration of interventions’.
Eighty-one of 303 (27%) studies adopted a theory or framework in the PE. Ten studies used classic qualitative approaches (ie, ground theory, phenomenology, content analysis, etc), eight studies used the MRC framework for PE, seven studies used the RE-AIM framework, four used the Donobedian model and three used the Consolidated Framework for Implementation Research (CFIR). Also, the Theoretical Domains Framework, the Andersen healthcare utilisation model and PRECEDE-PROCEED model were used by two studies each. The other 42 studies used other theories or frameworks such as the normalisation process theory, Saunders frameworks for cRCT, among others. When exploring how the theories or frameworks were used, 53 of 81 (66%) studies applied the given theory or framework; 15 (19%) studies were informed by the theory or framework; 11 of 81 (14%) studies tested the theory or framework; and 2 studies used more than one, for example, applied and tested the theory. Some examples of the use of theories and frameworks are Morrison et al41 who used the MRC framework to describe the implementation and fidelity of a Participatory Learning Action in an RCT in diabetes in Bangladesh; Gyamfi et al42 who developed their interview guide using the CFIR focusing on resources needed to implement a task-shifting intervention in Ghana; and Hashimoto et al43 who analysed the process and effects of implementing a vector surveillance system for Chagas diseases in the health centres from Honduras using the PRECEDE-PROCEED model (online supplemental material 5).
Two hundred and sixty-six of 301 (89%) studies stated the purpose of the PE and 177 of 299 (59%) mentioned the causal assumption of how the intervention will work. Also, 53 of 303 (19%) studies had a prespecified protocol.
Methods applied to conduct PE and PE outcomes
Most of the PE study designs were qualitative studies (97 of 303, 32.3%) or observational studies (92 of 303, 30.3%) (table 2). Ninety-seven studies of the 300 (32%) exclusively used qualitative methods and 64 of 300 (21%) used mixed methods according to the study design of the PE (table 2).
Between the qualitative methods applied, interviews were the most common with 166 of 222 (75.2%), followed by focus groups with 83 of 213 (39.2%). In the case of the quantitative approaches, descriptive analysis with 133 of 203 (66%) and routine monitoring data with 80 of 203 (39%) were the most common (table 2). Data collection for the PE was recorded only post-intervention in 92 of 301 (31%) studies, whereas it was recorded only during intervention in 75 of 301 (25%) studies, and in 31 of 301 (10%) studies it was not clear at which moment these data were collected (figure 3). Also, 212 of 303 (70%) studies integrated their main outcome results with the results of the PE and 53 of 303 (17%) studies had a completely independent team leading the PE.
As an example, one study evaluated the diabetes programme of the Mexican Institute of Social Security using only qualitative methods.44 This included interviews, focus groups, observations and case studies with different stakeholders. Another study in Colombia evaluated the Onchocerciasis Elimination Program for the Americas using routine data for its PE to confirm that the country had achieved the elimination of onchocerciasis.45 Another case was a self-management education programme in patients with chronic obstructive pulmonary diseases, where the authors used descriptive analysis, advanced statistical analysis and qualitative methods through focus groups.46
The most common measured outcomes in these PEs were acceptability, barriers and facilitators for the implementation of interventions, experiences and perceptions, and feasibility (table 3). Other outcomes, measured in less than 5% of the studies, were adaptation, penetration and mechanism of impact.
Barriers and facilitators for intervention implementation
The review of a subsample of 50 randomly selected articles enabled the identification of some barriers and facilitators to implement complex interventions in LMICs. The main themes identified were contextual factors including: health systems, human resources, medicine, equipment, attitudes, policy environment and others. For example, Anand et al47 mentioned the following barriers: ‘Waiting time and out of pocket expenditure for the blood glucose testing emerged as major issues. The general reaction of the participants to disclosure of high-risk status was that of distress and denial’. Nicholls et al45 highlighted as facilitator ‘the health education component also addressed improving the inhabitants’ living conditions and health literacy’. Details of barriers and facilitators are provided in table 4.
Implications for PE in LMICs
Of the 50 articles, some mentioned how the PE data would be used to inform research, decision-makers or other stakeholders’ next steps. For example: (1) to improve the intervention without mentioning the next step or to improve the intervention for a future RCT (17 studies); (2) future implementation of the study in the health system (7 studies); (3) scale up the intervention (5 studies); (4) further research to expand the intervention to other settings or other diseases (4 studies); and (5) identification of PE outcomes relevant for future studies (2 studies). Some examples include Venables et al48 who mentioned possible next steps: ‘We could also consider expanding antiretroviral treatment and NCD refills into community venues, as has been done in other contexts, although this would depend upon the strength of the group dynamic’ and Perez et al49 who declared intentions to scale up the intervention: ‘The process-oriented fidelity research adopted offers insights for both strategy developers and Aedes aegypti Control Program managers to design implementation plans for further scaling-up and institutionalization of the participatory strategy.’
In this scoping review, 303 studies were identified, most of them focused more on NCDs than on NTDs. More than 50% of the studies were conducted in upper middle-income countries. There was also a wide variety of methods used to conduct the PEs, particularly in terms of data collection approaches and analysis, as well as use of theories and frameworks. The most common interventions described were for health education with focus on management of the diseases. Findings show that 1 of 10 studies used other labels for PE studies and about one-third of the studies used a theory or framework for their PE. Additionally, 42% used mixed methods to report their findings. The most frequent outcomes of PEs were acceptability, barriers and facilitators, experiences and perceptions, and feasibility. In a subsample of 50 studies, the most common barriers for implementation of the interventions were contextual factors including: the social determinants of health, health system factors, related to infrastructure, organisation and human resources, for example, the scarcity of health providers or limited medical skills. In contrast, facilitators mentioned were support from decision-makers, community engagement and social support.
Comparison with other studies
Recently, two systematic reviews explored the use of PEs.23–25 The study by Liu et al23 24 included 69 studies and only 22 (31%) were labelled as PE, which is higher in comparison with the current scoping review where only 12% used this label. It is noteworthy that many studies are using other terms for the same methodology in LMICs. This review found only 27% of studies adopted a theory or framework in the PE, which is close to the figures reported by Liu et al23 24 and Scott et al,25 31% and 38%, respectively. It is possible that this is related to the scarce use of the label ‘PE’, because if a study is not recognised as a PE, it will not use a theory or framework for PE. Additionally, it could be related to the fact that some researchers do not think that the current theories or frameworks fit their design or findings. Also, other potential reasons are poor knowledge about the existence of these theories and/or frameworks, poor matching between them and data collected, difficulties to select one between the huge number of available theories and/or frameworks or it could be related to the fact that most of them were developed in high-income countries. For example, a previous systematic review published in 201850 found 108 feasibility studies or PE studies that used the normalisation process theory, but were conducted only in high-income countries. The critiques identified in 10% of the included studies were the complexity of the technical vocabulary and constant overlap between the definitions or interpretations of the theory domains. One study in the review51 complained that the focus is in the participants and the mechanism of action instead of the interaction of the participants with the context, and another study52 mentioned that the theory focused too much in the professional participants and not in the patient.50
Concerning the design of PEs, our scoping review found a higher proportion of mixed or multimethod analysis (42%) in comparison with previous systematic reviews.24 25
Our scoping review found that 177 (59%) studies mentioned the causal assumption of how the intervention will work; this is similar to Liu et al24 where 46 (67%) studies have a clear description of the intervention and/or clarification of the causal assumption. This feature is important because complex interventions have many components, therefore, it is relevant to understand the intervention and how it will achieve the expected change. Also, only 53 (19%) articles in this review included a prespecified protocol, which is a lower percentage than Liu et al24 found, where 30 (48%) studies had a prespecified protocol. This could be related to the fact that Liu et al24 only included RCTs and it is more possible that the PE protocols would be published independently or alongside the intervention protocol.
PE outcomes reported in the current scoping review were acceptability, barriers and facilitators, experiences and perceptions, and feasibility. This is different to Scott et al25 who focused on PE studies in different diseases areas, where barriers and/or facilitators to implementation were by far the most reported outcomes in 72% studies; and experiences and perceptions and adherence and fidelity were the next more commonly reported outcomes. On the other hand, Liu et al24 identified that 20% of studies explored the influence of contextual factors in the intervention and they highlighted as a strength that they included studies that triangulated qualitative and quantitative results of acceptability; another strength was the determination of the intervention fidelity, for example, use of routine data or e-health indicators, among others, to determine this outcome. There is a huge variety of PE outcomes and there is certain overlap between definitions, for example, adoption and adherence are outcomes that in practical terms are the same. Also, the measure of these outcomes could be made through diverse methods (qualitative, quantitative or both) without knowing exactly which method is more appropriate to measure a specific outcome. Additionally, the use of strategies to incorporate the measurement of PE outcomes into routine is a good option, however the quality of the data and the overload of work for health professionals could be problematic.
In terms of barriers and facilitators for implementation of interventions identified in our study, we found a variety of factors related to the context, health system, human resources and policy. The most common barriers related to health systems were low availability of equipment, supplies and drugs as well as the organisation of the health centres or hospitals. This is similar to other studies, where one systematic review of studies that integrate the management of HIV and chronic services found that the main barriers in the health system were: (1) the need of effective collaboration and coordination between health teams, health personnel and patients; (2) limited trained health workers with the adequate skills and incentives; and (3) scarce institutional support and resources.53 Acknowledging and identifying the main barriers could allow the identification of modifiable barriers which can be addressed, and barriers which are not modifiable and need to be monitored to understand the impact in the success or failure of the implementation of complex interventions. Although this is key for the future implementation of complex interventions, few studies look at contextual factors and health system indicators.
Implications for research and practice
It has been described that it can take 17 years between innovation to the actual application in practice, and only 14% of the evidence is implemented.54 55 In order to alleviate this, PEs can serve as a useful approach to identify what needs adjustment and/or changes required before implementation of an intervention to avoid wasting opportunities to improve the translation of knowledge into daily practice. In addition, it is important to consider that the health system is complex and dynamic and measuring changes through PE is relevant in order to understand the interaction between the intervention and the health system.
This review has allowed the identification of several weaknesses in the literature in PE and proposes the following four recommendations. First, as for other methodologies or study designs such as systematic reviews or clinical trials, there is a need for standard protocols such as PRISMA56 or Consolidated Standards of Reporting Trials57 for PEs to guide researchers in the presentation of their work with clear aims, design, data collection tools and analysis. Next, there is need to explore and understand the reasons for the scarce use of theories and/or frameworks. Third, another element is the importance of conducting and publishing PE of studies with ‘negative’ efficacy results to identify if some components of the intervention were positive or if the problem was the implementation of the intervention or the inherent lack of efficiency of the intervention. For example, a study with the aim of comparing thermometry and mHealth versus thermometry alone to prevent diabetic foot ulcers was negative.58 However, the PE identified that the adherence to using thermometry was high in both groups showing that some aspects of the overall intervention were positive even though the overall efficacy result was negative. Another area that needs to be addressed is the importance of capacity building in research teams to strengthen their skills in qualitative and mixed methods. Also, additional resources needed in order to collect and analyse diverse types of data are necessary to foster the use of PEs. Another recommendation is the development of a guideline for the reporting of PEs, including PE outcomes in order to make these studies more homogeneous and comparable, which would be very valuable. Moreover, this tool could guide researchers on how to select theories and/or frameworks. Finally, another finding in this scoping review was the small number of PE studies in the area of NTDs, even when this group includes 20 conditions and represents a high burden for many LMICs. There is need for this type of research to close gaps to achieve better prevention, diagnosis, management or even elimination of NTDs.
Given that PE is key for different stakeholders (funders, researchers institutions, policymakers and communities), it is imperative for these stakeholders to take on board the recommendations presented above to support the improvement in the design of this methodology in order to achieve better results.24 25 Different stakeholders can take different actions according to their roles: funders can provide financial resources to conduct PE and request a proper design; researchers can provide protocols and reporting guidelines in order to improve their quality and to strengthen the capacity building of their teams in PE, qualitative research and mixed methods; policymakers can get involved in the PE and commit to apply the results for future implementation of the intervention; and finally communities to be active in providing information and critical thinking to improve the diverse components of the complex intervention.
Strength and weaknesses
Some limitations and strengths of our study need to be reported: (1) our search strategy was not peer reviewed,59 but it was designed with an experienced librarian and it was built based on the systematic review of Liu et al24; (2) we did not include all languages due to limited human resources but we expect that the four major selected languages covered the majority of relevant literature; (3) we are using the definition of LMIC from the World Bank 2019; however, it is possible that since 2008 there have been some changes in the categories or that the data collection was conducted when the country was not classified as an LMIC; (4) grey literature was not included but six databases from different regions ensure that most of the literature in the topic has been included; (5) the data extraction was not duplicated because of the high number of studies; this was mitigated by previous specific training of the two reviewers and frequent meetings to clear doubts; (6) only a subset (n=50) of studies were included in the subsample analysis of barriers and facilitators, which may have limited the number of identified barriers or facilitators, (7) due to the limited use of the term PE (many studies used other labels), it is possible that some articles were not identified by our search strategy despite our use of a broad terminology. However, in the current scoping review, only 12% of the articles were labelled as PE, so we were able to identify a good proportion of studies without this label. Other strengths of this study are that it explores PE of RCTs and non-RCTs, also the number of articles included allowed us to have a proper overview of the research in the topic.
PE is a methodology that allows researchers, practitioners and decision-makers to understand how a complex intervention is implemented. This information is particularly relevant in LMICs because of the challenges faced during the implementation of complex interventions due to their context and health systems.
Multiple gaps were identified such as scarce use of the labelled PE, poor use of theories and frameworks, and heterogeneous reporting of these studies. The mentioned findings from the current scoping review reinforce the findings from the two previous systematic reviews.24 25 Guidelines are needed in order to standardise the reporting of PE studies, which should improve the identification of barriers and facilitators that will inform future research or scale up of complex interventions in NCDs and NTDs.
Data availability statement
Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Patient consent for publication
Contributors Protocol definition—ML-P, HL, PP, JJM, GM, FC and DB. Search terms definition—ML-P and HL. Abstract screening—ML-P, HL, MO, XY and AM. Data extraction—ML-P, TB, WCG-V, NP and MB. Data analysis—ML-P, PP and DB. Manuscript writing—ML-P, HL, PP, JJM, GM, FC and DB. All the authors read and approved the final manuscript. ML-P is the author responsible for the overall content as the guarantor.
Funding COHESION (COmmunity HEalth System InnovatiON) Project was funded by the Swiss National Science Foundation and the Swiss Development Cooperation under the Swiss Program for Research on Global Issues for Development (40P740-160366). ML-P receives funding from the Swiss Excellence Government Scholarship (2018.0698).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.