You are here

  • Home
  • Archive
  • Volume 12, Issue 9
  • Process evaluation of complex interventions in non-communicable and neglected tropical diseases in low- and middle-income countries: a scoping review

Article Text

Article menu
Global health
Original research
Process evaluation of complex interventions in non-communicable and neglected tropical diseases in low- and middle-income countries: a scoping review
  1. Maria Lazo-Porras1,2,
  2. Hueiming Liu3,
  3. Menglu Ouyang3,4,
  4. Xuejun Yin3,4,
  5. Alejandra Malavera3,
  6. Tiana Bressan5,
  7. Wilmer Cristobal Guzman-Vilca6,7,
  8. Niels Pacheco6,
  9. Melissa Benito1,
  10. J Jaime Miranda1,6,
  11. Graham Moore8,
  12. François Chappuis2,
  13. Pablo Perel9,
  14. David Beran2
  1. 1CRONICAS Centre of Excellence in Chronic Diseases, Universidad Peruana Cayetano Heredia, Lima, Peru
  2. 2Division of Tropical and Humanitarian Medicine, University of Geneva and Geneva University Hospitals, Geneva, Switzerland
  3. 3The George Institute for Global Health, Faculty of Medicine, University of New South Wales, Sydney, New South Wales, Australia
  4. 4The George Institute China at Peking University Health Science Center, Beijing, China
  5. 5University of Guelph, Ontario, Canada
  6. 6School of Medicine, Universidad Peruana Cayetano Heredia, Lima, Peru
  7. 7Sociedad Científica de Estudiantes de Medicina Cayetano Heredia (SOCEMCH), Universidad Peruana Cayetano Heredia, Lima, Peru
  8. 8DECIPHer, UKCRC Centre of Excellence, Cardiff University, Cardiff, UK
  9. 9Epidemiology and Public Health, London School of Hygiene & Tropical Medicine, London, UK
  1. Correspondence to Dr Maria Lazo-Porras; maria.lazo.porras{at}gmail.com

Abstract

Objectives The aim of this review is to map out the use of process evaluation (PE) in complex interventions that address non-communicable diseases (NCDs) and neglected tropical diseases (NTDs) to identify gaps in the design and conduct, as well as strengths, limitations and implications, of this type of research in low- and middle-income countries (LMICs).

Design Scoping review of PE studies of complex interventions implemented in LMICs. Six databases were searched focused on studies published since 2008.

Data sources Embase, PubMed, EbscoHost, Web of Science (WOS), Virtual Health Library (VHL) Regional Portal and Global Index Medicus: Regional Indexes AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), WPRIM (WPRO) Global Index Regional Indexes, MEDLINE, SciELO.

Eligibility criteria Studies conducted in LMICs on PEs of randomised controlled trials (RCTs) and non-RCTs published between January 2008 and January 2020. Other criteria were studies of interventions for people at risk or having physical and mental NCDs, and/or NTDs, and/or their healthcare providers and/or others related to achieve better health for these two disease groups. Studies were excluded if they were not reported in English or Spanish or Portuguese or French, not peer-reviewed articles, not empirical research and not human research.

Data extraction and synthesis Data extracted to be evaluated were: available evidence in the utilisation of PE in the areas of NCDs and NTDs, including frameworks and theories used; methods applied to conduct PEs; and in a subsample, the barriers and facilitators to implement complex interventions identified through the PE. Variables were extracted and categorised. The information was synthesised through quantitative analysis by reporting frequencies and percentages. Qualitative analysis was also performed to understand facilitators and barriers presented in these studies. The implications for PEs, and how the information from the PE was used by researchers or other stakeholders were also assessed in this approach.

Results 303 studies were identified, 79% were for NCDs, 12% used the label ‘PE’, 27% described a theory or framework for the PE, and 42% used mixed methods to analyse their findings. Acceptability, barriers and facilitators to implement the interventions, experiences and perceptions, and feasibility were the outcomes most frequently evaluated as part of the PEs. Barriers and facilitators themes identified were contextual factors, health system factors, human resources, attitudes and policy factors.

Conclusions PEs in NCDs and NTDs are used in LMICs with a wide variety of methods. This review identified many PEs that were not labelled by the authors as such, as well as a limited application of PE-related theories and frameworks, and heterogeneous reporting of this type of study.

  • diabetes & endocrinology
  • hypertension
  • coronary heart disease
  • infectious diseases
  • public health

Data availability statement

Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2021-057597

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Six databases focused on process evaluation of complex interventions implemented in low- and middle-income countries in the area of non-communicable diseases and neglected tropical diseases were searched.

    • The search strategy was designed with an experienced librarian and it was built based on the systematic review of Liu et al.

    • All languages were not included due to limited human resources but we expect that the four major selected languages covered the majority of relevant literature.

    • Data extraction was not duplicated because of the high number of studies; this was mitigated by previous specific training of the two reviewers and frequent meetings to clear doubts.

    • Only a subset (n=50) of studies were included in the subsample analysis of barriers and facilitators, which may have limited the number of identified barriers or facilitators.

    Introduction

    In low- and middle-income countries (LMICs), the double burden of chronic long-term non-communicable (NCDs) and neglected tropical diseases (NTDs) represents a major challenge to vulnerable populations and their health systems.1 2 NCDs kill 41 million people each year and one-third of them are between 30 and 69 years old.3 This group of diseases is having an increasing impact in LMICs with variability in terms of burden and mortality within and between these countries.4–6 NTDs affect more than 1 billion people worldwide and they are the most vulnerable populations with poor access to healthcare services.7 8 NCDs and NTDs impact individuals,9 10 communities and health system responses.4 11 12 Complex interventions are needed to address these complex and multifaceted health challenges13 14 in order to tackle global health problems,15 16 and therefore achieve the Sustainable Development Goal 3 to ‘Ensure healthy lives and promote well-being for all at all ages’.17 However, interventions with multiple components face diverse challenges when these interventions are implemented on the ground,18 especially in LMICs, where the context and health system barriers add additional complexity.19 Given this, process evaluation (PE) is a type of study that can be used to understand how complex interventions are implemented, clarify causal mechanisms and identify contextual factors associated with variation in outcomes rather than simply measuring effectiveness outcomes.20 PEs are also useful to close translation gaps and reduce research waste,21 because they can inform future interventions, policy and practice.20 22

    Two recently published systematic reviews explored the use of PEs in the literature. One study focused on primary healthcare interventions addressing NCDs in randomised controlled trials (RCTs) worldwide.23 24 Another systematic review was conducted on PE studies of knowledge translation interventions without restriction of health conditions.25 These reviews found that PEs used both quantitative and qualitative methods of data collection, and that they can be conducted at different moments during a project such as feasibility testing phases, alongside evaluations of effectiveness or alongside scale-up studies. These reviews were restricted to publications in English language only, and found limited LMIC representation. Liu et al23 24 in their study that included PEs conducted in primary care settings found only three RCTs for NCDs in LMICs (one study in India, Malaysia, and Zambia, respectively). Scott et al25 did not report the regions of the implementation of the studies included in their review. Neither of these reviews explored the main barriers and facilitators of implementing complex interventions on NCDs and NTDs in LMICs.

    This scoping review aims to identify gaps in the design and conduct of PE of complex interventions for NCDs and NTDs in LMICs. These findings expand the knowledge of previous systematic reviews by focusing on LMICs, where barriers to implement complex interventions are frequent and challenging. The objectives of this scoping review are: (1) determining the proportion of studies labelled as PE and identifying other terms used instead of PE in the literature; (2) identifying theories and frameworks used in PEs of complex interventions for NCDs and NTDs in LMICs; (3) examining the design and conduct of PE of complex interventions for NCDs and NTDs in LMICs; and (4) summarising barriers and facilitators identified in a subsample of studies of PEs of complex interventions for NCDs and NTDs in LMICs.

    Methods

    The full protocol for this scoping review was published elsewhere.26 Results are reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews.27

    Eligibility Criteria

    Studies on PEs of RCTs and non-RCTs (feasibility studies, observational studies, quasi-experimental studies) were included. Given that findings in previous systematic reviews23–25 showed that PEs are often not explicitly labelled as such, we included studies with data that: tried to understand how complex interventions were implemented in the field; measured PE outcomes; clarified causal mechanisms; and/or identified contextual factors beyond measuring effectiveness outcomes. Studies conducted in LMICs according to the World Bank definition in 201928 and published between January 2008 and January 2020 were included. This date was chosen as it was when the UK’s Medical Research Council (MRC) updated its guidance for developing and evaluating complex interventions.29 ,30 If a study was conducted in high-income countries and LMICs, only the data from the LMICs were included.

    Studies considered for this scoping review had interventions targeted at people at risk or having physical and mental NCDs (type 2 diabetes mellitus, type 1 diabetes mellitus, cardiovascular disease, depression, chronic obstructive pulmonary disease and chronic kidney disease),31 and/or NTDs (online supplemental material 1),32 and/or healthcare providers (physicians, nurses, technicians) and/or others related to achieve better health (reforms in universal coverage, public policy, service delivery and leadership) for these two disease groups.

    The types of interventions included were complex interventions according to the MRC guideline that considered ‘intervention with several interacting components’ but also could include other dimensions of complexity such as: ‘number and difficulty of behaviours required by those delivering or receiving the intervention’; ‘number of groups or organisational levels targeted by the intervention’; and ‘degree of flexibility or tailoring of the intervention permitted’.29 30 In addition, we included studies implemented in diverse settings such as community, health system or at policy level. The comparators were treatment as usual, active control or placebo control, as well as absent control condition (eg, pre/post-evaluations). In addition, studies with multiple active interventions were included.

    Studies were excluded if they were not reported in English or Spanish or Portuguese or French, not peer-reviewed articles, not empirical research and not human research.

    Information sources

    Embase, PubMed, EbscoHost, Web of Science (WOS), Virtual Health Library (VHL) Regional Portal and Global Index Medicus: Regional Indexes AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), WPRIM (WPRO) Global Index Regional Indexes, MEDLINE, SciELO.

    Search strategy

    A different search strategy (online supplemental material 2) was designed for each database and with an experienced librarian.

    Study records

    Data management

    Covidence (Veritas Health Innovation, Melbourne, Australia)33 was used in the screening process. Titles and abstracts were screened after the importation of citations from the databases, and manuscripts were selected to proceed to data extraction.

    Selection process

    Five reviewers (ML-P, HL, MO, XY, AM) in pairs considered the potential eligibility of all abstracts and titles identified from executing the search strategy. Disagreements were solved by a third reviewer. Agreement on the screening selection among reviewers was between 90% and 92%. Also, teams of two independent and blind reviewers evaluated full texts of all potential eligible studies. First, the reviewers calibrated their judgements using a smaller set of articles to ensure the same understanding of inclusion and exclusion criteria. Subsequently, they reviewed the complete set of articles. Any disagreements were resolved by group discussion or arbitration until reaching consensus. Agreement on the screening selection among reviewers was between 70% and 81%. Additionally, we included studies: (1) if the search strategy identified protocols with a published PE article and (2) if a PE study was identified during the full-text screening through further reference checking. Also, these potential eligible articles were reviewed by duplicated too.

    Data collection process and data items

    Data extraction included:

    1. Full description of the study and complex intervention.

    2. Details about the PE such as clear labelling as a PE, purpose, prespecified protocol, stage when the PE was applied, specific framework and/or theory of the PE18 20 34–38 and how the framework and/or theory of the PE was used (informed by, applied theory or tested theory),39 methods used, leading team of the evaluation and type of analysis (quantitative, qualitative, mixed or multimethod), as well as PE outcomes defined by the literature that were measured (online supplemental material 3).40

    3. Lessons learnt about the PE: barriers and facilitators for implementation of complex interventions and implications for PE in LMICs (how the information of the PE was used). Data extraction on lessons learnt about the PE was literally extracted. The tools and detailed information for the data collection can be found in the protocol.26

    Reporting of results

    Variables were extracted and categorised (eg, data related to the methods of the PE and details of the intervention). The information was synthesised through quantitative analysis by reporting frequencies and percentages.

    Qualitative analysis was also performed to understand facilitators and barriers to implement the complex interventions in the field as well as the implications for PEs in LMICs (how the information from the PE was used by researchers or other stakeholders). Fifty studies were randomly chosen from the total sample of included articles for the qualitative analysis using the web platform Research Randomizer (https://www.randomizer.org) to guarantee saturation. Based on the information extracted from these studies, codes were generated and used to synthesise and analyse the data.

    Results are presented in the following four sections:

    1. General characteristics of the interventions.

    2. Available evidence in PE in the areas of NCDs and NTDs, including frameworks and theories used.

    3. Methods applied to conduct PE.

    4. Findings from the PEs related to barriers and facilitators of implementation of the complex interventions, as well as findings about implications of the PE.

    Patient and public involvement

    There was no patient nor public involvement in this study.

    Results

    Our search strategy identified 11 297 articles, 9977 were screened after removing 1320 duplicates, and 285 studies were included. The reasons for exclusions are available in figure 1 and the main reason was that they were not PE studies. Additional 18 articles were included, 11 studies that the search strategy identified as protocols but with published PE results and 7 articles which were found during the full-text screening through further reference checking (figure 1). Thus, a total of 303 studies were included in this review (online supplemental material 4) and 18 of them were articles published in Spanish or Portuguese.

    Figure 1
    Figure 1

    Flow chart. LMIC, low- and middle-income country; PE, process evaluation.

    General characteristics of the interventions

    Disease and region

    Of the 303 studies, 239 of 303 (79%) focused on NCDs, 62 of 303 (20%) on NTDs and 2 of 303 (1%) included both groups of diseases or had a health system perspective. Among the most common NCDs, studies focusing on cardiovascular diseases were 86 of 239 (36%) and type 2 diabetes with 50 of 239 (21%) studies. Regarding NTDs, dengue was the most common condition with 19 of 62 (31%) studies.

    In regard to the geographical regions where the studies were conducted, 83 of 303 (27%) were from sub-Saharan Africa, 65 of 303 (21.5%) from Latin America and 16 of 303 (5%) from countries in the Middle East and North Africa. More than 50% of the studies were conducted in upper middle-income countries (table 1).

    View this table:
    Table 1

    General characteristics of the studies

    Study design and context

    Seventy-five studies (75 of 303, 25%) were RCTs or cluster RCTs (cRCTs), 19 of 303 (6%) were qualitative studies and 11 of 303 (3%) were mixed/multimethod studies (table 1). Furthermore, 139 of 303 (48%) studies were implemented in primary healthcare setting, and 85 of 303 (28%) studies were conducted in urban areas (31% NCDs and 18% NTDs).

    Characteristics of the evaluated interventions and study results

    Most of the interventions aimed at management (139 of 298, 47%) or prevention (99 of 298, 33%) of specific conditions, with few studies focusing on rehabilitation (13 of 298, 4%) or including more than two types of interventions (eg, prevention and management, 14%) (figure 2). Additionally, the most common interventions were: health education (92 of 300, 31%); clinical practice guidelines (56 of 300, 18%); task-shifting (55 of 300, 17%); self-management (50 of 300, 17%) and use of technology (42 of 300, 14%). Health education was frequently reported with other interventions, such as self-management (21 of 300), task-shifting (20 of 300) and training of health personnel (17 of 300). Conversely, the least common interventions were peer support, vector control, screening and epidemiological surveillance. In terms of target population, 62 of 300 (21%) of the studies’ interventions focused on health workers and patients; and other groups were intervened such as community members or lay people. Most of the studies reported positive results of the efficacy of their studies (180 of 303, 59%), whereas only 22 of 303 (7%) reported negative results. More information is shown in table 1.

    Figure 2
    Figure 2

    Type of the interventions in the included studies.

    Frameworks and theories

    Only 35 of 303 (12%) studies defined their study design as ‘PE’ (13% for NCDs and 6% for NTDs). Other terms used by researchers included ‘mixed methods study’, ‘qualitative study’, ‘feasibility study’, ‘pilot study’ or terms related to PE outcomes such as ‘fidelity’, ‘acceptability’, ‘evaluation of interventions/programmes’ or ‘implementation/integration of interventions’.

    Eighty-one of 303 (27%) studies adopted a theory or framework in the PE. Ten studies used classic qualitative approaches (ie, ground theory, phenomenology, content analysis, etc), eight studies used the MRC framework for PE, seven studies used the RE-AIM framework, four used the Donobedian model and three used the Consolidated Framework for Implementation Research (CFIR). Also, the Theoretical Domains Framework, the Andersen healthcare utilisation model and PRECEDE-PROCEED model were used by two studies each. The other 42 studies used other theories or frameworks such as the normalisation process theory, Saunders frameworks for cRCT, among others. When exploring how the theories or frameworks were used, 53 of 81 (66%) studies applied the given theory or framework; 15 (19%) studies were informed by the theory or framework; 11 of 81 (14%) studies tested the theory or framework; and 2 studies used more than one, for example, applied and tested the theory. Some examples of the use of theories and frameworks are Morrison et al41 who used the MRC framework to describe the implementation and fidelity of a Participatory Learning Action in an RCT in diabetes in Bangladesh; Gyamfi et al42 who developed their interview guide using the CFIR focusing on resources needed to implement a task-shifting intervention in Ghana; and Hashimoto et al43 who analysed the process and effects of implementing a vector surveillance system for Chagas diseases in the health centres from Honduras using the PRECEDE-PROCEED model (online supplemental material 5).

    Two hundred and sixty-six of 301 (89%) studies stated the purpose of the PE and 177 of 299 (59%) mentioned the causal assumption of how the intervention will work. Also, 53 of 303 (19%) studies had a prespecified protocol.

    Methods applied to conduct PE and PE outcomes

    Most of the PE study designs were qualitative studies (97 of 303, 32.3%) or observational studies (92 of 303, 30.3%) (table 2). Ninety-seven studies of the 300 (32%) exclusively used qualitative methods and 64 of 300 (21%) used mixed methods according to the study design of the PE (table 2).

    View this table:
    Table 2

    Methods applied in process evaluation

    Between the qualitative methods applied, interviews were the most common with 166 of 222 (75.2%), followed by focus groups with 83 of 213 (39.2%). In the case of the quantitative approaches, descriptive analysis with 133 of 203 (66%) and routine monitoring data with 80 of 203 (39%) were the most common (table 2). Data collection for the PE was recorded only post-intervention in 92 of 301 (31%) studies, whereas it was recorded only during intervention in 75 of 301 (25%) studies, and in 31 of 301 (10%) studies it was not clear at which moment these data were collected (figure 3). Also, 212 of 303 (70%) studies integrated their main outcome results with the results of the PE and 53 of 303 (17%) studies had a completely independent team leading the PE.

    Figure 3
    Figure 3

    Time of data collection for the process evaluation.

    As an example, one study evaluated the diabetes programme of the Mexican Institute of Social Security using only qualitative methods.44 This included interviews, focus groups, observations and case studies with different stakeholders. Another study in Colombia evaluated the Onchocerciasis Elimination Program for the Americas using routine data for its PE to confirm that the country had achieved the elimination of onchocerciasis.45 Another case was a self-management education programme in patients with chronic obstructive pulmonary diseases, where the authors used descriptive analysis, advanced statistical analysis and qualitative methods through focus groups.46

    The most common measured outcomes in these PEs were acceptability, barriers and facilitators for the implementation of interventions, experiences and perceptions, and feasibility (table 3). Other outcomes, measured in less than 5% of the studies, were adaptation, penetration and mechanism of impact.

    View this table:
    Table 3

    Process evaluation outcomes

    Barriers and facilitators for intervention implementation

    The review of a subsample of 50 randomly selected articles enabled the identification of some barriers and facilitators to implement complex interventions in LMICs. The main themes identified were contextual factors including: health systems, human resources, medicine, equipment, attitudes, policy environment and others. For example, Anand et al47 mentioned the following barriers: ‘Waiting time and out of pocket expenditure for the blood glucose testing emerged as major issues. The general reaction of the participants to disclosure of high-risk status was that of distress and denial’. Nicholls et al45 highlighted as facilitator ‘the health education component also addressed improving the inhabitants’ living conditions and health literacy’. Details of barriers and facilitators are provided in table 4.

    View this table:
    Table 4

    Barriers and facilitators to implement complex interventions identified by the process evaluation

    Implications for PE in LMICs

    Of the 50 articles, some mentioned how the PE data would be used to inform research, decision-makers or other stakeholders’ next steps. For example: (1) to improve the intervention without mentioning the next step or to improve the intervention for a future RCT (17 studies); (2) future implementation of the study in the health system (7 studies); (3) scale up the intervention (5 studies); (4) further research to expand the intervention to other settings or other diseases (4 studies); and (5) identification of PE outcomes relevant for future studies (2 studies). Some examples include Venables et al48 who mentioned possible next steps: ‘We could also consider expanding antiretroviral treatment and NCD refills into community venues, as has been done in other contexts, although this would depend upon the strength of the group dynamic’ and Perez et al49 who declared intentions to scale up the intervention: ‘The process-oriented fidelity research adopted offers insights for both strategy developers and Aedes aegypti Control Program managers to design implementation plans for further scaling-up and institutionalization of the participatory strategy.’

    Discussion

    Main findings

    In this scoping review, 303 studies were identified, most of them focused more on NCDs than on NTDs. More than 50% of the studies were conducted in upper middle-income countries. There was also a wide variety of methods used to conduct the PEs, particularly in terms of data collection approaches and analysis, as well as use of theories and frameworks. The most common interventions described were for health education with focus on management of the diseases. Findings show that 1 of 10 studies used other labels for PE studies and about one-third of the studies used a theory or framework for their PE. Additionally, 42% used mixed methods to report their findings. The most frequent outcomes of PEs were acceptability, barriers and facilitators, experiences and perceptions, and feasibility. In a subsample of 50 studies, the most common barriers for implementation of the interventions were contextual factors including: the social determinants of health, health system factors, related to infrastructure, organisation and human resources, for example, the scarcity of health providers or limited medical skills. In contrast, facilitators mentioned were support from decision-makers, community engagement and social support.

    Comparison with other studies

    Recently, two systematic reviews explored the use of PEs.23–25 The study by Liu et al23 24 included 69 studies and only 22 (31%) were labelled as PE, which is higher in comparison with the current scoping review where only 12% used this label. It is noteworthy that many studies are using other terms for the same methodology in LMICs. This review found only 27% of studies adopted a theory or framework in the PE, which is close to the figures reported by Liu et al23 24 and Scott et al,25 31% and 38%, respectively. It is possible that this is related to the scarce use of the label ‘PE’, because if a study is not recognised as a PE, it will not use a theory or framework for PE. Additionally, it could be related to the fact that some researchers do not think that the current theories or frameworks fit their design or findings. Also, other potential reasons are poor knowledge about the existence of these theories and/or frameworks, poor matching between them and data collected, difficulties to select one between the huge number of available theories and/or frameworks or it could be related to the fact that most of them were developed in high-income countries. For example, a previous systematic review published in 201850 found 108 feasibility studies or PE studies that used the normalisation process theory, but were conducted only in high-income countries. The critiques identified in 10% of the included studies were the complexity of the technical vocabulary and constant overlap between the definitions or interpretations of the theory domains. One study in the review51 complained that the focus is in the participants and the mechanism of action instead of the interaction of the participants with the context, and another study52 mentioned that the theory focused too much in the professional participants and not in the patient.50

    Concerning the design of PEs, our scoping review found a higher proportion of mixed or multimethod analysis (42%) in comparison with previous systematic reviews.24 25

    Our scoping review found that 177 (59%) studies mentioned the causal assumption of how the intervention will work; this is similar to Liu et al24 where 46 (67%) studies have a clear description of the intervention and/or clarification of the causal assumption. This feature is important because complex interventions have many components, therefore, it is relevant to understand the intervention and how it will achieve the expected change. Also, only 53 (19%) articles in this review included a prespecified protocol, which is a lower percentage than Liu et al24 found, where 30 (48%) studies had a prespecified protocol. This could be related to the fact that Liu et al24 only included RCTs and it is more possible that the PE protocols would be published independently or alongside the intervention protocol.

    PE outcomes reported in the current scoping review were acceptability, barriers and facilitators, experiences and perceptions, and feasibility. This is different to Scott et al25 who focused on PE studies in different diseases areas, where barriers and/or facilitators to implementation were by far the most reported outcomes in 72% studies; and experiences and perceptions and adherence and fidelity were the next more commonly reported outcomes. On the other hand, Liu et al24 identified that 20% of studies explored the influence of contextual factors in the intervention and they highlighted as a strength that they included studies that triangulated qualitative and quantitative results of acceptability; another strength was the determination of the intervention fidelity, for example, use of routine data or e-health indicators, among others, to determine this outcome. There is a huge variety of PE outcomes and there is certain overlap between definitions, for example, adoption and adherence are outcomes that in practical terms are the same. Also, the measure of these outcomes could be made through diverse methods (qualitative, quantitative or both) without knowing exactly which method is more appropriate to measure a specific outcome. Additionally, the use of strategies to incorporate the measurement of PE outcomes into routine is a good option, however the quality of the data and the overload of work for health professionals could be problematic.

    In terms of barriers and facilitators for implementation of interventions identified in our study, we found a variety of factors related to the context, health system, human resources and policy. The most common barriers related to health systems were low availability of equipment, supplies and drugs as well as the organisation of the health centres or hospitals. This is similar to other studies, where one systematic review of studies that integrate the management of HIV and chronic services found that the main barriers in the health system were: (1) the need of effective collaboration and coordination between health teams, health personnel and patients; (2) limited trained health workers with the adequate skills and incentives; and (3) scarce institutional support and resources.53 Acknowledging and identifying the main barriers could allow the identification of modifiable barriers which can be addressed, and barriers which are not modifiable and need to be monitored to understand the impact in the success or failure of the implementation of complex interventions. Although this is key for the future implementation of complex interventions, few studies look at contextual factors and health system indicators.

    Implications for research and practice

    It has been described that it can take 17 years between innovation to the actual application in practice, and only 14% of the evidence is implemented.54 55 In order to alleviate this, PEs can serve as a useful approach to identify what needs adjustment and/or changes required before implementation of an intervention to avoid wasting opportunities to improve the translation of knowledge into daily practice. In addition, it is important to consider that the health system is complex and dynamic and measuring changes through PE is relevant in order to understand the interaction between the intervention and the health system.

    This review has allowed the identification of several weaknesses in the literature in PE and proposes the following four recommendations. First, as for other methodologies or study designs such as systematic reviews or clinical trials, there is a need for standard protocols such as PRISMA56 or Consolidated Standards of Reporting Trials57 for PEs to guide researchers in the presentation of their work with clear aims, design, data collection tools and analysis. Next, there is need to explore and understand the reasons for the scarce use of theories and/or frameworks. Third, another element is the importance of conducting and publishing PE of studies with ‘negative’ efficacy results to identify if some components of the intervention were positive or if the problem was the implementation of the intervention or the inherent lack of efficiency of the intervention. For example, a study with the aim of comparing thermometry and mHealth versus thermometry alone to prevent diabetic foot ulcers was negative.58 However, the PE identified that the adherence to using thermometry was high in both groups showing that some aspects of the overall intervention were positive even though the overall efficacy result was negative. Another area that needs to be addressed is the importance of capacity building in research teams to strengthen their skills in qualitative and mixed methods. Also, additional resources needed in order to collect and analyse diverse types of data are necessary to foster the use of PEs. Another recommendation is the development of a guideline for the reporting of PEs, including PE outcomes in order to make these studies more homogeneous and comparable, which would be very valuable. Moreover, this tool could guide researchers on how to select theories and/or frameworks. Finally, another finding in this scoping review was the small number of PE studies in the area of NTDs, even when this group includes 20 conditions and represents a high burden for many LMICs. There is need for this type of research to close gaps to achieve better prevention, diagnosis, management or even elimination of NTDs.

    Given that PE is key for different stakeholders (funders, researchers institutions, policymakers and communities), it is imperative for these stakeholders to take on board the recommendations presented above to support the improvement in the design of this methodology in order to achieve better results.24 25 Different stakeholders can take different actions according to their roles: funders can provide financial resources to conduct PE and request a proper design; researchers can provide protocols and reporting guidelines in order to improve their quality and to strengthen the capacity building of their teams in PE, qualitative research and mixed methods; policymakers can get involved in the PE and commit to apply the results for future implementation of the intervention; and finally communities to be active in providing information and critical thinking to improve the diverse components of the complex intervention.

    Strength and weaknesses

    Some limitations and strengths of our study need to be reported: (1) our search strategy was not peer reviewed,59 but it was designed with an experienced librarian and it was built based on the systematic review of Liu et al24; (2) we did not include all languages due to limited human resources but we expect that the four major selected languages covered the majority of relevant literature; (3) we are using the definition of LMIC from the World Bank 2019; however, it is possible that since 2008 there have been some changes in the categories or that the data collection was conducted when the country was not classified as an LMIC; (4) grey literature was not included but six databases from different regions ensure that most of the literature in the topic has been included; (5) the data extraction was not duplicated because of the high number of studies; this was mitigated by previous specific training of the two reviewers and frequent meetings to clear doubts; (6) only a subset (n=50) of studies were included in the subsample analysis of barriers and facilitators, which may have limited the number of identified barriers or facilitators, (7) due to the limited use of the term PE (many studies used other labels), it is possible that some articles were not identified by our search strategy despite our use of a broad terminology. However, in the current scoping review, only 12% of the articles were labelled as PE, so we were able to identify a good proportion of studies without this label. Other strengths of this study are that it explores PE of RCTs and non-RCTs, also the number of articles included allowed us to have a proper overview of the research in the topic.

    Conclusions

    PE is a methodology that allows researchers, practitioners and decision-makers to understand how a complex intervention is implemented. This information is particularly relevant in LMICs because of the challenges faced during the implementation of complex interventions due to their context and health systems.

    Multiple gaps were identified such as scarce use of the labelled PE, poor use of theories and frameworks, and heterogeneous reporting of these studies. The mentioned findings from the current scoping review reinforce the findings from the two previous systematic reviews.24 25 Guidelines are needed in order to standardise the reporting of PE studies, which should improve the identification of barriers and facilitators that will inform future research or scale up of complex interventions in NCDs and NTDs.

    Data availability statement

    Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    References

    1. ↵
      1. Aya Pastrana N,
      2. Cárdenas MK,
      3. Pesantes MA
      . Addressing the double burden of non-communicable and neglected tropical diseases in lmics. Global Health, Epidemiology and Genomics [Internet]. 2017, 2021. Available: http://gheg-journal.co.uk/2017/11/burden-ncds-ntds-lmics/.
    2. ↵
      1. Krishnapillai V,
      2. Sreedevi A,
      3. Ramakrishnan D
      . From neglect to equity vis‐a‐vis noncommunicable diseases and neglected tropical diseases. Indian J Community Med 2019;44.doi:10.4103/ijcm.IJCM_304_19
    3. ↵
      . Noncommunicable diseases. Geneva World Health Organization; 2021.
    4. ↵
      1. Beaglehole R,
      2. Bonita R,
      3. Horton R, et al
      . Priority actions for the non-communicable disease crisis. Lancet 2011;377:1438–47.doi:10.1016/S0140-6736(11)60393-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/21474174
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Ziraba AK,
      2. Fotso JC,
      3. Ochako R
      . Overweight and obesity in urban Africa: a problem of the rich or the poor? BMC Public Health 2009;9:465.doi:10.1186/1471-2458-9-465pmid:http://www.ncbi.nlm.nih.gov/pubmed/20003478
      OpenUrlCrossRefPubMed
    6. ↵
      1. GBD 2015 Risk Factors Collaborators
      . Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks, 1990-2015: a systematic analysis for the global burden of disease study 2015. Lancet 2016;388:1659–724.doi:10.1016/S0140-6736(16)31679-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/27733284
      OpenUrlCrossRefPubMed
    7. ↵
      1. Marchal B,
      2. Van Dormael M,
      3. Pirard M, et al
      . Neglected tropical disease (NTD) control in health systems: the interface between programmes and general health services. Acta Trop 2011;120 Suppl 1:S177–85.doi:10.1016/j.actatropica.2011.02.017pmid:http://www.ncbi.nlm.nih.gov/pubmed/21439930
      OpenUrlPubMed
    8. ↵
      1. WHO
      . Investing to overcome the global impact of neglected tropical diseases: third WHO report on neglected diseases 2015. In: . Geneva: World Health Organization, 2015.
    9. ↵
      1. Mehta P,
      2. Hotez PJ, NTD and NCD Co-morbidities
      . The example of dengue fever. PLoS neglected tropical diseases. 2016;10:e0004619.doi:10.1371/journal.pntd.0004619
    10. ↵
      1. Hotez PJ
      . Linking Tropical Infections to Hypertension: New Comorbid Disease Paradigms in Our Era of "Blue Marble Health". J Am Heart Assoc 2019;8:e03984.doi:10.1161/JAHA.119.012313pmid:http://www.ncbi.nlm.nih.gov/pubmed/30879371
      OpenUrlPubMed
    11. ↵
      1. UNDP
      . Addressing the social determinants of noncommunicable diseases. New York: United Nations Development Programme, 2013.
    12. ↵
      1. Horton R
      . Offline: nightmare on Downing street. Lancet 2011;378:1986.doi:10.1016/S0140-6736(11)61853-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/30060392
      OpenUrlPubMed
    13. ↵
      1. WHO
      . A guide to implementation research in the prevention and control of noncommunicable diseasesWorld Health Organization, 2016.
    14. ↵
      1. Breda J,
      2. Wickramasinghe K,
      3. Peters DH, et al
      . One size does not fit all: implementation of interventions for non-communicable diseases. BMJ 2019;347:l6434.doi:10.1136/bmj.l6434
      OpenUrl
    15. ↵
      1. Moyer JD,
      2. Hedden S
      . Are we on the right path to achieve the sustainable development goals? World Dev 2020;127:104749.doi:10.1016/j.worlddev.2019.104749
    16. ↵
      1. Khalil H,
      2. Kynoch K
      . Implementation of sustainable complex interventions in health care services: the triple C model. BMC Health Serv Res 2021;21:143.doi:10.1186/s12913-021-06115-xpmid:http://www.ncbi.nlm.nih.gov/pubmed/33588823
      OpenUrlPubMed
    17. ↵
      1. NCDC collaborators,
      2. Countdown NCD, NCD Countdown 2030 collaborators
      . Ncd countdown 2030: pathways to achieving sustainable development goal target 3.4. Lancet 2020;396:918–34.doi:10.1016/S0140-6736(20)31761-Xpmid:http://www.ncbi.nlm.nih.gov/pubmed/32891217
      OpenUrlCrossRefPubMed
    18. ↵
      1. May C,
      2. Finch T,
      3. Mair F, et al
      . Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res 2007;7:148.doi:10.1186/1472-6963-7-148pmid:http://www.ncbi.nlm.nih.gov/pubmed/17880693
      OpenUrlCrossRefPubMed
    19. ↵
      1. Kruk ME,
      2. Gage AD,
      3. Arsenault C, et al
      . High-Quality health systems in the sustainable development goals era: time for a revolution. Lancet Glob Health 2018;6:e1196:252–1252.doi:10.1016/S2214-109X(18)30386-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/30196093
      OpenUrlCrossRefPubMed
    20. ↵
      1. Moore GF,
      2. Audrey S,
      3. Barker M, et al
      . Process evaluation of complex interventions: medical Research Council guidance. BMJ 2015;350:h1258.doi:10.1136/bmj.h1258pmid:http://www.ncbi.nlm.nih.gov/pubmed/25791983
      OpenUrlFREE Full Text
    21. ↵
      1. Murdoch J
      . Process evaluation for complex interventions in health services research: analysing context, text trajectories and disruptions. BMC Health Serv Res 2016;16:407.doi:10.1186/s12913-016-1651-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/27538946
      OpenUrlPubMed
    22. ↵
      1. Steckler A,
      2. Linnan L,
      3. Overview A
      . In: Steckler a, Linnan L, editors. process evaluation for public health interventions and research. United States of America: Jossey-Bass, 2002.
    23. ↵
      1. Liu H,
      2. Muhunthan J,
      3. Hayek A, et al
      . Examining the use of process evaluations of randomised controlled trials of complex interventions addressing chronic disease in primary health care-a systematic review protocol. Syst Rev 2016;5:138.doi:10.1186/s13643-016-0314-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/27526851
      OpenUrlCrossRefPubMed
    24. ↵
      1. Liu H,
      2. Mohammed A,
      3. Shanthosh J, et al
      . Process evaluations of primary care interventions addressing chronic disease: a systematic review. BMJ Open 2019;9:e025127.doi:10.1136/bmjopen-2018-025127pmid:http://www.ncbi.nlm.nih.gov/pubmed/31391188
      OpenUrlCrossRefPubMed
    25. ↵
      1. Scott SD,
      2. Rotter T,
      3. Flynn R, et al
      . Systematic review of the use of process evaluations in knowledge translation research. Syst Rev 2019;8:266.doi:10.1186/s13643-019-1161-ypmid:http://www.ncbi.nlm.nih.gov/pubmed/31699136
      OpenUrlPubMed
    26. ↵
      1. Lazo-Porras M,
      2. Liu H,
      3. Miranda JJ, et al
      . Process evaluation of complex interventions in chronic and neglected tropical diseases in low- and middle-income countries-a scoping review protocol. Syst Rev 2021;10:244.doi:10.1186/s13643-021-01801-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/34488877
      OpenUrlPubMed
    27. ↵
      1. Tricco AC,
      2. Lillie E,
      3. Zarin W, et al
      . PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018;169:467–73.doi:10.7326/M18-0850pmid:http://www.ncbi.nlm.nih.gov/pubmed/30178033
      OpenUrlCrossRefPubMed
    28. ↵
    29. ↵
      1. Skivington K,
      2. Matthews L,
      3. Simpson SA, et al
      . A new framework for developing and evaluating complex interventions: update of medical Research Council guidance. BMJ;2021:n2061.doi:10.1136/bmj.n2061
    30. ↵
      1. Craig P,
      2. Dieppe P,
      3. Macintyre S, et al
      . Developing and evaluating complex interventions: the new medical Research Council guidance. BMJ 2008;337:a1655.doi:10.1136/bmj.a1655pmid:http://www.ncbi.nlm.nih.gov/pubmed/18824488
      OpenUrlFREE Full Text
    31. ↵
      1. Firth J,
      2. Torous J,
      3. Nicholas J, et al
      . The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017;16:287–98.doi:10.1002/wps.20472pmid:http://www.ncbi.nlm.nih.gov/pubmed/28941113
      OpenUrlCrossRefPubMed
    32. ↵
      1. Savioli L,
      2. Daumerie D
      . Accelerating work to overcome the global impact of neglected tropical diseases: A roadmap for Implementation. In: Diseases DoCoNT. Geneva, Switzerland: World Health Organization, 2012.
    33. ↵
      1. Veritas Health Innovation
      . Covidence systematic review softwareMelbourne, Australia. Available: www.covidence.org
    34. ↵
      1. Pawson R,
      2. Tilley N
      . Realistic Evaluation. In: . London: Sage Publications Ltd, 1997.
    35. ↵
      1. Atkins L,
      2. Francis J,
      3. Islam R, et al
      . A guide to using the theoretical domains framework of behaviour change to investigate implementation problems. Implement Sci 2017;12:77.doi:10.1186/s13012-017-0605-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/28637486
      OpenUrlCrossRefPubMed
    36. ↵
      1. Damschroder LJ,
      2. Aron DC,
      3. Keith RE, et al
      . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50.doi:10.1186/1748-5908-4-50pmid:http://www.ncbi.nlm.nih.gov/pubmed/19664226
      OpenUrlCrossRefPubMed
    37. ↵
      1. Crosby R,
      2. Noar SM
      . What is a planning model? an introduction to PRECEDE-PROCEED. J Public Health Dent 2011;71 Suppl 1:S7–15.doi:10.1111/j.1752-7325.2011.00235.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/21656942
      OpenUrlPubMed
    38. ↵
      1. Glasgow RE,
      2. Vogt TM,
      3. Boles SM
      . Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999;89:1322–7.doi:10.2105/AJPH.89.9.1322pmid:http://www.ncbi.nlm.nih.gov/pubmed/10474547
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. McIntyre SA,
      2. Francis JJ,
      3. Gould NJ, et al
      . The use of theory in process evaluations conducted alongside randomized trials of implementation interventions: a systematic review. Transl Behav Med 2018;8:168–78.doi:10.1093/tbm/iby110
      OpenUrl
    40. ↵
      1. Proctor E,
      2. Silmere H,
      3. Raghavan R, et al
      . Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011;38:65–76.doi:10.1007/s10488-010-0319-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/20957426
      OpenUrlCrossRefPubMed
    41. ↵
      1. Morrison J,
      2. Akter K,
      3. Jennings HM, et al
      . Implementation and fidelity of a participatory learning and action cycle intervention to prevent and control type 2 diabetes in rural Bangladesh. Glob Health Res Policy 2019;4:19.doi:10.1186/s41256-019-0110-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/31312722
      OpenUrlPubMed
    42. ↵
      1. Gyamfi J,
      2. Allegrante JP,
      3. Iwelunmor J, et al
      . Application of the consolidated framework for implementation research to examine nurses' perception of the task shifting strategy for hypertension control trial in Ghana. BMC Health Serv Res 2020;20:65.doi:10.1186/s12913-020-4912-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/31996195
      OpenUrlPubMed
    43. ↵
      1. Hashimoto K,
      2. Zúniga C,
      3. Nakamura J, et al
      . Integrating an infectious disease programme into the primary health care service: a retrospective analysis of Chagas disease community-based surveillance in Honduras. BMC Health Serv Res 2015;15:116.doi:10.1186/s12913-015-0785-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/25889097
      OpenUrlCrossRefPubMed
    44. ↵
      1. Morales-Pérez MI,
      2. Urbina-Aguilar BA,
      3. Zavala-Cruz GG
      . Estudio cualitativo del Programa DiabetIMSS en Una Unidad de medicina familiar de San Luis Potosí. Revista de Enfermería del Instituto Mexicano del Seguro Social 2017;2:101–10.
      OpenUrl
    45. ↵
      1. Nicholls RS,
      2. Duque S,
      3. Olaya LA, et al
      . Elimination of onchocerciasis from Colombia: first proof of concept of river blindness elimination in the world. Parasit Vectors 2018;11:237.doi:10.1186/s13071-018-2821-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/29642939
      OpenUrlPubMed
    46. ↵
      1. Ng WI,
      2. Smith GD
      . Effects of a self-management education program on self-efficacy in patients with COPD: a mixed-methods sequential explanatory designed study. Int J Chron Obstruct Pulmon Dis 2017;12:2129–39.doi:10.2147/COPD.S136216pmid:http://www.ncbi.nlm.nih.gov/pubmed/28790816
      OpenUrlPubMed
    47. ↵
      1. Anand T,
      2. Kishore J,
      3. Isaakidis P, et al
      . Integrating screening for non-communicable diseases and their risk factors in routine tuberculosis care in Delhi, India: a mixed-methods study. PLoS One 2018;13:e0202256.doi:10.1371/journal.pone.0202256pmid:http://www.ncbi.nlm.nih.gov/pubmed/30138331
      OpenUrlPubMed
    48. ↵
      1. Venables E,
      2. Edwards JK,
      3. Baert S, et al
      . "They just come, pick and go." The Acceptability of Integrated Medication Adherence Clubs for HIV and Non Communicable Disease (NCD) Patients in Kibera, Kenya. PLoS One 2016;11:e0164634.doi:10.1371/journal.pone.0164634pmid:http://www.ncbi.nlm.nih.gov/pubmed/27764128
      OpenUrlCrossRefPubMed
    49. ↵
      1. Pérez D,
      2. Lefèvre P,
      3. Castro M, et al
      . Process-oriented fidelity research assists in evaluation, adjustment and scaling-up of community-based interventions. Health Policy Plan 2011;26:413–22.doi:10.1093/heapol/czq077pmid:http://www.ncbi.nlm.nih.gov/pubmed/21149346
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      1. May CR,
      2. Cummings A,
      3. Girling M, et al
      . Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci 2018;13:80.doi:10.1186/s13012-018-0758-1pmid:http://www.ncbi.nlm.nih.gov/pubmed/29879986
      OpenUrlCrossRefPubMed
    51. ↵
      1. Clarke DJ,
      2. Godfrey M,
      3. Hawkins R, et al
      . Implementing a training intervention to support caregivers after stroke: a process evaluation examining the initiation and embedding of programme change. Implement Sci 2013;8:96.doi:10.1186/1748-5908-8-96pmid:http://www.ncbi.nlm.nih.gov/pubmed/23972027
      OpenUrlPubMed
    52. ↵
      1. Segrott J,
      2. Murphy S,
      3. Rothwell H, et al
      . An application of extended normalisation process theory in a randomised controlled trial of a complex social intervention: process evaluation of the strengthening families programme (10-14) in Wales, UK. SSM Popul Health 2017;3:255–65.doi:10.1016/j.ssmph.2017.01.002pmid:http://www.ncbi.nlm.nih.gov/pubmed/29302612
      OpenUrlPubMed
    53. ↵
      1. Watt N,
      2. Sigfrid L,
      3. Legido-Quigley H, et al
      . Health systems facilitators and barriers to the integration of HIV and chronic disease services: a systematic review. Health Policy Plan 2017;32:iv13–26.doi:10.1093/heapol/czw149pmid:http://www.ncbi.nlm.nih.gov/pubmed/28666336
      OpenUrlCrossRefPubMed
    54. ↵
      1. Balas EA,
      2. Boren SA
      . Managing clinical knowledge for health care improvement. Yearb Med Inform 2000;1:65–70.doi:10.1055/s-0038-1637943pmid:http://www.ncbi.nlm.nih.gov/pubmed/27699347
      OpenUrlPubMed
    55. ↵
      1. De Geest S,
      2. Zúñiga F,
      3. Brunkert T, et al
      . Powering Swiss health care for the future: implementation science to bridge "the valley of death". Swiss Med Wkly 2020;150:w20323.doi:10.4414/smw.2020.20323pmid:http://www.ncbi.nlm.nih.gov/pubmed/33022070
      OpenUrlPubMed
    56. ↵
      1. Page MJ,
      2. McKenzie JE,
      3. Bossuyt PM, et al
      . The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 2021;10:89.doi:10.1186/s13643-021-01626-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/33781348
      OpenUrlCrossRefPubMed
    57. ↵
      1. Schulz KF,
      2. Altman DG,
      3. Moher D
      . Statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010;2010:c332.doi:10.1136/bmj.c332
      OpenUrl
    58. ↵
      1. Lazo-Porras M,
      2. Bernabe-Ortiz A,
      3. Taype-Rondan A, et al
      . Foot thermometry with mHeath-based supplementation to prevent diabetic foot ulcers: a randomized controlled trial. Wellcome Open Res 2020;5:23.doi:10.12688/wellcomeopenres.15531.2pmid:http://www.ncbi.nlm.nih.gov/pubmed/32923686
      OpenUrlPubMed
    59. ↵
      1. McGowan J,
      2. Sampson M,
      3. Salzwedel DM, et al
      . PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol 2016;75:40–6.doi:10.1016/j.jclinepi.2016.01.021pmid:http://www.ncbi.nlm.nih.gov/pubmed/27005575
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @wilmer_guzman_v

    • Contributors Protocol definition—ML-P, HL, PP, JJM, GM, FC and DB. Search terms definition—ML-P and HL. Abstract screening—ML-P, HL, MO, XY and AM. Data extraction—ML-P, TB, WCG-V, NP and MB. Data analysis—ML-P, PP and DB. Manuscript writing—ML-P, HL, PP, JJM, GM, FC and DB. All the authors read and approved the final manuscript. ML-P is the author responsible for the overall content as the guarantor.

    • Funding COHESION (COmmunity HEalth System InnovatiON) Project was funded by the Swiss National Science Foundation and the Swiss Development Cooperation under the Swiss Program for Research on Global Issues for Development (40P740-160366). ML-P receives funding from the Swiss Excellence Government Scholarship (2018.0698).

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.