Article Text
PDF
Abstract
Introduction In 2006, the Congolese Ministry of Health developed a health system strengthening strategy focusing on health district development. This strategy called for reforming the provincial health administration in order to better support the health district development through leadership and management capacity building of district health management teams. The implementation is currently underway, yet, more evidence on how, for whom and under what conditions this capacity building works is needed. The proposed research aims to address this gap using a realist evaluation approach.
Methods and analysis We will follow the cycle of the realist evaluation. First, we will elicit the initial programme theory through a scoping review (completed in December 2022, using MEDLINE, Health Systems Evidence, Wiley Online Library, Cochrane Library, Google Scholar and grey literature), a review of health policy documents (completed in March 2023), and interviews with key stakeholders (by June 2023). Second, we will empirically test the initial programme theory using a multiple-embedded case study design in two provincial health administrations and four health districts (by March 2024). Data will be collected through document reviews, in-depth interviews, non-participant observations, a questionnaire, routine data from the health information management system and a context mapping tool. We will analyse data using the Intervention-Context-Actor-Mechanism-Outcome configuration heuristic. Last, we will refine the initial programme theory based on the results of the empirical studies and develop recommendations for policymakers (by June 2024).
Ethics and dissemination The Institutional Review Board of the Institute of Tropical Medicine and the Medical Ethics Committee of the University of Lubumbashi approved this study. We will also seek approvals from provincial-level and district-level health authorities before data collection in their jurisdictions. We will disseminate the study findings through the publication of articles in peer-reviewed academic journals, policy briefs for national policymakers and presentations at national and international conferences.
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- Health policy
- Human resource management
- Organisation of health services
- PUBLIC HEALTH
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- Health policy
- Human resource management
- Organisation of health services
- PUBLIC HEALTH
Strengths and limitations of this study
Realist evaluation is a theory-driven approach that seeks to open the ‘black box’ in order to explain how an intervention brings about outcomes in a specific context.
This context-sensitive approach fits well with evaluating complex interventions (such as capacity building interventions) within complex systems (such as health systems).
Potential limitations of this study include recall bias (for the designers of the intervention), social desirability bias, poor completeness of routine data and the risk of tunnel vision as data collection is guided by the initial programme theory.
Introduction
Most African countries did not achieve the health-related Millennium Development Goals and are facing challenges in the implementation of Sustainable Development Goals.1 Africa bears the highest burden of under-5 children’s death, maternal mortality ratio, HIV/AIDS, malaria and tuberculosis.2 Weak health systems are one of the leading causes of this situation.3 This includes weak leadership and management, resource constraints, reliance on poorly coordinated funding donors, limited public accountability, excessive centralisation of power, political instability and insecurity3 4 and low capacity to deal with dynamic environments.5
The Democratic Republic of Congo (DRC) is no exception to this situation. It is a post-conflict country with a fragile health system characterised by poor leadership and governance, weak integration, fragmented and inefficient financing and low resilience.6 7 As a result, the health outcomes of the population remain worrying. The maternal mortality ratio (846 per 100 000 live births) is one of the highest in Africa,8 and the under-5 mortality rate (70 per 1000 live births) remains high.9 The prevalence of HIV/AIDS is stagnating at around 1.2% since 2007,8 and the burden of malaria is the second-highest in Africa after Nigeria. The tuberculosis burden is the ninth-highest in the world.10 Besides this burden of infectious diseases, there are increasing non-communicable diseases. The prevalence of hypertension and diabetes in Kinshasa has been estimated at 11% and 15.5%, respectively.10
This situation calls for health system strengthening to meet the population’s health needs and move towards achieving the Sustainable Development Goals. However, while there is a clear diagnosis of the determinants of health system weakness in several countries, the approaches to address them are not evident.4 Among these approaches is decentralisation, one of the major reforms most sub-Saharan countries have been embarked on over the past four decades.11–14 It involves the transfer of responsibility for planning, management and decision-making from national to subnational level in order to overcome institutional, physical and administrative barriers to development.15–17
The DRC has been engaged in a decentralisation process since February 2006, after a constitutional reform.18 Within this framework, the Ministry of Health has developed the health system strengthening strategy, adopted in 2006 and revised in 2010.7 18 This strategy considers the development of the district health system as the ‘conditional priority for strengthening the health system’ in the DRC.18 It recognises the importance of developing the leadership and management capacities of the district health managers by the provincial health administration (PHA). It emphasises the necessity to reform the PHA to better respond to this mission within the perspective of decentralisation18 (box 1).
The reform of the provincial health administration in the DRC
The constitution of the DRC embraces the concept of decentralisation in 2006. The Ministry of Health subsequently developed the health system strengthening strategy, which called for reforming the national and provincial health administrations to allow them to better play their roles within the decentralisation context.
The PHA reform consisted of a triple reorganisation. First, the functional reorganisation involved separating the inspection and control function of the provincial level from technical support to the health districts.84 Second, the structural reorganisation aimed at transforming the organisational structure of the PHA to suit its role better. Previously characterised by the replication of central structures (65 directorates including vertical programmes), the PHA was restructured into two distinct structures: the provincial health inspectorate, a deconcentrated structure responsible for monitoring and controlling the application of the national’s norms and guidelines, and the provincial health division, a devolved structure whose primary function becomes the technical support to the health districts.84 In this paper, we will refer to the PHA as the provincial health division. Following positive results of an action-research conducted in the provinces of North Kivu and Oriental Kasai between 2008 and 2011, the PHA reform was scaled up to all the 26 new provinces of the DRC between 2014 and 2015. The structure of the PHA was reorganised around four core professions: the technical support to health districts, the health information, communication and research, the control and inspection and the management of resources.84 The new structure of the PHA office includes also working groups, which are ad hoc structures where PHA staff meet to discuss and reflect on specific issues, such as technical support to health districts, health information management, medicine supply to health districts, health financing coordination and epidemiological surveillance. Such an institutional arrangement is expected to enable synergy among staff, participation in decision-making and individual and collective learning.84 Third, the cultural reorganisation supposed a gradual shift from a hierarchical culture dominant in public administrations to an adhocracy culture better adapted to the complex nature of health systems.
DRC, Democratic Republic of Congo; PHA, provincial health administration.
Evidence suggests that leadership and management are crucial for improving the performance of health systems.3 19–25 The health districts need effective leadership and management. The health district is considered as the central element of a health system. The proximity of district health managers to their population enables them to adequately capture their needs and priorities and integrate them in health planning (‘bottom-up’ planning) and, at the same time, translate national policies and resources into effective services to meet population needs (‘top-down’ planning).26–28 Building the leadership and management capacities of district health managers is likely to improve the stewardship and performance of health districts.20 29–31 However, capacity building is a complex intervention20 32 33 and (still) remains an elusive concept.29 34 35
Capacity building programmes (CBPs) are inherently complex.29 36 They bring together various actors (policymakers, managers, providers, funders, patients and communities) from various institutions (national, provincial or district administrations, health facilities, community and funding agencies, etc).37–40 These actors have different values, norms, power and their objectives and expectations may be at odds. Health districts are complex, adaptive and open systems embedded in the larger (social, political and economic) environment with which they constantly interact.32 33 These interactions may lead to unpredictable and non-linear behaviours.40–43 This complexity explains why a CBP that works in one place may not necessarily work in another or may even not work anymore in the same place in a later stage.44
There is no consensus on the definition of capacity building among academics and practitioners. It is considered an umbrella concept associated with a range of (sometimes opposite) meanings.34 35 39 45–49 Some authors refer simplistically to training as capacity building,31 50 51 while others consider that capacity building should be a systematic approach.26 35 46 52 These different views of capacity building, ranging from the reductionist to the more holistic, result in practice in various approaches, models and tools.29 34 35 39 45 They also contribute to the methodological challenges associated with CBP evaluation49 53 and their effectiveness on organisational performance.34 46 54 55 Capacity building enables an individual or organisation to achieve its stated objectives.46 In the health sector, this includes improving the performance of health systems in terms of access, quality, equity and responsiveness.3 However, the link between a CBP and improved organisational performance is by no means straightforward. Several contextual factors shape this relationship at different levels. These factors include individual attributes (eg, motivation, knowledge, skills, self-confidence), organisational characteristics (eg, leadership style, nature of supervision, teamwork, organisational culture) and contextual factors (eg, political, economic, social).24 27
In the DRC, the health system strengthening strategy calls for the ‘technical support’ from PHA staff to district health management teams (DHMTs) members in order to strengthen their leadership and management capacities and subsequently improve the stewardship of their health districts as integrated systems, which offer accessible, equitable, quality healthcare and services responsive to the population’s needs. While in one province, this technical support was overall well appreciated by the DHMT members,56 the lack of a clear conceptual model to guide the operationalisation of this technical support was reported in another province.57 Moreover, to the best of our knowledge, no study has addressed how, for whom and under what conditions the technical support from the reformed PHA to the DHMTs works (or not) in the DRC.
Study objectives and research questions
Study objectives
The general objective of this study is to understand how, for whom and under what conditions the technical support from the reformed PHA to DHMTs, as a capacity building intervention, works to improve the management capacity of DHMT members and the health district performance in the DRC.
The specific objectives of the study are to:
Elicit the assumptions (ie, initial programme theory) underlying the technical support from the PHA to DHMTs.
Empirically test the initial programme theory through a set of substudies.
Refine the initial programme theory on the basis of the empirical findings.
Research questions
We will address the following research questions:
How is capacity building of district health managers being conceptualised, operationalised and evaluated in sub-Saharan Africa? What are the contextual factors that influence the capacity building process?
How is the technical support from PHA to DHMTs in the DRC supposed to work from the designers’ perspective? What are the conducive contextual conditions for this support?
How is the technical support from PHA to DHMTs implemented in the DRC? What are the contextual conditions that shape this implementation?
How do the DHMT members in the DRC perceive the technical support from the PHA staff?
What are the outcomes of technical support from PHA to DHMT in terms of management capacities and performance of health districts?
Methods and analysis
The methodological approach
The overall methodological approach of this study is realist evaluation.58 We choose this approach because of the complexity of both capacity building programmes and health districts. Realist evaluation goes beyond the ‘does it work’ question to understand ‘what works, for whom and under what conditions’.59–62 It seeks to open the ‘black box’ to explain how an intervention brings about outcomes in a specific context.60 63 This context-sensitive approach is a best fit for the evaluation of complex interventions.
Realist evaluation is based on scientific realism. It considers that the primary drivers of change are people. Social programmes do not bring about changes but rather offer resources and opportunities to people.58 Realist evaluation is theory-driven. It begins and ends with a programme theory. The programme theory is a set of assumptions explaining how the interactions between the intervention and actors within a given context are supposed to trigger the mechanisms that lead to the expected outcomes. Realist researchers use the Context-Mechanism-Outcome configuration as a heuristic tool to elicit the causal pathways underlying the observed outcome. In this study, we will use a finetuned variant of the traditional Context-Mechanism-Outcome configuration, the Intervention-Context-Actors-Mechanisms-Outcomes (ICAMO) configuration, to better differentiate intervention from context and emphasise the role of different actors in the change processes.64
We structure our protocol along the realist cycle (figure 1). We followed the RAMESES II guidance for reporting on realist evaluations,65 which we adapted to fit the format for a protocol paper (online supplemental table 1).
Supplemental material
The realist cycle.85 86
The intervention
In the DRC, the health sector reform has centred the role of the PHA around the technical support to the health districts. This support aims to strengthen the leadership and management capacities of the DHMT members and thus improve the health districts performance and ultimately the population health outcomes. The underlying assumption was that the development of district health systems depends on the leadership and management capacities of DHMT members.7 18 Indeed, the DHMT is in charge of managing the health district daily. The team’s leadership and management capacities determine the district’s performance. Therefore, it is essential to enhance the leadership and management capacities of DHMT members to enable them to carry out their responsibilities effectively. These capacities include coordination of stakeholders, planning and budgeting, monitoring and evaluation, hands-on training and supervision of health workers, health system information management, epidemiological surveillance, resources management (human, financial, material and medicines) and operational (or action) research.66 The technical support is expected to build DHMT members’ capacities through facilitative supervision, coaching and problem-solving support. It differs from the way supervision is usually organised in the DRC, that is, focusing on inspection and fault-finding.
The PHA reform assigned the technical support role to experienced PHA staff with public health and district management backgrounds. Their number depends on the number of health districts and the availability of qualified personnel at each PHA office. There is one PHA staff member for two to four health districts. PHA staff provide technical support to DHMT members through field visits. These visits are in principle based on problems identified by the PHA staff and/or raised by the DHMT members. Technical support visits are meant to take place on a quarterly basis, but their actual frequency depends on geographical accessibility and the availability of resources. Furthermore, the PHA reform established a technical support working group at the PHA office. It is an ad hoc structure where PHA staff meet to discuss and reflect on technical support issues.
Step 1: eliciting the initial programme theory
As a first step towards developing the initial programme theory (IPT), we conducted a scoping review and a policy document review. This will be followed by a series of interviews with key stakeholders.
The scoping review
In the first step, we carried out a scoping review analysis to understand how CBPs of district health managers in sub-Saharan Africa have been designed, operationalised and evaluated. The review focused on identifying the underlying assumptions or theories behind CBPs of district health managers. We searched five electronic databases (MEDLINE, Health Systems Evidence, Wiley Online Library, Cochrane Library and Google Scholar). The search strategies are outlined in the online supplemental table 2. We included all primary studies (a) reporting leadership or management capacity building of district health managers (b) in sub-Saharan Africa, (c) written in English or French and (d) published between 1 January 1987 and 13 October 2022.
Supplemental material
The scoping review indicated that the main assumption underlying most CBPs is that strengthening leadership and/or management capacities of district health managers would enhance their knowledge and skills, and therefore their competences to implement effective management and/or leadership practices in their workplaces. These enhancements would then improve health system performance (access, quality, equity and responsiveness) and ultimately improve the population’s health outcomes.
The scoping review pointed out six main features of an effective CBP for DHMTs: a learning-by-doing approach, an alternation of short workshops and on-the-ground follow-up visits through coaching, mentoring and supervision, a team-based approach, flexibility, supportive interactions between facilitators and participants and a long-term perspective. These intervention features can trigger a set of mechanisms (self-efficacy, mutual trust, perceived support relevance, openness to learning and a sense of ownership) that contribute to the intended outcomes in specific context conditions: adequate decision spaces of DHMTs, a supportive organisational culture, adequate resources, adequate support from the hierarchy and good stakeholder collaboration and involvement.
The scoping review also highlighted an increasing use of the action-learning approach as a capacity building approach. Action learning is grounded in Kolb’s experiential learning theory,67 which emphasises real-life experiences as a driver of learning. This theory posits a four-stage learning cycle: concrete experience, reflective observation, abstract conceptualisation and active experimentation. In this learning cycle, concrete experiences lead to reflective observations from which abstract concepts are developed and tested in new experiences. The experiential learning theory is close to two other learning theories which stress the importance of experience and reflection in the learning process: Knowles’ et al adult learning theory (or andragogy) and Mezirow’s transformative learning theory. Knowles’ et al adult learning theory underlines the centrality of self-direction, experiences (including errors), perceived relevance, problem-centredness and intrinsic motivation in the learning process.68 According to Mezirow’s transformative learning theory, learning occurs when people transform their values, beliefs and assumptions into new meaningful perspectives through critical reflection. It emphasises six core elements: individual experience, critical reflection, dialogue, holistic orientation, awareness of context and authentic relationships.69
Other lessons can be drawn from the scoping review: as the technical support is a learning process, PHA staff can act as learning catalysers within DHMTs. To do so, they need to be motivated and have the management, interpersonal and facilitation competencies to engage DHMT members in successful learning processes using the adult learning principles highlighted in the above three learning theories.
The policy document review
Following the scoping review, we reviewed the health policy documents. They were drawn from the Ministry of Health, supporting partner institutions and previous studies in the DRC. We included 21 documents (online supplemental table 3) based on their relevance, that is, documents that provide appropriate information related to PHA reform and technical support to health districts.70 This review provided us with general information about the implicit logic model of the technical support from PHA staff to DHMT members (figure 2).
Supplemental material
Logic model of technical support from provincial health administration to DHMTs. DHMT, district health management team.
Using a realist lens, we further thematically analysed these documents to identify any information about the intervention (policy), context, actors, mechanisms and outcomes (table 1).
Elements of ICAMO configuration from review of the policy documents
The preliminary initial programme theory
From the scoping review and the review of programme documents, we formulated the preliminary IPT as follows (figure 3):
If PHA staff [A] are well trained [I], have regular meetings [I] and are adequately rewarded [I], then their motivation, commitment and competencies [O] to provide regular and effective technical support to DHMT members will improve, because training improve their self-efficacy [M] as they become more knowledgeable and skilful, meetings enable their reflexivity [M] as they share, reflect on and learn from their experiences and rewards improve their commitment [M]. Contextual conditions for such changes to occur include an optimal functioning of PHA office [C] characterised by effective leadership, which creates a good work climate and promotes positive values, supportive working conditions, adequate resources, shielding of negative political influences and which achieves support from the national Ministry of Health and provincial political leaders.
If PHA staff are motivated, committed and competent [A], then they will provide effective technical support to DHMT members [A] through participative needs assessment, priority setting and goal setting [I], helpful information sharing [I], problem-solving support [I], stimulating meaningful reflection and providing constructive feedback [I]. Such support would increase the DHMT members’ motivation and commitment [O] to the technical support process and then improve their management competencies [O] because of the resulting positive perceived support [M]. Conducive contextual conditions include an enabling environment for learning [C] which is judgement-free, fault-accepting, non-threatening and less-hierarchical and supportive relationships between PHA staff and DHMT members.
If DHMT members have good management competencies [A], then they will improve the performance of their health districts [O] because of increased self-efficacy [M] and autonomy [M] in implementing sound management practices [I]. Conducive contextual conditions [C] include effective leadership, which creates a good work climate and promotes positive values within the DHMTs, optimal decision space, supportive working conditions, adequate resources and the absence of negative political influences.
The preliminary initial programme theory. PHA, provincial health administration; DHMT, district health management team.
Refining the preliminary initial programme theory
We will conduct 10–15 interviews with purposely selected stakeholders (policymakers, researchers and technical assistants) involved in the design of PHA reform in order to explore their assumptions and thus better understand how and why the intervention would bring about expected changes. We will use an interview guide with open-ended questions. The interviews will be recorded, transcribed verbatim and analysed thematically using ICAMO configuration as a heuristic. The end result will be the final IPT, which will be tested in the subsequent stage of the research.
Step 2: study design
We will conduct a multiple embedded case study, which fits dynamic and complex situations in which multiple actors and factors interact.71 We define the case as the technical support from PHA staff to DHMT members. The unit of analysis is the PHA and the subunit is the health district. Within each province, we will focus on (1) the actual implementation of the technical support, (2) the perceptions of DHMT members on this technical support, (3) the DHMTs’ management capacity, (4) the performance of health districts and (5) the contextual conditions required for effective and sustainable capacity development.
We purposively selected two PHAs (Tshopo and Kasai Central). This purposive selection was informed by the preliminary ICAMO elements, the principle of maximum variation and the extreme case sampling strategy as described by Patton.72 Based on these principles, we used the following criteria. (1) The performance of PHA as ranked by the Ministry of Health in its annual report 2021.73 According to this ranking, Kasai Central province has attained the top position for its very high performance, while Tshopo province has been ranked the lowest due to its low performance (box 2); (2) The presence of a funder (respectively, Enabel and World Bank). We have identified the availability of resources as a crucial contextual factor. In the DRC, external aid is the second source of health system financing (representing 37% of the current health expenditure) after the households who fund 43% of the current health expenditure.74 The presence of a funder greatly affects the functioning and performance of the PHA. (3) The geographical location in the country (in the east and centre, respectively). We have identified geographical accessibility as a contextual condition in table 1. It also allows to achieve maximum variation in the selection of units, as different provinces may have significantly different contexts, especially in a country with continental dimensions like the DRC. (4) The geographical access for the PhD candidate as this study protocol is part of a PhD study. Indeed, the feasibility in conducting research is also considered a common criterion to facilitate the data collection for the PhD candidate.
The assessment of PHAs’ performance by the Ministry of Health in the DRC
The Ministry of Health used a benchmarking approach to evaluate the performance of 26 PHAs in the country. Eleven indicators of the health information management system were used to measure their performance in 2021. These indicators include the rate of outpatient consultations, proportion of referrals at district hospitals, in-hospital mortality beyond 48 hours, in-hospital maternal death audited, four visits of antenatal care at 36 weeks, BCG vaccination coverage, vaccination coverage against measles, DTC-HepB-Hib 3 vaccination coverage, completeness, timeliness and data quality score of DHIS2 reports. The Ministry of Health classified PHA performance as very high (score 90–100%), high (80–89%), average (60–79%), low (50–59%) or very low (<50%). The PHA offices also assess the performance of their health districts in a similar manner.
PHA, provincial health administration.
Similarly, in each province, we will select two health districts based on the following criteria: (1) the health district location (urban and rural). We chose this criterion to ensure the maximum variation and to capture the specificities of urban and rural settings that may influence the technical support process and the performance of health districts; (2) the performance of health district as ranked by the PHAs in their annual report 2021 (high and poor) using the benchmarking approach (box 2) and (3) the geographical access for the PhD candidate for the same reasons as mentioned above.
Step 3: data collection
We will collect data through document review, in-depth interviews, non-participant observations and questionnaire. We will also collect routine data from the health information management system and use a context mapping tool.
Document review
A document review will allow us to analyse the technical support process (research question 3) and to assess the DHMTs’ management capacity (research question 5). All paper and electronic documents (technical support reports, annual reports, meeting minutes, internal evaluation reports, external independent evaluation reports (if any), etc) available at the PHA and health district offices will be screened.
For research question 3, we will gather data related to the actual implementation of the technical support: preparation, frequency, duration, intensity, content, tools, quality process, actors, contextual factors, etc.
For research question 5, we will use a scorecard for assessing the DHMTs’ management capacity. This scorecard will be developed and piloted in collaboration with PHA cadres to make sure that all included items are relevant to the context of the Congolese health system. It will include items from each core management function assigned to the DHMTs in the DRC: coordination of stakeholders, planning, monitoring and evaluation, supervision of health workers, hands-on training of health workers, health information management, epidemiological surveillance, resources management (human, material, financial and medicines) and research. For each item, we will assign a score from 1 to 4 (1: very poor, 2: poor, 3: good and 4: very good) according to the established criteria.
In-depth interviews
We will carry out in-depth interviews with purposively selected PHA staff and DHMT members to collect data related to the implementation of the technical support (research question 3), the perceptions of the DHMT members about the technical support (research question 4) and the outcomes of the technical support (research question 5). Based on our IPT, we will refine, pilot and use interview guides for PHA staff and DHMT members (online supplemental material 1). Interviews will be held at the preferred times and places of the respondents.
Supplemental material
Non-participant observations
In addition to thein-depth interviews, we will conduct non-participant observations to collect data related to the technical support process (research question 3). Where possible, We will observe the technical support working group meeting at the PHA offices and the technical support visits at the health districts. Using an observation guide, we will collect data related to the nine social situations’ dimensions defined by Spradley75: space, actor, activity, object, act, event, time, goal and feeling.
Questionnaire
We will use a questionnaire administered to all DHMT members in the selected health districts (exhaustive sampling) to assess their perceptions about the process and outputs of technical support from the PHA cadres (research question 4). Based on our IPT, we will refine a questionnaire (online supplemental material 1) with 5-point Likert scale (1=strongly disagree, 2=disagree, 3=neutral, 4=agree; 5=strongly agree) and pilot it before use.
Routine data from health information management system
We will collect routine data from health information management system in DHIS2 software and other relevant sources (PHA or districts’ annual reports) to assess the health district performance during the last 5 years (research question 5). In collaboration with the PHA cadres, we will develop another scorecard with key performance indicators related to the access, quality and equity of healthcare and services. For each item, we will assign a score from 1 to 4 (1: very poor, 2: poor, 3: good and 4: very good) according to the established criteria.
Context mapping tool
At each PHA and health district, we will develop and use a context mapping tool in order to document key contextual issues that may influence the provincial technical support to DHMTs (research question 3). These issues include organisational culture, organisational structure, decision spaces, availability of resources, key stakeholders and events and their influence on the technical support, etc.
Step 4: data analysis
Realist evaluation is method neutral, that is, it follows the guidance of the disciplines from which the used methods originate. In general, the ICAMO configuration serves as the analytical heuristic.
We will use R software to manage quantitative data from the questionnaire and the health information management system. Analysis will be essentially descriptive (percentage, means or medians with 95% CI). We will use NVivo software to manage the qualitative data from documents reviews, interviews, field-notes and memos. In line with realist principles, we will thematically code data based on core elements of our IPT. We will perform a framework analysis76 as it allows for both a priori and emergent concepts.
In the first round of analysis, we will classify data into categories corresponding to the ICAMO configuration: intervention, context, actors, mechanisms and outcomes. In the subsequent rounds of analysis, we will adopt a retroductive approach in order to identify the link between the observed outcomes and the other elements of the ICAMO configuration. Retroduction in realist approach means ‘going back from, below, or behind observed patterns or regularities to discover what produces them’.77 This starts from identifying the outcomes (both intended and unintended, positive and negative), and finding out how the actual intervention triggered mechanisms among the actors in specific context conditions, which contributed to the outcomes.
In order to enhance the internal validity of our study’s findings, we will use various kinds of triangulation described by Patton,72 including (1) the method triangulation by using a mixed methods, (2) theory triangulation by constant searching for rival theories, (3) triangulation of data sources through a scoping review, a document review, in-depth interviews, a questionnaire and routine data from DHIS2, (4) triangulation between study sites, such as between contrasting PHAs or health districts, (5) expert audit review in order to obtain insightful feedback from supervisors, colleagues, peer reviewers, etc and (6) respondent validation.
Step 5: synthesis
We will summarise the findings of each province in reports. Subsequently, we will carry out a cross-province analysis. To do so, we will compare their ICAMO configurations in order to identify similarities or differences and explore how they challenge, refute or confirm the IPT. This will result in a refined programme theory, which provides plausible explanations of how the actual intervention leads to the observed outcomes by activating underlying mechanisms in (a group of) actors under specific contextual conditions. This theory will be discussed with relevant stakeholders for further fine-tuning.
Table 2 provides a summary of the research process (study objectives, research questions, data collection and analysis).
Summary of research process
Patient and public involvement
There is no patient or public involvement in this study.
Ethics and dissemination
We obtained ethical approvals for this study from the Institutional Review Board of the Institute of Tropical Medicine, Antwerp and the Medical Ethics Committee of the University of Lubumbashi. Approvals from provincial-level and district-level health authorities will be sought and obtained before data collection in their jurisdictions. We will conduct this study following the principles of the Declaration of Helsinki as amended in 2013.78 Written informed consent will be obtained from each participant in the study prior to data collection. We will pseudonymise all quantitative and qualitative data by giving each participant a unique personal identifier to ensure confidentiality. All research data will be stored in a password-protected drives accessible only by the research team members.
We will disseminate the study findings through publication of articles in peer-reviewed academic journals, development of policy briefs for national policymakers and delivering presentation at national and international conferences.
Discussion
The health sector reform in the DRC has assigned the PHA the central role of supporting the development of health districts as integrated systems offering comprehensive, continuous, coordinated care adapted to the population’s needs. This role, known as technical support, is carried out by PHA cadres with public health and district management backgrounds and is expected to strengthen the management capacity of DHMT members and improve the performance of health districts. Our study adopts the realist evaluation approach to provide an explanatory account of the mechanisms triggered by the technical support from PHA cadres to improve (or not) the management capacities of DHMT members and the performance of health districts within the specific context of the DRC. This explanatory account will contribute both to policymaking and scientific research. It corresponds to Schmitt’s two-pronged causal mechanism claim: increased policy relevance and strengthened causal capacity.79
On the one hand, the results of this study will inform policymakers on possible adjustments to the ongoing PHA reform in the DRC, thus contributing to the promotion of (more) evidence-informed health policymaking. Furthermore, this research is in line with the health system strengthening strategy,7 18 which recognised the health system’s complexity and emphasised the importance of health systems and policy research. On the other hand, the refined programme theory from this study will provide insights for subsequent realist evaluations of capacity building of DHMT members in the DRC and other contexts, thus contributing to cumulative theory development80 which may gradually result in a middle range theory.81 The lessons learnt from this evaluation will feed into methodological discussions to improve the quality of evaluations of complex interventions.
While designing this study, we encountered some challenges. The first was related to the development of preliminary IPT. Indeed, the documents of the health sector reform in the DRC are quite evasive on what can be understood by technical support to DHMT members and the causal pathways by which it improves their management capacities. This may be explained by the fact that the technical support is not a specific programme but a part of a larger portfolio of interventions to be carried out to strengthen the health districts. Moreover, the document review showed that the official documents address the context conditions in a rather generic way. They point for instance to the importance of a conducive organisational culture at provincial and district level, without stating the current situation and critically analysing how the current prevailing non-conducive elements can be addressed. The scoping review provided us with insights into the features of effective CBPs and the set of mechanisms that can, in a given context, lead to the intended outcomes. However, the scoping showed that few studies were carried out in fragile settings, comparable to the DRC, which raises questions about the context conditions required for capacity development processes to work. Second, methods and tools to measure the DHMTs’ management capacities and the performance of health districts are not standardised. We will involve PHA staff in designing scorecards to ensure that the included items are relevant to the development of health districts in the context of the DRC. Beyond increasing the research relevance, this co-construction of data collection tools between the researchers and key health actors may boost their participation in the research and uptake of its findings.82
In conducting this research, we are aware of potential limitations. These include the recall bias, particularly for the designers of the intervention (which is over 5 years old), the social desirability bias, and poor completeness and quality of data because of poor archiving of documents at both PHA and health district offices, as well as insufficient encoding of routine data in the DHIS2 software. We will minimise these limitations and enhance the research credibility through the different triangulations proposed by Patton.72 Another limitation is inherent in the realist interview technique. The fact that the IPT guides data collection may expose researchers to the risk of tunnel vision, that is, the tendency to look only for the elements that corroborate the IPT.83 To reduce this risk, we will constantly search for rival theories or alternative explanations throughout the research process.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @BosongoS, @drbelrhiti
Contributors SB, ZB and BM conceptualised the study protocol. SB drafted the initial manuscript. SB, ZB, BM, FC and BC contributed to manuscript revision. All authors read and approved the final version of this manuscript.
Funding This work was supported by the Directorate-General Development Cooperation and Humanitarian Aid, Belgium, in collaboration with the Institute of Tropical Medicine, Antwerp, as a part of the doctoral programme of SB, grant number 911063/70/130. The funder had no role in the whole process of the review from the design to the publication.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.