Article Text
PDF
Abstract
Objectives This paper presents the iterative process of participatory multistakeholder engagement that informed the development of a new national tuberculosis (TB) policy in Georgia, and the lessons learnt.
Methods Guided by realist evaluation methods, a multistakeholder dialogue was organised to elicit stakeholders’ assumptions on challenges and possible solutions for better TB control. Two participatory workshops were conducted with key actors, interspersed by reflection meetings within the research team and discussions with policymakers. Using concept mapping and causal mapping techniques, and drawing causal loop diagrams, we visualised how actors understood TB service provision challenges and the potential means by which a results-based financing (RBF) policy could address these.
Setting The study was conducted in Tbilisi, Georgia.
Participants A total of 64 key actors from the Ministry of Labour, Health and Social Affairs, staff of the Global Fund to Fight AIDS, TB and Malaria Georgia Project, the National Centre for Disease Control and Public Health, the National TB programme, TB service providers and members of the research team were involved in the workshops.
Results Findings showed that beyond provider incentives, additional policy components were necessary. These included broadening the incentive package to include institutional and organisational incentives, retraining service providers, clear redistribution of roles to support an integrated care model, and refinement of monitoring tools. Health system elements, such as effective referral systems and health information systems were highlighted as necessary for service improvement.
Conclusions Developing policies that address complex issues requires methods that facilitate linkages between multiple stakeholders and between theory and practice. Such participatory approaches can be informed by realist evaluation principles and visually facilitated by causal loop diagrams. This approach allowed us leverage stakeholders’ knowledge and expertise on TB service delivery and RBF to codesign a new policy.
- health policy
- tuberculosis
- qualitative research
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Participation of a wide range of stakeholders provided a reality check to the initial plans of policymakers and the evaluation design of the researchers.
Concept mapping and causal mapping strengthened the participatory process of policy formulation.
Applying realist methodology further informed policymaking by stimulating a reflection on the influence of the policy context.
A study limitation was the lack of inclusion of participants without a biomedical or health programme background.
Absence of consistency in attendees from the first to the second workshop limited continuity of the reflection process.
Introduction
Although Georgia has made significant progress in controlling tuberculosis (TB), with reported success rates of 83% for new and relapsed TB cases, some challenges remain, including high rates of drug-resistant TB (DR-TB) (11% incidence in new patients and 31% in those previously treated) and relatively low cure rates for DR-TB.1 Following the collapse of the Soviet Union, several waves of health system reforms were implemented in Georgia from 1994 onwards. These led to a provider–purchaser split whereby the Ministry of Labour, Health and Social Affairs (MoLHSA) took on the role of priority setting, policy formulation, programme development and funds allocation,2 while the actual TB programme delivery was relegated to private sector providers. During Soviet times, these functions were administered through the highly centralised Semashko system; healthcare personnel were employed by the state and were redistributed on a hierarchical basis at district, regional and republican levels. The system was tax-based with a guiding principle of granting the population universal access to healthcare free of charge, although illegal out-of-pocket payments were extremely common.2 In the aftermath of the Rose Revolution (2003–2007), healthcare delivery was further privatised and many disease control components were transferred to a diverse range of private providers, from small rural healthcare facilities to networks of healthcare services.3 Although the National TB programme remained responsible for policies, programmes, surveillance and monitoring, TB control and service delivery activities were effectively devolved to private providers.
In 2015, health authorities and TB control programme officials of Georgia, a country with 3.7 million inhabitants in the Caucasus region, initiated the process of developing a new policy to improve the performance of the TB control programme. Policymakers identified the poor integration of TB services at primary care level and low remuneration of service providers as barriers to effective care and treatment of TB patients.4 Reports of low coverage and poor adherence to treatment were also linked to low interest of private providers in sustaining TB service provision. Poor motivation of TB professionals further exacerbated the challenges.5 6 In response, through a concept note submitted to the Global Fund to Fight AIDS, TB and Malaria (GFATM), the Government of Georgia articulated its intention to pilot a results-based financing (RBF) scheme to motivate providers and improve TB service delivery in private settings.
Although the concept note indicated the Government’s readiness to test this new financing model, it provided little clarity on its design or on the evaluation model to be used. In response, in March 2017, the 4-year RBF for TB care (Results4TB) project was launched as a collaborative project between researchers and national policy makers. This Economic and Social Research Council/Medical Research Council/Department for International Development/Wellcome Trust-funded project brings together policymakers, TB programme managers and national and international researchers to inform the policy formulation, document and evaluate its implementation and assess its cost-effectiveness. The project intends to introduce monetary incentives for health providers, rolled out alongside existing incentive schemes for patients, to improve medication adherence and as such improve treatment success rates for both drug-susceptible and multidrug-resistant TB patients.
The Results4TB project adopted a participatory, codesign approach to developing the study protocol that incorporates an impact evaluation, a cost-effectiveness analysis and a realist research component. We refer to Marchal et al7 for the protocol of the realist component, Chikovani et al for the trial protocol8 and to the project’s website9 for further information.
Since the publication of guidance on evaluation of complex interventions by the MRC in 2000 and of revised guidance in 2008,10 trial studies often include implementation fidelity evaluations, process evaluations and/or context mapping exercises. These are often adopted in trials in an effort to enhance understanding of the actual implementation processes and the influence of context on the observed outcomes.11 In 2015, the MRC issued guidance on process evaluations of complex interventions,11 which was followed by several studies that combined realist process evaluations with trials12–14, as well as by calls to go a step further and develop theory-informed trials.15 16 One innovation the Results4TB project pursues is to test how theory-informed trials can be designed and implemented.
Realist evaluation (RE) is a theory-based approach that is recognised for its usefulness in informing evidence-informed policy and practice.17 Based on a realist philosophy of science, it responds to a need for actionable evidence for decision-making. Rather than merely confirming if policies and programmes work, realist inquiry identifies ‘what works in which circumstances and for whom?’.18 RE is increasingly used in the field of health policy and systems,19 for instance in the study of policy implementation,20 leadership and management,21 22 clinical decision-making support,23 adherence to treatment,24 and infection control.25
Besides including a realist research substudy, the Results4TB project adopted the RE perspective to inform the overall study design. A premise of the project is that this would have a number of advantages. First, the development of a detailed hypothesis in the form of an initial programme theory (IPT). The IPT can best be described as a detailed hypothesis that specifies which mechanisms a programme is expected to trigger among different groups of actors in order to achieve results. It also hypothesises the context conditions under which the programme would work. The IPT thus explains how the programme is expected to generate outcomes and in which conditions.18
In the case of Results4TB, we expected RE would stimulate researchers to develop a shared understanding of the policy, its objectives and the way it would work. This in turn would facilitate the integration of project sub-studies on its impact, cost-effectiveness and the underlying processes of implementation and evaluation, contribute to leaner data collection and a more integrated data analysis strategy.26 Second, the RE approach can stimulate participation of key actors from the very start: eliciting the assumptions of the actors would both enrich the researchers’ understanding of the policy, and also establish strong links with the commissioners, designers, implementers and beneficiaries of the new policy. This engagement would allow existing tacit actor knowledge, evidence and scientific analysis to feed into the policy formulation, leading to a realist formative process.27 Through these processes, critical issues and underdeveloped elements can be identified, leading to better intervention planning or implementation, and indirectly to better programme uptake.28 29 Such participatory processes can also help in better designing the evaluation by making the assumed causal pathways explicit and identifying potential intermediate variables and interim outcomes that may help in monitoring proximal and distal outcomes.30 In the field of theory-driven evaluation, group model building techniques or other methods to jointly develop impact pathways or logic models are frequently used.31 32 To the best of our knowledge, there is little documentation of how such participatory processes are being applied in REs and perhaps even less for simultaneously informing policy formulation and evaluation design within the framework of a trial.
In this paper, we present the iterative process of participatory multistakeholder engagement that we organised at the start of the project to elicit the IPT of the RBF policy. We also discuss the lessons learnt and compare our experience with other projects.
Methods
The first step in any realist study is eliciting the IPT. There are generally three ways to elicit the programme theory (figure 1). First, in line with the participatory outlook of RE, the implicit assumptions of the key actors about how the programme is supposed to work can be unearthed. Understanding these assumptions is important since they influence the choices, design and implementation of a programme.18 Second, literature reviews can be carried out to identify existing evidence, concepts and theories.29 Third, if there is no or little documented evidence, exploratory research can be carried out in lieu of a literature review.33
Methods to develop the initial programme theory.
In this project, we elicited the assumptions of key actors regarding the problem of the ‘poor’ performance of the TB programme through a policy document review, concept mapping workshops and interviews with policymakers (figure 1).
Document review
We first collected and reviewed National TB Programme regulatory documents, the Strategic Plan for TB control for 2016–2020, TB programme guidelines, WHO mission reports and study reports on TB. The document review focused on the identification of key components of the existing TB control policy in Georgia, its objectives and beneficiaries and on how the TB programme operates throughout the country.34 This resulted in a preliminary root cause analysis of the problem.
First concept mapping workshop
We conducted two concept mapping workshops with key actors from the MoLHSA, staff of the GFATM Georgia Project, the National Centre for Disease Control and Public Health (NCDC), the National TB programme and with TB service providers. Participants were grouped in a way that ensured stakeholder representation, and that facilitated a ‘confrontation’ of different actors’ perspectives (service managers, providers, programme managers, policymakers, researchers, representatives of funding agency).
In the first workshop, in May 2017, six members of the Georgian research team facilitated the group dialogues. Table 1 presents the background of the 31 participants.
Overview of workshop participants
Participants were taken through three steps: (1) identifying the key elements of the problem (identification and analysis of the bottlenecks); (2) identifying the solutions (pathways of change) and (3) identifying the priorities for TB control. Figure 2 presents the conceptual framework, adapted from Vassall et al,35 that was used to guide the group dialogues. Also see online supplemental file 1.
Supplemental material
Conceptual framework (adapted from Vassall et al).35
Interviews with key respondents
After the first workshop, we carried out unstructured interviews with nine key stakeholders, including policymakers, funders, implementers and programme managers involved in the policy formulation. The main objective was to validate the issues raised during the first workshop and to identify potential solutions. Although no formal interview guide was used, the facilitation probes from the first workshoponline supplemental file 1 guided this informal inquiry (online supplemental file 1). Essentially, we aimed to explore their ideas and assumptions regarding different aspects of the policy. For example, on the need to improve collaboration between TB doctors and family doctors to encourage an integrated care approach, the distribution of roles in the proposed integrated care team, the policy verification system and the data availability for tracking process and outcome indicators.
Questions on TB and integration into primary healthcare (PHC) inquired on side effects and management of comorbidities—which have been raised as important issues affecting overall TB treatment care success rates. Respondents shed light on challenges to service integration, opportunities for improvement and the role of various actors at the different levels of care. Questions on intervention verification aimed to gain insight on existing health information systems collecting TB programmatic data, their weaknesses and opportunities for addressing these, including how to deter fraudulent practices within the system.
Due to the position of the respondents, interviews were not recorded nor systematically transcribed. Rather, notes were taken to inform the revision of the intervention description. Based on the information collected through these discussions, the intervention design was updated, documented, and later on presented to the second workshop participants for further discussion with a wider group of stakeholders.
The second workshop
The second workshop was organised in August 2017 and was facilitated in the same manner (see online supplemental file 2). It was attended by 28 representatives and five research team members, split in two groups. Of those in attendance, five persons in each of the two groups had not attended the first workshop. These included a representative from the Social Service Agency (the State service purchaser under the MoLHSA) and family doctors and managers from primary care facilities (see table 1). During this workshop, participants were asked to analyse the main elements of the proposed new national TB policy—which had been redrafted after the first workshop, focusing on (1) the policy’s causal pathway, (2) the underlying processes of change (ie, mechanisms) and (3) the required context conditions.
Supplemental material
Analytical strategy
After the first workshop, we organised a preliminary analysis meeting involving the research team, following which results from the workshop were summarised into tables. As mentioned above, we adopted a realist approach. RE offers the researcher a methodological approach that allows the discernment of how interventions work by exploring mechanisms underlying outcomes and the contextual conditions in which these occur. We used concept mapping and causal mapping techniques36 37 to analyse how the actors presented their understanding of the challenges of TB service provision and the potential mechanisms or means by which the policy could address these. Our exploration of mechanisms focused on those underlying the uptake of the policy by providers and managers, assuming that these two groups of stakeholders would have a major influence on actual policy implementation. Both intrinsic and extrinsic motivation drivers were explored—the latter including remuneration, staffing levels and general working conditions. ‘Context’ was represented by facility-related factors (eg, scale, resource availability, geographical location, quality of care provided) and broader cultural and social factors. Results from the interviews and the document review were used to triangulate results of the workshop. The research team drew on their collective expertise in TB programmes, RBF, clinical care, the Georgian context and health systems. This analysis resulted in causal loop diagrams (CLD) structured to aid visualisation of the patient pathway for TB care within the health system (figure 2). These initial diagrams were reiteratively refined during extensive discussions within the research team and subsequently framed as an IPT (see results section).
After the second workshop, a similar process was carried out, leading to a refined CLD, a reformulated initial programme theory and a new policy package proposal (see results section). Figure 3 depicts the various processes that were combined to develop the IPT of the TB policy and the policy package proposal.
Stages of eliciting the initial programme theory and development of the policy package.
Patient and public involvement
Public engagement was critical to the redesign of the government policy to improve TB care, because it would give room for the issues experienced by various actors in the health system to be voiced and potentially addressed. While TB patients were not involved in the workshops, there was representation from civil society. The policy formulation workshops allowed for confrontation of perspectives between different actors including service managers, providers, programme managers, policymakers, researchers, representatives of funding agency without whose input there would not have been a revised and robust policy.
Data sharing agreement
No additional data available.
Results
In this section, we present the results of the concept mapping workshops and interviews with key respondents, and how these contributed to the formulation of the IPT and the policy package.
The first workshop
During this workshop, participants first identified the current bottlenecks and problems with the National TB programme. These included low patient awareness, which delays patients seeking care, the high cost of the first visit for patients if they are not properly referred by a family doctor, inappropriate management of the side-effects of the TB treatment and associated high costs, the complexity of the directly observed therapy regimen, as well as inadequate coordination between providers (ie, TB doctors and family physicians) for the management of comorbidities/side-effects and the efficient use of services covered under the Universal Health Coverage programme in this process (table 2)
Current bottlenecks and problems, and solutions proposed during the workshops
In the second phase of the workshop, participants moved from bottleneck analysis to the identification of solutions they thought would need to be integrated in the new policy package. The following solutions were identified: better integration of TB care in general health services, training of first-line providers and specialists, institutional and team-based incentives instead of individual incentives, better patient management information systems and motivating poorly remunerated and ageing TB providers (table 2).
After the workshop, we identified the concepts mentioned or referred to by the participants in the discussions. We mapped these on the framework we adapted from Vassall et al35 (figure 2), adding the layer of ‘facility managers’ and ‘providers’. Next, we expanded this concept map into a CLD, which allowed us to identify context factors, potential negative effects and causal relations. Finally, we ‘pulled’ the diagram to a higher level of abstraction by grouping the causal loop nodes into the categories of policy, actors, context factors, outcomes and mechanisms. The resulting IPT indicated where the new policy would interact with the clinical pathway and how it could contribute to the TB programme outcomes of treatment success, treatment failure and loss to follow-up. Unsurprisingly perhaps, we found that the policy as proposed did not address any demand-side issues (figure 4).
The initial programme theory resulting from the first workshop. TB, tuberculosis.
The narrative version of the IPT is presented in box 1.
Narrative version of the initial programme theory (first workshop)
Incentives targeting the organisational level would incite service managers to increase the motivation of their providers, while training would improve the latter’s competences.
Incentives for providers would increase their extrinsic motivation but could lead to crowding out their intrinsic motivation.
All in all, a change of provider practices would occur, which in turn would affect how patients would take decisions regarding care for their health problem. Their decisions would also be influenced by context factors such as accessibility of the facility, and the acceptability and affordability of tuberculosis care.
Potential negative effects include gaming, fraud and crowding out of intrinsic motivation, which would be influenced by the degree of supervision, control and regulation.
The results of the first workshop informed the policy formulation process at the national level. The Curatio International Foundation (CIF) team and international team members were involved in drafting the intervention concept and held several meetings with representatives of the NCDC and the MoLHSA. In essence, the research team argued that there was a need for a comprehensive package, including performance-based incentives for teams as opposed to incentivising only individual providers, a reallocation of roles and responsibilities of healthcare providers (including family doctors) in the provision of TB care, the implementation of new tools (including patient management plans) and training for specialists on the management of side effects. These elements were taken into account by the policymakers. The inputs from the first workshop thus contributed to the formulation of a new TB policy package (figure 5).
Changes in the policy package after the first workshop. TB, tuberculosis.
Second workshop
At the start of this workshop, the CIF team presented the updated policy package, its objectives and the implementation modalities. The participants first discussed how they expected the policy would result in better TB control in the long term. To do so, they discussed the policy’s causal pathway, starting from components of the policy package and from the beneficiaries to expected effects and potential negative effects. Where opinions diverged, workshop facilitators stimulated further exploration of the issue. Participants proposed some modifications of the incentives package, for instance, calling for a refined bonus system. However, they did not challenge the main thrust of the proposed policy package.
In a second step, participants were asked to identify the mechanisms of change (ie, the processes that explain why a change would occur at the level of providers and patients, the healthcare facility or the health system), the essential conditions for the policy to succeed, the contextual constraints and finally, the conditions necessary for a sustained effect. Box 2 presents the key points raised by the stakeholders.
Summary of key points raised during the second workshop
Every tuberculosis (TB) patient should be assigned to a primacy healthcare (PHC) facility with an integrated TB unit where the patient will receive general care from a family doctor as well as TB care by a TB specialist. While this could create geographical barriers for some patients, it would ensure an integrated approach.
An integrated patient-centred TB care team should link patients to social services as needed.
The role and functions of rural doctors should be well defined and different bonus payment schemes should be created for them.
Incentives for family doctors should be a fixed top up to their salary as opposed to incentives per patient.
Facilities with low numbers of patients need stronger incentives than facilities with a higher patient load.
Managers of health facilities should ensure adequate service provision (eg, laboratory investigations, consultant visits) through contracts with other PHC facilities if they are unable to offer these services on site.
The RBF policy and intervention should be incorporated as a component of the National TB Programme.
The TB state programme budget should be revised to ensure adequate funding for routine tests and investigations. Mechanisms for distributing drugs to manage the side-effects of TB treatment should also be provided for within the TB programme’s budget.
The second workshop resulted in fine tuning the policy package (figure 6).
Changes in the policy package after the second workshop. TB, tuberculosis.
Final analysis
In terms of research, the results from the second workshop were incorporated and mapped onto the initial CLD (figure 7). This exercise helped us in formulating a final version of the IPT (box 3).
The initial programme theory resulting from the second workshop. DR, drug-resistant; DS, drug-susceptible; TB, tuberculosis.
Narrative version of the reformulated initial programme theory
Financial incentives will stimulate facility managers to (1) continue providing TB services, (2) remit incentives to TB providers within their facilities and (3) optimise their TB service activities (mechanism of organisational self-interest), on the condition that incentives are considered to be adequate and that sufficient monitoring and regulation systems are in place, in the absence of which gaming and fraud could occur.
Incentives for providers will increase their extrinsic motivation (mechanism) and if the staffing and working conditions are adequate (contextual prerequisite) they will contribute to better care for TB patients (outcome). Motivated and competent providers will provide better adherence support to patients, if supervision, monitoring and regulation are adequate. If not, crowding out of intrinsic motivation and gaming may occur (unintended consequence).
Training for providers (eg, on side effect of drugs, management of comorbidities and patient-centred care as well as on how the RBF scheme will work) will enhance their competence, which leads to higher self-efficacy (mechanism). The latter, combined with an increase in extrinsic motivation, contributes to behaviour change, which will enable patients to better adhere to the treatment (outcome).
TB patients who are informed, motivated and supported by providers, and trained to develop the needed skills, will initiate adherence if the social and cultural context is favourable (eg, absence of stigma), if the facility is accessible, acceptable and affordable and if they consider the treatment regime as acceptable.
The integration of TB care will improve the continuum of care between general and specialised TB services and thus facilitate the patient to correctly follow the treatment and care pathway towards being cured.
TB patients who experience positive effects, feel capable of managing their health condition (mechanism of self-efficacy) and feel in control of decisions about their care are more likely to maintain long-term adherence, on the condition that they consider the treatment regime to be acceptable.
Analysis from the second workshop also led to revision of some elements of the policy package following discussions with the policymakers. These included defining functions of each type of provider involved in the scheme, reporting requirements, distribution of bonus payment scheme between team members, monitoring and verification. The process was followed by corresponding changes to the design of the Results4TB pilot phase, the implementation manual and other supporting documents (eg, guidelines for Results4TB programme managers, reporting forms to the Social Service Agency, the patient management plan).
Discussion
In this paper, we set out to report the process of eliciting the IPT of a new RBF policy that the Georgian authorities hope will address the persisting burden of TB in Georgia and improve service delivery. The Results4TB project took the opportunity of being asked to participate in the formulation and evaluation of the new policy to develop and implement a theory-informed trial design. Our assumption was that applying RE to the process from the very start would both contribute to develop a shared understanding of the policy among the researchers (and thus a theory-informed study design), and also allow us to identify the assumptions of the key actors, including the research team, and thus inform the policy package leveraging the strengths of diverse stakeholder perspectives.
The workshops proved helpful in informing the policy content and implementation design. The active involvement of key stakeholders allowed tacit knowledge and expertise on TB service delivery and RBF from a range of stakeholders to be leveraged in codesigning the policy. Similar benefits of a participatory approach have been reported in health and non-health policymaking processes.38 39 Similarly, the workshops allowed for a bottom-up problem analysis whereby participants working at operational level provided feedback based on their daily experience.38 Workshop participants indicated that beyond provider incentives, additional policy components would be necessary to achieve desired goals. This included training of service providers to increase family doctors’ awareness on TB case detection, organising proper referral to TB services, better management of side-effects, a redistribution of roles between the PHC and TB specialised services and between care team members to support an integrated care model, training of integrated care team members on the principles of the new policy, creating a ‘patient card’ to increase patient involvement in TB care management and refinement of policy monitoring tools. These were issues not previously considered and could have led to programme failure if unaddressed.
To facilitate discussions during the first workshop, we adopted the patient pathway of Vassall et al,35 which proved useful in structuring the analysis. Participants were able to identify crucial issues which did not appear in the framework, including service organisation gaps, MoLHSA programme issues and social support—all factors impeding the delivery of TB care which the proposed policy would need to take into account.
CLDs were drawn with the aim of integrating findings from the workshops and the ideas of the researchers between the first and second workshops. We used CLDs to develop a graphical representation of the patient and health system pathways and the barriers and facilitators to the policy, more than to identify feedback loops as was done for instance by Renmans et al40 and Kwamie et al.22 The CLDs demanded systematic in-depth discussions among the researchers and stimulated them to make their assumptions explicit, which has been mentioned as a key strength of CLD development.41 42 We also used the CLDs to identify key health system elements for successful TB care and used them to identify specific factors that would drive TB service integration at various levels. A similar use of CLDs was reported by Renmans et al.40 While we did not (yet) exploit the CLDs to their full potential, the CLD development ultimately contributed to developing a more refined IPT and gave a visual overview that all stakeholders could appreciate and contribute to.
The research process we adopted is an example of embedded research. A senior researcher of the CIF team for instance played the dual role of being senior researcher, and a member of Parliament of Georgia and head of its Health and Social Issues Committee. Embedded research arguably facilitates the uptake of evidence in health policymaking.43 In our case, close interaction between policymakers and researchers helped both to refine the research design and also the actual policy. However, other factors shaped the actual formulation of the policy. The workshops, for example, played a critical role in identifying existing challenges in TB care management throughout the patient pathway from a bottom-up perspective. The first workshop led to the ‘discovery’ of contextual factors influencing TB treatment outcomes that were unique and/or specific to the Georgian context and which we did not identify in the literature reviews. One such factor is the lack of information sharing between general and specialised TB service providers. The second workshop contributed to examining the operational constraints to a successful policy implementation (eg, difficulties to develop a monitoring system caused by the non-functional TB electronic reporting module) and propose appropriate solutions. Embedded research has a number of conditions of success including the need to have a flexible approach and the ability to navigate contextual realities, including constraints.44
By taking a participatory approach, both workshops created an atmosphere where the inputs of all participants were valued and acknowledged as critical for designing the new TB policy. This approach eventually resulted in the creation of a more context-adapted policy package. All critical feedback received during the workshops was later used by the research team to refine the policy concept which led to the various policy documents, including the programme component of the RBF pilot phase, the implementation manual and other supporting documents to facilitate a smooth policy introduction. The limited time allocated to group work during the workshops did not facilitate in-depth discussions on some topics, like the quality of the evidence provided by participants to make a point.27
This study has some limitations. Ideally, both workshops would have been attended by the same participants to ensure continuity of the reflection. However, each workshop had slightly different objectives, and the results of the first workshop were presented at the start of the second one allowing for an additive process. We also took care to ensure that participants who were unavailable for the second workshop were replaced by representatives from the same category of stakeholders. A second issue concerns the selection of stakeholders: while it was quite wide ranging in terms of including representatives from all levels of the health system, most attendees had a biomedical or health programme background with limited representation from the community. We attempted to counter this professional bias by encouraging reflexion and critique, and used techniques to reduce group think as much as possible, for instance, always starting analysis with individual tasks, or by having facilitators challenging statements. In the next phases of the Results4TB programme, patient groups and community representatives will be engaged prior to actual policy trial.
Further research into participatory policymaking could explore the conditions in which more inclusive participation occurs and the role herein of providing safe spaces for participants from different hierarchical levels to speak out, or of ensuring that all types of information and evidence are considered.
Conclusion
Developing policies that address complex issues, such as TB care in Georgia, and conducting policy research requires methods that facilitate the linkages between multiple stakeholders including researchers, and between theory and evidence on the one hand and practice on the other hand. Adopting a participatory approach to developing the IPT and the policy package, we found that the realist methodology helped us to leverage the tacit knowledge and expertise on TB service delivery and RBF of a wide range of stakeholders to codesign the policy. The policy now reflects a more complex intervention, which will be tested using a combination of realist, trial and cost-effectiveness tools.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
Ethics statements
Ethics approval
The Results4TB trial protocol has been approved by the respective Institutional Review Boards (IRB) of the partner institutions. The realist evaluation component which forms the basis of this manuscript was granted ethical clearance by IRBs of the National Centre for Disease Control and Public Health in Georgia (ref. IRB # 2018-019) and the Institute of Tropical Medicine, Antwerp (ref. IRB #- 1240/18).
Acknowledgments
We would like to acknowledge the inputs of all workshop participants, which were essential for developing the programme theory as well as the policy package.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @lade_abeji, @sophie_witter
Contributors BM, LS and IC conceptualised the manuscript with input from I-OOA, MU, NS, KD, AV, AZ, ANG and SW, who collectively designed and executed the research study as members of the Results4TB research consortium. The participatory workshops were mainly facilitated by the research consortium and the CIF team (LS, IC, MU, NS and AZ) in particular. I-OOA and BM prepared the drafts of the article with additional contribution from the other authors (LS, IC, MU, NS, KD, AV, AZ, ANG and SW). All authors contributed to and agreed on the final manuscript.
Funding The Results4TB study is a 4-year (2017-2021) project funded under the Health Systems Research Initiative call 3 of the MRC/ESRC/DfID/Wellcome Trust consortium (Grant Ref: MR/P015018/1).
Competing interests AZ being the project PI, is both a member of the Results4TB research team and a member of parliament in Georgia.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.