Article Text
PDF
Abstract
Introduction The single biggest barrier for countries in sub-Saharan Africa (SSA) to scale up the necessary health services for addressing the three health-related Millennium Development Goals and achieving Universal Health Coverage is the lack of an adequate and well-performing health workforce. This deficit needs to be addressed both by training more new health personnel and by improving the performance of the existing and future health workforce. However, efforts have mostly been focused on training new staff and less on improving the performance of the existing health workforce. The purpose of this paper is to disseminate the protocol for the PERFORM project and reflect on the key challenges encountered during the development of this methodology and how they are being overcome.
Methods The overall aim of the PERFORM project is to identify ways of strengthening district management in order to address health workforce inadequacies by improving health workforce performance in SSA. The study will take place in three districts each in Ghana, Tanzania and Uganda using an action research approach. With the support of the country research teams, the district health management teams (DHMTs) will lead on planning, implementation, observation, reflection and redefinition of the activities in the study. Taking into account the national and local human resource (HR) and health systems (HS) policies and practices already in place, ‘bundles’ of HR/HS strategies that are feasible within the context and affordable within the districts’ budget will be developed by the DHMTs to strengthen priority areas of health workforce performance. A comparative analysis of the findings from the three districts in each country will add new knowledge on the effects of these HR/HS bundles on DHMT management and workforce performance and the impact of an action research approach on improving the effectiveness of the DHMTs in implementing these interventions.
Discussion Different challenges were faced during the development of the methodology. These include the changing context in the study districts, competing with other projects and duties for the time of district managers, complexity of the study design, maintaining the anonymity and confidentiality of study participants as well as how to record the processes during the study. We also discuss how these challenges are being addressed. The dissemination of this research protocol is intended to generate interest in the PERFORM project and also stimulate discussion on the use of action research in complex studies such as this on strengthening district health management to improve health workforce performance.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/
Statistics from Altmetric.com
Article summary
Article focus
-
There is a need for more research into how best to improve the performance of the existing health workforce in sub-Saharan Africa (SSA). However, the complexity of health systems (HS) and wider environment requires an understanding of how health worker performance can be improved in the real and complex world facing health managers and decision-makers in SSA.
-
The dissemination of this research protocol is intended to generate interest in the PERFORM project and, more importantly, stimulate discussion on the use of action research (AR) to improve health workforce performance within complex health systems in low-income and middle-income countries.
Key messages
-
Using AR to design and evaluate the effects of interventions to strengthen district health management and improve health workforce performance should encourage ownership of the results by the district health managers and ensure the sustainability of the interventions even after the project has ended.
-
Key challenges faced during the development of the methodology are: the changing context in the study districts, competing with other projects and duties for the time of district managers, complexity of the study design, maintaining anonymity and confidentiality of study participants as well as how to record the processes during the study.
Strengths and limitations of this study
-
Comparing the intervention effects across the nine unique study sites is likely to be methodologically complex. While this is a strength of the study, it requires a combination of different methodological expertise.
-
This study will generate new knowledge on ‘what HR/HS interventions work best for whom and in which contexts’ and how district health managers can be supported to do their work better.
Introduction
The single biggest barrier for countries in sub-Saharan Africa (SSA) to scale up the necessary health services for addressing the health-related Millennium Development Goals and achieving Universal Health Coverage is the lack of an adequate and well-performing health workforce.1 ,2 This deficit in health workforce capacity needs to be addressed both by training more new health personnel and by improving the performance of the existing and future health workforce. However, simply increasing the number of health workers or improving workforce performance will not necessarily result in the improvement of health-related processes and outcomes unless the organisational context provides an enabling environment in which to carry out the healthcare activities.
Much emphasis in research and development has been placed on training more new health personnel3 ,4 and less on strategies for improving workforce performance.5 WHO describes the dimensions of a well-performing workforce as availability, competence, responsiveness and productivity.6 Vujicic and Ohiri7 add quality of care and highlight the importance of absenteeism as a determinant of availability. We understand performance management as the process of creating a work environment or setting in which people are enabled to perform to the best of their abilities. While human resource (HR) management is understood as the process of facilitating and improving the performance of the collaborators by building a conducive work environment and providing maximum opportunities to the collaborators for participating in the organisational planning and decision-making process, we are specifically interested in the interactions of these processes with the broader health systems (HS) functioning.
In 2007, WHO introduced a model of six interdependent HS building blocks including HRs and the need for integrated strategic responses to challenges.8 De Savigny and Adams9 further developed the concept of health systems thinking in health systems strengthening and, more recently, the concept of complex adaptive systems has been described to highlight the complex nature of health systems.10 ,11 Projects with complex social interventions can adopt a realistic approach: “what works for whom, in what circumstances, in what respects, and how?” (p.V)12 and recognise that “every intervention, from the simplest to the most complex, has an effect on the overall system, and the overall system has an effect on every intervention” (p.19).9 The concept of systems thinking is also found in the mainstream HR management literature which recognises the complex interactions between different HR management strategies, the methods of implementation and the wider HS context.13 ,14 For example, the pay-for-performance of health workers in maternity care may be combined with learner-directed training, changes in processes (eg, consultation), supplies (eg, medicines), equipment (eg, laboratory) and infrastructure (eg, facilities). Therefore an integrated set, or a ‘bundle’, of HR and HS strategies can address complex areas such as improved workforce performance.15–17 A pay-for-performance initiative in one area may also lead to unintended effects such as the neglect of other service delivery areas, because health personnel are distracted from their essential duties.
The ability to adopt a systems approach—combining an integrated set of HR and complementary HS strategies with the aim of achieving synergies and avoiding negative unintended consequences—is only possible if managers have adequate room for manoeuvre of what Bossert and Beauvais18 refer to as ‘decision space’. The increasing HS decentralisation of planning and management authority to lower levels and, in particular, to districts in SSA can make this space available to managers.
Local health managers need to learn how to effectively use the available decision space. Owing to the complexity of all the influences, including resource constraints, to be considered, this may be best learnt through Action Research (AR) on the job19 as a form of management development (figure 1).
Action research cycle.
The PERFORM project, funded by the European Commission FP7 Framework, aims to enhance our understanding of how health workforce performance at the district level in three African countries can be improved and how AR as an approach can strengthen health management capacity. We are further interested in the relations between improved HR performance and its interactions with HS performance.
AR in PERFORM
AR has been defined as “an enquiry which is conducted by a group on a problem which is of interest to them. Its aim is twofold; to improve practice and to generate knowledge about the processes and strategies that work best to create that improvement.”20 ,21
Lewin22 is often credited with first describing the AR cycle. The cycle has further been described as a spiral or similar to an intertwined Russian wedding ring23 as the complexity of the cycles of intervention and evaluation deepens.
In contrast to a top-down research approach where project design and implementation are directed by outsiders, in AR studies, the people who are closest to the problem, in this case the district health management teams (DHMTs), will lead on planning, implementation, observation, reflection and the redefinition of the activities in the study. The country research teams (CRTs) will support the DHMTs and can ask for peer advice from the paired European partner (EP). The DHMTs will continuously work through systematic cycles of planning, acting, observing and reflecting to
-
Describe and analyse the initial and changing HR and HS situation they face (eg, analysing the evidence available to them).
-
Identify and plan strategies to improve the situation or problem (eg, using the literature to select interventions they can implement within their budget).
-
Implement the changes needed (eg, through decisions taken during DHMT meetings).
-
Observe and record the effects (eg, through monitoring and evaluation).
-
Explain and reflect on the processes and effects of changes made (eg, discuss identified outcomes during DHMT meetings).
The purpose of this paper is to disseminate the methodology for the PERFORM project and reflect on the key challenges encountered during the development of this methodology and how they are being overcome. We start by outlining the project's aim and objectives. We proceed to describe the project setting and partners, outline the research design and finally conclude with a reflection on the lessons learnt from this experience.
Methods
Project aim and objectives
The aim of the PERFORM project was therefore to identify ways of strengthening decentralised management to address health workforce inadequacies by improving health workforce performance in SSA. It seeks to answer the following key questions: how effective is AR for management strengthening in the context of decentralised managerial authority? And, what bundles of HR/HS strategies help improve workforce performance, for whom and in what contexts?
In addition to developing new knowledge, the specific research objectives are:
-
To support health managers to carry out a situation analysis on the health workforce, with a particular focus on performance, in the study districts.
-
To develop and test context-specific management strengthening processes, focused on improving workforce performance, which will
-
Identify areas of health workforce performance to be improved;
-
Implementing integrated HR and HS strategies feasible within the existing context, to improve health workforce performance;
-
Monitor the implementation of strategies and evaluate the intermediate processes and impact on health workforce performance and the wider HS.
-
-
To conduct comparative analyses across districts and countries of
-
The management strengthening intervention to support improved workforce performance;
-
Processes of implementing the integrated HR and HS strategies and the intended and unintended effects on health workforce performance and the wider HS.
-
-
To provide ongoing communication of the research process, findings and conclusions, as well as to improve the research capacity of partners on integrated approaches to workforce performance improvement and contribute to strengthening capacities of decentralised management of district HSs.
Project setting
The project will work at the district level in Ghana, Tanzania and Uganda. These three countries were selected because they have decentralised health systems where managers have adequate potential ‘decision space’18 to address the problems of poor performance of the health workforce.
Three districts in each country have been selected for this study using two broad predefined criteria. First, owing to the collaborative nature of the project, it was important to have a motivated and reasonably staffed DHMT with which to work. A second criterion was the inclusion of different district contexts including a mix of rural and urban as well as those with good and poor overall performance. As this was not an experimental study, no control districts were selected. However, information about wider contextual changes to inform the impact assessment of the interventions will be captured.
Partners
Six partner institutions make up the PERFORM consortium—three are based in Africa and three in Europe. Each partner from an African country (also referred to as CRTs) is paired with an EP (table 1).
Paired partners
The DHMTs are regarded as coresearchers in the project; rather than have the research conducted on them, the research is conducted with and by them. They will be involved in collecting and analysing data throughout the project as well as leading and participating in each phase of the AR cycle.
The CRTs will act as facilitators, supporting the DHMTs in identifying appropriate research tools and analysis methods, problem analysis and strategy development as well as facilitating reflection among the DHMTs on how to maximise the effectiveness of the interventions. Specific capabilities and strengths will differ from one DHMT to another, and therefore, ultimately, there will be room for negotiation of roles between the DHMTs and the CRTs.
The EPs as part of the paired partnership will provide support and offer research advice to their African pairs and are also expected to attend all national workshops held by their paired African partner. The University of Leeds is offering a methodology help desk to all partners during the implementation phase.
In a unique opportunity to transfer knowledge and skills in both directions and to document a reflection-action process which will be hopefully embedded in the routine management practices at the district level, PERFORM has brought together DHMTs and researchers from Africa and researchers from Europe. PERFORM provides opportunities for intercountry collaboration and learning.
Research design
This project will use AR to enhance understanding of how and under what conditions strengthening of district health management can improve health workforce performance. A systems approach using an integrated set of HR and other HS strategies that are feasible within the context and affordable within the DHMT budget will be used by district level managers through an AR approach (see figure 2) to strengthen district management in order to improve health workforce performance.
Overview of the research concept.
Through the AR cycles, the DHMTs, with the assistance of the CRTs, will evaluate the success of these HR/HS bundles to improve district workforce performance. The DHMTs, CRTs and EPs will reflect continually throughout the project and compare baseline and end-line data to assess the effectiveness of the use of AR in strengthening district management processes.
Project timeline
The project started in September 2011 and will last a total of 48 months as shown in figure 3. It is split into three phases.
Project timeline.
Phase 1 involves project start-up, district selection and methodology development. During phase 1, study districts will be selected in each country and the research methodology developed and finalised in collaboration with local and national stakeholders. The methodology will include agreements on the indicators for which data will be collected and the development of specific data collection tools, for subsequent adaptation in each country. The methodology will be updated periodically as the project progresses, based on need and the reflections of the PERFORM research team.
Phase 2 is the implementation phase of the project and will last about 24 months. During the first 9 months of this period, each DHMT will be supported to undertake a situation analysis of the state of the health workforce in their district, the DHMT management capacity including their decision-making space as well as the wider HS and the contextual environment. The findings of the situation analysis will be reviewed during National workshop 1 involving the members of each DHMT, relevant stakeholders such as members of the regional health management team and supported by the respective CRT and EP. At consortium workshop 2, CRTs and EPs will discuss the findings of the situation analyses and possible HR/HS bundles to address the problem areas in each district explored. Following this workshop and supported by the CRTs, the DHMTs in each district will go on to select relevant HR/HS bundles to address their respective problems. Each DHMT will have the freedom to choose which problems to work on. This is primarily to foster ownership for the process and the results. Each DHMT will be encouraged to integrate a plan for implementing and monitoring the effects of the bundles into their annual district health plans; this includes identification of what data to collect, as well as the methods and timing for the data collection.
The selected HR/HS bundles will be implemented in each district by the DHMT. Using the AR cycles, the effects of the actions of the DHMTs will be observed and reflected on by the DHMTs. In the light of this, elements of the HR/HS bundles may change or new ones may be added during the implementation period. At consortium workshop 3, CRTs and EPs will review the progress of the HR/HS bundles in each district and plan for comparative analysis. The effects of the HR/HS bundles on the health workforce and service delivery indicators will be evaluated in each district. At the end of this phase, the DHMTs will undertake another situation analysis which will form part of the evaluation of the HR/HS bundles and will serve as a comparison with the baseline dataset from the initial situational analysis. Following this, the DHMTs and CRTs will meet at National workshop 3 to discuss and share the findings across the three districts.
The main focus of phase 3 is the intracountry and intercountry comparative analysis of the extent to which, and under what conditions, the implemented interventions work well for strengthening the management skills of DHMTs and improving the health workforce performance. Furthermore, the content of strategies and their intended and unintended effects will be explored—both on the workforce performance and on the wider health system. At consortium workshop 4, all partners will meet to review the preliminary findings from the comparative analysis. Following this meeting, a small amount of additional data may be collected. The analysis will then be completed and the results compiled into a research report.
Data collection
Quantitative and qualitative data collection methods will be used during the project. Each DHMT can choose the methods they wish to use depending on the action being assessed within the AR cycle. Several data sources have been identified
-
Routine data from Health Management Information Systems;
-
A district questionnaire developed specifically for this project;
-
Key informant interviews and focus group discussions;
-
Observation of management processes;
-
Document review of official reports and minutes of meetings.
Where possible, routine data will be used to assess the effectiveness of the strategies, with the aim being to support DHMTs in the use of routinely available data to inform their planning and decision-making. This is seen as a more sustainable approach to strengthening the management capacity of DHMT. Data collection tools have been adapted to suit the context and needs in each country. The processes and learning from observations are being recorded by DHMTs in learning histories with reflection supported by the CRTs.
The CRTs will provide research advice to the DHMTs during data collection and also take part in collecting and analysing data where necessary.
Situation analysis
The purpose of the initial situation analysis is to serve as the baseline for the project as well as to inform the subsequent AR cycles in each district. It is being carried out in a two step process:
-
The first step is common to all study sites: data on basic information on the health workforce, DHMT, HS and the local and national context will be collected. This information will be used by the CRTs to prepare a report for the before/after comparative analysis in and between all districts.
-
The second step is specific to each study site and will focus on particular issues or problems emerging in step 1, for example, lack of training opportunities. These will become part of the first AR cycle in the district. A toolkit of data collection methods is available for the DHMTs to use in consultation with the CRTs, and to adapt to the district context, and includes: key informant interviews, focus group discussions, surveys, observations of management processes and document reviews. CRTs are providing advice on research strategies.
AR cycles
In the observation and reflection parts of the AR cycle, data will be collected to record the implementation process, the observed effects of the implemented changes and the reflection and learning which are taking place. The DHMTs, with support from the CRTs, are selecting the methods to collect this data.
Final situation analysis
This will take place after the implementation of the HR/HS bundles is complete. The data collected will be the same as that collected during the initial situation analysis.
Evaluation of intervention effects
In order to evaluate the effectiveness of the selected HR/HS bundles as well as their effects, it is necessary to collect data on outputs relevant to the intervention and analyse these data. It is planned that this will take place on an ongoing basis as part of the AR cycle. The data collection method to be used for this will be decided by the DHMTs and CRTs.
Evaluation of the DHMT strengthening process
In order to evaluate the effectiveness of the DHMT strengthening process, it is necessary to have the above information on the bundles, analyse the learning histories in each district and organise a focus group and individual interviews with DHMT members.
Data analysis
Several analyses will be undertaken
-
Situational analysis before and after the implementation of HR/HS bundles aimed at capturing baseline information on key indicators in the districts.
-
Comparative analyses focused on measuring and recording the differences over time and between geographical regions as the project progresses.
-
Ongoing analysis of the effects of implemented HR/HS bundles.
Generic indicators and measurements will be used to describe each study site. The choice of indicators and outcomes on selected problem areas is made by the DHMTs with agreement and support from the CRTs and EPs. The analysis of the data obtained during the project follows the AR approach. This means that while conventional qualitative and quantitative analysis methods are used, the process of the analysis aims to be participatory, with DHMTs discussing and reflecting on how to analyse quantitative data and the interpretations of qualitative data.
Ethics
The study protocol was approved by the appropriate Institutional Review Boards in Ghana, Tanzania and Uganda. In addition, ethics approval was also obtained from the Institutional Review Boards of the Liverpool School of Tropical Medicine (project coordinators) and the University of Leeds (project methodology lead).
All questionnaires and interview transcripts will be coded in order to protect the confidentiality of participants. No names or other identifying information of participants will be used in any results in order to ensure their anonymity.
All data collection tools such as the district questionnaire will be developed in English. Where English is not the official language such as in Tanzania, data collection tools will be translated by approved translators.
All participants will receive the project information sheet prior to being approached for written consent to participate in the project. Participants have the right to withdraw consent at any point during the project. All DHMTs have agreed to their participation as coresearchers and informants through a Memorandum of Understanding which was signed by the director of the DHMTs and the leader of the CRTs.
Discussion
Having summarised the research protocol, we now reflect on the key challenges encountered during the methodology development and discuss these alongside the lessons learnt for future similar studies.
During the development of the methodology, the consortium partners reflected on the application of the design and methods through face-to-face meetings, online discussions, email exchanges and workshops. These iterative reflections helped the team to adapt the methodology to the context of the three countries. The key lessons from this reflection include five broad issues.
-
The contexts in which we are working are constantly changing; each district has different starting points and preferences for the issues they would like to address in their district. Hence, the researchers are faced with the challenge of, as it were, trying to ‘jump on a moving train’. Changes to key DHMT members such as the District Medical Officer due to transfers or study leave make it difficult to collaborate and plan ahead. Turnover of DHMT members means that new relationships have to be established, and it can take time for new members of staff to buy into the reflective AR process, though most DHMTs have around 10 members, so in most cases a critical mass will remain. The way out of this dilemma for the researchers has been to adopt a flexible approach regarding agreements and accommodate DHMTs constraints and priorities. While this approach may constrain the attainment of project objectives (given the limited time we have), we believe it is essential for fostering ownership and participation from the DHMTs.
-
AR requires continuous substantial time inputs and strong commitment from all involved parties. DHMTs face multiple priorities and are faced with a substantial number of requests from internal and external bodies. For example, on a few occasions, DHMTs have been called away to attend other meetings and workshops during a scheduled PERFORM workshop. This, coupled with the changing context described previously, has created a challenge in time management for the DHMTs. This means that in order to achieve project goals and encourage continuity with the gains from the project, DHMTs need to integrate the project tasks within their routine work and duties. Each DHMT is integrating their selected HR/HS bundles into their annual health plans. Also, the DHMTs have made a commitment to include the PERFORM project in the agenda for their routine meetings. This is likely to be challenging initially, as DHMTs adapt to a different way of doing things. However, in the long term, this may result in a more efficient use of their time and other resources.
-
Developing the methodology for comparing what in essence are nine unique AR studies in a meaningful way has presented a challenge. As it is not possible to know in advance which interventions each district will choose to implement, it is impossible to predetermine which outcomes will be measured and therefore compared across districts and countries. Our solution to this has been to compare the process of planning and implementing the interventions in each study district as well as comparing the extent to which each intervention achieves the desired effects. We plan to also compare how the effects of the interventions change DHMT management and health HR performance in the district.
-
The researchers have learnt that dealing with ethical issues such as maintaining confidentiality and anonymity of participants is not straightforward. The difficulty stems from the fact that the DHMTs are coresearchers in this AR-based study, leading to two challenges. First, they are expected to also take part in data collection and analysis. This presents a dilemma where participants such as health facility staff or other local stakeholders express a view which portrays the DHMT, either as a group or individually, in less than favourable terms. In order to protect the confidentiality and anonymity of participants, the research team has adopted strict guidelines on obtaining, storing and referencing personal information to be adhered to by all researchers on the project, thus ensuring that only the necessary researchers have the ability to match the names of individuals to their responses. Second, it is quite likely that DHMTs will find out about difficulties in the other study districts in their country during joint meetings and workshops. The research team agreed to anonymise information and data as much as possible and remind each DHMT of their duties as researchers to keep certain information confidential and not share it outside the research group.
The PERFORM project will work towards introducing new ways of working in order to challenge and improve hierarchical HSs where gender and power interplay.
-
Recording the processes and outcomes during the project is essential in order to facilitate learning. Deciding how records are kept (ie, learning histories, journals or reflective logs, written or oral recordings) and who is responsible for keeping these records (ie, CRTs or DHMTs or both) and the depth of information to be recorded and who is synthesising all collected data has proved to be challenging. Our response to this challenge is to be guided by the DHMT preferences in relation to the format of a diary, ensure the link with existing planning and decision processes within the districts and utilise CRTs-DHMT visits to support and encourage reflection by the DHMTs on the processes.
Despite the above challenges, the research team has found the DHMTs to be enthusiastic about their work and dedicated to the project and its AR approach. This positive attitude is an important catalyst in projects like this one where there is little financial incentive for the DHMTs to participate in the project. Also, there is a growing level of collaboration across the participating districts in each country—and collaboration and sharing of experiences and research skills across the countries through the research partners.
Conclusion
This paper presents the methodology for the PERFORM project, an AR-based study being undertaken in three African countries. We share five broad challenges encountered in, and the corresponding lessons learnt from, the development of the methodology: dealing with the changing context in each study site, competing priorities of the DHMTs, the complexity of this multisite study, facilitating and recording learning and processes during the project and the ethical issues.
The dissemination of this research protocol is intended to generate interest in the PERFORM project and, more importantly, stimulate discussion on the use of AR in complex studies such as this project on strengthening district health management to improve health workforce performance.
Acknowledgments
This document is an output from the PERFORM project: improving health workforce performance in Ghana, Tanzania and Uganda, funded by the European Commission. The project involved a consortium of six partners: Liverpool School of Tropical Medicine, University of Leeds, University of Ghana, Swiss Tropical and Public Health Institute, Institute of Development Studies, University of Dar-es-salaam, School of Public Health, Makerere University.
Footnotes
-
Contributors RH, TM, MA, SOB, PK and TMa are the principal investigators on the grant and made substantial contributions to the conception and design of the project. CM wrote the first draft of the present manuscript and updated further drafts based on feedback from the coauthors. All authors reviewed and commented on the first and subsequent drafts of the manuscript. All authors read and approved the final manuscript.
-
Funding This study is funded by the European Commission Seventh Framework programme (FP7 Theme Health: 2010.3.4-1, grant agreement number 266 334).
-
Competing interests None.
-
Ethics approval Faculty of Medicine and Health Research Ethics Committee, University of Leeds (Ref: HSLTLM/11/053).
-
Provenance and peer review Not commissioned; internally peer reviewed.