Article Text
PDF
Abstract
Objective A multidisciplinary support team for general practice was established in April 2014 by a local National Health Service (NHS) England management team. This work evaluates the team's effectiveness in supporting and promoting change in its first 2 years, using realist methodology.
Setting Primary care in one area of England.
Participants Semistructured interviews were conducted with staff from 14 practices, 3 key senior NHS England personnel and 5 members of the support team. Sampling of practice staff was purposive to include representatives from relevant professional groups.
Intervention The team worked with practices to identify areas for change, construct action plans and implement them. While there was no specified timescale for the team's work with practices, it was tailored to each.
Primary and secondary outcome measures In realist evaluations, outcomes are contingent on mechanisms acting in contexts, and both an understanding of how an intervention leads to change in a socially constructed system and the resultant changes are outcomes.
Results The principal positive mechanisms leading to change were the support team's expertise and its relationships with practice staff. The ‘external view’ provided by the team via its corroborative and normalising effects was an important mechanism for increasing morale in some practice contexts. A powerful negative mechanism was related to perceptions of ‘being seen as a failing practice’ which included expressions of ‘shame’. Outcomes for practices as perceived by their staff were better communication, improvements in patients' access to appointments resulting from better clinical and managerial skill mix, and improvements in workload management.
Conclusions The support team promoted change within practices leading to signs of the ‘green shoots of recovery’ within the time frame of the evaluation. Such interventions need to be tailored and responsive to practices' needs. The team's expertise and relationships between team members and practice staff are central to success.
- PRIMARY CARE
- QUALITATIVE RESEARCH
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Realist methodology is recognised as appropriate and relevant for evaluating complex interventions in healthcare environments. Recently published reporting standards informed its use in this work.
All practices which engaged with the team in its first 2 years were involved in the evaluation and a range of staff were interviewed, not just those in leadership roles.
Practices which did not engage at all with the team also declined to participate in the evaluation; the views of this important group of practices are therefore not represented.
The study is embedded in the context of a highly inspected, highly regulated service: generalisability to less heavily managed systems cannot be assumed.
Two of the evaluators work as general practitioners in the medical community in which the team works and this may have led to bias; however, their practice of reflexivity reduces the impact of this.
Introduction
Universal access to high-quality primary care is an important driver of the health of a population.1 Although the UK offers universal access to primary care, there are substantial differences in the quality of care provided by its general practices.2 In the UK, general practice quality is measured and monitored using a range of quality indicators3–7 and inspections by the Care Quality Commission (CQC),8 ,9 a regulatory body which has a statutory responsibility to ensure that health and social care in the UK is safe and effective. The quality data and the CQC's reports are publicly available and failures to reach these standards consequently become public knowledge; this ‘naming and shaming’ has become part of the process of improving accountability within the UK's National Health Service (NHS).10 ,11
While staff are a key determinant of the quality of care provision, a 2006 literature review12 regarding team effectiveness in healthcare concluded that there is no single model of an ‘effective team’; team composition and function need to be tailored to their purpose. No clear direction could be given for the creation or maintenance of a ‘high-functioning team’. Nevertheless, team leadership is important: Aranzamendez et al13 concluded that ‘psychological safety’ for teams arising from the characteristics of leaders has a high impact on healthcare quality. Other literature focuses on the effectiveness of teams, facilitators or learning programmes which aim to promote specific quality improvement initiatives14–16 rather than general improvements in practice function. There is some evidence from Canada that such initiatives improve communication, collaboration and leadership.16 However, Dean et al17 reported primary care teams' concerns about wasted effort and resources being barriers to their adoption of quality improvement activities.
In 2014, an NHS England Local Area Team (the management ‘layer’ of the NHS which then had responsibility for commissioning primary care medical services) established and funded a ‘Supporting Change in General Practice’ team (the SCGP team). The SCGP team's stated aim is to ‘improve the quality of working life for practice staff, achieve sustainable positive change and improve care for patients in the locality’, a population of 1.4 million people which was served by ∼200 general practices. It was not established specifically to support practices to prepare for or remediate after a CQC inspection,8 but for any practice which wanted or needed help to change. The SCGP team has five members, all of whom are employed solely to undertake this work and all have relevant experience in the NHS: a general practitioner (GP), a practice manager, a nurse, an administrator and an analyst. Practices do not contribute to its costs. The SCGP team does not have a regulatory role and does not impose solutions or sanctions on practices; engagement is voluntary even in the situation of a practice having failed a CQC inspection. The SCGP team collates data from a variety of sources during a ‘scoping’ stage with the practices. These data inform discussions between the SCGP team and practice staff in an action planning stage, which is followed by the implementation of the action plan, in which the SCGP team may or may not be involved and for which it maintains an ‘open door’ policy, meaning that there is no point at which it formally ends its relationship with a practice. Clinical backfill is available, provided by the team's GP.
Little has been published about support teams for general practices. The only similar intervention of which we are aware is the UK-based Royal College of General Practitioners (RCGP) pilot18 in which a team provides support for practices in ‘special measures’ after CQC inspections.8 There are important differences in the way they operate; for example, the RCGP team has ‘two or three advisors’ and practices are expected to contribute to its costs.18 No evaluation of this team has as yet been published.
The Keele School of Medicine was commissioned by NHS England to evaluate the team's effectiveness over a 2-year period (April 2014 to March 2016).
Methods
We used realist methodology19 ,20 which seeks to explain how, why and when an intervention leads to change by considering configurations of contexts, mechanisms and outcomes (CMO).19 A realist evaluation is a cyclical process of making and testing hypotheses with the aim of specifying CMO configurations in which change has occurred. It is increasingly used for the evaluation of complex interventions in socially constructed settings such as healthcare.19 ,20
We considered using a ‘quasi-experimental’ quantitative methodology20 involving NHS performance indicators.3–5 However, the way these performance indicators are applied and used in the NHS is not stable and they are no longer reliable measures of change.21 ,22 We also considered quantitative measures of team function23 but the likelihood of abreactions by practice teams early in the process of change24 ,25 meant that these were unlikely to be reliable indicators of change within the timescale of the evaluation.
We describe one cycle of the realist evaluation process using the RAMESES II reporting standards.20
Interviews with key personnel from NHS England, the members of the SCGP team, and practice staff to inform the development of a conceptual foundation or programme theory19 ,20 for the intervention.
Data analysis from a realist perspective19 ,20 to specify contexts, mechanisms and outcomes. We grouped mechanisms into ‘resource’ and ‘reasoning’ mechanisms, as willingness and ability to change depends on the resources available and the reasoning behind the choices people make.19
Further interviews with practice staff in order to test hypotheses about the CMO configurations which resulted in change.
Development of a programme specification.19
The interviews
We used purposive sampling for NHS England staff and SCGP team members, and stratified sampling for practice staff. For practice staff, we aimed for a representative variety of professional groups in the practices (GP principals and sessional doctors, nurses, healthcare assistants, managers, administrators and receptionists). All interviews were semistructured, audio-recorded and transcribed with participants' consent. The questions were open and invited comment on experiences and perceptions.
Initial interviews with practice staff took place after the SCGP team's scoping stage and follow-up interviews several months later when it was conceivable that change would be identifiable. Their purpose was to test hypotheses, thus refining the programme theory.19 ,20 All the SCGP team members were interviewed throughout the evaluation period, and key people from NHS England were interviewed once.
The data were analysed thematically by two researchers independently and then several iterations of discussion organised and refined the themes in CMO configurations.
Descriptive data relating to the practices were obtained from the SCGP team.
Ethics
According to the NHS research ethics decision tool,26 this work is an evaluation rather than research, and approval from an NHS Research Ethics Committee was not required. The project was discussed with the Primary Care Research and Development Manager of the National Institute for Health Research (NIHR) Clinical Research Network (West Midlands, UK), who gave permission for it to proceed on 4 April 2014. It was discussed with Keele University's Research Governance Officer on 19 March 2014; approval of the University's Ethics Review Panel was not required.
The work was carried out with the intention of, as far as possible, maintaining the confidentiality of practices and individuals.
Results
We report on data relating to 14 practices. Four practices had approached the SCGP team, one because staff were concerned about preparing for a CQC inspection.8 ,9 Nine were identified and approached by the SCGP team as a result of being outliers in performance data sets. One practice engaged with the team as a direct result of an unsatisfactory CQC inspection report.
Initial and follow-up interviews were conducted for five practices. For the rest, for reasons of timing or staff agreement, only one set of interview data was obtained.
The practices
There were three rural practices (located in settlements of <10 000 people)27 and 11 urban ones. The registered list sizes varied between 3000 and 15 000. The numbers of whole time equivalent (WTE) doctors ranged from 1.75 to 8.125 and the WTE:list size ratios from 1335 to 3050 (the mean for England in 2014 was 1678.)28
Interviews with the SCGP and NHS England staff
All the SCGP team members were interviewed individually twice (April 2014, January 2015) and as a group three times (June 2014, August 2015, February 2016).
Three key people from the local team of NHS England were interviewed: the Medical Director individually, and the Head of Primary Care and the Primary Care Lead together.
Interviews with practice staff
A total of 72 interviews were carried out with staff from 14 practices:
15 practice managers;
18 GP partners who ‘led’ their practice's work with the team;
6 other GP partners;
17 nurses;
6 receptionists;
8 administrators;
2 sessional GPs.
Constructing a programme theory
The SCGP team's purpose was consistently viewed as being to provide help and support for practices which were ‘struggling’ or ‘vulnerable’. It was perceived by two NHS England staff as having a specific purpose in supporting practices to develop action plans as required by the CQC,8 ,9 but this had not been originally given as a primary reason for establishing it. They had clear ideas about criteria for success, for example, patient survey data and other performance indicators, the focus being on outcomes for patients. The Medical Director (NHS England local team) reported that he had ‘no clear set criteria for success’, but that the team's work would lead to “visible green shoots of recovery, maybe manpower, maybe patient satisfaction, maybe something else…” and that it would be specific to each practice. His view was that “the NHS is changing too quickly to rely on some of the indicators such as the Quality and Outcomes Framework]3 they are no longer a stable measure…”.
The SCGP team's own criteria for success changed over the 2 years; from initially being focused on outcomes for patients, by February 2016 the emphasis was on better communication within practice teams, increased morale, and sustainability for the practices and the individuals within them. The team members developed a perception that these might take time to achieve. The SCGP team itself did not perceive the CQC work as ever being ‘the majority’ of its workload.
Practice staff perceived that they needed to change and were hoping that they would perform better as a result of the work. They speculated about benefits to themselves in terms of better morale and a more managed workload, and concomitant benefits for patients.
The analysis of the interviews, with the incorporation of concepts described in the King's Fund's ‘Exploring the CQC's Well-led Domain’,29 led to the development of a programme theory:
“When practices are identified as performing poorly, work with a multi-disciplinary team to help them to identify areas and strategies for improvement and then support them to implement these strategies, may lead to better practice performance and sustainability for individuals and practices. As the purpose of general practices is to provide good quality care for patients, better performance will lead to better outcomes for patients in the future.” (see figure 1).
The programme theory. Definitions: CQC, Care Quality Commission.
Contexts, mechanisms and outcomes
Contexts
All practices were operating in the contexts of the NHS in England and their local medical communities. Each practice had its own context of patient demographics, staffing structures, service provision and challenges. Most staff were working in a context of perceived suboptimal management and leadership, and many with suboptimal clinical team skill mix and staff deployment, particularly in their nursing teams. A significant aspect of context was the route to engagement with the team; some had actively asked for help while some had accepted it willingly or reluctantly when it was offered to them. Some felt that they were individually or collectively ‘at breaking point’ and that help was essential. Some expressed concern about why they had been identified as needing help and some that the reason they were offered it was never made clear.
The context of the NHS in England was perceived by all as challenging; practice staff described themselves as being on a ‘hamster wheel’, the work being ‘pressured’ and ‘frantic’. The team described a set of problems common to all practices, including workload, staff recruitment, financial viability and personal sustainability.
Mechanisms
Identified mechanisms were grouped into ‘resource’ and ‘reasoning’ mechanisms, as willingness and ability to change depends on the resources available and the reasoning behind the choices people make (table 1).19 ,20
The mechanisms
The most powerful resource mechanism was the practical expertise of the SCGP team, particularly of its manager, which was mediated by the nature of the relationships between practice staff and team members. This practical expertise included knowledge of human resources and business management, NHS processes, professional regulation and clinical skill development.
The SCGP team described an action plan, owned and constructed by the practice staff, as being ‘primary’; however, from the practice staff's perspectives, the value was variable and dependent on context; those practices concerned about CQC8 ,9 inspections placed more value on it and there was evidence that they used it more in their implementation of change, particularly the practice in special measures after a CQC inspection. For others, action plans were more evolutionary and more fluidly used, or not used at all.
The provision of clinical backfill was perceived as a positive resource by some practice staff in that it gave doctors time away from clinical duties to take part in discussions and developmental work, but it was perceived by some as being part of the inspection and giving the SCGP team an opportunity to look at doctors' clinical practice and find out what the patients thought about the doctors, and therefore caused some anxiety.
The most powerful reasoning mechanism was the ‘external view’ provided by the team and its corroborative and normalising effects. In some practice contexts, this corroboration was perceived negatively by practice staff; they felt that the team's work added nothing new to their knowledge about the difficulties they were facing. Others expressed relief that no unexpected problems were found by the team.
The most prominent negative reasoning mechanism was a concern associated with engagement with the team, which was expressed most strongly by the practice managers; they were worried about being perceived in the local medical community as ‘failing practices’ if they were known to be working with the team. We interpreted this as an expression of shame.
Further negative reasoning mechanisms were the perceived generic nature of the intervention, and some negative aspects of the relationships between team members and practice staff which were related to issues of trust, both of individuals in the team and ‘the NHS’.
Outcomes
Within the timescale of this evaluation, the principal outcomes perceived by the practice staff were better skill mix and deployment of clinicians improving access for patients, better workflow arising from better deployment of administrative staff, better communication within practice teams and increased morale (table 2). The fact that staff were reporting these positive perceptions is in itself evidence of improved morale, and a variety of professional groups are represented, suggesting that changes were not only being perceived by those at managerial level (who had made the decision to work with the SCGP team).
The outcomes
There were perceptions in some practices that though some change had taken place, it was the beginning of a process and more input was needed.
One practice described a temporarily negative outcome in which the report produced by the SCGP team at the end of its scoping phase created some divisions between staff groups which contributed to two staff members leaving the practice; however, the overall perception of the outcome was that communication had improved as a result.I think it caused a break-up, certainly, for a short period of time… Practice manager practice A
it helped to improve things because it helped to have a lot more open conversations…I think we've established a better structure. GP partner practice A
Two GPs (from different practices) expressed some anxiety about the provision of continuing support.NHS England have now got to help us with a rescue plan…it's become more of a rescue plan than an action plan. GP partner practice H
I do hope nobody gets rid of them before we're finished. It's a valuable asset really…we can see some light at the end of the tunnel but we're not out of the woods by any means. GP partner practice K
CMO configurations
We present CMO configurations19 ,20 relating to seven practices chosen to show a variety of different situations and practice needs. Among this group, there are practices with each of the following contexts (table 3):
Practices which chose to approach the team for help;
Practices which were approached by the team:
Those who took up the offer of help,
Those who did not take up the offer;
A practice which had ‘failed’ a CQC inspection.
The CMO configurations
Practices are included from the first two groups identified above which had an initial negative reaction but which took up the offer of help and reported good experiences overall, practices in which the reaction stayed negative and further input was declined, practices in which there were mixed responses from staff, and practices where change was clearly attributed to the SCGP team.
Apart from the practice which was involved with the SCGP team because of a CQC inspection, these practices were initially engaged with the team in its early existence, which meant that there was sufficient time between the first and follow-up interviews for some change to take place in the practices and practice staff to be able to evaluate the impact of the team.
The other practices are not included either because it was considered that inclusion would not contribute anything more to the evaluation, or if data were incomplete because of timescales or practices' choices about participation in the evaluation.
The programme specification
The following programme specification was developed from the CMO analysis (see figure 2).
A summary of the contexts, mechanisms and outcomes. NHS, National Health Service.
When practices are identified as performing poorly work with a well-functioning multidisciplinary team which has appropriate expertise (knowledge, experience and skills) to help them to identify areas and strategies for improvement and then support them to implement these strategies, some practices will accept and benefit from the engagement. For others, especially where they have not actively sought the help of such a team, acceptance is difficult due to negative perceptions including shame, and these practices are less likely to continue with engagement or perceive a value in it. However, practices which engage as a result of a CQC inspection are likely to view engagement positively. Tailoring the intervention to the specific needs of the practice is critically important. The principal positive mechanism for change is the expertise of members of the team. Practices may not engage in a linear manner, and outcomes may be delayed.
Discussion
This evaluation has shown that when general practices engage with a team whose purpose is to support change, change can take place even in a short timescale. There are clear signs of the ‘green shoots of recovery’ in the practices manifested in better communication, morale and skill mix (table 2).
Using realist methodology, we have considered CMO configurations (table 3) to inform a programme specification. Mechanisms either belonged to the ‘resource’19 of the team or to the ‘reasoning’19 of the practice staff. While some mechanisms were predominantly positive or negative, the effects of others were contingent on factors in individual practices' contexts, such as the route to engagement with the SCGP team, influences of inspection and regulation, and the daily challenges experienced by the practices. The expertise of the SCGP team was consistently positive while the ‘external view’ provided by the SCGP team was variable and more likely to be positive when a practice had actively sought the input of the team; even then some individuals viewed it negatively early in their involvement.
In the current context of the NHS in England, there is an emphasis on inspection and regulation3–11 and negative feelings are known to arise in relation to this.11 ,30 Where there is inspection and judgement against a standard, there is the possibility of failure to reach the standard. For this group of practices, the external standards which led to their engagement with the SCGP team were those of the NHS3–5 and the CQC.8 The ‘naming and shaming’ which takes place in the NHS10 ,11 is a process involving humiliation31 and feelings of shame and anger may result, especially when there is a perceived risk of loss of public reputation and ‘standing within a given social sphere’.32 The process of change itself, even when desired, also induces negative emotions25 which may be explained by the Kubler-Ross Change Curve;24 shock, anger and denial tend to be followed by acceptance and eventually more positive emotions. These negative reactions may have become focused on the SCGP team itself rather than on the need to change, which may have been the real cause of them, and they may have been responsible for the difficulty some practice staff had in engaging with the SCGP team. For some practices, a specific negative emotion was a disincentive for engagement with the team, particularly related to concerns about how engagement might be perceived in local medical communities. We interpreted and labelled this as ‘shame’, as it was related to the practices' perceptions that the SCGP team was for ‘failing practices’. It is possible that being seen to be needing help also contributed to this.33 Both self-referring and practices which had been approached by the team reported these negative reactions.
The usefulness of reasoning and resource mechanisms depended heavily on the quality of the relationships between practice staff and the team, dependent in their turn on skilled and sensitive communication in the contexts of the practices' reactions to their situations. There are examples in table 1 of perceptions of suboptimal communication which affected the effectiveness of the SCGP team in promoting change, the negative mechanisms being ‘mistrust’ and ‘relationships’.
For this group of practices, the issues which had led to their engagement with the team were very similar to those described by Rendel et al:18 low morale, leadership and management issues and staffing issues.
Strengths and limitations
A strength of this work was the use of a recognised and relevant research methodology. Though there has been a lack of agreement over what makes for a realist evaluation of high quality, the recently published reporting standards20 go some way towards addressing this, and these standards have been used to inform this work. We have described both the data collection and analysis and the care we took to ensure rigour in both. Though the numbers are small, all practices which engaged with the team in its first 2 years were involved in the evaluation. A range of practice staff was interviewed, not just those in acknowledged leadership roles. Some practices which did not engage with the team beyond the initial stage participated in the evaluation, which gave balance to the data.
There are limitations; 2 years is a short timescale in which to evaluate change, and review of the practices' progress in the future is likely to yield useful data. Some of the more negative emotions identified as having been generated by the need to change are difficult to discuss, and the researchers are aware that a deeper discussion would contribute much to the evaluation. Practices which did not engage at all with the team also declined to participate in the evaluation; the views of this important group of practices are therefore not represented. Those practices which had engaged with the SCGP team for reasons related to CQC inspections (which carry the ultimate sanctions of prosecution or closure8) are likely to be highly motivated to change. The study is embedded in the context of a highly inspected, highly regulated service: generalisability to less heavily managed systems cannot be assumed.
The evaluation team included two GPs who work in the same medical community as the SCGP team. This meant that they were exposed to the informal perceptions of others in that community regarding the team's work, however, neither worked in practices with which the team was involved, and both are experienced in the practice of reflexivity as part of qualitative research methodology. Though this is likely to have reduced the possibility of subjective bias, it is not possible to remove the effects of this completely.
Conclusion
In the current context of the English NHS, this support team has been effective in promoting change within practice teams via the mechanisms of an external view on their situations and shared expertise in their individual contexts of a need to improve. There were clear signs of ‘green shoots of recovery’ in that participants reported better morale, skill mix and communication within all of the practice teams. Nevertheless, practice staff may have negative reactions to such teams, related to perceptions of shame and feelings of mistrust arising from the current context of inspection and regulation in the NHS, which may impact on relationships with team members.
This study demonstrates that, to be successful, such interventions need to be carefully tailored and responsive to the practices' needs. Relationships between team members and practice staff are central to success and team members need to be sensitive to the multiple and, at times, conflicting drivers for practice engagement and change. Careful communication is essential but change can be achieved.
Acknowledgments
The authors would like to acknowledge Dr Chris Harrison of Keele School of Medicine for his help as a critical friend on a late draft of this paper and Dr Geoff Wong of Oxford University for his advice on the use of Realist methods in evaluation.
References
Footnotes
Contributors The evaluation was jointly designed by MB and RKM and led by MB. The data were collected and analysed by MB and RB. MB wrote the first draft, while RB and RKM contributed to subsequent revisions. All authors approved the final draft.
Funding This work was commissioned and funded by NHS England, grant number IR00502.
Competing interests All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare that the work was funded by a grant paid by NHS England to Keele University. MB and RB received payment from Keele University for the work they did on the project. MB and RKM are general practitioners in practices from which NHS England commissions general medical services; however, since neither are GP principals, they are not involved in strategic or financial decision-making within the practices.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.