Article Text
PDF
Abstract
Objectives Occupational therapy, physiotherapy and speech and language therapy are central to rehabilitation after a stroke. The UK has introduced an audited performance target: that 45 min of each therapy should be provided to patients deemed appropriate. We sought to understand how this has influenced delivery of stroke unit therapy.
Design Ethnographic study, including observation and interviews. The theoretical framework drew on the work of Lipsky and Power, framing therapists as ‘street level bureaucrats’ in an ‘audit society’.
Setting Stroke units in three English hospitals.
Participants Forty-three participants were interviewed, including patients, therapists and other staff.
Results There was wide variation in how therapy time was recorded and in decision-making regarding which patients were ‘appropriate for therapy’ or auditable. Therapists interpreted their roles differently in each stroke unit. Therapists doubted the validity of the audit results and did not believe their results reflected the quality of services they provided. Some assumed their audit results would inform commissioning decisions. Senior therapy leaders shaped priorities and practices in each therapy team. Patients were inactive outside therapy sessions. Patients differed regarding the quantity of therapy they felt they needed but consistently wanted to be more involved in decisions and treated as individuals.
Conclusions and implications Stroke unit therapy has different meanings in different hospitals. Measuring therapy time is problematic due to varied interpretations of ‘what counts’ and variation in reporting practices. Although stroke policy, guidelines and audit are potential tools of improvement, their benefits are not automatic. Their actual effects depend largely on the attitudes and values of local influential ‘street level leaders’. More work is needed to promote an integrated whole team approach to rehabilitation. Further research into contextual and human factors, including the roles and views of therapy leaders, would enable a better understanding of implementation of guidelines and service improvement.
- stroke
- rehabilitation medicine
- therapy
- quality In health care
- clinical audit
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
First study to use an ethnographic and theory-based approach to investigate therapy practice in the context of a newly implemented guideline and audit.
Large-scale study with over 300 hours of observational data and 43 participants interviewed.
Ethnographic methods provided rich data and allowed the researchers to compare what participants said with what they did and question them about any differences between the two.
Theoretical framework used for data analysis offers insights that have wide application across the field of healthcare, as national audits are increasingly being used for evaluation of services.
We offer one interpretation, but data could be interpreted differently by different researchers using different theory to analyse findings.
Introduction
A stroke is a sudden and potentially catastrophic brain event that can lead to any combination of difficulties in movement, cognition, perception and behaviour.1–3 Since 1995, the Stroke Programme at the Royal College of Physicians has been driving service improvements across multiple areas of stroke care in the UK.4 Therapy is considered to be effective in increasing independence and reducing disability after a stroke. It is widely agreed that more is better,5–7 although the specifics regarding how therapy should be provided and the required intensity remain unclear.6 7 Increasing the intensity of therapy provided to patients who had a stroke has become a target for improvement. The therapy intensity guideline, which aimed to increase the amount of therapy offered to patients who had a stroke, was set out in the National Clinical Guidelines for Stroke8 and incorporated into the National Institute for Health and Care Excellence guidelines for Stroke Rehabilitation.7 It applies to occupational therapists (OTs), physiotherapists (PTs) and speech and language therapists (SLTs). The guideline stated:
Patients with stroke should be offered a minimum of 45 min of each appropriate therapy that is required, for a minimum of 5 days per week, at a level that enables the patient to meet their rehabilitation goals for as long as they are continuing to benefit from the therapy and are able to tolerate it.8
The recommendation of a specific intensity of therapy treatment is one among many stroke standards yet proved controversial. A consensus meeting held by the Intercollegiate Working Party for Stroke and the Stroke Research Network9 showed therapists continued to oppose the guideline. Some criticised it on the grounds of being unachievable due to resource issues. Others questioned the desirability of the recommendation, criticising the rationale and evidence base.10
The Sentinel Stroke National Audit Programme (SSNAP) began auditing stroke services against the therapy intensity guideline in 2013. SSNAP results showed national variation in the amount of each therapy patients who had a stroke were receiving and in the proportion of patients each team recorded as appropriate for each therapy. SSNAP provides a detailed guide covering the definitions and methods that should be used to complete the therapy data.11 Although there have been steady improvements in therapy results since its inception, many services are still measured as not meeting the 45 min guideline. There continues to be wide variation in the proportion of patients considered appropriate for therapy.
Despite the proliferation of data generated through the audit, there is little information about how the national policy is being interpreted or implemented locally in practice. It is recognised that despite an assumption that guidelines will lead to improved care, implementation of guidelines can be problematic, with poor compliance and underutilisation in practice.12 Given the context of a new stroke therapy intensity guideline being measured and monitored in a national audit, we sought to investigate the influence of the guideline and audit on therapy practice: specifically, how it was adopted, its influence on care and the role of the audit in these processes. There is an identified need for qualitative research examining how therapists negotiate the different and sometimes conflicting factors shaping delivery of therapy.13 Using ethnographic research with a theoretical framework to shape data analysis is regarded as a valuable approach to investigate healthcare.14 15 Despite large quantities of numerical data regarding therapy intensity, there is little understanding of how therapists interpret and enact their roles on stroke units or of how they interpret and enact the relevant guideline and audit. This study therefore sought to investigate the delivery of therapy on stroke units in the policy context of the 45 min guideline and auditing of therapy intensity.
Methods
An ethnographic approach was used to study therapy practice in three different stroke units. Ethnographic research uses a combination of observation and interviews to elicit descriptive information about a given group or setting and was considered an appropriate method for examining how therapy decisions are made and acted on in everyday settings. Its use in healthcare research has been found to be valuable, particularly for understanding differences in healthcare delivery.14 15 This approach allows comparisons to be made between what participants say in interviews and what they do in practice.15 Participant observation can be viewed as a continuum with full immersion at one extreme and detached observation at the other.16 Adler and Adler17 describe three different types of membership role in fieldwork: peripheral, active and complete. Our researcher membership role was peripheral, with the primary researcher assisting with general tasks (such as cleaning equipment) but not working as a therapist.
The ontological position for this study is that the application of any specific therapy intervention involves factors that are not objectively ‘out there’ in the world. The epistemological position is that attempts to quantify ‘therapy’ (which could involve any variety of interventions) in general terms could involve misleading oversimplification. Quantifying the amount of time spent in therapy can reveal little about what is being offered, what is being received or why it is or is not beneficial. Instead, in keeping with constructivist and constructionist paradigms, the most appropriate means of furthering understanding in this area is to seek and interpret participants’ views and observe their behaviour, while being mindful of how the researcher’s own background and perspective might shape this interpretation.18–21
Research team and reflexivity
ET conducted the study as part of a PhD in Health Services Research and previously conducted and published qualitative research in the area of stroke rehabilitation. CM and FJ supervised the research and have extensive experience of leading on and publishing findings of qualitative research in healthcare settings, including research specifically regarding stroke rehabilitation. ET had a previous background as a senior OT in stroke rehabilitation and had last worked in a stroke unit 8 years prior to the study. She had previous connections with one of the hospitals and knew some participants across the sites. FJ had a clinical background in physiotherapy. CM had been a member of the Intercollegiate Stroke Working Party responsible for developing national guidelines for stroke and had links with key influencers in the Stroke Programme and SSNAP. It was important to acknowledge and consider the potential influence of the research team’s previous roles, relationships and experiences on participants, as well as on the analysis and interpretation of findings. Relationships were established with participants either prior to or at the start of fieldwork at each site. Previous knowledge of one site and familiarity to some staff might be beneficial for building trust and gaining access, but across sites, staff were equally open and trusting. The influence of the main researcher’s background on patients only caused an issue on one occasion, when a carer sought advice about the therapy team’s decisions. Reflexive field notes were made on a daily basis during fieldwork, and these were shared with the research team along with interview transcripts to ensure rigour. A basic knowledge of the field was an advantage as it was possible to understand the terminology and jargon used in meetings, and the fact that the prior experience of this setting was not recent gave it sufficient unfamiliarity to be viewed from an outsider’s perspective.
Participants and recruitment
Purposive and pragmatic sampling methods were used to select stroke units with different characteristics that were considered by the team to have the potential to influence the response to the research question, allowing a wide range of perspectives.22 For example, we sought to include sites within and outside London, with different levels of performance reported in the therapy domains of the SSNAP audit. The decision to use three sites for fieldwork was based on the need to balance rich, detailed data from each site with diversity from a range of sites, within the timescales afforded by the study. Therapy leaders at each site gave initial approval for the study to take place, in consultation with relevant managers. Local approvals were sought and provided at each site. Fieldwork was carried out one site at a time, and at the beginning of each episode of fieldwork, a meeting was arranged to explain the project to the team. Posters were placed in ward areas to explain the study to staff, patients and visitors and invite them to contact the research team. Verbal consent from staff/patients was sought for observations, and written consent was provided for interviews.
Staffing in all the stroke units included therapy assistants (TAs) who worked across the therapy professions, often working with patients on activities delegated to them by therapists. Some TAs had more of a focus on SLT or OT and PT, but most of them worked to support all three therapy professions. All OTs, PTs, SLTs and TAs working in each site and the patients they were working with during the fieldwork were considered for observation and invited to participate in interviews.
(Note: National Health Service (NHS) therapy posts in the UK are banded according to levels of knowledge, skills and responsibility required. Band 5 is the entry level for a qualified therapist. Band 6 is a senior clinical post. Band 7s are expected to have a higher level of knowledge, skills and responsibility, and these posts often involve team leadership. The inclusion and structuring of Band 8 posts varies across services. Band 8s are likely to be clinical specialists or therapy managers.)
For interviews, the core sample sought in each site included:
Staff from each of the three therapy professions (OT, PT and SLT) and TAs.
Staff with diversity in years of experience and seniority.
Patients working with therapists, with contrasting characteristics such as level of impairment/dependence, social situation, discharge destination, ethnicity and age.
Participants meeting these criteria were approached face to face and selected based on their availability and willingness to participate. In each site we sought the same core range of interview participants, with an openness to interview others who were found to play a key role relevant to the enquiry, such as a medical consultant, nurse, manager, administrator or relative. On the advice of the research ethics committee who approved the study, the researcher checked with the team on a case-by-case basis to ensure they did not have any concerns about patients being approached based on factors such as cognition or medical status. All interview participants were observed in practice prior to being interviewed. This meant that there had been establishment of some rapport between interviewer and interviewee, and it was possible to question participants about areas that had been noticed during observations.
Data collection and analysis
Data collection included observations of therapy work and interviews with therapists, TAs, managers, patients and carers in three stroke units.
Observations were unstructured, and the aim was to become immersed in the day-to-day working of therapists in each stroke unit in order to understand how they worked, how they made decisions and how they prioritised their time. Approaches to observation varied. For example, in the first site, therapists used timetables to plan their weekly sessions with patients, including individual and group work. Initially, the primary researcher joined in with timetabling, using the same timetable template and to book joint sessions with therapists. Frequently, these sessions were cancelled or rearranged, and we therefore changed strategy to accompany individual therapists for a morning or afternoon. This was more useful, as it enabled emersion in the pace and pattern of therapists’ work time rather than just joining in with certain sessions, and was used in the second and third sites. All aspects of therapists’ working day were observed, including meetings, administration and lunch times.
Observational data were gathered using detailed field notes and were used to document events as well as to prompt further questions for consideration or investigation. Topic guides were used for interviews (see online supplementary appendix A), which were audio-recorded and transcribed for analysis. NVivo software was used to manage the data.
Supplementary file 1
A constructionist approach to thematic analysis was used to identify, analyse and report latent themes.22 Field notes and interview transcripts were analysed inductively alongside deductive use of theory to support and shape the analysis. To ensure rigour, a sample of transcripts was coded by all three researchers, and coding processes were regularly reviewed and discussed. Field notes differed from interview data in that they often included the researcher’s interpretation of the observations. Therefore, interview data were coded without the field notes, and field notes were consulted as a reminder of activities and events observed and experienced and any early interpretations of these. Data analysis took place in between data collection at each site, and preliminary findings were shared with teams shortly after fieldwork at each site had ended, the interpretation of the data and links with underlying theory developed during this process.
Working closely with the data, the primary researcher coded and grouped data using NVivo, sticky notes on flip chart paper, mind maps and writing prose. NVivo was used for the first round of coding, although this was subsequently repeated by hand. Most data analysis was conducted using Word or on paper, but NVivo was used at later stages to conduct word counts on terms that appeared to have arisen frequently, for example, ‘commissioners’. Themes were derived from the data and connections made between theory and emerging findings. At all stages, data analysis was discussed with coauthors CM and FJ, as well as other colleagues, members of research groups, stroke survivors and participants in the research for member checking. This often occurred in the form of a presentation followed by a discussion. COREQ guidelines for reporting qualitative research were used.23
Patient and public involvement
The King’s College London Stroke Research Patients and Family group was involved in the development of the research question and design, and emerging findings were discussed with the group during data analysis.
Ethics
The study was conducted using the principles of ‘ethical mindfulness’ to navigate the unanticipated ethical decisions that inevitably arise in the field.24 Site-specific approval was obtained from each hospital’s Research and Development team. Written consent was provided by all interview participants.
Theoretical framework
In ethnographic research, theory is used inductively and deductively to broaden and deepen insights into the subject of study. Various potentially relevant theories were considered during the course of data collection, and appraising their usefulness in illuminating the driving forces underpinning the findings was a part of the ongoing data analysis. The theoretical framework for the analysis presented here drew on the work of Lipsky25and Power,26 framing therapists as street-level bureaucrats in an audit society. Lipsky’s theory of street-level bureaucracy concerns the implementation of policy through direct encounters between front line public service workers and citizens. Lipsky claimed that policy becomes distorted in its implementation, as the use of discretion and autonomy by public service workers in complex interactions is inevitable. The current study, constructing therapists as street-level bureaucrats (ie, public service workers on the front line who use their autonomy in the implementation of policy) sought to unpick what therapists do and why.
Power’s concept of audit society critiques the ‘audit explosion’ occurring within contemporary western society.27 Power associates the rise of audit with new public management and neoliberal governmentality and suggests it is an example of the public sector adopting private sector principles and practices. The power relation of audit is hierarchical and paternalistic, involving the scrutiniser and the observed. The observed are not involved in discourse, but instead become objects of information. The focus is to produce a quantifiable score and rank departments and institutions against each other. Use of this theory enabled a broader perspective and prompted an understanding of SSNAP as part of a wider context of audit culture.
Results
Sites A and C were located in different NHS hospitals in London. Site B was located in a town in the South East of England. All the sites differed in terms of referral pathways into and out of the stroke units. For example, one was located in the same building as the Hyper-acute Stroke Unit, in which patients stay for the first 72 hours poststroke and which was its only source of referrals. Another accepted patients from a number of other hospitals, and patients had sometimes been to multiple hospitals before being transferred there. The ratio of therapy staff to patients varied, with site A having the highest ratio of therapy staff to patients, and site C having the lowest. There was variation in the community services available to patients, and this influenced the point at which patients were considered ready to be discharged. The sites varied in their SSNAP results for therapy intensity. Site A consistently performed well on their scores (scoring A grades), site B had dramatically improved from low scores (eg, E grades) to good scores in the four most recent quarterly reports and site C was in the average range.
Over 300 hours of fieldwork were carried out across the three sites. Pseudonyms are used for the hospitals, places and participants to protect their identity. The pseudonyms given to interviewees reflect the name most commonly used to address them, that is, if a person introduced themselves with their first name, then we have chosen an alternative first name. Doctors have been given full names as they would sometimes be referred to formally and sometimes by their first name. Information about participants is restricted to details considered relevant to the study in order to reduce the risk of identification (see online supplementary appendix B). Forty-three participants were interviewed including therapy staff, doctors, managers, a nurse, patients and a patient’s wife. Interviews typically lasted for approximately 1 hour. In each site, there were different prominent figures who appeared relevant to interview in addition to these core participants. For example, in one site, a lead nurse was influential in decisions about when to withdraw therapy and was a driving force for a focus on SSNAP within the wider multidisciplinary team; therefore, it was considered valuable to interview her. Nobody declined an invitation to participate; therefore, interviewees were selected based on availability. One patient who had been keen to be interviewed became too unwell, and his wife consented to be interviewed. During data collection and analysis in the third site, it was evident that common themes were recurring. There were differences in all the sites, but this variation was seen as a finding in itself.
Supplementary file 2
Overall, we found:
There were key differences in the delivery of therapy in each site, including differences in the scope of activities therapy encompassed and differences in the perceived remit of stoke units and role of therapist.
Measuring therapy was therefore problematic, as there was a lack of consensus about what counted as therapy. There was no uniformity in the way therapy time was recorded and reported for the audit.
Therapists did not believe that their audit results reflected the quality of therapy provision.
There was an absence of an integrated, patient-centred approach to rehabilitation in the multidisciplinary teams.
Therapists associated the SSNAP audit and the monitoring of therapy time with the commissioning of their services. They expressed mistrust about auditing practices in other services, and they worried about commissioners taking these results at face value.
Therapy practice, including implementation of guidelines, was shaped by local clinical leaders.
What counts? Who counts?
The SSNAP audit records the quantity of therapy time provided to patients, but there were key differences in what was considered to count as therapy in each site. In one stroke unit, therapy was interpreted broadly. It could include groups and individual sessions in a range of environments, such as the gym, kitchen or outdoors. There, building therapeutic rapport and listening to patients’ concerns were considered to be valid use of therapy time. A narrower conception of therapy was evident in the two other stroke units, where there was a stronger emphasis on getting patients to the minimal level of physical ability required in order to discharge them. The influence of the local contextual factors on the delivery of therapy came through strongly in observations at each hospital.
Where I used to work, rehab was the ethos. You go there for rehab. So the way you come in you should go out at a different level, a better level, hopefully. Here we’re just a stepping stone to having your rehab at home. Joanne, Band 7 PT, Site C
‘Rehab happens in the community’ was a mantra in site C. This frequently caused ethical tensions for therapists who were keen to point out that the required rehabilitation would not be provided to many of their patients on discharge, depending on their home address. Nevertheless, provision of stroke unit rehabilitation beyond the essentials required for discharge was considered an ‘old-fashioned model’. A shift of emphasis from treatment to discharge planning was acknowledged by leaders in Sites B and C.
We don’t use the word ‘rehab’ in relation to inpatient stroke services at [NHS organisation] anymore because the concept is about community. Rehab happens in the community… I think I’m very clear… yes, the therapists don’t do therapy, but they get their patients home. Rona, Clinical Lead, Site C
Rona was referring to the fact that therapists needed to prioritise administration to facilitate discharge planning rather than providing rehabilitation. In our observations, we saw that therapists often set out to assess and treat patients, but then abandoned their plans when the pressure of expediting discharge mounted. The following field notes from observations at a multidisciplinary meeting illustrate the focus on discharge rather than rehabilitation.
The lady in bed 5 is cortically blind, fatigued, confused, anxious. OT says she was unwell when she tried to see her, and she would like to see her again as she really needs more assessment. She needs assistance of two for transfers, and the community team where she lives won’t see people who need assistance of two. Nevertheless, discharge date is tomorrow. It seems to me that ideally she would have more time and input either in hospital or at home, but she will get neither. Field notes from Site C
Individuals in all sites expressed mixed feelings about the apparent trajectory of improvement in stroke services. The nurse specialist in site B was driving the nursing team to improve on various processes that were audited for SSNAP, and she was sure that the audit had led to improvements that would be ongoing in these aspects of care. While the early medical management of stroke was seen as continually improving, there was less positivity as people discussed changes in therapy over recent years, including its scope and quality and the reduced availability of therapy spaces. Dr Adams echoed the comments of many site B therapists when he stated about inpatient rehabilitation:
In some respects, I think we were doing it better at some stage in the past than we are now. Dr Adams, Lead Medical Consultant, Site B
For therapists in all stroke units, there was ambiguity about what counted as auditable therapy. The team based at site B had fully engaged with the guidance and support offered by SSNAP, but other teams had not. Therapists in all stroke units made individual decisions about how to record their time for the audit. Some strictly adhered to their perception of the rules of the audit that only face-to-face time should be counted. Others would say things like ‘his discharge paperwork will be his session today’. They would justify the recording of administration as therapy time based on the argument that facilitating the patient’s discharge was their therapy priority and should therefore be seen as valuable use of their therapists’ time.
The calculation of SSNAP scores for therapy intensity takes into account the proportion of the case load deemed appropriate for therapy. This is measured against set benchmarks: that 80% of patients will require OT, 85% will require PT and 50% will require SLT. The stated rationale for these benchmarks is that they have been guided by previous audit data. From SSNAP guidance, a patient’s therapy time should only be included in the data reporting if that patient was deemed appropriate by the team. We observed stark contrasts in how teams recorded whether patients were appropriate for therapy. Therapists in sites A and C were unaware that this was an audit question. Unknown to the therapists, administrators in these teams were reporting that 100% of patients were auditable and were appropriate for therapy. In contrast, in Site B ‘appropriateness for therapy’ had become a daily clinical consideration for therapists, and they referred to patients who were appropriate for therapy according to the rules of the audit as ‘SSNAPing’. As one OT said, SSNAP had become ‘part of the jargon’, and in their daily morning meetings, we observed that it would be noted whether each patient was SSNAPing or not. Unless a patient was participating actively in 45 min of goal-focused therapy every day, and was improving, they were not counted as appropriate for therapy on the SSNAP audit for this team. Often therapists would ask each other, ‘Are they ‘SSNAPing?’, and this would become a point of debate and discussion.
‘The quality beneath’
Therapists in each site expressed a lack of confidence in the SSNAP therapy data, both nationally and locally, and they did not believe the data reflected the quality of therapy provided either for their own teams or at a national level. They perceived wide variation in the way different teams interpreted audit requirements and managed their data. Site A had been a consistent high scorer at the time of data collection, but senior therapists stated that their local data were ‘skewed’ as it was easy to accidentally duplicate data entries on the local computer system. Site B’s therapy scores had improved in response to the changes they made to data reporting, yet therapists there did not believe their grades reflected their practice. Several gave an example of a Christmas period during which they scored their best SSNAP grades despite feeling they were providing an inferior service due to staff pressures. A number of OTs felt that over the Christmas period, the OT score should have been an E (a low score) instead of an A (the top score), if it reflected the quality of service that was being provided. This was also raised by the clinical lead therapist.
[The OTs] said we did really prioritise when we were really short staffed so that SSNAP did not suffer… I think patients were perhaps being SSNAP-stopped prematurely. So, I think they were making SSNAP-stop decisions on resource availability as opposed to patient need. Lucy, Clinical Lead, Site B
Lucy charted the changes that she had initiated and the subsequent improvements in their SSNAP scores. When asked whether the improvements in their audit results reflected ‘real life’ improvement, she and her colleagues consistently responded with a clear ‘no’, explaining that most of their changes had been in their audit processes.
I just am concerned about the value being attached to [SSNAP] in its raw kind of sense, so its overall grading system doesn’t allow you to see the quality beneath. Lucy, Clinical Lead, Site B
We observed that Site B staff had detailed knowledge of their SSNAP performance, and their SSNAP results were regularly presented to the team. Therapists at site C had a much more vague perception of their SSNAP performance but still held the opinion that their score did not reflect their practice.
Obviously the data that we’re getting doesn’t reflect our practice. So something is not quite right. So I think they’re just trying to figure out what the problem is and have a bit more effective way of collecting that data… [B7 PT] has told me that we’re complying. To be honest, I know it’s not right, and she said, yes and that’s why we need to actually look into it. Ghita, B6 PT, Site C
This perspective was teamwide at site C and was raised in interviews as well as observed meetings and informal discussions with the researcher. Therapists believed that their SSNAP score was too high, compared with their perception of the service they provided.
Apparently we were getting like 100% and we were like, ‘no way’… because there’s no way that we’re seeing every patient 45 min a day. No way. You’ve seen it. Nancy, B7 OT, Site C
Therapists in all sites discussed having internalised the message that ‘more is better’, but this had become a voice of guilt in the backs of their minds rather than something that changed their practice.
I’m always waiting [for] when somebody comes heavy handed and says, ‘You haven’t been doing this!’, and I’ll be punished. It always feels like that, the guilt is there. There’s lots of guilt. ‘Oh I haven’t been seeing patients as often as I would like to. Agata, Band 6 OT, Site A
We observed that in all sites for the majority of the day patients were lying in bed or sitting at their own bedside, as one patient said, ‘just gazing’. We noticed that in team meetings, patients were ascribed different functional levels for therapists and nurses, meaning that nursing staff could not enable patients to do the things they had achieved in therapy sessions until the therapists gave their approval. Therapists’ and nurses’ work was hidden from each other behind the closed curtains around each patients’ bed area, or in the therapy spaces that therapists took patients to for their designated therapy sessions. In informal discussions, staff and patients frequently referred to the lack of an integrated approach to rehabilitation and the wasted time experienced by patients and staff. Some suggested that the SSNAP audit had encouraged a unidisciplinary focus, with professions focusing on their own scores rather than working cohesively as a team with the patient at the centre.
Patients varied in the extent to which they reported feeling happy with the amount of therapy they received. Some wanted more, some thought they were receiving too much or it was ‘too heavy’. Patients were less concerned about the quantity of therapy offered to them than the quality of care and the nature of the therapy they received.
Depends on the nature of the therapy. If you were in the therapy that I was telling you when I first came into the room, about ‘come on come on you can do it, stand up stand up’, that nonsense therapy, that’s not therapy. That’s bullying. Not 45 min - God! People wouldn’t come out of the therapy… You should be able to attune yourself to the patient. And you can’t train somebody to do that. They’ve either got it, because they love people, and they’ve got an empathy, it’s natural it’s innate in their nature. Some people are not like that… They’ve got to have that disposition. Eddie, Site A
In general, patients felt that the professionals involved should know best about what they needed, but they consistently wanted to be involved in the discussion and treated as individuals, and this was not their experience.
Competition and commissioner-centred care
In all sites, teams expressed scepticism about neighbouring services’ SSNAP practices. Therapists attended regional meetings and heard about how colleagues in other services were reporting SSNAP data, so were aware of the variation in audit practices across services. They questioned the quality of the national audit data for therapy, and they used language such as ‘bending the rules’, ‘playing the numbers game’ or ‘lying’ when discussing the practices of other teams. Some had visited neighbouring hospitals to find out about their audit practices.
It was really interesting to get insight into how other people do it… So that was interesting to come away thinking: this is a high performing A rated unit. What I took away from that is, do we really want to be one of those? Lucy, Clinical Lead, Site B
Rivalry and mistrust were observed to go hand in hand with discussion of the audit ratings. Many staff mentioned funding and commissioning when asked about their SSNAP scores. In most cases, when asked what the implications of SSNAP results were, therapists expressed concerns about how they might be used to inform commissioning decisions.
I worry that one day they’ll look at our stats and say, ooh speech therapy isn’t meeting the [45 min] standard…. So if that was the case, if they were to take the contracts off us then some of us could lose our jobs. Claire, SLT B7, Site A
Across all sites, fears were expressed about potential implications of SSNAP for service commissioning. In interviews, hospital therapy managers and consultants endorsed this as a reasonable concern.
Well there’s a little bit of paranoia there but at the same time … what we don’t want staff to do is to be naïve, and you know, shielded or protected from any sort of other conversation. So when the [neighbouring borough] stroke beds came here it was a tender for a service which this organisation won, and it’s a tender for 3 years, so at any, you know, and obviously we’re 2 years or so into that. So it will need to be reviewed at some point. So obviously as it goes increasingly closer to review, then people will become anxious. Ann, Therapy Manager, Site A
Ann talked about the ‘new way of providing healthcare’, with tenders coming out for very short-term contracts, sometimes just for 1 year. Many changes therapists had perceived in their work were linked to service contracts and commissioning and, for them, SSNAP was associated with these changes in the wider context of healthcare delivery. Few therapists associated SSNAP scores with quality of care, while most saw them as something services needed to use to ‘please the commissioners’, suggesting that the way the audit was implemented encouraged commissioner-centred rather than patient-centred therapy delivery.
The influence of local clinical leadership
In each site, it was evident that local clinical therapy leaders shaped priorities regarding the delivery of therapy and influenced attitudes regarding the 45 min guideline and SSNAP audit. Their specific roles differed, but in each site, there was someone influential who clinicians respected due to their clinical experience, but who also had responsibility for ensuring implementation of top-down mandates. They would filter the many policies and mandates coming through to them, and promote, emphasise or soften them according their own judgement.
Clinical leads in all sites talked about not wanting to put pressure on therapists to meet the target of therapy intensity. They gave various reasons for not prioritising this among the different top-down mandates they were expected to reinforce to their teams. These included believing that using session length as a measure of the quality of therapy was problematic; believing it was unachievable; and wanting to protect therapists from additional pressure.
Many therapists knew what was expected of them by their clinical leaders but did not know the origins of the protocols and guidelines they were expected to follow.
The local target kind of protocol that’s been put together I think by [clinical lead], that is in the forefront of my mind, which I always kind of get a little bit confused with, whether that is what is the kind of national targets. Nancy, B7 OT, Site C
Therapy staff identified opportunities for quality improvement at a local level, and this appeared to be more influential on them than national policy. National stroke guidelines and audit were used at management and service-commissioning levels to protect stroke services. Clinical leaders acted as an interface between the multiple local and national policies and imperatives and the therapists practising on stroke units.
Discussion
This study sought to investigate the delivery of therapy on stroke units in the policy context of the 45 min guideline and auditing of therapy time. The study illuminated experiences of stroke unit therapists at a specific point in time when the national auditing of therapy was new. It offers insights into the factors influencing the delivery of therapy and the influence of guidelines and audit on therapy delivery. Strengths of the study included its scale, with 300 hours of observational fieldwork completed as well as 43 interviews. The ethnographic approach of sustained periods of observation as well as interviews allowed insider insights into what participants actually do, as well as what they say they do.15 The use of theory allowed deeper insights into the findings, and suggests that the findings are likely to have broader applicability. A possible criticism of this design is that our account is interpretative and open to discussion and alternative analyses. During fieldwork, we noticed some practices and attitudes change, therefore completing the study at a different time could have captured different findings. The selection of three stroke units with contrasting features was a strength, and it was useful that one of the teams had consciously addressed their audit scores, and staff there were able to describe this process. However, it is a limitation of the study that we did not include a stroke unit that was performing poorly on the audit at the time of the study. It is also a limitation that the stroke units were all located in the South East of England. However, our findings have similarities to those of a recently published mixed methods case study evaluation of eight stroke units,27 and this suggests the issues identified are not specific to the time or regions of the UK at which our study took place. The global relevance of our study could be challenged on the basis that it took place in the UK. Many countries now have a therapy intensity guideline contained within their stroke guidelines, and the question of whether this should be audited is timely. Further research into the influence of similar guidelines and audit in other countries would allow useful comparisons to be made.
We found that the term ‘therapy’ was interpreted and delivered differently by therapists in different sites, and audit practices varied widely. Therapists were aware of this variation and reported that audit results did not reflect the quality of their service. These factors undermined the credibility they attributed to the audit. There was mistrust regarding the auditing practices of neighbouring teams, and therapists were concerned that audit results would influence commissioners’ decisions about service contracts, potentially leading to a negative outcome for their particular service. Therapists wanted to provide more rehabilitation and felt guilty about not doing so. Meanwhile, a focus on integrated multidisciplinary rehabilitation was absent, and patients were often observed as inactive outside their designated therapy sessions. The guideline and audit were among many local and national policies and mandates that clinical leaders filtered for their teams. Therapists were strongly influenced by these leaders in their delivery of therapy and their interpretation of the guideline and audit.
Analysing these findings with inductive and deductive reference to the theoretical framework enabled rich insights into the influence of policy and audit on therapy delivery at ‘street level’. In The Audit Society,27 Power claims that the use of audit in healthcare is prolific and increasing and that this follows a trend in public services and Western society that he termed ‘the audit explosion’. The SSNAP can be seen as one example of this. Within SSNAP, the auditing of the 45 min therapy standard is an example of using time as a performance measure. Street-level bureaucracy was introduced as a concept by Lipsky25 as a way of understanding the implementation of policy by the people who actually implement it. Lipsky noted that in the case of complex interventions provided by street-level bureaucrats, calculating use of time is the simplest way of measuring performance but is problematic and reveals nothing about the quality or appropriateness of the way that time has been used.
There is mixed evidence regarding the efficacy of using performance measurement to improve quality.28–30 It has been suggested that a good performance indicator should have reliability and validity; be based on agreed, fully described definitions; and be relevant and actionable for those using it.28 31 32 The UK therapy intensity guideline is based on consensus, and this may be a reason for some of the confusion regarding its rationale and evidence base. It has been noted that internationally, recommendations regarding the intensity and appropriateness of stroke rehabilitation vary.33 Therefore, there is a global need for more clarity regarding what is being recommended and why.
The lack of consensus regarding ‘what counts’ as therapy, or how therapists should be using their time, also calls for the attention of policy makers and those funding services. The various pressures on staff are sometimes in conflict, and clear and consistent messages are needed regarding what is expected of them. The need for a broader interpretation of therapy that includes listening to patients’ concerns echoes recent findings from the Family-led rehabilitation after stroke in India (ATTENDS) trial.34 If discharge from hospital is to be the primary focus of inpatient therapists, then more work is needed to reduce the evident disparities in community services to prevent patients from missing out on the opportunity of rehabilitation.
In keeping with criticisms of performance measurement,25 26 35 we found examples of ‘hitting the target but missing the point’. ‘The point’ was to improve rehabilitation for patients who had a stroke, but stroke units are not universally functioning as rehabilitative environments. This finding is in line with those of various observational studies that have quantified the amount of time stroke unit patients spend active or in therapy and suggests that this has not improved over time.27 36–40 In fact, our findings suggest that profession specific guidelines and measures may encourage siloed working, rather than a team approach focused on the individual needs of each patient. In an era of audit and big data, it is important to recall that ‘not everything that can be counted counts, and not everything that counts can be counted’.41 The challenge of determining how best to measure and monitor what matters to patients remains unsolved.
Wider contextual factors regarding the marketisation of healthcare were inextricably linked to much of the data. The perceived consequences of SSNAP discussed by therapists were less about patients’ experiences and outcomes, and more about team reputations, rivalries and the vulnerability of their commissioned services. This links with critiques of the rise of audit in Western society, New Public Management (NPM) and neoliberalism.26 NPM refers to the public sector’s adoption of certain private sector principles and practices.26 42 This includes a style of management that seeks effectiveness and efficiency through top-down control, a shift to greater competition43 and an emphasis on performance management.44 Although the language of neoliberalism, NPM and marketisation was not used by participants in the current study, they were mindful of working in a competitive market and the audit itself engendered a spirit of competition.
Market competition is hailed by some as a driver for improvement in healthcare, and this is often based on the premise that patients (as consumers) can actively choose between providers for elective interventions, such as in Bloom et al’s study.45 However, stroke unit patients in this study did not have ‘consumer choice’ but were processed through local stroke pathways after the sudden and unanticipated event of a stroke. Rather than being used as tool for patients to choose their provider, therapists feared that ratings were used by funders to select services for investment. In this sense, the audit had potential to be a tool of commissioner-centred care. This possibility calls for further exploration, and further research should also incorporate the perspectives of commissioners and funders. It has been claimed that focusing on numbers and statistics instead of people is a threat to person-centred, humanising practice.46 Our findings suggest that guidelines and audit do not hold power on their own to improve patient care. Their implementation and impact is dependent on people with influence conveying a message about what is important and why, and attention to potentially important contextual factors is essential. Alongside quantitative measures, there is a need to encourage creativity and bottom-up improvement to address local problems in order to improve patients’ experiences.
Lipsky’s theory of Street Level Bureaucracy emphasises the autonomy of individual front-line public service workers. Lipsky distinguished between workers and managers, but he did not account for clinicians in leadership positions, who act as an interface between policy and practice. We found that street-level leaders filter diverse top-down expectations and understand that it is impossible to demand that therapists give them all equal weighting. They therefore prioritise and amplify the messages they consider to be most important. Hupe and van Kooten47 noted that despite an abundance of literature regarding public management, this tends not to focus on middle management or work supervisors. They suggested that in processing rules, public managers either formulate additional rules, pass on rules or buffer rules and that in this way first-line supervisors are also discretionary actors.47 Our findings support this claim in the case of stroke unit therapists, and this highlights a need for consideration of their role as clinical leaders.
The role of clinical leaders in improving or maintaining quality has been widely discussed.48–51 Some have claimed the importance of leaders being ‘actually in the arena’48 and discussed the role that embedded leaders can have in ensuring that values, such as putting the patient first, are upheld.48 Little attention has been paid to the role leaders have in filtering or prioritising the conflicting demands placed on front-line staff. Furthermore, the clinical leadership literature predominantly discusses medics or nurses but not therapy leaders. This study offers new insights into the unexplored area of clinical therapy leaders as agents of discretion with a key role in shaping the delivery of policy on the ground. This is an area that warrants further investigation.
Conclusion
National audit results have identified variations in the delivery of therapy to patients who had a stroke. This study contributes to the literature by illustrating the problematic nature of auditing therapy time. The guidelines and audit of adherence to guidelines were intended to increase therapy intensity. There were local and individual variations in interpreting guidelines and recording inputs. Therapists were aware of this and expressed cynicism about the audit results. They described a mismatch between their results and their actual performance and did not feel that changes in their audit results reflected the quality of therapy delivered. In the wider context of healthcare organisation and changes, the audit was associated with concerns about investment in services. We found this led to a focus on commissioners of services, rather than the experiences of patients.
We conclude that although stroke policy, guidelines and audit are potential tools of improvement, their benefits are not automatic. Their actual effects depend largely on the attitudes and values of local influential ‘street level leaders’. This study highlights the importance of attending to contextual factors and potential negative consequences when implementing strategies for improvement. Approaches to health services research are needed that investigate whole systems and the human factors involved in improvement and implementation. Further work is needed to determine how best to ensure that the aspiration of improving quality for patients is not lost in the process of implementation.
Acknowledgments
We are grateful to all study participants for their involvement in the study. We would like to thank the members of the King’s College London Stroke Research Patients and Family Group for their valuable comments on the study proposal and the findings.
References
Footnotes
Contributors The first author (ET) conducted the study, supervised by CM and FJ. All authors made substantial contributions to the conception and design of the work. ET was primarily responsible for the acquisition, analysis or interpretation of data for the work and drafting the work and revising it critically. CM and FJ viewed and commented on all drafts and gave final approval of the version to be published. All authors agree to be accountable for all aspects of the work.
Funding The Stanley Thomas Johnson Foundation provided funding for the doctoral study. The research was supported by the National Institute of Health Research Comprehensive Biomedical Research Centre at Guy’s and St Thomas’ NHS Foundation Trust, King’s College London and National Institute of Health Research Collaboration for Leadership in Applied Health Research and Care (CLAHRC) South London.
Competing interests None declared.
Patient consent Not required.
Ethics approval Ethical approval was obtained from National Research Ethics Committee on 18 July 2014.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
Correction notice Since this paper was first published online the conclusion section has been added to this paper.