Article Text

Download PDFPDF

Care pathway and organisational features driving patient experience: statistical analysis of large NHS datasets
  1. Kelsey Flott1,
  2. Ara Darzi2,
  3. Erik Mayer2
  1. 1 NIHR Imperial Patient Safety Translational Research Centre (PSTRC), Imperial College London, London, UK
  2. 2 Centre for Health Policy, Institute of Global Health Innovation, Imperial College London, London, UK
  1. Correspondence to Dr Kelsey Flott; k.flott14{at}imperial.ac.uk

Abstract

Objective The aim of this study was to identify the care pathway and organisational factors that predict patient experience.

Design Statistical analysis of large National Health Service (NHS) datasets.

Setting andparticipants England; acute NHS organisational-level data.

Primary and secondary outcome measures The relationship of care pathway and organisational variables to organisation-level patient experience.

Results A framework of 18 care pathway and organisational variables were created based on the existing literature. 11 of these correlated to patient experience in univariate analyses. Multicollinearity tests resulted in 1 of the 11 variables holding a correlation to another variable larger than r=0.70. A significant multilinear regression equation, including the final 10 variables, was found (F(10,108)=6.214, p<0.00), with an Embedded Image of 0.365. Two variables were significant in predicting better in patient experience: Amount of support to clinical staff (beta=0.2, p=0.02) and the proportion of staff who would recommend the trust as a place to work or receive treatment (beta=0.26, p=0.01). Two variables were significant in predicting a negative impact on the patient’s rating of their experience: Number of patients spending over 4 hours from decision to admit to admission (beta=−1.99, p=0.03) and the percentage of estates and hotel services contracted out (beta=−0.23, p=0.01).

Conclusions These results indicate that augmenting clinical support and investing in the mechanisms that facilitate positive staff experience is essential to delivering appropriate, informative and patient-centric care. Reducing wait times and the extent of external contracting within hospitals is also likely to improve patient ratings of experience. Understanding the relationship between patient experience and objective, measurable organisational features promote a more patient-centric interpretation of quality and compel a better use of patient experience feedback to drive improvement.

  • health policy
  • quality in health care
  • organisational development

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

View Full Text

Statistics from Altmetric.com

Strengths and limitations of this study

  • This analysis unearths patterns that can guide quality improvement and debunk myths about what organisational features are most important to invest in to achieve patient centricity.

  • This study accounts for a wide range of variables, which have never before been analysed together to understand their relative importance to patient experience.

  • This work advances, through quantitative findings, an understanding of experience as intrinsically linked to objective, measurable aspects of care pathways and organisational culture.

  • The limitations of the relevant publicly available data inhibited this analysis.

  • Due to the number of variables that could influence experience, many of which relate to individual expectations and cannot be measured retrospectively, there are limitations inherent in any study that sets out to determine what drives patient feedback on an aggregate level.

Introduction 

Patient experience is a relative newcomer to the triad of concepts that define quality: clinical effectiveness, patient safety and patient experience.1 Its prominence has grown rapidly over the two past decades and is now embedded within healthcare quality discourse. The integration of experience into quality could be the result of a more enlightened service delivery model, or it could be the effect of a disease burden that is shifting towards the need for continual care rather than single cures.2 3 It has been in part due to the recognition of patients as experts in their lived experience of care, and, in certain healthcare systems like the USA, because of movement away from fee-for-service medicine and towards value-based payment models.4 5 Regardless, it has become a mainstay of quality, and its evaluation is deemed to be as important as that of effectiveness and safety.1 Frameworks have been established to delineate the principles of patient experience like receipt of information, desired levels of involvement and relational aspects of care that determine engagement and pathway adherence.6 These frameworks have helped move away from overly simplistic satisfaction surveys and guided the development of more nuanced patient experience feedback tools.6 7 Despite progress around patient experience, and the repository of patient feedback it has facilitated, the trend has not gone far enough. Patient experience feedback is collected and analysed, but often remains dormant, underused in efforts to improve service quality.4 8–10 There is an enduring scepticism around patient experience that leads many to consider it immaterial, comprised inexact concepts and driven by highly individual circumstances.

As a result, experience is consistently portrayed as the least important aspect of quality. It is overshadowed in quality improvement plans by the more objective, more easily measurable safety and effectiveness. Since the development of large-scale, patient experience surveys, patient feedback and the methods used to obtain it have been subject to considerable scrutiny. For instance, the impact of survey mode, the inescapable reality of bias and fact that certain groups score systematically worse are all valid considerations when interpreting patient experience data.11–14 However, the architecture behind many national surveys like those in the National Health Service (NHS) National Patient Survey Programme (NPSP) and the US Consumer Assessment of Healthcare Providers and Systems are robust, grounded in rigorous protocols for designing, testing, administering and achieving a high response rate.15–19 The concerns articulated above are therefore caveats to the absolute reliability of data; they do not, however, diminish the overall messages that patients deliver in their feedback. While there are legitimate questions around what really influences experience, the incredulous sentiment undermines the authority of patients’ feedback and devalues organisational attempts to use that feedback. It perpetuates an understanding of experience as being the result of ambiguous factors, extraneous to the fundamentals of healthcare.

However, this perception is unfounded. First, there are well-defined components of quality across sectors which indicate the relevance of certain, measurable attributes a service must provide to garner positive perceptions form its users. Parasuraman et al define a framework for service quality including determinants like reliability, responsiveness, communication and courtesy.20 These determinants are mirrored in determinants of patient experience as well: The Picker Domains of Patient Experience as well as a host of literature about what matters to patients, indicate the importance of information giving, communication and relational aspects of care.5 21 Furthermore, the perception of patient feedback being a result of elusive, unmeasurable factors is flawed because patient experience has not been thoroughly explored within the context of the objective care pathway and organisational factors that might influence it. Care pathway factors can be defined as the concrete steps in a patient’s journey through hospital or any healthcare interaction, including things like route of admission and whether they had a surgical procedure. Organisational factors can be defined as the objective characteristics of an organisation including, but not limited to size, type, budgets and staffing.

A more useful inquiry into what drives patient experience revolves less around survey factors and more around care pathway and organisational factors. Examples of care pathway features like wait times and route of admission, as well as organisational factors like staffing, estates, maintenance and facilities budgets, hotel factors (ie, hospital amenities), mortality rates and staff experience, feature heavily in the literature as possible influencers of experience.22–26 Understanding the significance of a range of possible factors that might predict experience, while accounting for certain demographic factors, will help direct focus to the root causes of patients’ negativity and provide useful business intelligence from which to make concrete organisational improvements.

The aim of this study was to identify the care pathway and organisational factors that influence patient experience. Using the NHS as a case study, in order to achieve these aims, two specific objectives were set: (1) to determine which factors are significant predictors of patient experience through a multilinear regression analysis and (2) to analyse how these care pathway and organisational features could be leveraged for local quality improvement.

Methods

Metrics

In the NHS patient experience is measured in a variety of ways, often with conflicting results.4 The largest survey within the NPSP, the National Inpatient Survey, is a robust source of experience data and the best suited to this type of organisation-level analysis. In 2015, the National Inpatient Survey was sent to a sample of 1250 patients at each of the 149 acute NHS organisations (aside from a few very small specialist organisations that could not reach this number). A total of 83 116 people responded, yielding a 47% response rate.27 Patients were eligible to be included in the sample if they spent one night in hospital during July 2015. Fieldwork took place between September 2015 and January 2016.

The 2015 National Inpatient Survey contained approximately 60 measures of different aspects of experience, divided into 10 discrete sections (box 1).28 This study used an average of National Inpatient Survey section scores from the 2015 survey as a composite metric for patient experience (equation 1).

Equation 1: Patient Experience Composite score calculation

Embedded Image

Box 1

Sections on the National Inpatient Survey (2015)

  1. Admission to hospital.

  2. The accident and emergency department.

  3. Waiting list or planned admission.

  4. The hospital and ward.

  5. Doctors.

  6. Nurses.

  7. Your care and treatment.

  8. Operations and procedures.

  9. Leaving hospital.

  10. Overall levels of dignity and respect.

See full survey questionnaire: National Inpatient Survey.

Topics from the literature were quantified using publicly available metrics reported annually at the organisation level for all acute NHS organisations. The primary datasets holding these metrics were the Care Quality Commission (CQC) Intelligent Monitoring Report, The Health and Social Care Information Centre (HSCIC) (now NHS Digital) Estates Returns, HSCIC Workforce Statistics, The National Staff Survey (NSS), The English Indices of Deprivation and National Inpatient Survey.29–32 All metrics were taken from 2015 in order to correspond with the most current publicly available National Inpatient Survey data; however, some variables were measured at different points throughout the year (table 1). All variables were continuous or categorical and assumptions were tested for a multilinear regression model.

Table 1

Variables and sources

Correlations and linear modelling

Data were filtered for non-specialist acute organisations only, as specialist organisations had substantially different characteristics than standard organisations. By design, specialist organisations have relatively narrow clinical remits, meaning they only treat a select few conditions and do not have emergency departments. Historically, specialist organisations have historically high patient ratings of experience.33 Thirteen organisations were excluded based on this filtering. These organisations were excluded in order to understand the relationship between characteristics of non-specialist organisations (which account for the vast majority of acute organisations in the country) and patient experience. All remaining 136 acute organisations were included in the analysis.

First, univariate regression was undertaken to determine if care pathway, organisational and patient demographic variables correlated with organisation-level patient experience. Second, variables that did share a significant correlation with experience were then tested for multicollinearity. Third, variables were removed if they held a correlation of over r=0.70 to any other variables in the model. Fourth, remaining variables were incorporated into a multilinear regression model in order to determine the relative importance of each. Fifth, a linear relationship between the two variables was assessed by plotting the regression standardised residuals against the regression standardised predicted value (online supplementary appendix 1).34

Supplementary file 1

Finally, given the result of the regression, which heavily pointed to staff experience as a key factor in driving patient experience, a final exploratory multilinear regression model using the same assumptions was conducted to better understand the predictors of staff experience. Staff experience was measured using the continuous variable for the proportion of staff responding to the NSS who would recommend the organisation as a place to work. Predictor variables were taken from the key findings about communication, leadership and culture from the same survey.

A 95% CI was set for all analyses.

Patient involvement

This project concerned secondary data analysis and did not involve patients.

Results

Eighteen care pathway, organisational and patient demographic variables were derived from the existing literature to form a framework of factors to analyse (table 3). Seven of these did not correlate to the patient experience composite score (equation 1) in univariate analyses and were therefore excluded from the model. A final variable held a strong correlation (r>0.70) with another variables and was excluded from the model to avoid multicollinearity. Ten variables were included in the final analysis (table 2).

Table 2

Pathway, organisational and patient characteristic variables, sources and their correlations to experience

Table 3

Predictors of organisation-level patient experience scores

A multilinear regression was calculated to predict patient experience based on the 10 care pathway, organisational and patient demographic factors. A significant regression equation was found (F(10,108)= 6.214, p<0.05), with an Embedded Image of 0.37.

Two variables were significant in predicting more positive patient experience scores: Amount of support to clinical staff (beta=0.22, p=0.02) and the proportion of staff who would recommend the trust as a place to work or receive treatment (beta=0.26, p=0.01). Two variables were significant in predicting a decline in patient experience: Number of patients spending over 4 hours from decision to admit to admission (beta=−1.99, p=0.03) and the percentage of estates and hotel services contracted out (beta=−0.23, p=0.01) (table 3). The regression model demonstrated a linear relationship between the outcome variable and predictor variables.

Finally, in order to further understand the quality implications for these results, a final regression analysis was conducted to understand the specific aspects of staff experience that are important and which can be measured and improved. A significant regression equation was also found regarding staff experience. Staff recommending their organisation as a place to work or receive treatment could be predicted from key questions on the NSS (F(3,144)=132.17, p<0.05), with an Embedded Image of 0.73. Although this was conducted on an exploratory basis, assumptions for a valid multilinear regression were met. Staff recommending their organisation as a place to work was normally distributed using a Shapiro-Wilks test (p=0.37) and the predictor variables correlated with staff experience, but did not exhibit multicollinearity.

Three variables were significant in predicting more positive staff experience: The proportion of staff reporting good communication between senior management and staff (beta=0.47, p=0.00), the proportion of staff reporting receiving support from immediate managers (beta=0.20, p=0.01) and the proportion of staff who stated that the incident reporting procedure was fair and effective (beta=0.28, p=0.00) (table 4). This equation also demonstrated a linear relationship between the outcome and predictor variables.

Table 4

Predictors of staff recommending the organisation as a place to work or receive treatment

Discussion

Statement of principle findings

The results of the multilinear regression demonstrated that care pathway and organisational factors were in fact significant predictors of patient experience at the organisation level. The variables that predicted more positive patient experience scores related primarily to staff experience (beta=0.26, p=0.01) and the amount of support clinical staff received (beta=0.22, p=0.02). The variables associated with lower patient experience were waiting times (beta=−1.99, p=0.03) and the value of estates and hotel services contracted out to external companies (beta=−0.23, p=0.01).

When staff experience was further explored, staff-reported measures of positive communication with senior managers and direct line managers, as well as fair and effective incident reporting systems, all predicted higher staff experience (as measured by rates of staff recommending the organisation as a place to work).

Strengths and weaknesses

The limitations of the relevant publicly available data inhibited aspects of the analysis. Data sources on staffing information report totals rather than rates per patient, meaning some could relate more to the size of the organisation than the nature of its staffing. These numbers also do not capture staff skill mix, which is likely to be relevant. Furthermore, the reliance on quantitative data at the organisation level does not go as far as it needs to in order to understand what impacts patient experience. This analysis would benefit from analysis of the qualitative patient input about what has impacted their experience that is included in the open-ended portion of the National Inpatient Survey. Due to the number of variables that could influence experience, many of which relate to individual expectations and cannot be measured retrospectively, there are limitations inherent in any study that sets out to determine what drives patient feedback on an aggregate level.

However, the enquiry beyond highly personal factors is valid, as it unearths patterns that can guide quality improvement and debunk myths about what features are most important to invest in to achieve patient-centricity. This study accounts for a wide range of variables, which have never before been analysed together to understand their relative impact on experience. The primary strength of this approach is its ability to advance, through quantitative findings, an understanding of experience as intrinsically linked to objective, measurable aspects of care pathways and organisational culture. It reveals what drives patient experience and positions patient experience feedback itself as a useful source of organisational intelligence.

Differences in relation to other studies

Compared with other studies and common perceptions of patient experience, this study calls into question the role of hotel factors and demographic characteristics in driving patient experience. Studies outside the field of patient experience often conflate satisfaction and experience and promote the idea that experience relates to factors that are irrelevant to care, like the availability of luxury food items.35 While many hotel factors, such as comfortable hospital amenities, are important and helpful, more focused literature on what matters most to patients clearly demonstrates that communication, involvement and information hold primacy for patients rather than aesthetic factors.36 37 The results of this study corroborate such messages by demonstrating that experience, in its totality, is driven by care pathway and organisational factors that can be measured and improved. It is important to note, however, that ‘hotel factors’ can be a misleading term. These results do not suggest that the features that make a hotel stay pleasant do not also make a hospital stay pleasant; rather, they indicate that when accounting for many variables, total spending on luxury services does not improve overall experience (r=0.00). From a quality improvement perspective, it does not suggest disregard for certain hotel factors, but encourages improvement of patient experience without demanding more money to be spent on luxury services.

Currently, there is also concern among clinicians around how much demographic factors are likely to influence patient experience.38 Certain demographic factors have been shown to influence scores: women and ethnic minorities have been shown to report lower levels of experience, while older patients have tended to report high levels of experience.13 14 As a result, these characteristics have become a scapegoat for poor experience feedback at the organisation level. However, this study concludes that, at an organisation level, neither deprivation, gender nor age drive composite experience scores.

Rather than hotel or demographic factors, this analysis suggests that patient experience is driven by pathway factors, like high proportions of people waiting over 4 hours from decision to admission. This conclusion resonates with existing findings about wait times and emergency care experience, but goes further to suggest that these factors are important regardless of hotel factors and estates budgets.22 Finally, this study concludes that organisational features like positive staff experience, support to clinical staff and low rates of external contracting for estates and hotel services drive patient experience. The relationship between positive staff experience and positive patient experience has been established and well documented.24 39 However, studies exploring this relationship have never before accounted for this range of other factors, which are commonly thought to influence patient experience.24 The relationship between external contracting and patient experience does not have a substantive evidence base and opens an important line of inquiry for future research.

These findings corroborate a growing body of evidence around the importance of organisational culture to quality, outlining that the more relational and procedural aspects of working life have a significant impact on staff experience. Effectively reducing blame and increasing corporation requires bringing clinicians and managers into a conversation about best practice in incident reporting in order to avoiding the perception of a bureaucratic task, as well as implementing fair repercussions for wilful disruption or negligence.40–42 This harkens back to early evidence around patient safety and effectiveness, demonstrating that adversarial communication, lack of role clarity and poor training, increase the risk of error and harm.40 41 It has become clear through extensive research that ‘no blame’ culture combined with adequate time and training inspires more honest and accurate incident reporting, a central feature of safer hospitals.39

Implications for policy-makers and health services

Too often strategies for improving patient experience rely on ‘question chasing’, focusing on improving scores to one single survey question rather than investigating the root cause behind poor scores. This might boost positivity around one survey question, but it does not necessarily unpick why a certain score is poor in the first place.42 Question chasing and ignoring feedback because of patient characteristics will not deliver a change to experience and will not lead patients to have better interactions with the health service such that they achieve better results and avoid costly complications.

An improved use of patient experience data would involve a strategy to nurture the most influential organisational drivers of patient experience. As a result, systemic change of patient experience requires deeper inquiry into what drives staff culture and how to improve it. As other studies have stipulated, this includes a more targeted investigation into leadership, communication and management to understand how organisations can develop a culture that supports positive patient experience.43 Investigating the intricacies of outsourcing, staff environment, the communication staff receive from management and the fairness of procedures by which they operate is likely to improve staff experience. In return, these features will positively impact the patients they treat such that those patients are better equipped to be active, educated patients, ready to embark on an effective patient journey.

Conclusion

The results of the regression analysis do not simply state that a friendly staff environment helps make patients happy and comfortable. This would be a trivialisation of patient experience scores. The patient experience scores used in this analysis account for how well-informed thousands of patients feel in hospital, whether or not patients and their carers are appropriately involved in decisions and the extent to which patients receive critical education about their treatment plan.6 44 In fact, these results suggest that eliminating entrenched cultures of egotism—the type of cultures that prohibit constructive communication and effective, blame-free incident reporting—is essential to delivering care that is appropriate, informative and capable of supporting patients to adhere to the most effective pathway.

The challenge is dispelling myths that organisation-level patient experience is primarily driven by demographics, budgets and the more trivial hotel factors. The ambition needs to be promoting the role of culture and staffing support, factors which are not always considered the heart of quality. Improving quality for patients requires digging deeper into the organisational features—especially staff culture—that pervade the hospital experience and influence patients’ successful adherence to a pathway. Understanding the relationship between patient experience and objective, measurable organisational features promote a more patient-centric interpretation of quality and compel a better use of patient experience feedback to drive improvement.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
View Abstract

Footnotes

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.