Objectives To examine the relative importance of patient and centre level factors in determining self-reported experience of care in patients with advanced kidney disease treated by maintenance haemodialysis (HD).
Design Analysis of data from a cross sectional national survey; the UK Renal Registry (UKRR) national Kidney patient-reported experience measure (PREM) survey (2018). Centre-level data were obtained from the UKRR report (2018).
Setting National survey of patients with advanced kidney disease receiving treatment with maintenance HD in UK renal centres in 2018.
Participants The Kidney PREM was distributed to all UK renal centres by the UKRR in May 2018. Each centre invited patients receiving outpatient treatment for kidney disease to complete the PREM. These included patients with chronic kidney disease, those receiving dialysis—both HD and peritoneal dialysis, and those with a functioning kidney transplant. There were no formal inclusion/exclusion criteria.
Main outcome measures The Kidney PREM has 38 questions in 13 subscales. Responses were captured using a 7-point Likert scale (never 1, always 7). The primary outcome of interest was the mean PREM score calculated across all questions. Multilevel modelling was used to determine the proportion of variation of the mean PREM score across centres due to patient-related and centre-related factors.
Results There were records for 8253 HD patients (61% men, 77% white) from 69 renal centres (9–710 patients per centre). There was significant variation in mean PREM score across centres (5.35–6.53). In the multivariable analysis there was some variation in relation to both patient- and centre-level factors but these contributed little to explaining the overall variation. However, multilevel modelling showed that the overwhelming proportion of the explained variance (45%) was explained by variation between centres (40%), only a small proportion of which is identified by measured factors. Only 5% of the variation was related to patient-level factors.
Conclusions Centre rather than patient characteristics determine the experience of care of patients receiving HD. Further work is required to define the characteristics of the treating centre which determine patient experience.
- general medicine (see internal medicine)
- quality in health care
Data availability statement
Data may be obtained from a third party. Data are held by the UK Renal Registry. To apply to access Registry data see https://www.renalreg.org/about-us/working-with-us/.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
A large sample size deriving from a national study involving all but two UK renal centres.
We employ multilevel analysis to explore the factors that inform patient experience of renal care at both patient-level and centre-level and provide accurate estimates of their relative contributions.
Patient-reported experience measure results do not take into account the potential impact of negative non-response bias, that is, the tendency to participate in the survey being negatively associated with problem reporting.
While distribution of patient scores is skewed, sensitivity modelling indicated that the reported models and interpretation are robust.
As the survival of patients improves, attention is shifting from clinical and biomedical ‘hard metrics’ as primary indicators of healthcare quality1 towards gathering feedback from patients on aspects of their experience consistent with their values; including health, well-being, respect and dignity, quality of life (QoL) and independence.2 Patient experience of care, defined as ‘patients’ perception of the range of interactions they have with the healthcare system they use’3–5 is increasingly recognised as an indicator of healthcare quality.5–7 The patient perspective has traditionally been assessed through measures of patient satisfaction.2 Patient-reported experience measures (PREMs) are emerging as a favoured metric for assessing healthcare quality.3 7 8 PREMs are standardised self-report tools designed to tap into specific domains of care (ie, communication, care environment, healthcare team). Unlike satisfaction measures, patient-reported experience captures patient perspective of quality by assessing the extent to which specific care processes occur over time3 9 (eg, ‘In the last 12 months how often did your doctor give you the information you needed?’). Patient experience allows providers to monitor the extent to which fundamental standards of healthcare quality have been met, through objective reporting.3 7 In contrast, patient satisfaction measures assess the extent to which patients are happy with care, which is a reflection of patients’ own subjective standards, expectations and affect, which is highly influenced by patient gratitude for the care received, and provides a less reliable measure of healthcare quality.3 7 10 PREMs aim to minimise subjective bias that arises from patient satisfaction measures.6 Nevertheless, patient experience influences patient satisfaction, demonstrating a causal link between these quality metrics.10 Measuring patient experience offers a unique understanding of how we appraise healthcare quality, by responding to the voice of those at the centre of care.7 Evaluated alongside patient safety and effectiveness, patient experience contributes to an overall holistic representation of healthcare quality.6 7 11
PREMs can be used by healthcare providers, policymakers and regulators to assess provider performance and facilitate quality improvement initiatives.1 PREMs are also used to inform pay-for-performance schemes, by which services may receive funding increments in line with their performance.8 9 Survey results allow for comparisons across providers1 and can be made publicly available for use by patients and the general public. Patient experience is linked to a range of outcomes across healthcare populations; including clinical effectiveness, self-rated and objectively measured health outcomes, patient activation and treatment adherence.9 12 This demonstrates the clinical utility of measuring patient-reported experience towards optimising healthcare delivery and patient outcomes. In the UK, approximately 6% of the adult population have a diagnosis of chronic kidney disease (CKD). Most have mild to moderate disease (stages 1–3), with around 0.4% having advanced CKD (stages 4 and 5). Renal replacement therapy (RRT) is required for many patients with stage 5 disease. Around 60 000 patients in the UK receive RRT, approximately half of whom are living with a renal transplant and the remainder receiving dialysis.13 Haemodialysis (HD) is the most prevalent dialysis modality. HD patients require long-term and burdensome treatment throughout their lifetime.6 The quality of this care can have a significant impact on these patients’ wellbeing, health and QoL.6 14 Health-related QoL is often indicated by patients as being more important to them than survival.15
Introduced in 2016, the UK Renal Registry (UKRR) Kidney PREM (online supplemental file 1) is part of a national programme that collects annual feedback from patients with kidney disease on their experience of National Health Service (NHS) renal services, including those on HD. However, the implementation of PREMs in clinical practice has been relatively limited.6 As a result, responding to patient experience feedback remains a challenge for providers.16 To translate patient feedback into actionable information to drive quality improvements,6 we must be able to define which factors inform patient experience. Whether patient experience is mainly determined by patient characteristics17 18 or by those of the centre remains unknown.19 There is a growing body of literature addressing patient experience of care (and the factors which inform it), but few studies acknowledge the hierarchical framework of patients nested within centres. No previous studies have taken account of this framework in the HD population. The objective of the current study was to examine the relative importance of patient- and centre- level factors in determining self-reported experience of care in patients with advanced kidney disease treated by maintenance HD. We used clustered analysis (multilevel regression models) to examine the association between patient demographic and centre-level factors and patient experience, using secondary data from the national 2018 Kidney PREM survey. We have focused on patients receiving HD to reduce the heterogeneity of the sample and because data suggest poorer experience of care in this group.20 By using analytical methods that account for grouping within treatment centre, our study aims to provide evidence that treating centre makes a larger contribution to patient experience than patient characteristics, thereby supporting a systems perspective towards understanding patient experience.1
Kidney PREM data
The validated Kidney PREM questionnaire was developed by a group of patients, clinicians and academics as a disease specific measure of patient experience. The third survey, conducted in 2018,21 involved 71 adult renal centres across the UK, with kidney patients given the option of a paper version of the questionnaire, distributed and collected by centres, or completion online. Patient age, gender and ethnicity were recorded along with current treatment (peritoneal dialysis; HD and its location (at home, in hospital, in satellite); transplanted or attending kidney clinics but not on dialysis or in receipt of a transplant). Patients reported their use of hospital transport by answering transport-specific questions. Information regarding patient experience of care was collected from 38 questions grouped into 13 subscales consisting of between two and five questions, with a final question asking the patient to score their overall experience of the service provided. Responses were captured using a 7-point Likert scale from never (1) to always (7), with additional options of don't know and not applicable.. The research reported here investigated only those patients receiving HD. The sample size was determined by the number of HD patients providing a completed Kidney PREM questionnaire in the 2018 survey.
Centre-level population characteristics
In addition to the patient-level survey data, the UKRR provides centre-level population characteristics (centre-level data). UKRR has collected centre-based data and published annual reports since 1998. Information provided for this study was from 2017,13 the most recently available. Centre demographic variables included mean patient age, percentage male, ethnicity distribution (Asian, Black, White,Other, Rather not say) and mean index of multiple deprivation. Follow-up information was obtained on the percentage of patients alive at the end of 2016 that died during 2017. Other centre-level information related to dialysis access for patients receiving RRT, current or previous smoking, comorbidities at the start of RRT (eg, angina, previous myocardial infarction, diabetes), the percentage of patients who were late presenters (ie, less than 90 days between first nephrology appointment and commencing RRT), and mean systolic and diastolic blood pressures. Mean blood test levels were provided for phosphate, bicarbonate, potassium, ferritin and urea reduction ratio. Information on transplant waiting lists and transplant rate was also obtained.
The primary outcome of interest was the mean PREM score calculated across all 38 questions across the 13 subscales, analysed as a continuous variable, with a difference of greater than 10% (0.7) of the scale range (1–7) being regarded as clinically important. Scores were calculated where patients had answered five or more of the 33 PREM questions which are not filtered for treatment type. Details of how the mean score was adjusted for missing responses is provided in the technical annex to the 2018 report.21
The information available from the completed PREM questionnaires was used to obtain the patient-level variables, these being: age in years (17–21, 22–30, 31–40, 41–55, 56–64, 65–74, 75–84, 85+), gender (male, female, rather not say), ethnicity (Asian, Black, White, Other, Rather not say), use of hospital transport (yes, no) and HD location (in-hospital, in-satellite, at home). Clinically relevant centre-level variables were selected. As the range of possible values differs between the constituent countries of the UK, for comparability, deciles for the Index of Multiple Deprivation were calculated. The strength of association between mean PREM score and prevalent mortality measures was assessed. These were the percentage of patients alive at the end of 2016 who died during 2017 (prevalent 1-year mortality unadjusted) and prevalent 1-year mortality adjusted to age 60 years.13
A multivariable linear regression, which assumes independence of observations, was performed on the patient-level and centre-level variables that were at least 85% complete and available for each centre. Variables with a p value of less than 0.1 were selected for multilevel modelling. Multilevel analyses were then performed on the mean scores by using the iterative generalised least squares method of estimation. Variance partition coefficients were calculated. These represent the proportion of total variation in an outcome that is due to differences occurring at each level; a high coefficient at centre-evel indicates that more of the variation in the model is due to differences between centres than between patients. First, the variance partition coefficients were calculated for the null model to give raw coefficients. Then a model was fitted adjusting for patient-level characteristics to give patient adjusted coefficients. Next, both patient-level and centre-level characteristics were entered. Finally, any interactions between variables identified were added in to give fully adjusted coefficients.
Sets of models were compared using likelihood ratio tests. In order to assess whether the patients entered into the analysis were similar to those excluded due to incomplete information, the two groups were compared on age group, gender, ethnicity and location of treatment.
Sensitivity analyses were performed as part of the modelling. As patient PREM scores were negatively skewed, an exponential transformation was applied to bring the observations closer to a normal distribution. Additionally, results were compared with and without the robust estimation of errors. Snijders-Bosker R2 values were estimated after fitting the multilevel model to determine the proportion of variance in scores attributable to centre-level and patient-level differences.
All analyses were conducted using Stata V.IC/15.1.
Patient and public involvement
The UKRR Patient Council was consulted regarding how best to share this research, as well as the Kidney PREM and other related research, with patients, medical professionals and the public in general. The study was presented at a Patient Council meeting in March 2019 and the members’ views sought in particular on our dissemination plans to maximise the impact of the research on patient care.
Feedback from the Council was that there are many ways to engage with the kidney community such as through public user group fora, patient blogs or via the Kidney Patient Associations. It was also felt that it would be beneficial to send results directly to centre staff, as well as the hospital chief executive, medical director and clinical director to ensure the findings were received at all organisational levels. In addition, the Council were keen that treatment centres were sent their own results from the Kidney PREM, highlighting how they compare to other centres, to ensure that best practice was shared across settings.
There were records for 8253 HD patients from 69 renal centres (9–710 patients per centre) with two centres providing no responses. Overall, reported experience of PREM was positive (mean=5.93, SD=1.00). There was noticeable variation between centres. Mean scores were normally distributed, ranging from 5.35 to 6.53 (median 5.96, IQR 5.79–6.13). In addition, centres differed markedly in the proportion of patients giving a low experience score. Taking scores of less than 5 to be low, there was a 10-fold difference between centres (3.0%–33.3%). The range of PREM scores (1.18) is far greater than 10% (0.7) of the scale range (1–7) and more than 1 SD of the overall score, giving an indication of the impact of the centre-level variation. Table 1 shows PREM summary scores for groups based on demographic characteristics. Of note, those unwilling to disclose their gender or ethnicity (selecting ‘rather not say’) on average gave lower experience scores. Some variation was seen across age groups, with those over 65 years reporting a better experience. However, the difference was small, ranging from 5.71 (18–21 years) to 6.01 (5+ years).
Those centre-level variables with at least 85% completeness and recorded by all renal centres are shown in table 2. These consisted of sociodemographic variables at both patient summary and centre levels, centre size, transplanting centre (yes/no), blood test results and waiting list information. Although the data for percentage White at renal centre level were more than 85% complete, they were unavailable for seven centres, all in Scotland, and so were excluded. However, there was a strong negative correlation between percentage White and size of centre (r=–0.7922) suggesting that centre size is a good proxy variable for percentage White. No association was found between mean PREM score and prevalent mortality, either unadjusted (r=0.0482), or age-adjusted (r=−0.0500).
The results from the multivariable linear regression are shown in table 3. All patient-level variables were retained in the model, along with selected centre-level variables as indicated in table 2. Adjusting for other patient- and centre-level variables increased the effect of age on the PREM scores, with a wider range of 0.53 across the age groups (compared with 0.3 as shown in table 1). Several centre-level variables were found to be reliably associated with mean scores in the adjusted model, however, the range of effect for any variable was limited (95% CI 0.03 to 0.44).
Table 4 provides details of relationships between the explanatory variables at the level of centre and patient, and the mean patient experience score. Data for the multilevel modelling were obtained from 6945 (84.2%) patients, with 69 centres represented. Patients who were included differed from those excluded in terms of ethnicity (White: 77.8% vs 72.6%) and location of treatment (hospital: 45.4% vs 42.0%). However, age (65+: 58.8% vs 57.9%), gender (male: 60.8% vs 59.5%) and use of hospital transport (yes: 62.9% vs 65.8%) were similar.
At the centre level, higher mean scores were associated with a higher percentage placed on the waiting list for a transplant within two years of starting RRT. Lower scores were associated with larger centres. At the patient level, lower mean scores were associated with younger patients (<56 years), refusal to disclose gender, refusal to disclose ethnicity, use of hospital transport and receipt of HD in-centre or in-satellite (compared with dialysis at home). Most of these effects were relatively small (<0.4 on a scale from 1 to 7), the main exceptions being refusal to disclose gender and refusal to disclose ethnicity. Patient-level variables had a similar effect on scores in the multilevel model as they did in the multivariable linear regression model (tables 3 and 4). The sensitivity analyses performed by firstly dropping robust estimation and then using the transformed data each produced similar results to those of the main analysis.
Table 5 shows the estimated variance for each model. The total variance decreases as the patient-level variables are added and decreases further once adjusted for centre-level variables (ie, as more of the variance in patient scores was explained). The Snijders-Bosker R2 values22 suggest that 40.1% of variation in scores is explained by centre variation and just 5.3% of variation is accounted for by differences in patient characteristics.
Statement of principal findings
This study provides important insights into what influences patient experience of care in HD patients in this national sample of respondents of the UKRR/KCUK Kidney PREM 2018 survey. In this study, patient characteristics—younger patients (<56 years), refusal to disclose gender, refusal to disclose ethnicity, use of hospital transport and receipt of treatment in centre or satellite—were only modestly associated with patient experience. These effects are small in magnitude, indicating a limited impact on patient experience. Collectively, patient characteristics account for just 5% of the observed variance in the PREM score. The only important effects were observed consistently in those patients who refused to disclose their gender or ethnicity. Previous studies have also described small variations in ratings for self-reported patient experience among older patients, those experiencing shortened treatments18 and some minority ethnic populations.19
In contrast, the centre accounts for an overwhelming proportion of the variance (40%). However, our findings indicate that while centre plays a major role in determining patient experience of care, the centre-level factors that drive patient experience are unknown. Any associations between collected centre factors and patient experience were modest and of doubtful clinical importance (ie, <10% (0.7) of the 1–7 scale range) and likely to have limited impact on patient experience. For example, lower scores were associated with larger centres (−0.027 per 100 patients, p<0.001) and higher scores associated with a higher percentage of patients placed on the transplant waiting list within 2 years of starting RRT (0.055 per 10%, p=0.007). These findings challenge the concept that providing care focused on patient characteristics is the key determinant of patient experience. In keeping with this, Brady et al19 have reported variation in mean patient experience scores between groups of US facilities with different characteristics, for example, size of facility, hospital-based versus free-standing, for-profit versus non-profit. Differences were relatively small, yet still outweighed those attributed to patient characteristics. Factors at a group (cluster) level (eg, hospitals or GP practices) can have an influence on an individual’s outcome of interest. This is typically handled in analysis using a group variable, or multilevel models. In sample size estimation where no current estimate of the ‘cluster effect’ is published it is generally accepted that a reasonable conservative estimate of the intra-cluster correlation is in the order of r=0.05.23 Often, the effect of clustering is smaller. In this paper however we present evidence that indicates that for patient experience the variation between clusters (eg, treating centres, ie, the centre effect) is larger by several orders of magnitude. In fact, variation attributable to the observed patient-level variables (age, ethnicity and so on) is approximately 5%, while the variation attributable to the centre effect is approximately 40%.
Strengths and limitations of the study
Our study has considerable strengths, most notably the large sample size deriving from a national study involving all but two of the UK renal centres making it more likely that the sample was representative, raising statistical power and hence the strength of the findings. The study is also the first to employ multilevel analysis to explore the factors that inform patient experience of renal care at both patient-level and centre-level and provide accurate estimates of the relative contributions for factors at both these levels. It has been suggested that centre-level numbers in excess of 50 are required to prevent bias in estimates of centre-level SEs.24 It is worth noting that the very large sample size means that a number of so-called statistically significant effects (p values <0.05) represent small differences unlikely to be of clinical importance.
The study also has some limitations. Though PREM scores were generally high, the results do not take into account the potential impact of negative non-response bias, that is, the tendency to participate in the survey being negatively associated with problem reporting.25 Furthermore, the distribution of scores is skewed, but sensitivity modelling indicated that the reported models and interpretation are robust. More importantly, the observed variation between centres was largely unmeasured. While there have been some calls to focus on variation between centres, there is as yet little effort to understand what these factors are. Based on our study findings, the need to address this gap is urgent.
Meaning of the study: possible explanations and implications for clinicians and policymakers
These findings have implications for care delivery, suggesting that the benefits on patient experience of personalising care delivery according to patient characteristics may be far outweighed by efforts focused on reducing variation between centres as suggested by others.1 26 However, it is unclear just what the centre characteristics are which drive this variation. Certainly, the factors examined in this study do not provide an adequate explanation. Wider systemic problems such as understaffing, lack of managerial support, and increased pressure to meet ambitious national targets can promote a culture of care delivery which is task- rather than patient-focused.27 Though these issues reflect challenges faced universally by the NHS, the centre-wide variation in patient experience observed in this study may indicate that some centres are better equipped than others to cope with increasing pressures, while others may lack the culture and mechanism to deliver the same standard of care.1
Unanswered questions and future research
Further study is needed to determine those centre characteristics which most influence the experience of care reported by the UK HD population. The recognition of the need for ‘deep cultural change’28 in promoting quality improvement is consistent with these interpretations. Additionally, further study is needed to ascertain if this is the case across other service provisions, not just in renal settings.
In summary, this large national study of patients receiving HD suggests that it is centre rather than patient characteristics which are important in shaping patient experience of care. Our findings suggest that a shift in focus from patient characteristics to the service itself is required to understand and improve patient experience of HD and potentially of many other areas of healthcare. This study provides a clear direction for future research aimed at determining the nature of the centre characteristics which drive patient experience.
Data availability statement
Data may be obtained from a third party. Data are held by the UK Renal Registry. To apply to access Registry data see https://www.renalreg.org/about-us/working-with-us/.
The study involved secondary analysis of robustly anonymised UK Renal Registry data. The UK Renal Registry has NHS Health Research Authority research ethics approval (16/NE/0042).
Thank you to all the patients who completed the 2018 Kidney PREM, and the UK renal centres for providing data to the UK Renal Registry. We are grateful to the UK Renal Registry for supplying the data from the 2018 PREM and 2017 UK Renal Registry Report for analysis for this study, and to the UK Renal Registry Patient Council and for Kidney Care UK for their input.
JH and NS are joint first authors.
Contributors Study design: KF, JH, JJ, NS, DW. PPI: CC, JH, JJ. Aggregation of data: AB, JH, NS, RS. Analysis of data: AB, KF, NS, SNvdV, DW. Interpretation of data/results: AB, CC, KF, RG, JH, JJ, BR, CS, NS, SNvdV, DW. Drafting manuscript: AB, KF, JH, JJ, BR, NS, SNvdV, DW. Approval of final manuscript: all authors.
Funding This project was funded internally by the University of Hertfordshire with a cross-school collaboration research grant.
Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Author note Guarantor: Janine Hawkins.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.