Article Text

Original research
Variability and performance of NHS England’s ‘reason to reside’ criteria in predicting hospital discharge in acute hospitals in England: a retrospective, observational cohort study
  1. Elizabeth Sapey1,2,
  2. Suzy Gallier1,3,
  3. Felicity Evison3,
  4. David McNulty3,
  5. Katherine Reeves3,
  6. Simon Ball4,5
  1. 1PIONEER Data Hub, University of Birmingham, Birmingham, UK
  2. 2Department of Acute Medicine, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
  3. 3Department of Research Informatics, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
  4. 4Renal Medicine, University Hospitals Birmingham NHS Foundation Trust, Birmingham, West Midlands, UK
  5. 5Better Care Programme and Midlands Site, HDR UK, Birmingham, West Midlands, UK
  1. Correspondence to Dr Elizabeth Sapey; e.sapey{at}bham.ac.uk

Abstract

Objectives NHS England (NHSE) advocates ‘reason to reside’ (R2R) criteria to support discharge planning. The proportion of patients without R2R and their rate of discharge are reported daily by acute hospitals in England. R2R has no interoperable standardised data model (SDM), and its performance has not been validated. We aimed to understand the degree of intercentre and intracentre variation in R2R-related metrics reported to NHSE, define an SDM implemented within a single centre Electronic Health Record to generate an electronic R2R (eR2R) and evaluate its performance in predicting subsequent discharge.

Design Retrospective observational cohort study using routinely collected health data.

Setting 122 NHS Trusts in England for national reporting and an acute hospital in England for local reporting.

Participants 6 602 706 patient-days were analysed using 3-month national data and 1 039 592 patient-days, using 3-year single centre data.

Main outcome measures Variability in R2R-related metrics reported to NHSE. Performance of eR2R in predicting discharge within 24 hours.

Results There were high levels of intracentre and intercentre variability in R2R-related metrics (p<0.0001) but not in eR2R. Informedness of eR2R for discharge within 24 hours was low (J-statistic 0.09–0.12 across three consecutive years). In those remaining in hospital without eR2R, 61.2% met eR2R criteria on subsequent days (76% within 24 hours), most commonly due to increased NEWS2 (21.9%) or intravenous therapy administration (32.8%).

Conclusions Reported R2R metrics are highly variable between and within acute Trusts in England. Although case-mix or community care provision may account for some variability, the absence of a SDM prevents standardised reporting. Following the development of a SDM in one acute Trust, the variability reduced. However, the performance of eR2R was poor, prone to change even when negative and unable to meaningfully contribute to discharge planning.

  • Information management
  • Health policy
  • Quality in health care

Data availability statement

Data are available on reasonable request. The anonymised dataset used for analysis is available on reasonable request from the PIONEER Data Hub on submission of a data request form, see www.pioneerdatahub.co.uk for a copy of the form and processes for data access.

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • The intracentre and intercentre variability of reason to reside (R2R) reporting was based on national data and included >6.6M patient bed-days.

  • Standardised data model to form eR2R was based on nationally agreed criteria for each clinical question.

  • All admissions >24 hours were included for eR2R performance review, reducing bias.

  • eR2R data based on one centre only, although one of the largest National Health Service Trusts nationally serving a diverse population and including >1M patient bed-days.

Introduction

In 2021, the UK Government published its policy and operating model for hospital discharge and community support within the National Health Service in England (NHSE).1 This policy responded to concerns about bed capacity during the COVID-19 pandemic.

A National Audit Office report recognised the potential to release acute hospital beds in 2016, finding that older patients no longer needing acute treatment accounted for 2.7 million NHS hospital bed days per year.2 The report concluded that a lack of planning delayed discharge, recognising research that highlighted adverse outcomes during prolonged hospital stay.3 4

The aforementioned policy mandates using set criteria to identify in-patients in whom discharge home, or to a less acute setting, should be considered. These criteria have been referred to interchangeably, as ‘Reason[s] to reside’ (R2R), ‘right to remain’ or ‘criteria to reside’ (see table 1A). Since April 2020, NHS hospitals have been required to provide daily reports on the numbers of people leaving hospital, to where and the reasons for those remaining in hospital. The proportion of in-patients not meeting R2R criteria and the proportion of patients without R2R discharged that day are also reported. These metrics are considered to be measures of organisational efficiency.

Table 1

Reason to reside (R2R)

R2R appears to have emerged heuristically from the clinical experience of those involved in its development. A series of questions are posed that might prompt consideration of individual patients for discharge. However, there are no standardised data definitions, there has been no validation of R2R, no investigation of its role as a clinical decision support tool or of its value in evaluating hospital performance. A further barrier to evaluating the performance of R2R is that there is no gold standard definition that identifies patients who could be discharged from hospital against which to compare R2R performance. This lack of a reference standard limits, but does not preclude assessment of the validity of a clinical test, provided a ‘fair’ measure of performance can be defined.5 The set of patients actually discharged in the subsequent 24 hours is one potentially ‘fair’ test of performance of R2R.

In the current study, we show the degree of variation in R2R-associated metrics reported across centres in England. Second, we propose precisely defined, interoperable data definitions corresponding to the elements of R2R. This allows for consistent, generalisable analysis. Third, we evaluate the performance of R2R to predict discharge over the subsequent 24 hours.

Methods

All studies activities followed the World Medical Association’s Declaration of Helsinki. The R2R criteria are as described1 and are also provided in table 1.

National data

National NHS England data were accessed via The UK Health Facts and Dimensions database6 for all reporting Trusts in England. Assessment of variability in national R2R reporting included data from 29 November 2021 to 20 February 2022. Online supplemental table S1 provides the names of the Trusts whose data are presented anonymously. Data were collected daily during the censor period for 121 centres, yielding a total of 10 164 potential data points (centre-days). For each of these, the total number of occupied and unoccupied beds, and the number of patients with no right to reside were extracted. The number of patients with no right to reside were submitted once a day by each NHS trust, based on the local hospital interpretation of the definition provided by NHSE.1 This required none of the criteria to be met at the time of local data collection. The numbers of patients with right to reside were then calculated by subtracting the number with no right to reside from the total number of occupied beds on that day. The number of general and acute beds occupied in any given centre, on any given day (in-patients), was used as a surrogate for the number of patients eligible for evaluation using the R2R criteria. Review of the dataset found some missing and potentially spurious data, which were excluded prior to analysis. This included instances where R2R data were not recorded (n=184 data points), where the total numbers of beds were either zero, missing or clearly spurious (n=37 data points) or where there were more patients with no R2R than the total number of beds (n=3 data points). The national data are shown for the other n=121 centres, excluding UHB.

Local data

In-depth analysis of R2R criteria were performed using data from the Queen Elizabeth Hospital Birmingham (QEHB). QEHB is an NHS, urban, adult, acute hospital in England, which in 2019 had 1269 beds including 80 level 2/3 intensive care unit (ICU) beds, an emergency department that assesses >300 patients per day and a mixed secondary and tertiary practice that includes all major adult specialities except for obstetrics and gynaecology. The electronic healthcare record (EHR) at QEHB (PICS, Birmingham Systems) contains time-stamped, structured records that include demography, location, admission and discharge, comorbidities, physiological measurements supporting NEWS2 and Glasgow Coma Scale, operation noting, prescribing and investigations.

The R2R criteria in table 1 were mapped to computable definitions derived from the EHR (see table 2), to generate an electronic R2R (eR2R). The OPCS Classification of Interventions and Procedures codes mapped to criteria 9–11 are described in online supplemental table S2. The concept ‘acute functional impairment in excess of home/community care provision’ had no direct correlate. Safer Nursing Care Tool (SNCT) levels of care were however available.7 SNCT levels 2 and 3 correspond closely with the requirement for HDU or ICU.8 Level 1a identifies patients requiring enhanced nursing reflecting acuity of illness, and level 1b identifies a group with increased nursing dependency. Level 1b is likely to include those who would and would not be considered to require ongoing care in acute hospital. SNCT level 1 was included in the definition of eR2R in two ways, including (eR2Rab) and excluding (eR2Ra) level 1b, to determine if this affected performance.

Table 2

Data definitions used to operationalise R2R for EHR

The primary analysis of eR2R was for patients who had been in hospital for more than 24 hours at midnight. Discharge over the course of the subsequent 24 hours was evaluated. Secondary analyses were undertaken for the set of patients in a bed at 08:00 and at 16:00 to define any change in eR2R performance in these different cross-sections of the in-patient population. Three calendar years were analysed separately to assess the effects of the COVID-19 pandemic.

Statistics

Initially, daily numbers of patients with R2R quantified both as absolute numbers and a proportion of the total number of beds were plotted for national centres and used to calculate between-centre and within-centre variation. These data are analysed as beds occupied at the specified time of day, where the bed inherits the demographics, comorbidities and other qualities of the occupying patient. This represents the in-patient population in cross-section.

For the local analysis of eR2R: the term patient-day was used to refer to a bed with the qualities of the occupying patient at the time of the analysis. The in-patient population is described as means of patient-days thereby representing a cross-section of the group. The performance of eR2R as a predictor of remaining in hospital (or absence of eR2R as a predictor of discharge) was reported as a true positive rate (TPR) and true negative rate (TNR), positive predictive value (PPV), negative predictive value (NPV) and Youden’s J statistic (TPR+TNR-1), where positive is remains in hospital and negative is discharge from hospital within 24 hours.

Normally distributed variables are reported as arithmetic means±SD, with medians and ranges used otherwise. Between-centre variation was assessed by analysis of variance. This included a model accounting for day of the week as a fixed effect and the centre as a random effect. All analyses were performed using IBM SPSS V.22 (IBM Corp), with p<0.05 deemed to be indicative of statistical significance throughout.

Patient and public involvement

The research question and topic were agreed following patient/public discussion groups about NHSE discharge policies. Patients/public reviewed the data fields included in the study, with the PIONEER Data Trust Committee providing support for the project (a group of patient/public members who review studies using health data9). A patient/public group has reviewed the results and has written a lay summary for study dissemination to patient groups.

Results

R2R reporting in England, November 20–February 21

Across 10 164 available centre-days, accounting for 6 602 706 patient-days, the number of patients reported without R2R as a proportion of in-patients varied significantly between centres (p<0.0001). Individual centre means ranged from 6.7%±2.5% to 59.9%±13.8% (figure 1A). There was also marked within-centre variation (figure 1A), with coefficients of variation (CV) ranging from 8.2% up to 59.3%. Of patients not meeting R2R criteria, the proportion discharged over the following 24 hours, varied significantly between centres (p<0.0001). Individual centre means ranged from 14.0%±7.4% to 85.8%±25.2% (figure 1B). There was also marked within centre variation, with CV ranging from 6.4% up to 83.2%. These data are shown as median and IQR in online supplemental figure S1A,S1B). The proportion of patients without R2R and the proportion of that group discharged within 24 hours were only weakly correlated (R2=0.12; Online supplemental figure S2).

Figure 1

National reporting of R2R criteria. The proportion of patients with no R2R (A) and of that group the proportion of patients discharged within 24 hours (B) reported to Strategic Data Collection Service (SDCS) NHS Digital, UK from 29 November 2021 to 20 February 2022 across 121 centres. Each dot represents result for a single centre-day. We have ordered centres in both A and B according to the median value of proportion of patients with R2R (see online supplemental figure S3 for median and IQRs). R2R, reason to reside.

Performance of eR2R at QEHB

Standardised definitions corresponding to the elements of R2R (table 2) were used to analyse data from QEHB, on 1 214 480 in-patient days, between 1 January 2019 and 31 December 2021. The demographic and clinical details of that population are summarised in table 3, which also shows that those meeting the definition of eR2Rab were older and more likely to have one or more comorbidities than those who did not. Variation in the daily number of patients with or without an eR2R is shown in online supplemental figure S3.

Table 3

Demographics of patients meeting and not meeting R2R criteria on presentation to QEHB in the censor period

Criteria contributing to eR2R

Given the potential for the COVID-19 pandemic to affect R2R, calendar years were analysed separately. The number of patients meeting any given eR2R criterion are shown in table 4A. The progressive contribution of different elements of the definition of eR2R assessed daily in a modified Consort table are summarised in table 4B. The proportion of patients not meeting eR2R criteria exhibited relatively little day-to-day variation in 2019 (eR2Rab, CV=11.2%; eR2Ra, CV=6.3%), although somewhat higher in the context of case mix variation consequent on peaks of patients admitted with COVID-19 in 2020 (eR2Rab, CV=23.3%; eR2Ra, CV=14.4%) and 2021 (eR2Rab, CV=17.1%; eR2Ra, CV=9.9%). The criteria contributing most to eR2R status included acuity level (NEWS2>3), SNCT level nursing requirement, being on intensive care and requiring intravenous medications or fluids.

Table 4

(A) The number (percentage) of patient-days on which each eR2R data definition was met. (B) A phased analysis undertaken for each day and presented as a modified Consort diagram

Informedness of eR2R for discharge in the next 24 hours

For the outcome discharge (remain −)/no discharge (remain +) within 24 hours, across the three different years, the eR2Ra TPR lay between 0.63 and 0.65, TNR between 0.46 and 0.47, the PPV was 0.91 and NPV between 0.12 and 0.15; the eR2Rab TPR lay between 0.88 and 0.91, TNR between 0.18 and 0.24, the PPV between 0.90 and 0.91 and NPV between 0.18 and 0.20 (table 5). The J statistic for both definitions lay between 0.09 and 0.12. In secondary analyses based on the in-patient population at 08.00 and at 16.00 the J-statistic ranged between 0.10–0.14 and 0.10–0.15, respectively (online supplemental table S3A,S3B).

Table 5

Contingency tables showing the number of patients meeting criteria for (A) eR2Ra and (B) eR2ab

In-patients not meeting eR2R

The demographic and clinical details of patient who did not meet the eR2Rab definition, stratified by discharge in the subsequent 24 hours, are shown in online supplemental table S5. For patient-days on which discharge occurred within 24 hours, there was significantly higher representation of those with no documented comorbidities 29.2% vs 24.0% (p<0.0001). In those that remained in hospital, 61.2% met eR2R criteria on subsequent days (76% within the next 24 hours). Of all those that remained, 21.9% acquired a NEWS2>3, 32.8% received iv fluids or drugs >3 times/day and 1.9% were admitted to ICU.

Discussion

Assessment of an individual patient’s R2R has been promoted as a tool to improve the identification of those who could be discharged from acute hospitals in England. The proportion of in-patients with R2R and their rate of discharge has then been used to evaluate the operational efficiency of acute hospitals and their adjacent health and social care system.1 10 This paper presents findings to suggest that as currently constituted, R2R is of limited value for these purposes.

The high levels of variation in R2R-related metrics, within and between centres in England, has been attributed to variation in case mix and operational efficiency.11 However, such extremes of variation are not observed in other metrics that use established data standards. Furthermore, the proportion of patients not meeting R2R criteria correlates poorly with their rate of discharge over the subsequent 24 hours, whereas one might anticipate that such closely related measures of operational efficiency would reflect one another. These findings are most obviously accounted for by the fact that R2R does not constitute a semantic data model. It is therefore susceptible to differing interpretation by individuals and centres. This applies to all the concepts described by R2R, but most obviously those that are necessarily subjective, such as ‘acute functional impairment in excess of home/community care provision’ and ‘diminished level of consciousness where recovery is realistic’.12 13

We therefore developed machine readable data definitions corresponding to each concept, allowing consistent analysis of R2R at scale, using data derived from the EHR in our centre. The SNCT is a global nursing assessment of acuity and dependency that was developed to guide workforce deployment. It is regularly recorded within the EHR at our centre. Because level 1b describes a group of patients who are highly dependent on nursing care for daily activities, this was mapped onto the R2R concept ‘acute functional impairment in excess of home/community care provision’. However, since the definition of level 1b could include a group of patients suitable for discharge to a less acute setting, two definitions or eR2R were tested, with and without SNCT 1b. Our analysis is therefore likely to represent two extremes of inclusion of patients with acute functional impairment.

Within centre variation in eR2R was low, consistent with it minimising individual interpretation of each data element. eR2R was a poor predictor of discharge within 24 hours.14 Youden’s Index was consistently <0.15 across three calendar years, three different times of day and two eR2R definitions. For a dichotomous test such as eR2R, a Youden’s Index >0.50 is generally considered the empirical benchmark for a test to support clinical decision making.15 eR2R is therefore unsuited to the provision of clinical decision support tool for discharge. It does not define a subpopulation on which to assess discharge performance.16 The limitations of R2R are not entirely surprising, given the need to interpret concepts that are not semantically defined. Although addressed by eR2R, it nevertheless remains a simple series of binary responses to questions that have not been validated for the purpose of discharge prediction. For example, NEWS2 was validated as an acuity score to quantify physiological instability on initial presentation to hospital.17 It was not developed and has not been validated, as a triage tool to assess fitness to leave hospital, at any threshold.

Importantly, more than half of those who remain in hospital without eR2R, subsequently acquired eR2R. This group of patients were older and had multiple long-term health conditions, suggesting that there were clinical grounds for that decision, although undefined. This subpopulation requires further study.

There are limitations to our analysis. The eR2R was assessed in only one centre, although one that serves a diverse, multiethnic, urban population, in which more than 1.2 million patient days were assessed. Patients admitted for <24 hours at the time of analysis were excluded to allow clinical decisions to be made and executed. The first day postadmission is a highly dynamic situation, with frequent clinical review, a setting in which this embodiment of clinical decision support is arguably less relevant. Another more intrinsic problem is that there is no gold standard by which to define all patients suitable for discharge so that actual discharge was used as a fair test when evaluating the performance of eR2R.18 This assumes that patients actually discharged are part of a continuous population of all those who could be discharged. It is also the case that each R2R element could be defined or operationalised in slightly different ways by healthcare professionals when being applied in clinical settings. Our data analysis, with clear definitions for each parameter within the eR2R does not include the ‘art’ of clinical medicine but does enable consistent comparisons to be made across time and localities.

It is important to validate and evaluate tests within their intended setting. The effects of embedding new care pathways or tools within clinical service delivery, without appropriate evaluation, are increasingly described. There is significant opportunity for unintended consequences to arise from the implementation of poorly considered clinical decision support,19 particularly when there is competition for clinical resource. This has been recently discussed for NEWS2,20 sepsis alerting and COVID-19 virtual wards.21 R2R has been endorsed and adopted but without validation or consideration of the unintended consequences of its application. This is not to contend that a significant number of in-patients could not be discharged earlier, simply that there is no evidence that R2R can support clinical decision making. The collective limitations of R2R identified are likely to account for variation in nationally reported metrics that are difficult to explain.

Our study highlights the need for reproducible standardised data definitions to support both implementation and validation of any tool that purports to support clinical decision making. Further research should focus on building, validating and refining tools to inform clinical decisions.

Data availability statement

Data are available on reasonable request. The anonymised dataset used for analysis is available on reasonable request from the PIONEER Data Hub on submission of a data request form, see www.pioneerdatahub.co.uk for a copy of the form and processes for data access.

Ethics statements

Patient consent for publication

Ethics approval

This study used unconsented, anonymous health data, and all study activity was approved by the East Midlands–Derby REC (reference: 20/EM/0158). Specific approvals were provided by East Midlands–Derby REC (reference: 20/EM/0158) to use unconsented, anonymised health data.

Acknowledgments

This work was supported by the PIONEER Data Hub (see www.pioneerdatahub.co.uk) and by the PIONEER patient and public advisory group and Data Trust Committee.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • ES and SG are joint first authors.

  • Contributors SB and ES conceived the study; SG, FE, DM and KR conducted data analysis. ES and SG wrote the first draft of the study. All authors contributed to the study manuscript. SB is senior author and manuscript guarantor.

  • Funding This work was funded by Health Data Research - UK (PIONEER2019).

  • Competing interests FE, DM and KR have no relevant conflicts of interest. SG reports grant funding from HDR-UK. ES reports grant funding from HDR UK, Innovate UK, MRC, NIHR, British Lung Foundation and Alpha 1 Foundation. SB reports funding from HDR-UK.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.