Article Text
PDF
Abstract
Objectives To evaluate the impact of comorbidities, acute illness burden and social determinants of health on predicting the risk of frequent hospital admissions.
Design Multivariable logistic regression was used to associate the predictive variables extracted from electronic health records and frequent hospital admission risk. The model's performance of our predictive model was evaluated using a 10-fold cross-validation.
Setting A single tertiary hospital in Singapore.
Participants All adult patients admitted to the hospital between 1 January 2013 and 31 May 2014 (n=25 244).
Main outcome measure Frequent hospital admissions, defined as 3 or more inpatient admissions within 12 months of discharge. Area under the receiver operating characteristic curve (AUC) of the predictive model, and the sensitivity, specificity and positive predictive values for various cut-offs.
Results 4322 patients (17.1%) met the primary outcome. 11 variables were observed as significant predictors and included in the final regression model. The strongest independent predictor was treatment with antidepressants in the past 1 year (adjusted OR 2.51, 95% CI 2.26 to 2.78). Other notable predictors include requiring dialysis and treatment with intravenous furosemide during the index admission. The predictive model achieved an AUC of 0.84 (95% CI 0.83 to 0.85) for predicting frequent hospital admission risk, with a sensitivity of 73.9% (95% CI 72.6% to 75.2%), specificity of 79.1% (78.5% to 79.6%) and positive predictive value of 42.2% (95% CI 41.1% to 43.3%) at the cut-off of 0.235.
Conclusions We have identified several predictors for assessing the risk of frequent hospital admissions that achieved high discriminative model performance. Further research is necessary using an external validation cohort.
- Frequent hospital admission
- comorbidities
- illness burden
- social determinants of health
- depression
- readmission
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- Frequent hospital admission
- comorbidities
- illness burden
- social determinants of health
- depression
- readmission
Strengths and limitations of this study
Study identified discriminative predictors for frequent admission risk which are easily retrievable from the electronic health record.
High discriminatory power with acceptable sensitivity and specificity.
Predictors on functional status and patient behaviour were not available.
Introduction
Patients with frequent admissions consume a disproportionately large amount of healthcare resources and contribute to bed shortages in hospitals.1 Moreover, frequent admitters (FAs) experience significant psychological stress and financial burden.2 In countries with a rapidly ageing population, increasing life expectancy, rising chronic disease burden and decreasing working population, are factors that contribute to straining the finite healthcare resource further. There is consensus that reducing avoidable hospital readmissions require identification of patients at highest risk for readmissions, followed by identification of potentially modifiable risk factors that increase readmission risk.3 Resource intensive discharge planning and postdischarge monitoring can then be targeted at these risk factors.3 Finally, community-based home care services, primary care and case management could be provided to such high-risk individuals to optimise their care in the community and keep them out of hospitals.
In the USA, published prediction models have focused on predicting for 30-day readmission risk, in line with avoiding Medicare penalties for excessive readmission rates under the Readmission Risk Prevention Program.4 However, there is a case to be made for taking a longer term view, especially in predicting high-cost users. It is well established that a high proportion of health expenditures are consumed by a small percentage of individuals. In the USA, the top 10% of residents in terms of healthcare costs accounted for 64% of all health expenditures.5 Beyond the postdischarge transitional period, a longer term strategy in the form of community-based case management and care integration are important to build healthy communities and cope with the complex healthcare needs of rapidly ageing populations. The ability to predict future high-cost users in the following 12 months will allow time for intervention by targeted transitional care programmes aimed at modification of behavioural risk factors, addressing social and functional care needs, with potentially large benefit. So far, most models are derived using administrative data and perform poorly in predicting 30-day readmission risk, with only six studies reporting a C-statistic >0.7.6 To date, no published literature has investigated the effect of severity during acute illness, such as the treatment with high-risk medications and requirement for dialysis during the hospitalisation episode, on readmission risk. Similarly, the index discharge diagnosis would be an important determinant of readmission risk4 but is rarely investigated in the published literature.
Singapore is a developed city state in Southeast Asia with a multiethnic population of 5.6 million people. Its population is one of the most rapidly ageing in Asia with an increasing chronic disease burden.7 Healthcare expenditure is expected to triple from S$4 billion in 2011 to S$12 billion in 2020. Like most countries in the developed world, the main cost driver of healthcare in Singapore is inpatient care.8 Reducing unnecessary readmissions and providing care at the most appropriate setting is the focus of health systems reforms aimed at reining in escalating healthcare costs. In Singapore, FAs are defined as patients with three or more inpatient admissions in 12 months. FAs are high-cost patients with an average cost per patient approaching SG$30 000.9 Interestingly, a cumulative direct cost of 30 000 Canadian dollars (SG$32 260) marks the top 5% of healthcare resource users in Ontario, Canada.10 Taking a population health approach, the Ministry of Health (MOH) Singapore has created six regional health systems (RHSs) in 2011, each being responsible to integrate care for a specific geographic region. The Singapore Health Services (SingHealth) RHS is the largest healthcare cluster in Singapore providing care for the south-central part of Singapore. The Singapore General Hospital (SGH) is the flagship hospital of the SingHealth RHS and the largest tertiary care hospital in Singapore with 37 clinical specialities and 88 000 inpatient admissions each year. Each RHS will receive capitated funding to manage and reduce readmission risk of patients at high risk of frequent admissions in its catchment population. Since 2014, SingHealth RHS has trained patient navigators11 to function as case managers and reorganised its transitional home care service into an integrated practice unit12 to coordinate the care of patients through the complex healthcare system for the 1 year after hospital discharge.
For interventions to be targeted at patients at highest risk, we sought to identify predictors of frequent hospital admission risk. These candidate predictors should be readily available in the electronic health records (EHR) for a predictive score to be automated or can be easily retrieved from patient medical records and entered into an online spreadsheet or smartphone/tablet application to facilitate clinical use.
Our primary objective was to investigate the impact of comorbidities, acute illness burden and social determinants of health on predicting frequent readmission risk (defined as three or more inpatient admissions within 12 months of discharge) and measure the performance of our prediction model using receiver operating characteristic (ROC) analysis, sensitivity, specificity and precision.
Materials and methods
Study design and population
This was a retrospective single-centre cohort study designed to investigate the impact of comorbidities, acute illness burden and social determinants of health on the risk of frequent hospital admissions. All admitted adult patients (≥21 years of age) from 1 January 2013 to 31 May 2014 at the SGH were enrolled. Only index admissions were included. Patients were excluded if they were non-residents or died during the index admission. Patients were also excluded if their admission specialty was obstetrics, emergency medicine, dentistry or ophthalmology. Emergency medicine admissions were excluded as these were observation ward admissions rather than true hospital admissions. Patients admitted to the emergency medicine observation ward are typically monitored up to 24 hours and subsequently discharged home or converted to hospital admissions according to their clinical circumstances. Patients converted to hospital admissions would be captured in our data set. We excluded patients admitted to obstetrics as these admissions are pregnancy related; and admissions to dentistry and ophthalmology which are usually elective in nature.
Outcomes
The primary outcome was frequent hospital admissions, which were defined as three or more inpatient admissions within 12 months of discharge.
Predictive variables
Candidate variables were identified a priori and according to the medical literature.6 ,13–15 A total of 44 variables were selected and extracted from the hospital's EHR, including patient demographics (age, gender and ethnicity) and prior healthcare usage (number of hospital admissions during the past year, number of emergency department visits in the past 6 months and number of specialist outpatient clinic visits in the past year). Comorbidities (major diseases listed under the Charlson Comorbidity Index (CCI) and Elixhauser Comorbidities)16–18 were derived by extracting the International Classification of Diseases (ICD)-9 and ICD-10 codes dating back to 7 years. We believe that this is most comprehensive among the published literature19 and would account for potential lapses in diagnostic coding.
We included social determinants of health in our list of predictors. We used eligibility for admission to subsidised ward class and assistance by Medifund as predictors of low socioeconomic status. Singapore uses a tiered system of healthcare subsidies that is adjusted using means testing. Patients in the lowest socioeconomic tier receive the highest tier of subsidy. Such patients are characterised by their entitlement for admission to the ward class that receive the highest subsidy. In addition they are eligible to receive financial support from Medifund. Medifund is an endowment fund set up by the Singapore government to help needy Singaporeans.20 Medifund is a safety net for patients who face financial difficulties with their remaining hospital bills after receiving government subsidies and drawing on other means of payments including insurance. Therefore, patients who needed financial assistance using Medifund would belong to the neediest in society.
We studied markers of acute illness burden as predictors. These included length of stay of the index admission; ‘treatment with intravenous furosemide 40 mg or more’; ‘treatment with second-line intravenous antibiotics’ (defined as piperacillin-tazobactam, meropenem, vancomycin and moxifloxacin) and ‘required dialysis’. Our study is also novel in weighting the index discharge diagnoses using the CCI, which is already a validated marker for mortality within 1 year.17
The psychological state of patients is known to be an important factor in coping with illness. The presence of a debilitating level of mood disorders was inferred from treatment with antidepressants in the past 1 year. We used treatment with antidepressants as it is a more accurate indicator of depressive mood disorders, of which many patients are treated as outpatients and would not have discharge ICD codes for depression. The other comorbidities (table 1) were identified using ICD-10 codes18 in any primary or secondary diagnosis fields dating back to 7 years preceding the index admission.
Baseline characteristics by frequent hospital admission status
Statistical analysis
The demographics and clinical characteristics of the recruited population were described and compared among patients who were FAs and who were not. We analysed continuous variables using Student's t-test and the categorical variables using χ2 test or Fisher's exact test when appropriate. Univariable logistic regression was implemented to evaluate the associations between each of the candidate predictors and the outcome. Multivariable logistic regression with stepwise variable selection was used to build predictive models. Statistical significance was set as p<0.05.
To assess the predictive performance, 10-fold cross-validation was conducted for model evaluation. First, the whole data set was evenly divided into 10 non-overlapped subsets. Second, a predictive model was built with nine subsets of data and validated on the remaining one subset. Third, the second step was repeated another nine times so that all subsets were tested. Finally, the ROC analysis was conducted to evaluate the model performance. Further measures of diagnostic accuracy such as sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were presented. Data analyses in this study were conducted in R V.3.2.3 (R Foundation, Vienna, Austria).
Results
A total of 25 244 patients were included in this study, of which 4322 patients (17.1%) had three or more inpatient admissions within 12 months of discharge and were classified as FA. Table 1 describes the baseline characteristics of the study population. The mean age of FAs was 65 (SD=15) years and the mean age of non-FAs was 57 (SD=18) years. There were 13 010 male patients (51.5%). Most admissions were emergent (76.5%, n=19 320). Mean length of stay was 5.59 days. On average, FAs had a mean CCI of 3.62 and non-FAs had a mean CCI of 1.06. The readmitted patients were older compared with the non-readmitted patients, had significantly longer length of stay during the index admission, higher CCI and more emergency department admissions in the preceding 6 months.
The multivariable logistic regression was used in predicting frequent hospital admissions. Eleven variables were found to be significantly associated with frequent hospital admissions, that is, three or more inpatient admissions within 12 months of discharge. As shown in table 2, these variables were age, gender, eligibility for financial assistance using Medifund, number of visits to the emergency department 6 month before index admission, number of visits to specialist clinic 1 year before index admission, number of hospital admissions 1 year before index admission, admission to a subsidised ward class during index admission, required dialysis during index admission, treatment with intravenous furosemide 40 mg and above during index admission, treatment with antidepressants in the past 1 year, and CCI. The top two strongest independent predictors were treatment with antidepressants in the past 1 year (adjusted OR 2.51, 95% CI 2.26 to 2.78) and ‘required dialysis’ during index admission (adjusted OR 2.33, 95% CI 1.98 to 2.74). The odds of frequent hospital admission in the following 12 months for patients who were treated with intravenous furosemide 40 mg and above were 1.49 times higher than the odds of those who were not treated with the medication (adjusted OR 1.49, 95% CI 1.308 to 1.701). Every increase of one point on the CCI score was associated with 33% increased odds of becoming a FA (adjusted OR 1.33, 95% CI 1.312 to 1.347).
Multivariable logistic regression analysis
Based on 10-fold cross-validation, the final logistic regression model had high discriminative ability (area under the curve (AUC) 0.84, 95% CI 0.83 to 0.84; figure 1). The optimal cut-off for the regression model was 0.235, which achieved a sensitivity of 73.9% (95% CI 72.6% to 75.2%) and a specificity of 79.1% (78.5% to 79.6%). By choosing three different cut-offs, we stratified the patients into four groups, namely high risk, moderate risk, low risk and minimal risk. Table 3 shows the sensitivity, specificity, PPV and NPV according to these cut-offs. Table 4 elaborates the number of patients stratified into four risk groups as well as the corresponding FA ratios. For example, a cut-off score of 0.442 predicted 2525 patients as FAs while 1455 (57.6%) patients were true positives.
Prediction results by the multivariable logistic regression model using ROC analysis
Patient stratification in four different risk groups
Area under the curve of final logistic regression model.
Discussion
In this retrospective cohort study of 25 244 eligible adult patients, we identified 11 predictors associated with frequent admission risk after hospital discharge in Singapore. In deriving these predictors, we have intentionally selected variables that are readily available in the EHR. This will facilitate the development of a practical predictive scoring tool that can be easily derived from readily available data. This in turn will enable automation and deployment as online spreadsheets or smartphone/tablet application to facilitate clinical use. Our study intended to identify predictors of frequent hospital admission risk that has the potential to be developed into a parsimonious risk score as future work. A risk score can function as a rapid risk stratification tool to identify patients at high risk for frequent admissions among the many admissions to our health system each day. This risk stratification must then be followed by an assessment of the high-risk patient to identify potentially modifiable risk factors for intervention. Similarly, the assessment will identify patients who are going to be in rapid decline with unavoidable readmissions but can benefit from appropriate interventions that are aligned with the patients' goals of care.
Owing to the large sample size of our study, most p values in univariable analysis were very small and we conducted multivariable analysis to select significant predictors. We started with 44 variables, and we found that 11 factors explained the majority of risk associated with frequent admissions. This supports the growing evidence that simplified prediction models using a few critical predictors can achieve a high level of accuracy in its predictions. While no studies have specifically evaluated predictors for FA risk, our findings that increasing age, prior hospital usage and increased FA risk were consistent with the existing literature on 30-day, 90-day and 12-month readmission risk.6 ,13 In the final multivariable logistic regression model, chronic disease comorbidities were not independently associated with FA risk. A likely explanation is that the impact of comorbidities is confounded by usage in the past 1 year. Another explanation is that a patient's past comorbidities may have lesser significance on his current risk than recent events. We found that the effort needed to identify and retrieve individual diseases listed in the Charlson and Elixhauser comorbidity measures is not efficient in determining FA risk.
Contrary to expectations, metastatic solid tumours and tumours without metastases did not increase FA risk, in contrast to the published literature on 30-day readmission risk.21 ,22 Study population differences could have accounted for the difference. Another possible explanation could be that these patients may be in remission or had received appropriate management such as advance care planning and did not become FA. It is also possible that these patients passed away after the index hospital discharge and did not accumulate three or more hospital readmissions in the following 12 months to become a FA. Interestingly, having a pressure ulcer trended towards significance for increasing FA risk. A pressure ulcer may be a proxy marker for poor functional status and the quality of nursing care for the patient. Patients who develop a pressure ulcer tend to be bedbound and have a high level of dependency on caregivers for their activities of daily living, including regular turning in bed to prevent the development of pressure ulcers. Functional status is a known risk factor for 30-day readmission.23 Caregiver availability is likely to be an important predictor and its inclusion is likely to improve the model performance in future studies.
We explored the use of proxy markers of severity of the acute illness episode as a predictor of FA risk. To the best of our knowledge, no previous studies have evaluated the impact of high-risk medications used during the hospitalisation although there was a study that focused on high-risk medications at discharge.22 Intravenous furosemide is used when prompt and effective diuresis is required and its use is an indicator of a state of severe fluid overload which required urgent reversal. Second-line antibiotics are used to treat severe hospital-acquired infections.24 Its use during hospitalisation indicates the presence of a serious infection. Similarly, patients with end-stage renal failure (ESRF) requiring dialysis suggests that they are undergoing active treatment and therefore at higher risk for FA. We found that the odds of becoming FA in the following 12 months for patients with ESRF requiring inpatient dialysis and patients who were treated with intravenous furosemide 40 mg and above were 2.33 times and 1.49 times higher than the odds of patients who did not require dialysis and were not treated with the medication, respectively. It is possible that undergoing inpatient dialysis and intravenous furosemide better reflect current illness severity and FA risk compared with pre-existing comorbidity burden. CCI is a validated measure of 1 year mortality17 and we adapted its utility to weight the index admission diagnoses as a composite measure of current illness burden. We found that every increase of one point on the CCI score was associated with 33% increased risk of becoming a FA. It would be interesting to evaluate if these markers of acute illness severity also increase the risk of 30-day readmission.
In Kansagara's review of readmission risk prediction models, only 6 out of the 26 unique models used 12 months as the outcome period of interest,19 ,25–29 while the majority used a shorter time interval of 30–90 days.6 The current preoccupation with 30-day readmission is only of value if it determines patients who return to hospital because of bad care, and not those who would otherwise need frequent hospital admissions that our model strives to identify. Five were on general medical inpatients while one focused on patients with congestive heart failure. There are significant differences worth highlighting between these models and ours. First, five of the models predicted readmission in the 12 months following index discharge making it difficult to compare their models with ours. The remaining model by Bottle and colleagues predicted ‘high-impact users’ who will have at least two or more emergency hospital admissions in the following 12 months, but the model was developed from administrative data. The model performance in terms of discriminatory ability was also poor, with five of the models having an AUC of <0.7. In comparison, our final regression model had high discriminative ability with an AUC of 0.83.
There is potential to explore other variables of interest in the future. These include functional status, health literacy, markers of social instability and caregiver availability.6 ,30 At the moment, these data are not routinely collected in most health systems although there is an intention to do so as part of a population database in our RHS. In the interim, we have focused on identifying predictors that can be easily retrieved from a patient's medical records or can potentially be automated through the EHR. Therefore, we have ignored variables that require additional collection by healthcare workers. Taha et al31 explored polypharmacy and problem medications such as anticoagulants and opioids on discharge. While these are available in our EHR system, the discharge prescription is among the finalised documents given to a patient on discharge. Delays in obtaining this information would have limited its usefulness in deriving a risk score on admission for the case managers and clinicians.
Limitations
We believe that our study had revealed many novel predictors that can be used in developing more effective and practical predictive models. However, there are limitations to our study. First, variables in our data set are restricted to those routinely collected in the EHR and administrative databases. As such, information about the social economic status of the patients is not detailed and we used tested access to financial support as a proxy. Functional status, caregiver availability and degree of social support could have improved the performance of our model but we did not include these predictors as such data are not routinely collected in our healthcare settings and our intention was to develop an easy-to-use and generalisable risk score. Second, due to the retrospective nature of the study, we were unable to prove a causal association between the predictor variables and frequent hospital admissions. After our predictive model has identified patients at high risk for frequent hospital admissions, intervention programmes would have to identify potentially modifiable risk factors. Finally, we did not exclude patients who might have deceased after index hospital discharge. We felt that that would have biased the prediction model as these patients could have had frequent hospital admissions before death.
Conclusion
In this retrospective cohort study, we investigated the impact of comorbidities, acute illness burden and social determinants of health on the risk of frequent hospital admissions and evaluated the performance of these variables on predictive modelling. We identified several strong predictors such as treatment with antidepressants and requiring inpatient dialysis during index admission. Our final regression model had high discriminative ability for the prediction of frequent hospital admission risk with an AUC of 0.84. We intend to follow-up with implementation of such a predictive tool in our RHS; however, a carefully designed model and external validation are needed to ensure reproducibility.
References
Footnotes
Contributors LLL and NL conceived and designed the study. LLL, NL and SW performed the study and analysed the data, including modelling and cross validation. LLL, NL, SW and KHL interpreted the results. LLL and NL wrote the first draft of the paper and all authors critically revised the paper and gave final approval for publication.
Funding This research received funding from Singapore Ministry of Health (MOH) Health Services Research New Investigator Grant HSRNIG14nov002 and SingHealth Foundation Health Services Research (Aging) Startup Grant SHF/HSRAg004/2015.
Competing interests None declared.
Ethics approval This study was approved by Singapore Health Services (SingHealth) Centralized Institutional Review Board CIRB 2015/2696 with a waiver of informed consent.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Details of the variables and derived predictive model are available from the corresponding authors at low.lian.leng@singhealth.com.sg or liu.nan@singhealth.com.sg.