Eligibility criteria

We have systematically identified peer-reviewed original studies, with internally or externally validated clinical scores of unplanned hospital readmissions in elderly inpatients. We excluded reviews and studies identifying readmission risk factors, developing a statistical prediction model for readmission and clinical score not having been, previously, internally validated. Indeed, statistical prediction models give a probability of a patient being readmitted, whereas clinical risk scores give a number which can be compared with a threshold from which the patient is at high risk of readmission. Clinical scores are, thus, easier to use routinely.

We included studies:

• In which participants were elderly inpatients (65 years old and over) discharged to home after a hospitalisation or returning home after an accident and emergency department (AED) visit.

• In which the main outcome measured was unplanned readmission or return to the AED within a month (28, 30 or 31 days) after discharge. We excluded studies that focused on specific conditions.

• Where the score discrimination was evaluated using the c-statistic. The graphic representation of the c-statistic is the receiving operating characteristic (ROC) curve. We excluded studies which had used neither the c-statistic, nor the area under the curve (AUC) of the ROC curves for the validation.

Information sources and search strategy

We identified potentially eligible studies by searching in four databases MEDLINE EMBASE, COCHRANE and Web of Science. Our search was restricted to original human studies, published in French or English, between 1966 and 2017. The search strategies of the four databases were approved by a university librarian (online supplementary file 2). The electronic search was conducted on 13 January 2017 and updated on 19 January 2018. In addition to searching databases, we checked reference lists of selected studies and researched also the studies citing the selected studies, on Web of Science.

Study selection

After identification, studies were screened independently by two reviewers (CS and VK-S) on the basis of title and abstract. Then, the full text was read for potential inclusion in the systematic review and agreement between the two reviewing authors (CS and VK-S) was assessed using Cohen’s κ statistic.13 A Cohen’s κ coefficient comprised between 0.60 and 0.74 was considered as good, and very good if superior to 0.75. In the case of disagreement, a third reviewer (PH) was a priori designed to be consulted.

Data collection process

We developed a data extraction grid based on The Strengthening the Reporting of Observational Studies in Epidemiology statement: guidelines for reporting observational studies14 and the Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement.15 General data extracted from full-text included: authors, year of publication, design, setting, participants, data sources, study size, quantitative variables and statistical analysis methods. For each selected study, CS extracted the general data and completed the grid. Independently, VKS checked all the data collected in the grid. Disagreements were resolved by discussion and consensus.

Using the same process, we extracted metrological data (AUC of the ROC curves, sensitivity, specificity, positive predictive value, negative predictive value, negative and positive likelihood ratio and accuracy) of the selected scores. Discrimination was considered fair for a c-statistic comprised between 0.5 and 0.69, and good when the c-statistic was superior to 0.7.

Quality assessment

Assessment of quality in individual studies was performed independently by two reviewers (CS and VK-S), using the validated Effective Public Health Practice Project (EPHPP) quality assessment tool.16 This instrument, usable for any quantitative study design, was adapted to our study design in order to evaluate the quality through five sections: selection bias, study design, data collection method, statistical analysis and withdrawals and dropouts. The quality was individually rated for each component (strong, moderate or weak) and added to assign a global rating to each study. The global rating followed the EPHPP guidelines: a strong global rating means there is no weak rating, a moderate global rating means there is one weak rating and a weak global rating means there are two or more weak ratings.

The quality across studies was assessed using a graphic representation of the percentage of studies with a strong, moderate or weak rating for each sections considered in the evaluation of the quality in individual studies.

Patient and public involvement statement

No patient was involved in this systematic review.

Result of search strategy

From the four databases, we extracted 2484 references (online supplementary file 3). After removal of 320 duplicates, we screened 2164 studies on the basis of titles and abstracts leading to the exclusion of 2075 irrelevant publications. A total of 93 publications were reviewed for eligibility. Nine studies finally met the inclusion criteria. We found three additional records by manual search (by checking the reference lists). Thus, a total of 12 studies17–28 and five different scores were included in the present analysis. The degree of agreement between the two reviewers was good, with a κ coefficient of 0.72 (95% CI 0.50 to 0.93).

Study characteristics

Study characteristics are presented in table 1. The 12 included studies were published between 2000 and 2017. Six were conducted in Europe,17–20 26 27 four in North America (the USA and Canada),21–23 28 one in Asia (Singapore)24 and one in Australia.25 Design, population, interventions and outcome will be detailed below.

Scores characteristics

The scores characteristics will be detailed bellow, by presenting predictors and measure of predictive performance (online supplementary files 4, 5 and 6).

Predictors

The median number of items included in the scores was 5 (range 3–8). As depicted in figure 1, the predictors composing the five scores can be divided into five categories: demographics, functional impairment, hospitalisation, polymedication and comorbidity. Functional impairment and comorbidity are the most used predictors, whereas demographics are used in one score (nomogram). The assessment of each category differs between scores, eg, the functional impairment can be measured with the capacity to walk (TRST), the ability to use a phone (variable  indicative of placement (VIP)) or the presence of a home carer (nomogram). Likewise, polymedication varies from more than three drugs to more than 6. All the categories are not use in every score, eg, the LACE index assesses readmission risk with hospitalisation characteristics (length of stay, admission type and history of AED visit) and comorbidity.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Categories of predictors included in risk scores.

Each predictor can be obtained at admission, except for the length of stay (LACE index) which will be obtained at discharge.

Measure of predictive performance: As depicted in figure 2, the c-statistics for the ISAR score were close and comprised between 0.608 and 0.690, except for Braes et al study,18 whereas the c-statistics of the TRST score were scattered and comprised between 0.48 and 0.65. The TRST score had a better discrimination (AUC of the ROC curve of 0.647) in the internal validation study than in the external validation studies. The three other scores (VIP, nomogram and LACE) have fair discriminations: the c-statistics were comprised between 0.5 and 0.65, the nomogram having the best c-statistics (0.65). Moreover, the AUC of the ROC curves of the ISAR score were quite similar: 0.55 (95% CI 0.49 to 0.61) and 0.595 (95% CI 0.581 to 0.608), respectively.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

c-Statistics and sample sizes of included studies.

As depicted in figure 3, TRST had a higher specificity and a lower sensitivity—the sensitivities varied from 0.62 to 0.871 and the specificities from 0.215 to 0.63—whereas ISAR has a higher sensitivity and a lower specificity—the sensitivities varied from 0.79 to 0.918 and the specificities from 0.19 to 0.40—and they showed good reproducibility. The two other scores, LACE and VIP, with only one study for each, showed better sensitivity and specificity than ISAR and TRST but had been externally validated only once. Finally, the specificity and sensitivity of the nomogram were not available.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Sensitivity, specificity and c-statistics of the scores Identification of Senior At Risk (ISAR), lengths  of stay,  admission, Charlson,  emergency (LACE), Triage Risk Screening Tool (TRST) and variable  indicative of  placement (VIP).

In the external validation study by Braes et al,18 the VIP score showed an AUC of the ROC curve of 0.516, a sensitivity of 0.22 and a specificity of 0.81 (online supplementary file 6). The others metrological qualities are presented in the supplementary files, but they were not calculated for each score (online supplementary files 4–6).

The sensitivities and specificities were evaluated based on thresholds defined in the studies. Different thresholds could be defined to achieve different results for these two statistics.

Quality assessment within studies

All included studies were observational studies. Thus, we evaluated whether the study was prospective (low risk of bias) or retrospective (moderate risk of bias).

All the studies were at low risk of bias (prospective studies), except Low et al 24 which was a retrospective cohort study, and Cotter et al,27 where the design was not specified.

As we assume that multicentric studies have a more representative population than monocentric studies, we identified four studies with a low risk of bias (multicentric studies)21–23 28 and eight studies with a moderate risk of bias (monocentric studies).17–20 24–27

To determine if patients had an unplanned readmission, the data collection for these observational studies was made from the medical records (high risk of bias), with the patient (moderate risk of bias) or both or with a national clinical data base (low risk of bias). The medical records rarely indicated if an unplanned readmission occurred in a different hospital, because about 20% of readmissions can be to a different hospital to the index admission.29 But the telephone interview and the national clinical data bases, they make it possible to overcome this classification bias.

We then evaluated whether the withdrawals and drop-outs were reported in terms of number and/or reasons per group (low risk of bias) or not (high risk of bias). Four studies had not reported the withdrawals and drop-outs.20 24 25 28

We evaluated the quality of the statistical analysis as to whether the discrimination and the metrological data (low risk of bias), or only the discrimination (moderate risk of bias) were calculated.

Finally, we rated the global quality: three studies17 18 23 had a global low risk of bias, seven19–22 24 26 27 had a moderate risk of bias and two24 25 had a global high risk of bias (table 2).

Quality assessment across studies

The global quality assessment, as presented in figure 4, was assessed through a graphical representation of risk percentages for each source of bias. It shows a low risk of bias for the design, the withdrawals and dropouts and the statistical analysis. The risk was moderate for the setting and high for the data collection.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Risk of bias across studies.

Summary of evidence

In this systematic review of the literature, we identified five validated risk clinical scores (ISAR, TRST, nomogram, LACE and VIP) identifying elderly patients at risk of unplanned readmission within 1 month after discharge. These five scores showed only fair discrimination (c-statistic less than 0.7). Among these five scores, the ISAR and the TRST are the most common scores, having undergone several external validations. Those two scores have used electronic medical records, and have predicted unplanned readmissions and AED visits. In the included studies, presenting similar design, the predictive validities of the ISAR and TRST scores were constant, showing that these scores are reproducible.

We have considered scores identifying elderly patients at risk of early unplanned readmission. Some other clinical scores identify adult inpatients at risk of unplanned readmission. For example, the HOSPITAL score, a clinical score developed in the USA in 2013, was internationally externally validated in an adult population, including elderly30 and has shown good performance with a c-statistic of 0.71. Unfortunately, it has not been specifically validated in an elderly population. Similarly, the LACE score was initially developed for adult inpatients.31 However, the LACE score showed, in the two external validation studies on an elderly population included in this systematic review,24 27 a lower discrimination (c-statistics of 0.55 and 0.595) than in the initial study (c-statistic=0.7).31 Thus, some clinical scores identifying patients at risk of unplanned readmission might be more effective on an adult population, rather than on an elderly population. In a recent study, Cooksley et al 32 externally validated the LACE score in an adult inpatient population from two hospitals in Denmark. In this study, the authors evaluated the modification in discrimination of LACE score by age group and found that the discriminatory power of the LACE score decreased with increasing age. The authors also evaluated the change in discrimination of the HOSPITAL score, and came to the same conclusion: the discriminatory power of the HOSPITAL score decreases with increasing age.

We only included studies dealing with general elderly population. On the contrary, some clinical risk scores focus on specific condition. For example, Pack et al 33 developed and validated a score predicting readmission after heart valve surgery and Echevarria et al 34 described a score predicting readmission after hospitalisation for acute exacerbation of chronic-obstructive pulmonary disease. These two scores predicted readmission with fair discrimination: c-statistic=0.67 and 0.70, respectively. Echevarria et al 34 also externally validated the LACE score for the prediction of readmission after hospitalisation for acute exacerbation of chronic-obstructive pulmonary disease and found a c-statistic of 0.65. The LACE score was also externally validated to predict readmission in patients hospitalised with heart failure, and showed a fair discrimination, with a c-statistic of 0.59.35 Hence, clinical scores focusing on specific patient populations or general risk scores used on specific patient population have not shown a better discrimination than general scores used on general populations.

For the ISAR and TRST scores, the c-statistics seem to be related to the outcome: the c-statistics appeared, indeed, lower for a prediction of an unplanned readmission, compared with an AED visit or a composite outcome.

Limitations

Our systematic review has some limitations. Unlike the review by Kansagara et al,10 we excluded risk prediction models, in order to include only clinical risk scores which might be more convenient for the current use. Clinical risk scores indeed give a numerical result and a cut-off, which could be automatically calculated by the medical software.

Moreover, we excluded validation studies on adult inpatients, which implies that studies with subanalyses that examine a score among the elderly population while the primary analyses were performed for a broad population were excluded. This is the case, for example, of the study of Cooksley et al,32 which have shown that the discriminatory power of the LACE index and the HOSPITAL score30 decreased with increasing age.

Similarly, we excluded validation studies that focus on specific conditions or where the outcome was readmission within more than 1 month. These exclusion criteria have limited the number of scores presented in this review, whereas some scores have shown good discrimination ability, and thus might be interesting to externally validate in other conditions. For example, the 80+score,36 an internally validated score on an elderly population which focuses on the patients ‘prescription, predicts the readmission within 1 year, with a good discrimination (c-statistic=0.72 (95% CI 0.66 to 0.77)).