Article Text
PDF
Abstract
Background Prediction models for trauma outcome routinely control for age but there is uncertainty about the need to control for comorbidity and whether the two interact. This paper describes recent revisions to the Trauma Audit and Research Network (TARN) risk adjustment model designed to take account of age and comorbidities. In addition linkage between TARN and the Office of National Statistics (ONS) database allows patient's outcome to be accurately identified up to 30 days after injury. Outcome at discharge within 30 days was previously used.
Methods Prospectively collected data between 2010 and 2013 from the TARN database were analysed. The data for modelling consisted of 129 786 hospital trauma admissions. Three models were compared using the area under the receiver operating curve (AuROC) for assessing the ability of the models to predict outcome, the Akaike information criteria to measure the quality between models and test for goodness-of-fit and calibration. Model 1 is the current TARN model, Model 2 is Model 1 augmented by a modified Charlson comorbidity index and Model 3 is Model 2 with ONS data on 30 day outcome.
Results The values of the AuROC curve for Model 1 were 0.896 (95% CI 0.893 to 0.899), for Model 2 were 0.904 (0.900 to 0.907) and for Model 3 0.897 (0.896 to 0.902). No significant interaction was found between age and comorbidity in Model 2 or in Model 3.
Conclusions The new model includes comorbidity and this has improved outcome prediction. There was no interaction between age and comorbidity, suggesting that both independently increase vulnerability to mortality after injury.
- Trauma
Statistics from Altmetric.com
Key messages
What is already known on this subject?
The trauma population is getting older and the incidence of pre-existing diseases is rising but the interaction between age and comorbidity is unclear.
Outcome prediction models are useful in the assessment of patient care and system development. However, the complexities of data collection and the lack of a comorbidity scale designed specifically for the trauma population limits their applicability to the new trauma demographic.
What might this study add?
The incorporation of a modification of the Charlson comorbidity index into our outcome prediction model improved its performance; using data linkage to measure 30 day outcome on all patients did not.
There was no significant interaction between age and comorbidity; both appear to be independent and separate influences on outcome.
Background
Trauma is a global disease affecting people of all ages and the leading cause of death in the first four decades of life. Care is improving1 ,2 but in order to understand the relative effectiveness of individual patient treatment and of trauma systems generally it is necessary to develop outcome prediction models and apply them to large data sets in trauma registries. The development of the trauma score/injury severity score methodology3 ,4 has been pivotal to these advances. This and other prediction models have been reported recently in a systematic review.5
The trauma population in the UK is ageing; the average age of patients on the Trauma Audit and Research Network (TARN) database has increased from 47 years to 57 years since 2008.6 Recorded comorbidity has also increased in the same period, from 37% to 62%.6 It therefore seems appropriate to include comorbidity in addition to age as a risk factor in outcome prediction models. Previous work suggests that comorbidity and age are independently associated with increased mortality after trauma but has not examined their interaction.7–9 A presence of interaction would mean that the effect of comorbidity on outcome is highly dependent of patient's age, for example, the impact of the presence of a serious comorbidity in reducing odds of survival after injury, could be higher in a younger patient than an older patient, or vice versa—if an interaction is present. If there is no interaction the relative reduction in odds of survival associated with serious comorbidity is the same regardless of patient age.
The outcome prediction model currently used by TARN6 defines survival as either ‘alive at hospital discharge’ or ‘alive in hospital at 30 days’; there is no accounting for the outcome of those discharged alive before 30 days.10 The model is ‘casemix adjusted’ using age, gender (and their interaction), injury severity score (ISS) and GCS as predictors. The 30-day threshold is considered appropriate because later death is less likely to be associated directly with the injuries which prompted hospital admission. However, including all deaths up to 30 days (not just those occurring in hospital up to 30 days) has been proposed as being more logical, despite demanding more complex data acquisition.11
This paper describes a new prediction model which uses ‘30 day outcome’ for all patients rather than just those still in hospital and incorporates a new comorbidity measure. It investigates the interaction between age and comorbidity and compares the new model's performance with that currently used by TARN for trauma quality assurance within England and Wales, Republic of Ireland and some hospitals in continental Europe.
Patients and methods
The TARN database is the largest trauma registry in Europe, holding data from all trauma receiving hospitals in England and Wales and some in Denmark, Switzerland and the Republic of Ireland. Patients of all ages are included on the database if they sustain injury resulting in any of: immediate admission to hospital for 3 days or longer, intensive or high dependency care, interhospital transfer for further care or death in hospital within 30 days. Patients aged over 65 years with an isolated fracture of the femoral neck or pubic ramus and those with isolated closed limb injuries (excepting the femoral shaft/condyles) are excluded. If transfers can be matched using data from both hospitals, these are linked for one patient to avoid duplication. Transfers out of the first hospital without a linked submission from a receiving hospital are excluded from mortality analyses.
Prospectively collected data were used in the current cohort study which includes eligible patients submitted to TARN from 1 January 2010 to 31 December 2013; presenting with blunt or penetrating trauma during those dates. A matching exercise was conducted in an attempt to obtain the records of patients transferred to a second hospital for specialist care. Where matching could not be accomplished the patients were excluded. The characteristics of the excluded and included patients were compared.
Comorbidity
Various comorbidity indices have been used in outcome prediction models. The Charlson comorbidity index (CCI) is one of the most frequently used as demonstrated in a recent systematic review.12 CCI uses weighted International Classification of Diseases (ICD) 10 based diagnoses summed to give a score for each patient. The original weights13 were updated by Dr Foster Intelligence in 2011 and incorporated into the hospital standardised mortality ratio.14 The CCI was also used in the recently developed Summary Hospital Mortality Index.15
The original 17 Charlson codes were developed from a sample of 559 patients;13 these did not cover the whole spectrum of medical conditions seen in trauma patients. The comorbidity dictionary separately developed earlier by TARN, and also based on ICD 10, contained many additional diagnoses. An extensive mapping exercise was therefore carried out to convert the TARN data points to Charlson compatible data points. The wider reach of the TARN data required the creation of five new groups which had not been represented in the original Charlson codes (in bold in table 1).
Modified Charlson comorbidity index with updated weights
The resulting 22 conditions were used as binary factors along with age and gender in a logistic regression model to derive the weights of each comorbidity group within the patients used for this study. The weights were obtained by dividing each of the regression coefficients by the coefficient with the smallest absolute value and then rounded up to the nearest whole number.16 The comorbidity index derived from this exercise can be described as a modified version of the original CCI and is therefore denoted as ‘mCCI’.
Outcome at 30 days
TARN does not hold data on patients’ outcome after hospital discharge. The Office of National Statistics (ONS) database does have such information and permission was given to carry out a data linkage exercise to obtain the outcome at 30 days after admission for all TARN registered patients (defined as ‘true’ 30 day outcome). The results were compared with the current TARN model7 which is based on a model using outcome at 30 days from admission or at discharge, whichever comes first.
Missing data
The GCS was not recorded in 10% of the selected cases. Previous studies17 have suggested that these patients are often the more seriously injured and must be included in the analyses. An imputation technique, based on chained equations and Rubin's rules, was therefore used on the assumption that the mechanism of missingness is random.18 Age, gender, mCCI and GCS were used as categorical variables. When GCS was missing for a specific non-random reason, for example, intubation, an extra category was added to GCS accordingly.
Comorbidity status was not recorded in 16% of patients. These were identified by adding a ‘missing’ category to the mCCI variable.
Statistical analysis
Three statistical models were compared each on the same number of TARN cases where the outcome from the last acute hospital was known.
Model 1 represents the current TARN model. It uses age and gender and their interaction, ISS, GCS and outcome at 30 days or at discharge, whichever comes first;7
Model 2 is the same as Model 1 with the addition of the TARN mCCI;
Model 3 is the same as Model 2 but using ‘true’ outcome at 30 days from admission as the dependent variable.
The outcome prediction models are based on multiple logistic regressions using age, gender, ISS, GCS, mCCI and age by gender interaction as predictors. The non-linearity of the ISS was corrected using fractional polynomials (FPs)7 to enable its inclusion in the logistic regression model,7 and was used as a continuous variable in the model, whereas age (eight levels), gender (two levels), age by gender interaction, GCS (seven levels) and mCCI (five levels) were used as categorical variables. The significance of the interaction between age and comorbidity was also assessed.
All the analyses were performed with Stata V.13.0 and R software.19 ,20
Validation
An internal validation for the model was carried out using a bootstrap21 procedure where 200 samples of the same size as the original data set were drawn with replacement using the library ‘bootstrap’ from the R package.20 The advantage of this procedure over split sample cross validation is that model performance and calibration are assessed in the original sample using the models developed in the bootstrap samples. A model which is developed on the whole sample produces stable estimates of the regression coefficients.22
Model performance or discriminant power was assessed through the area under the receiver operating curve (AuROC). Model calibration was assessed using calibration graphs instead of the Hosmer-Lemeshow (H-L) test.23 While the Hosmer-Lemeshow (H-L) test does not demonstrate poor calibration it is too sensitive with large sample sizes—any tiny deviation from perfect fit is detected. The same bootstrap technique used previously for the validation of the models was used to internally validate the modified Charlson weights.
TARN holds Health Research Authority Confidentiality Advisory Group (CAG) Section 251 approval.
Results
In the 4 years studied 129 786 patients meeting the TARN inclusion criteria from 207 trauma receiving hospitals were recorded with known outcomes; 10 810 patients were excluded because of unknown outcome after hospital transfer. Excluded patients were younger (51 years vs 57 years), more severely injured (ISS 16 vs 9), more likely to be male (68% vs 59%) but had similar GCS and comorbidity distributions (table 2). The characteristics of the patients used for the model derivation are shown in table 2.
Characteristics of the patients, 2010–2013
The sample mortality rate was 7%, median age 57 years, median ISS 9 and median GCS 15. Fifty-nine per cent of patients were male. More than 50% of the patients in the sample were recorded as having comorbidity; in 16% the comorbidity status was not recorded. Figure 1 shows the functional form of ISS on outcome (death) after FP transformation where the optimal powers obtained are (−0.5) and (0), representing the reciprocal of the square root and the (natural) logarithm of ISS, respectively. Female gender was associated with lower survival but as there was a significant interaction between age and gender which identified poorer survival in men than women over 65 years, the main effect for gender cannot be interpreted independently from age.
Fractional polynomials transformation for injury severity score (ISS).
The data linkage between TARN and ONS enabled the determination of the true final outcome of all patients within 30 days of admission. Five hundred and seventy-two patients died after discharge but within 30 days of admission; their characteristics are shown in table 3. The average age was 84 years, the GCS normal and the ISS low; the majority was female and most had a recorded comorbidity.
Characteristics of the patients who died after discharge within 30 days from admission, 2010–2013
Each of the outcome predictors was significant in each of the models. Model 2, incorporating our modification of the CCI, showed better prediction performance than Model 1 and Model 3. Model 2 was found to have a Brier score of 0.0451, demonstrating a good calibration and goodness-of-fit and validating the use of bootstrap simulation in its calibration. Model 2 also showed a significantly better discriminant power by means of AuROC=0.904 (95% CI 0.901 to 0.907, p value <0.001). The comparison between the three models is displayed in table 4.
Models comparison
The regression coefficients of Model 2 are shown in table 5; the effect of mCCI on outcome for the category ‘not known’ is similar to that of the ‘1–5’ group. Patients in this ‘not known’ category were younger (median age 48 years vs 65 years) but had similar median ISS and GCS.
Coefficients of the prediction model (Model 2) for outcome
The effects of the age/comorbidity interaction terms were not statistically significant; this is shown in figure 2 where the effect of mCCI on outcome is not modified by age.
Interaction effect of modified Charlson comorbidity index (mCCI) and age.
Discussion
We have augmented our trauma score/injury severity score based trauma outcome prediction model and shown that the incorporation of a modified comorbidity index significantly improves its prediction power (in terms of AuROC and other model performance statistics) for survival to discharge from acute care. Incorporating 30 day outcome data on all patents through data linkage was possible in 80% of cases. However, this did not improve outcome prediction. No significant interaction was found between age and comorbidity in their effect on survival to discharge.
The model retains the FP transformations of the ISS that we previously published.7 It also preserves the interaction between age and gender and suggests that increasing age and male gender related frailty independently predict adverse rates of survival after adjustment for comorbidity.
Some studies have shown that prediction of outcome after trauma has improved by including pre-existing medical conditions in models10 ,11 but one study found no benefit by adding the CCI and commented that the index was not developed specifically for trauma.24 In our study we have modified the original CCI and recalculated the weights specifically for trauma patients.
Two scoring components were examined in our study: the outcome at 30 days from admission for all patients, not just those remaining in hospital, and comorbidity using a modification of the CCI. Having demonstrated the added prognostic value of our ‘Model 2’, which incorporates comorbidity but not comprehensive 30 day outcomes, we have now adopted it in the TARN registry. We have updated the major trauma survival probability calculator for individual patients on our website6 and incorporated it in our institutional comparisons.
Strengths and limitations
Our study has a number of strengths. The data were extracted from the TARN database which is the largest registry in Europe; it uses standardised and centralised data coding and includes all hospitals in England and Wales and some in Denmark, Switzerland and the Republic of Ireland. Our outcome prediction models have shown good performance in terms of discrimination, calibration and internal validation.
There are some limitations. The outcome status of 8.3% of patients was not known. This is chiefly due to patients being transferred to other hospitals and lost to follow-up. Selection bias analysis showed that excluded patients were more severely injured; this is often the case with patients transferred to specialised centres. The effect of excluded patients on the model's performance cannot be assessed as their final outcome is unknown. However, transferred patents are referred for specialist and hopefully more effective care; this could impact on their outcome and therefore on the model's predictive power.
The problem of missing GCS data has been overcome by using the multiple imputation technique available in Stata.
Missing comorbidity data (16%) is recognised by creating a special category in the mCCI. Data linkage was successful in 80% of cases; failure to match the remainder was due to invalid patient identification numbers.
External validation of our models is currently being conducted in a collaborative project with the German trauma registry. This work will also compare our models with the model recently developed by the German trauma registry team.25 This uses a wider range of variables than the TARN model and represents the totality of sustained trauma with the worst and the second worst injuries rather than ISS. Risk Injury Severity Classification (RISK) II showed a good discrimination (AuROC=0.953) albeit in a more severely injured trauma population when compared with TARN.25
Conclusion
The accuracy of trauma outcome predictions, as assessed by the AuROC, was improved by adding comorbidity to the current TARN ‘survival to discharge’ model. However, using data linkage to derive 30 days postadmission outcome status for all patients and including this with comorbidity did not improve the model's accuracy; analysis of deaths after hospital discharge showed that they do not relate to severity of injury. We did not find a statistically significant interaction between comorbidity and age. If our data set is representative of the wider trauma population it suggests that both factors predict outcome after major injury independently.
References
Footnotes
Contributors OB wrote the manuscript and analysed the data. TJ and TL extracted and organised the data set. AE and MW mapped the TARN pre-existing medical conditions to the Charlson comorbidity index and directed the collection of data. DWY, RJ and FL reviewed this article for methodological content and made critical revisions to the final draft. All authors participated in the critical review of all versions of this article.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.