Original Article
A multipurpose comorbidity scoring system performed better than the Charlson index

https://doi.org/10.1016/j.jclinepi.2005.01.020Get rights and content

Abstract

Background and Objectives

To develop a comorbidity scoring system that out-performs the Charlson index.

Methods

Population-based cohorts of medical (n = 326,456), procedural (n = 349,686), and psychiatric (n = 16,895) inpatients in Western Australia were followed for 1-year mortality, 30-day readmissions, and length of stay (LOS) using data linkage. Conditions were identified at index admission and over the preceding 12 months. A Multipurpose Australian Comorbidity Scoring System (MACSS) was developed, based on the most frequent 102 comorbid conditions associated with a rate ratio (RR) ≥ 1.1 of death or readmission or a LOS difference ≥0.5 days. The performance of MACSS and the Charlson index in predicting mortality, readmission, and LOS, and in controlling confounding by comorbidity, was compared in five test scenarios involving asthma, myocardial infarction, mastectomy, transurethral prostatectomy, and major depressive illness.

Results

MACSS performed better than the Charlson index on all three outcomes in all five clinical groups. It reduced the failure of the Charlson index to discriminate on mortality and readmission outcomes by 5–40%, improved R2 in LOS models by up to fourfold and often doubled the correction of originally confounded effect measures.

Conclusion

The use of the MACSS and similar alternatives to the Charlson index are a new methodologic standard for adjustment of comorbidity risk.

Introduction

When comparing resource utilization and clinical outcomes between groups of patients undergoing different interventions, the groups should ideally be as similar as possible with respect to severity of illness, sociodemographic factors, and comorbid conditions [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]. Failure to assure homogeneity could lead to confounding that obscures the true relationship between treatment and outcome. But health services researchers usually lack the luxury of allocating patients to treatment protocols and must compare groups with disparate and complex pretreatment characteristics. Statistical methods of risk adjustment are then used in an effort to make valid comparisons, but their success depends on the predictive power of the statistical model and the accuracy of available data [16], [17], [18].

Although several comorbidity scales have proven useful for both patient classification in clinical research and prognostication in medical care [1], [2], [7], [8], [9], [11], [12], Charlson's comorbidity index [2], and its Deyo [3] and Dartmouth-Manitoba [4] adaptations for use with administrative data, are the most widely adopted systems. Charlson's method consists of a weighted index based on 17 comorbid conditions that predict 1-year survival. The index was derived from a training population of 559 medical inpatients at a New York hospital, and was validated on a testing population of 685 breast cancer patients at another hospital [2]. Subsequently, other researchers have refined the Charlson index with additional comorbid categories [15], a clearer distinction between complications and comorbid conditions in the index admission [15], or replacement of the original fixed integer scores with empirically derived, study-specific weights [10], [14]. But essentially, the Charlson index has remained the same conceptual entity that it was when first published in 1987.

Our conjecture was that contemporary methods of data linkage and statistical modeling techniques had potential to yield a comorbidity scoring system that would significantly outperform the Charlson index. This seemed likely from our studies of health care outcomes using the Western Australian (WA) Data Linkage System [19], [20], [21]. In one study, we found that transurethral prostatectomy (TURP) for benign prostatic hypertrophy had a higher postoperative mortality rate than open prostatectomy, but this relationship was confounded by greater comorbidity in the TURP group, such that after adjustment with the Charlson index using empirically derived weights, the rate ratio (RR) fell from 1.20 (95%CI 1.08–1.34) to 1.10 (0.99–1.23) [20]. A further improvement in goodness of fit of the Cox proportional hazards model (P < .001) and a further reduction in the RR to 1.07 (0.95–1.19) was achieved with the inclusion of fractional polynomials to account for a nonlinear relationships of the log(RR) with age and comorbidity [20]. We surmized that further enhancements in the measurement of and adjustment for comorbidity would lead to improved internal validity of similar analyses.

Table 1 gives an overview of possible avenues for the improvement of comorbidity scoring systems like the Charlson index. Problems 1–3 concern the composition of the index itself, problems 4–6 deal with the accuracy and detail of comorbidity information collected in the field, and problems 7–10 are potentially solved by improved methods of analysis, especially statistical modeling. Although some of these problems have received attention (we have referred already to numbers 1, 4, and 7), a comprehensive set of recommendations for optimal risk adjustment procedures is yet to emerge. The source of Charlson's index was limited to just 17 conditions based on a small unrepresentative sample of patients, unsegregated according to surgical, medical, or psychiatric subgroups, with no distinction made between the use of the index for different clinical and economic outcomes.

Using a large population-based sample of hospital admissions, the aim of this research was to develop a new and improved comorbidity scoring system, encompassing all comorbid conditions having any effect of practical importance on mortality, readmission, or length of stay (LOS) outcomes in medical, procedural, or psychiatric patients.

Section snippets

Western Australian data linkage system

The study used a system that links the administrative health data within a single Australian State of population 1.8 million [19], covering periods dating back to the 1960s. The system is updated on a continuous basis, and includes seven core databases and working links with more than 30 special-purpose research databases held externally. The WA hospital morbidity data are part of the core system and include demographic, diagnostic, and procedural information on all patients separated from

Results

When the MACSS guidelines were applied, 48.2% of medical patients, 33.7% of procedural patients and 61.2% of psychiatric patients had between 1 and 22 comorbid conditions. The proportions of patients with three or more comorbid conditions were 18.4, 7.7, and 22.0% in each subgroup, respectively. Comorbidity was consistently least prevalent in patients undergoing procedures with the exception that comorbid gynecologic conditions were most prevalent in the procedural subgroup. A high level of

Discussion

Following a process of extensive screening for comorbid conditions that had effects on mortality, readmission or LOS in a large training population, the resultant 102-item MACSS performed better than a risk adjustment procedure using the 17 comorbid conditions making up the Charlson index. For 1-year mortality, the relative reductions in failure to discriminate (1 – ROC statistic) achieved by the MACSS compared with the Charlson index ranged from 10 to 27% across the five test scenarios.

Conclusion

Contingent upon its validation in other data sets, and especially those offering improved capability to distinguish comorbidities from complications, it is possible that the use of the MACSS and similar alternatives to the Charlson index will become a new methodologic standard for adjustment of comorbidity risk. Given the technical problems that may be associated with inclusion of 102 terms in regression models, especially when applied to relatively small data sets, we anticipate that the MACSS

Acknowledgments

This research was supported by the National Health and Medical Research Council of Australia. The WA Data Linkage System was established with support from the WA Lotteries Commission, and is managed by the Data Linkage Unit of the WA Department of Health.

References (33)

Cited by (0)

View full text