Original ArticleA multipurpose comorbidity scoring system performed better than the Charlson index
Introduction
When comparing resource utilization and clinical outcomes between groups of patients undergoing different interventions, the groups should ideally be as similar as possible with respect to severity of illness, sociodemographic factors, and comorbid conditions [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15]. Failure to assure homogeneity could lead to confounding that obscures the true relationship between treatment and outcome. But health services researchers usually lack the luxury of allocating patients to treatment protocols and must compare groups with disparate and complex pretreatment characteristics. Statistical methods of risk adjustment are then used in an effort to make valid comparisons, but their success depends on the predictive power of the statistical model and the accuracy of available data [16], [17], [18].
Although several comorbidity scales have proven useful for both patient classification in clinical research and prognostication in medical care [1], [2], [7], [8], [9], [11], [12], Charlson's comorbidity index [2], and its Deyo [3] and Dartmouth-Manitoba [4] adaptations for use with administrative data, are the most widely adopted systems. Charlson's method consists of a weighted index based on 17 comorbid conditions that predict 1-year survival. The index was derived from a training population of 559 medical inpatients at a New York hospital, and was validated on a testing population of 685 breast cancer patients at another hospital [2]. Subsequently, other researchers have refined the Charlson index with additional comorbid categories [15], a clearer distinction between complications and comorbid conditions in the index admission [15], or replacement of the original fixed integer scores with empirically derived, study-specific weights [10], [14]. But essentially, the Charlson index has remained the same conceptual entity that it was when first published in 1987.
Our conjecture was that contemporary methods of data linkage and statistical modeling techniques had potential to yield a comorbidity scoring system that would significantly outperform the Charlson index. This seemed likely from our studies of health care outcomes using the Western Australian (WA) Data Linkage System [19], [20], [21]. In one study, we found that transurethral prostatectomy (TURP) for benign prostatic hypertrophy had a higher postoperative mortality rate than open prostatectomy, but this relationship was confounded by greater comorbidity in the TURP group, such that after adjustment with the Charlson index using empirically derived weights, the rate ratio (RR) fell from 1.20 (95%CI 1.08–1.34) to 1.10 (0.99–1.23) [20]. A further improvement in goodness of fit of the Cox proportional hazards model (P < .001) and a further reduction in the RR to 1.07 (0.95–1.19) was achieved with the inclusion of fractional polynomials to account for a nonlinear relationships of the log(RR) with age and comorbidity [20]. We surmized that further enhancements in the measurement of and adjustment for comorbidity would lead to improved internal validity of similar analyses.
Table 1 gives an overview of possible avenues for the improvement of comorbidity scoring systems like the Charlson index. Problems 1–3 concern the composition of the index itself, problems 4–6 deal with the accuracy and detail of comorbidity information collected in the field, and problems 7–10 are potentially solved by improved methods of analysis, especially statistical modeling. Although some of these problems have received attention (we have referred already to numbers 1, 4, and 7), a comprehensive set of recommendations for optimal risk adjustment procedures is yet to emerge. The source of Charlson's index was limited to just 17 conditions based on a small unrepresentative sample of patients, unsegregated according to surgical, medical, or psychiatric subgroups, with no distinction made between the use of the index for different clinical and economic outcomes.
Using a large population-based sample of hospital admissions, the aim of this research was to develop a new and improved comorbidity scoring system, encompassing all comorbid conditions having any effect of practical importance on mortality, readmission, or length of stay (LOS) outcomes in medical, procedural, or psychiatric patients.
Section snippets
Western Australian data linkage system
The study used a system that links the administrative health data within a single Australian State of population 1.8 million [19], covering periods dating back to the 1960s. The system is updated on a continuous basis, and includes seven core databases and working links with more than 30 special-purpose research databases held externally. The WA hospital morbidity data are part of the core system and include demographic, diagnostic, and procedural information on all patients separated from
Results
When the MACSS guidelines were applied, 48.2% of medical patients, 33.7% of procedural patients and 61.2% of psychiatric patients had between 1 and 22 comorbid conditions. The proportions of patients with three or more comorbid conditions were 18.4, 7.7, and 22.0% in each subgroup, respectively. Comorbidity was consistently least prevalent in patients undergoing procedures with the exception that comorbid gynecologic conditions were most prevalent in the procedural subgroup. A high level of
Discussion
Following a process of extensive screening for comorbid conditions that had effects on mortality, readmission or LOS in a large training population, the resultant 102-item MACSS performed better than a risk adjustment procedure using the 17 comorbid conditions making up the Charlson index. For 1-year mortality, the relative reductions in failure to discriminate (1 – ROC statistic) achieved by the MACSS compared with the Charlson index ranged from 10 to 27% across the five test scenarios.
Conclusion
Contingent upon its validation in other data sets, and especially those offering improved capability to distinguish comorbidities from complications, it is possible that the use of the MACSS and similar alternatives to the Charlson index will become a new methodologic standard for adjustment of comorbidity risk. Given the technical problems that may be associated with inclusion of 102 terms in regression models, especially when applied to relatively small data sets, we anticipate that the MACSS
Acknowledgments
This research was supported by the National Health and Medical Research Council of Australia. The WA Data Linkage System was established with support from the WA Lotteries Commission, and is managed by the Data Linkage Unit of the WA Department of Health.
References (33)
- et al.
The importance of classifying initial co-morbidity in evaluating the outcome of diabetes mellitus
J Chronic Dis
(1974) - et al.
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation
J Chronic Dis
(1987) - et al.
Adapting a clinical co-morbidity index for use with ICD-9-CM administrative databases
J Clin Epidemiol
(1992) - et al.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives
J Clin Epidemiol
(1993) - et al.
Validation of a combined comorbidity index
J Clin Epidemiol
(1994) - et al.
Development and validation of a claims based index for adjusting for risk of mortality: the case of acute myocardial infarction
J Clin Epidemiol
(1995) - et al.
Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data
J Clin Epidemiol
(1995) - et al.
Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data
J Clin Epidemiol
(1996) - et al.
Evaluation of two competing methods for calculating Charlson's comorbidity index when analysing short-term mortality using administrative data
J Clin Epidemiol
(1997) - et al.
Population-based linkage of health records in Western Australia: development of a health services research linked database
Aust NZ J Public Health
(1999)