Original ArticlesAssessing accuracy of diagnosis-type indicators for flagging complications in administrative data
Introduction
Hospital administrative discharge data are widely used to assess quality of health care and generate health system “report cards.” The reliability of the resuting outcome report cards, however, depends on the validity of the database and outcome measure chosen to determine the quality of health care [1]. The most widely studied outcome measure is mortality, because of its high reliability in administrative data. A major limitation of this measure, however, is that death is relatively rare and not sensitive to the minor quality of care problems [2], [3], [4]. Accordingly, use of mortality to assess quality of care requires a large sample size of patients with a high risk of death during hospitalization, as is seen in patients undergoing heart surgery [5]. Considering these limitations, complications have been proposed as a potentially more sensitive outcome measure than mortality [1], [3], [4], [6], [7], [8], [9], [10], [11]. A challenge to studying complications, however, is deriving a definition that captures complications objectively and reliably in administrative hospital discharge data.
A commonly used approach is to define complications using International Classification of Disease (ICD-9-CM) codes falling between 996.xx to 999.xx, which are specifically designated to code “complications of surgical and medical care, not elsewhere classified.” However, each code is very broad in scope (997.0 designates “nervous system complications”) so these codes are not very useful for defining specific complications. Another commonly used approach is to define complications using a combination of ICD-9-CM codes that include the above-mentioned complication codes falling between 996.xx to 999.xx and disease-specific codes that are selected by investigators based on medical content and clinical judgement [1], [10], [12], [13], [14]. This approach leaves some uncertainty as to whether the additional disease-specific codes were true complications as opposed to conditions present at the time of admission.
Canadian administrative hospital discharge data differ from American administrative data because they contain a diagnosis-type indicator for each diagnosis. This indicator distinguishes diagnoses arising during hospitalization from those pre-existing at admission [15]. Therefore, it can be used to identify complications in outcome studies and also can be used to exclude medical conditions that arose during hospitalization in the risk-adjustment analysis. Despite potential advantages of the Canadian data [15], no study has formally validated the indicator. Here, we address this important question by conducting a detailed chart review to evaluate the accuracy of diagnosis-type indicators for flagging complications in Canadian administrative data. The results of this study will inform other countries on the potential merit and/or problems associated with diagnosis-type indicators.
Section snippets
Administrative hospital data
We identified all inpatients discharged from the general medical and general surgical services of three adult acute care hospitals in Calgary, Alberta, Canada, between April 1, 1996 and March 31, 1997, by screening regional hospital administrative discharge data. The identified discharge records were stratified by speciality of most responsible physician (i.e., general internist vs. general surgeon) and hospital site. Then, 200 general medical discharge records and 200 general surgical
Agreement on presence of medical conditions
Table 2 presents the prevalence of the 12 medical conditions by data source. Compared to the chart data, the administrative data under-reported significantly for seven conditions and over-reported for one condition. The prevalence of the remaining four conditions was similar between the two data sources.
To assess whether the administrative hospital discharge data accurately reproduced what was recorded in the patient charts, we calculated kappa for each condition (see Table 2). The kappa value
Discussion
In this study, we assessed the validity of diagnosis-type indicators that are recorded in Canadian administrative hospital discharge data. To summarize, our series of analyses demonstrated the following: (1) There was a moderate agreement between the administrative data and the chart data for the overall presence of medical conditions regardless of diagnosis-type (see Table 2). (2) When the condition was present in both of the databases, agreement of diagnosis-type indicator coding between the
Acknowledgements
The conduct of this study was supported by an operating grant from the Calgary Health Region, Calgary, Alberta, Canada. Dr. Ghali is supported by a Health Scholar Award from the Alberta Heritage Foundation for Medical Research, Edmonton, Alberta, Canada, and by a Government of Canada Research Chair in Health Services Research.
References (29)
- et al.
Risk adjustment for complications of hysterectomy: limitations of routinely collected administrative data
Am J Obstet Gynecol
(1999) - et al.
The use of automated data to identify complications and comorbidities of diabetes: a validation study
J Clin Epidemiol
(1999) - et al.
Validation of a pneumonia prognostic index using the MedisGroups Comparative Hospital Database
Am J Med
(1993) - et al.
Comparing hospitals that perform coronary artery bypass surgery: the effect of outcome measures and data sources
Am J Public Health
(1994) - et al.
Outcomes research: measuring the end results of health care
Science
(1998) - et al.
Measurement of health outcomes in treatment effectiveness evaluations: conceptual and methodological challenges
Med Care
(2000) Using Medicare claims for outcomes research
Med Care
(1994)- et al.
Coronary artery bypass grafting in Canada: national and provincial mortality trends, 1992–1995
CMAJ
(1998) - et al.
Evaluation of the complication rate as a measure of quality of care in coronary artery bypass graft surgery
JAMA
(1995) - et al.
The relationship between adjusted hospital mortality and the results of peer review
Health Serv Res
(1993)