Original Articles
Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data

https://doi.org/10.1016/j.jclinepi.2003.01.002Get rights and content

Abstract

Objective

Canadian administrative hospital discharge data contain a diagnosis-type indicator for each coded diagnosis that allows researchers to distinguish complications from pre-existing diagnoses. Given that the validity of diagnosis-type indicators is unknown, we conducted a detailed chart review to evaluate the accuracy of diagnosis-type indicators for flagging complications.

Study design and setting

We obtained administrative hospital discharge data for 1,200 randomly selected adult inpatient separations in Calgary, Alberta, occurring between April 1, 1996 and March 31, 1997. Each discharge record contains up to 16 diagnoses and 16 corresponding diagnosis-type indicators (value of “2” = complication). The corresponding medical charts were reviewed for evidence of diagnoses and complications. A complication was defined as a new diagnosis arising after the start of hospitalization. We determined the extent to which the diagnosis-type indicator in the administrative data agreed with the chart reviewer's assessment (criterion standard) of whether a diagnosis was a complication or not.

Results

The agreement for complications between the two databases varied greatly across 12 conditions studied (kappa range: 0–0.72) and was often low (kappa <0.20 for six conditions). Sensitivity ranged from 0 to 57.1% (higher than 50% for only two conditions), indicating a tendency for complications to often be miscoded as baseline comorbidities. In contrast, specificity was generally high (range: 99.0–100%), suggesting that pre-existing conditions were usually appropriately coded as such in the administrative data.

Conclusion

The validity of diagnosis-type indicators in Canadian administrative discharge data appears to be poor for some types of complications. This is likely to be of greatest concern in studies that rely solely on diagnosis-type indicators to define complications as outcomes.

Introduction

Hospital administrative discharge data are widely used to assess quality of health care and generate health system “report cards.” The reliability of the resuting outcome report cards, however, depends on the validity of the database and outcome measure chosen to determine the quality of health care [1]. The most widely studied outcome measure is mortality, because of its high reliability in administrative data. A major limitation of this measure, however, is that death is relatively rare and not sensitive to the minor quality of care problems [2], [3], [4]. Accordingly, use of mortality to assess quality of care requires a large sample size of patients with a high risk of death during hospitalization, as is seen in patients undergoing heart surgery [5]. Considering these limitations, complications have been proposed as a potentially more sensitive outcome measure than mortality [1], [3], [4], [6], [7], [8], [9], [10], [11]. A challenge to studying complications, however, is deriving a definition that captures complications objectively and reliably in administrative hospital discharge data.

A commonly used approach is to define complications using International Classification of Disease (ICD-9-CM) codes falling between 996.xx to 999.xx, which are specifically designated to code “complications of surgical and medical care, not elsewhere classified.” However, each code is very broad in scope (997.0 designates “nervous system complications”) so these codes are not very useful for defining specific complications. Another commonly used approach is to define complications using a combination of ICD-9-CM codes that include the above-mentioned complication codes falling between 996.xx to 999.xx and disease-specific codes that are selected by investigators based on medical content and clinical judgement [1], [10], [12], [13], [14]. This approach leaves some uncertainty as to whether the additional disease-specific codes were true complications as opposed to conditions present at the time of admission.

Canadian administrative hospital discharge data differ from American administrative data because they contain a diagnosis-type indicator for each diagnosis. This indicator distinguishes diagnoses arising during hospitalization from those pre-existing at admission [15]. Therefore, it can be used to identify complications in outcome studies and also can be used to exclude medical conditions that arose during hospitalization in the risk-adjustment analysis. Despite potential advantages of the Canadian data [15], no study has formally validated the indicator. Here, we address this important question by conducting a detailed chart review to evaluate the accuracy of diagnosis-type indicators for flagging complications in Canadian administrative data. The results of this study will inform other countries on the potential merit and/or problems associated with diagnosis-type indicators.

Section snippets

Administrative hospital data

We identified all inpatients discharged from the general medical and general surgical services of three adult acute care hospitals in Calgary, Alberta, Canada, between April 1, 1996 and March 31, 1997, by screening regional hospital administrative discharge data. The identified discharge records were stratified by speciality of most responsible physician (i.e., general internist vs. general surgeon) and hospital site. Then, 200 general medical discharge records and 200 general surgical

Agreement on presence of medical conditions

Table 2 presents the prevalence of the 12 medical conditions by data source. Compared to the chart data, the administrative data under-reported significantly for seven conditions and over-reported for one condition. The prevalence of the remaining four conditions was similar between the two data sources.

To assess whether the administrative hospital discharge data accurately reproduced what was recorded in the patient charts, we calculated kappa for each condition (see Table 2). The kappa value

Discussion

In this study, we assessed the validity of diagnosis-type indicators that are recorded in Canadian administrative hospital discharge data. To summarize, our series of analyses demonstrated the following: (1) There was a moderate agreement between the administrative data and the chart data for the overall presence of medical conditions regardless of diagnosis-type (see Table 2). (2) When the condition was present in both of the databases, agreement of diagnosis-type indicator coding between the

Acknowledgements

The conduct of this study was supported by an operating grant from the Calgary Health Region, Calgary, Alberta, Canada. Dr. Ghali is supported by a Health Scholar Award from the Alberta Heritage Foundation for Medical Research, Edmonton, Alberta, Canada, and by a Government of Canada Research Chair in Health Services Research.

References (29)

  • A.J Hartz et al.

    The use of risk-adjusted complication rates to compare hospitals performing coronary artery bypass surgery or angioplasty

    Int J Technol Assess Health Care

    (1992)
  • J Daley et al.

    Complication rates as a measure of quality of care

    JAMA

    (1995)
  • D.S Raiford et al.

    Positive predictive value of ICD-9 codes in the identification of cases of complicated peptic ulcer disease in the Saskatchewan hospital automated database

    Epidemiology

    (1996)
  • J.M Geraci et al.

    International Classification of Diseases, 9th Revision, clinical modification codes in discharge abstracts are poor measures of complication occurrence in medical inpatients

    Med Care

    (1997)
  • Cited by (0)

    View full text