Objectives: To assess the intermethod reliability of medical chart review compared with directly observed care in patients presenting to emergency departments (EDs) for asthma care.
Methods: ED care practices for persons with asthma were evaluated by comparing chart review with trained observers. Fifty-one patients from five EDs participating in the Illinois Emergency Department Asthma Collaborative were studied. Practices in assessment, treatment, education, and referral were measured. Eighteen elements of care were assessed. Concordance between chart and observation was measured by using the kappa statistic.
Results: Of 51 subjects studied, nine were children. Kappa values varied depending on content. Kappa values ranged from 0.22 to 0.91 for items reflecting asthma assessment. Good concordances (kappa = 0.50 to 0.82) were found for items reflecting treatment practices. The lowest concordances were for items assessing educational activities (kappa = 0.04 to 0.34). Referral practices had fair to moderate concordances (kappa = 0.21 to 0.45).
Conclusions: Intermethod reliability of medical chart review and directly observed care varied depending on the element of care being measured. The use of chart review to measure quality of ED-based asthma care may only be appropriate for a limited number of care processes that are reliably and validly captured from chart review.