Article Text
PDF
Abstract
Objectives Cardiovascular magnetic resonance (CMR) imaging is the gold-standard test for the assessment of heart function. Despite its importance, many jurisdictions lack specific billing codes that can be used to identify patient receipt of CMR in administrative databases, limiting the ability to perform ‘big data’ CMR studies. Our objective was to identify the optimal billing code combination to identify patients who underwent CMR using administrative data in Ontario.
Design Retrospective cohort study.
Setting Quaternary care academic referral centre in Ontario, Canada.
Participants We tested all billing code combinations in order to identify the optimal one to determine receipt of CMR. The reference gold standard was a list of all cardiothoracic magnetic resonance scans performed at Sunnybrook Health Sciences Centre between 1 January 2014 and 31 December 2016, verified by chart audit. We assessed the diagnostic performance (accuracy, sensitivity, specificity, positive predictive value and negative predictive value) for all code combinations.
Results Our gold-standard cohort consisted of 2339 thoracic MRIs that were performed at Sunnybrook Health Sciences Centre from 1 January 2014 to 31 December 2016. Of these, 2139 (91.5%) were CMRs and 200 (8.5%) were chest MRIs. We identified the most accurate billing combination for the determination of patient receipt of CMR. This combination resulted in an accuracy of 95.3% (95% CI 94.4% to 96.2%), sensitivity of 97.4% (95% CI 96.6% to 98.1%), specificity of 86.4% (95% CI 83.1% to 89.6%), positive predictive value of 96.9% (95% CI 96.1% to 97.6%) and negative predictive value of 88.4% (95% CI 85.4% to 91.5%).
Conclusions Our study is the first to verify the ability to accurately identify patient receipt of CMR using administrative data, facilitating more robust population-based CMR studies in the future.
- cardiovascular imaging
- human resource management
- magnetic resonance imaging
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Cardiovascular magnetic resonance (CMR) imaging is the gold-standard test to assess heart structure and function.
Our study validated a combination of billing codes that can identify patient receipt of CMR in administrative databases in Ontario with an accuracy of approximately 95.3%.
This validated combination of codes can be used by researchers in the future in order to identify those who underwent CMR and conduct CMR-related population-based ‘big data’ studies.
Only data from a single centre in Ontario were used, thus potentially limiting generalisability. Further, the reported positive predictive values are very high due to 91.5% of the scans being true CMRs. If there was a setting in which there would be a higher prevalence of non-cardiac chest MRIs, the positive predictive value would be expected to be lower.
Introduction
Cardiovascular magnetic resonance (CMR) imaging is an increasingly used and important clinical imaging modality and is considered the gold standard in the assessment of the structure and function of the human heart.1–5 Despite the growth of CMR-related technology, there are no published population-based studies using CMR. Such studies are important in order to assess real-world outcomes related to this emerging technology as well as to understand health services issues such as access and cost. The main barrier to conducting such studies is the lack of a reliable mechanism to track those who undergo CMR on a population basis. In Canada, for example, most jurisdictions lack a unique billing code for CMR, thus limiting the accuracy of tracking. One of those jurisdictions is Ontario, the largest Canadian province and home to approximately 14 million people. In Ontario, CMRs are billed with a generic code for ‘thoracic MRI’ which includes both CMRs and chest magnetic resonance scans.6 7 Further, there are two additional billing codes that are commonly used for CMR scans; one for ‘cardiac gating’ (monitoring the patient’s ECG during the CMR scan) and the other for repeated MRI sequences that are sometimes performed as part of a comprehensive CMR scan.7 Administrative medical coding data are known to have inaccuracies. Therefore, it is important to validate these data using medical chart review as the gold-standard comparator. It is currently unknown if different combinations of the three billing codes can identify those who undergo CMR accurately. The objective of this paper was to identify the optimal billing code combination to identify patients who undergo CMR using administrative data in Ontario while using a gold-standard reference cohort composed of CMR scans that were confirmed based on a chart audit.
Methods
Administrative databases
Given Ontario’s universal healthcare, administrative databases in Ontario contain data on all patients in the population who undergo assessment and billing by a physician. In fact, the Ontario Health Insurance Plan database captures approximately 98% of physician billings in the province.8
Candidate billing codes in for CMR imaging
As previously mentioned, there is no specific billing code for CMR imaging in Ontario. Candidate codes were selected based on examination of local billing patterns and after extensive consultation with CMR specialists in six large referral centres across Ontario. Since CMR is a niche cardiac diagnostic test in Canada, these six centres perform that vast majority of all CMR scans in the province. All patients scanned are billed code X441 (‘MRI thorax-multislice sequence’). This is the generic code for thoracic MRI which includes both cardiovascular and chest MRIs. In addition, there are two other related billing codes that may also be billed. One of these is X445: (‘MRI thorax-repeat another plane, different pulse sequence’). This code is billed when scanning requires the performance of additional MRI sequences in different anatomical planes. This is often the case when the scan is performed to address clinical questions regarding cardiac anatomy. Finally, X486 (‘addition of cardiac gating’) may also be billed. This code is used when ECG recordings and gating are necessary while the patient is being scanned. In practice, this is required to improve image quality of movie images (cine images) of the beating heart and to monitor patients who may be at risk of arrhythmia. We sought to identify the diagnostic properties (accuracy, sensitivity, specificity, negative predictive value and positive predictive value) for the following combination of codes:
X441+X445.
X441+X486.
X441+X445+X486.
The gold-standard reference
All thoracic MRI (CMR and chest MRIs) scans performed during a 3-year period, between 1 January 2014 and 31 December 2016, at Sunnybrook Health Sciences Centre were included in this study and were deemed the gold-standard reference. Scans were identified using a list obtained from the records of the department of medical imaging. Next, via chart review, each scan on this list was checked by the principal author (IR), a trained CMR specialist, to ensure that the characterisation of cardiovascular versus chest MRI was indeed correct. There were no discrepancies between the list obtained by medical imaging and the chart review. Note that individual patients may have been represented twice within this cohort if they had multiple thoracic MRI scans. The unit of analysis was individual MRI scans, not patients.
The approach we employed to ascertain the validity of administrative data definitions using data obtained from chart review as the gold standard is similar to that used to validate administrative data definitions and algorithms for a number of medical conditions including myocardial infarction, rheumatoid arthritis, Alzheimer’s disease and multiple sclerosis.9–13
Statistical analysis
Sensitivity, specificity, accuracy, positive and negative predictive values were then calculated for each of the three combination of codes listed above, using the validated list obtained from the Department of Medical Imaging as the gold-standard reference. Sensitivity was calculated as true positives/true positives plus true negatives, specificity was calculated as false negatives/false negatives plus false positives, positive predictive value was calculated as true positives/true positives plus false positives and negative predictive value was calculated as false negatives/false negatives plus true negatives. Accuracy was calculated as true positives+true negatives/true positives+true negatives+false positives+false negatives. A 7-day window was allowed for lag time between the date of the scan on the gold-standard list and submission of billing codes to the ministry of health and long-term care, for payment. A sensitivity analysis using a window of <24 hours was also performed. Finally, subgroup analyses were performed according to inpatient/outpatient status. Binomial exact 95% CIs (by the Clopper-Pearson method) were calculated for all measures of diagnostic performance. All analyses were conducted using SAS V.9.4 (SAS Institute).
Patient and public Involvement
Patients were not involved in this study.
Results
Our gold-standard cohort consisted of 2339 cardiothoracic MRIs that were performed at Sunnybrook Health Sciences Centre from 1 January 2014 to 31 December 2016. Of these, 2139 (91.5%) were cardiovascular MRIs and 200 (8.5%) were chest MRIs. Patient characteristics of those undergoing cardiothoracic MRIs are summarised in table 1. All clinical patients undergoing cardiothoracic MRI (ie, excluding those with research scans) received code X441.
Characteristics of the gold-standard patient cohort of patients undergoing cardiothoracic MRIs at Sunnybrook Health Sciences Centre from 1 January 2014 to 31 December 2016
The results of our analyses are found in table 2. The optimal combination of codes was found to be X441+X486. This combination resulted in an accuracy of 95.3%. The combination of all three codes (X441, X486 and X445) had comparable results, with an accuracy of 95.1%. In contrast the combination of X441 and X445 produced inferior results with an accuracy of only approximately 80.5% and unacceptably low specificity and negative predictive value numbers.
Diagnostic performance of the different thoracic magnetic resonance code combinations for the identification of cardiovascular magnetic resonance scans
There was no substantive change when using a window of <24 hours between the billing of the individual codes with the best combination remaining X441+X486 and the worst performing combination remaining X441+X445 (see table 3). There were also no clinically important differences when examining subgroups of patients by inpatient/outpatient status with the best performing combination continuing to be X441+X486 (see table 4A,B).
Diagnostic performance of the different thoracic magnetic resonance code combinations for the identification of cardiovascular magnetic resonance (CMR) scans: sensitivity analysis using a window of <24 hours between performance of the CMR and submission of billing code for payment
Diagnostic performance of the different thoracic magnetic resonance code combinations for the identification of cardiovascular magnetic resonance scans among inpatients
Diagnostic performance of the different thoracic magnetic resonance code combinations for the identification of cardiovascular magnetic resonance scans among outpatients
Discussion
In the first study to examine the optimal mechanism to identify patients who underwent CMR using administrative data, we were able to identify the best and worst performing combinations of billing codes. The combination of the billing codes X441+X486 was optimal for identifying patients who underwent CMR, doing so with an accuracy of approximately 95.3%. This combination was marginally more accurate than the combination of X441+X445+X486, which provided an accuracy of 95.1%. In contrast, the combination of X441+X445 produced the worst results, with an accuracy of approximately 80.5%.
There has been a plethora of work validating diagnostic algorithms for a variety of cardiovascular indications including stroke, atrial fibrillation, cardiovascular procedures and major adverse cardiovascular events using administrative data.4 8 11 12 14–19 In contrast, there has been relatively little work attempting to validate patient receipt of cardiovascular imaging using administrative data. A recent study from Ontario demonstrated that in inpatients who suffered a stroke, administrative data alone were able to identify those who had MRIs or CTs of the brain in over 90% of cases.17 In contrast, in that study, administrative data were only able to identify transthoracic echocardiograms in approximately 40% of inpatients.17
We are not aware of any studies that examined the validity of identifying CMR using administrative data. CMR is an increasingly used technology and is now considered to be the gold standard for the assessment of right and left ventricular function, viability and flow. However, despite this status, CMR does not have a unique billing code in Ontario or most other Canadian jurisdictions.7 20 21 This has been perceived to be a barrier to identifying those who underwent CMR using administrative data, thus limiting the extent of ‘big data’ population-based CMR studies.
Importance
Our study is the first to demonstrate that CMRs can be accurately tracked using a simple combination of billing codes with high accuracy. Given the similarity of billing codes for CMR across Canada, our approach will be translatable to other jurisdictions. By using our validated coding combinations, one can identify patients who underwent this important diagnostic test in order to perform robust population-based outcomes studies related to CMR.
Limitations
This study must be interpreted in the context of its limitations. First, specificity and the related negative predictive value were modest even for the best performing of our algorithms. This is due to the fact that we have very few true and false negatives in our cohort. With that said, the accuracy for the best of our coding combinations exceeded 95% using the cohort of patients who underwent thoracic MRIs. Second, the Institute for Clinical Evaluative Sciences has strict policies on data confidentiality and minimising the risk of identification of subjects. In this context, patients who were analysed in a de-identified manner cannot be reidentified at a later time. Identification would allow us to access medical charts and determine the patients’ demographic characteristics. Due to these restrictions, we were unable to discern the patient characteristics of the approximately 5% of CMR scans in which the billing code combinations were inaccurate. Third, the gold-standard cohort was obtained from a single centre in Ontario. However, this is unlikely to affect the validity of our results given the similarities in billing practices across the province and the fact that the billing codes themselves are identical across the province. Fourth, while these results are generalisable to Canada, they may not be generalisable to other countries who use different, more specific CMR codes. Fifth, the reported positive predictive values are very high due to 91.5% of the scans being true CMRs. If there was a setting in which there would be a higher prevalence of non-cardiac chest MRIs, the positive predictive value would be expected to be lower. Finally, miscoding of administrative data, transcription errors by physicians or clerical staff submitting billing claims and in the case of those physicians who work under a salary model and ‘shadow bill’, improper documentation of diagnostic billing codes may all have affected the accuracy of the administrative data. Physicians in Ontario who are reimbursed under a salary structure often are required to send patient billings to the Ministry of Health. These billings are not used for directly reimbursing the physician but rather to ensure that the salary provided is relatively commensurate with the services provided. These ‘shadow’ billings are not counted by the Ontario Ministry of Health and thus are not traceable in administrative databases. However, the percentage of physicians who shadow bill in Ontario is <2% and our study found high accuracy for administrative data definitions using a gold-standard reference model based on chart review.
Conclusions
Our study is the first to report the ability to identify patient receipt of CMR using administrative data with high accuracy. These findings are important as they provide a validated method to identify those who undergo this diagnostic test in order to perform population-based ‘big data’ studies in the future.
Footnotes
Contributors IR and HCW carriedout the studies, participated in collecting data and drafted themanuscript. FQ performed the statistical analysis and participated inits design. IR, FQ, KAC ,GAW, MF, LJ-J and HCW helped draft the manuscript and/or made critical edits to themanuscript. All authors read and approved the final version of themanuscript.
Funding This study was funded by an Impact Challenge Grant in Cardiovascular Research provided by the Heart and Stroke/Richard Lewar Center of Excellence in Cardiovascular Research and the Banting and Best Diabetes Centre. This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. Data sets were linked using unique encoded identifiers and analysed at the Institute for Clinical Evaluative Sciences (ICES).
Competing interests None declared.
Patient consent Not required.
Ethics approval Ethics approval was obtained from the Sunnybrook Health Sciences Centre Research Ethics Board.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data set from this study is held securely in coded form at the Institute for Clinical Evaluative Sciences (ICES). While data sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS.