Article Text
PDF
Abstract
Objectives The aim of this study was to assess the frequency of appropriateness of inpatient CT and MRI scans performed in Southern Italy.
Methods The study was carried out by retrospectively reviewing medical records of adult patients admitted between 1 January and 31 December 2012 in two hospitals. The evaluation of appropriateness was performed according to the American College of Radiology Appropriateness Criteria, which assigns a score between 1 and 9.
Results Eight hundred and fifty-three medical records were reviewed. Six hundred and thirty-nine patients received CT examinations and 256 received MRI examinations. Four hundred and ninety-six (77.6%) of the patient population had appropriate CT and 202 (78.9%) received appropriate MRI examinations. The appropriateness was associated with: a confirmation of the diagnostic hypothesis, only one examination performed during hospital stay, the anatomical scan region, with musculoskeletal system being the least appropriate anatomical scan region. Moreover, for CT examinations, appropriateness was also associated with no use of contrast agent.
Conclusions Our findings highlight the need to reduce inappropriate use of CT and MRI. The study showed that the tool used is reliable to measure the extent of appropriateness of diagnostic imaging for inpatient examinations.
- appropriateness
- criteria
- CT
- inpatient
- Italy
- MRI
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Most prior studies focused on outpatient requests referred to diagnostic imaging departments. This is the first study exploring a large sample of imaging examinations requested during hospital stay.
In this study, appropriateness was exclusively evaluated through the American College of Radiology Appropriateness Criteria guidelines, which allow an objective appropriateness assessment.
Data were collected through a retrospective review of medical records. Therefore, the validity of results is influenced by the accuracy of clinical documentation.
Generalisability of results to all Italian hospitals is somewhat limited, since the data were collected from two hospitals in Southern Italy.
Introduction
The use of diagnostic imaging has increased significantly over the past decade, and expensive technologies such as CT and MRI have been extensively introduced into several diagnostic procedures. The clinical information acquired from their use, the decrease in time needed to perform them and greater accessibility to imaging facilities have benefited patients with significant improvements in diagnostic capabilities but at the same time have resulted in a substantial increase in healthcare costs.1 2
In addition, the increasing complexity of imaging has often been accompanied by inefficient use of diagnostic facilities, which has led to inappropriate patient management and unnecessary radiation exposure.3 CT delivers much higher doses of ionising radiation than conventional radiographs, and previous research has linked exposure to radiation levels in this range to the development of radiation-induced cancers.4 5
The increased utilisation of high-cost imaging examinations has motivated health systems worldwide to implement control mechanisms aimed at appropriate utilisation of imaging examinations.3 6 7
Assessing the appropriateness of individual medical imaging procedures is a complex issue involving several factors and may vary with the age, gender and physical limitations of the patient as well as with the condition and symptoms being investigated.8 The American College of Radiology (ACR) developed for the first time in 1993 an evidence-based set of appropriateness criteria (AC), which was revised in 2008, 2015 and 2017 and intended to guide physicians to the appropriate use of diagnostic and interventional radiology for given clinical situations.9
Previous studies have been conducted to estimate the proportion of outpatient examinations improperly prescribed and performed by using ACR-AC or other similar guidelines, and they showed a CT and MRI inappropriateness rate ranging from 26% to 44%.10 11 The causes of inappropriate utilisation include medical liability fears, patients’ demands, regional differences in practice style and physician experience and training in the appropriate use of newer imaging modalities.9 To the best of our knowledge, very limited research has targeted appropriate use of CT and MRI performed within the hospital healthcare setting.12–15
The primary aim of this study was to assess the frequency of appropriateness of inpatient CT and MRI scans performed in Southern Italy. The secondary aim was to identify possible variables that could affect the appropriateness, since we hypothesised that patient’s and examination’s characteristics, such as the investigated anatomical scan region, might be related to the appropriateness of CT and MRI scans performed in the hospital setting.
Materials and methods
Data collection was carried out from May 2013 to September 2014. Two trained physicians, who had experience in clinical documentation and were not involved in patient care, retrospectively reviewed medical records of patients aged 18 or more admitted between 1 January and 31 December 2012 to medical and surgical wards of a teaching hospital and a non-teaching acute care hospital located in Catanzaro (Italy). All medical records related to patients who received at least one CT or MRI examination were identified from an administrative database and were considered eligible for the study. Among these eligible medical records, we included in the study those reporting at least one clinical condition that matched with the list drawn up by the ACR-AC.
The sample size was determined prior to commencement of the study. It was calculated assuming an appropriateness rate of 50%, a margin of error of 5% and a 95% confidence level. Consequently, we sought to obtain a sample of 385 medical records. Anticipating an unavailability of clinical documentation in 30% of cases, a total sample size of 550 records was therefore needed. We decided to include an additional 300 medical records in case the clinical documentation was not complete.
To determine the sample size needed to evaluate the inter-rater agreement, we anticipated that there would be a 50% agreement and a relative error of 25%; thus, we calculated that less than 100 sample size was needed.
The following data were recorded for each patient: (1) sociodemographic characteristics (gender, age, marital status, education level and working activity); (2) characteristics of hospitalisation (date, diagnosis, mode and ward of admission and discharge); (3) clinical data (previous hospitalisations for the same disease, CT and MRI examinations for the same disease performed before admission and other diagnostic imaging examinations performed during hospital stay). To assess the clinical conditions of patients, we used the Charlson Comorbidity Index16 that predicts the 10-year mortality for a patient who may have a range of several comorbid conditions. Each condition is assigned a score of 1, 2, 3 or 6, depending on the risk of dying associated with each one. Scores are added up to provide a total score to predict mortality; and (4) additional information about CT and MRI examinations performed during hospitalisation (type, date, diagnostic question and its eventual confirmation, contrast agent use, radiation exposure dose and appropriateness).
For each clinical record, the available clinical and imaging data were retrieved to identify all the clinical and demographic factors that could contribute to the justification for the use of diagnostic imaging examinations. The ACR-AC addresses a large number of clinical conditions and their variants and assigns an appropriateness score to the radiological procedures performed for each clinical condition (online supplementary appendix 1), then the two physicians reviewed the ACR-AC to identify a match between the indication of the examination and a variant of a clinical condition reported in the ACR-AC, and the appropriateness score for the performed imaging examination was recorded. The rating of appropriateness was determined by ACR-AC on the basis of type and anatomical site of radiological procedure, use of intravenous contrast, setting of performing and so on. Therefore, there could not be any possibility for the physicians who reviewed the medical records to arbitrarily decide on the appropriateness in a way that would override the ACR-AC.
Supplementary file 1
The appropriateness is represented on an ordinal scale that uses integers from 1 to 9, which are grouped into three categories: if a radiological procedure is assigned a score from 1 to 3, it is classified as ‘usually not appropriate’; if from 4 to 6, it is classified as ‘may be appropriate’; if from 7 to 9, it is classified as ‘usually appropriate’.9 The application of ACR-AC assumes that the ultimate decision about the appropriateness of CT/MRI examinations is made in light of all the circumstances presented in an individual examination, including whether the examination is performed with the aim to confirm or exclude other pathology/conditions. When an examination received a score from 4 to 6, then physicians conservatively reclassified it as appropriate.
If a patient had received more than one diagnostic imaging examination, the judgement of appropriateness was carried out for each examination. If all examinations were judged appropriate, the patient was classified as being among those who received appropriate examinations. If at least one examination was inappropriate, we classified the patient as being among those who received an inappropriate examination.
At the time of writing the manuscript, the ACR-AC was revised. The latest release includes 11 new and 21 revised topics.9 However, since the main changes have involved imaging procedures other than CT and MRI that were included in this study, this update has not substantially modified the assignment of the appropriateness score in our study.
The two physicians concurrently and independently reviewed 85 medical records with the aim of evaluating the inter-rater reliability. Eventual disagreement in determining clinical conditions and variants that could affect the classification of the patients and, subsequently, the appropriateness rating of their examination, was resolved by discussion or if necessary by consensus in consultation with a third author (AB).
Statistical analysis
The overall agreement and the k statistic were used to assess the inter-rater reliability regarding the appropriateness of CT and MRI examinations. Multivariable stepwise logistic regression models were performed to determine the independent association of the several characteristics with the following outcomes of interest: appropriateness of CT examination (0=at least one inappropriate, 1=all appropriate or potentially appropriate) (model 1) and appropriateness of MRI examination (0=at least one inappropriate, 1=all appropriate or potentially appropriate) (model 2). The following explanatory variables were potentially included in all models: patient’s age (five categories: 18–45 years=1, 46–55 years=2, 56–65 years=3, 66–75 years=4, >75 years=5), Charlson Comorbidity Index (0=0 and ≥1=1), previous outpatient diagnostic examinations (no =0, yes =1), previous hospitalisations for the same admission disease (no =0, yes =1), ward of admission (medical=0, surgical=1), admission type (programmed=0, urgent=1), length of hospital stay (continuous), contrast agent use (no=0, yes=1) and diagnostic hypothesis confirmed (no=0, yes=1). For model 1, the following variables were also included: more than one CT examination performed (no=0, yes=1), anatomical scan region (abdomen/pelvis=1, chest=2, head=3, whole body=4, musculoskeletal system=5, vascular system=6). For model 2, more than one MRI examination performed (no=0, yes=1), anatomical scan region (abdomen/pelvis=1, chest=2, head=3, musculoskeletal system/spine/extremities=4, vascular system=5). If multiple diagnostic imaging procedures were performed, we chose the anatomical scan region of the first performed examination when all procedures were appropriate and of the first inappropriate examination if at least one examination was considered inappropriate.
The model building strategy included the following steps: (1) univariate analysis of each variable considered, using the appropriate statistic test (χ2 test, Fisher’s exact test or t-test) and (2) inclusion of any variable whose univariate test showed a P value lower than 0.25. The significance level for including variables in the two models was set at P=0.2, and P=0.4 for dropping variables from the models. The results of the logistic regression analysis are presented as ORs and 95% CIs.
Stata V.14 statistical software package was used in conducting all data analysis.17
According to the design of the present study, researchers were exempted from obtaining written consent by the patients who are requested during the hospitalisation to give permission for their personal data to be used for research, as detailed by the Italian rules (Legislative Decree 196/2003).
Results
One thousand eight hundred and seventy-four medical records of patients who received at least one CT or MRI were considered eligible; 937 of them reported at least one clinical condition included in the ACR-AC list and were included in the study. Eighty-four medical records were not available or did not report essential data for the appropriateness judgement. Therefore, 853 medical records were reviewed. In terms of test–retest reliability of the tool, the overall inter-rater agreement was excellent between the two reviewers, since the agreement and the k statistic for the assessment of the appropriateness of CT and MRI examinations were 92.5% and 0.84%, respectively. Indeed, only for six medical records the physicians had to discuss and to resolve the disagreement about the appropriateness classification. Six hundred and thirty-nine patients received at least one CT examination, and 256 received at least one MRI examination. Patient mean age was 62.7 years, the majority of the admissions were urgent, median length of stay was 11.3 days (range: 1–65 days) and the majority of the study population was admitted to medical wards (73.2%).
Overall, 751 CT examinations were reviewed, since 99 patients (15.5%) received more than one CT examination during the hospital stay. Among all CT performed, 596 (79.4%) were considered appropriate and 496 (77.6%) of 639 patients had all appropriate CT. Table 1 illustrates CT examinations by anatomical scan region and indications with relative appropriateness rates.
Inpatient CT-specific indications and relative appropriate rates
Three anatomical areas presented consistently higher CT scan rates: head (38.1%), abdomen/pelvis (22.9%) and chest (21.8%). In particular, a total of 286 brain, head and neck CT were performed during the study period.
More than half (55.3%) of head CT were requested for cerebrovascular disease; abdominal pain accounted for 34.9% of the CT scans of the abdomen/pelvis, whereas cancer, including screening purposes, staging and follow-up examinations, was the most frequent reason for performing a whole body CT.
Total body CT represented the least appropriate examination (62.3%). Other less appropriate site-related scans included musculoskeletal system CT (64.7%) and abdomen/pelvis CT (80.8%). The most frequent clinical conditions for which CT scans were deemed less appropriate included: abdomen/pelvis CT for kidney and urinary tract disease (61.4%); head CT for sensory loss (62.5%) and cerebrovascular disease (87.3%); and chest CT for acute respiratory illness (77.3%).
Three hundred and seventy-one MRI examinations were reviewed, since 92 patients (35.9%) received more than one MRI examination during hospital stay. Overall, 310 MRI examinations (83.6%) were considered appropriate, and 202 (78.9%) patients received appropriate MRI examinations. As shown in table 2, head MRI was the most requested examination (65.2%), primarily prescribed for suspected dementia (39.3%). Movement disorders accounted for 45.6% of spine MRI, whereas cancer and jaundice were the most frequent reasons for prescribing an abdomen/pelvis MRI.
Inpatient MRI-specific indications and relative appropriate rates
The lowest percentage of appropriate examinations (37.9%) was found to be for vascular system MRI. Indications for less appropriate MRI examinations included a broad array of clinical conditions, such as headache for vascular system (29.4%) and head MRI (86.4%), acute back pain for spine MRI (28.6%) and abdominal pain for abdomen/pelvis MRI (87.5%).
Table 3 shows the distribution of the appropriateness of diagnostic examinations (CT/MRI) according to various explanatory variables.
Descriptive characteristics of study population and distribution of appropriateness of diagnostic examination (CT/MRI) according to various explanatory variables
After univariate analysis, appropriate CT examinations were significantly more likely in subjects with an urgent admission (χ²=6.36, 1 df, P=0.012), with a shorter hospital stay (t=−1.98, 637 df, P=0.047), in those who received CT examinations without contrast agent (χ²=48.49, 1 df, P<0.001), or only one CT examination (χ2=27.09, 1 df, P<0.001), whereas they were significantly less likely in those whose musculoskeletal system or whole body were investigated compared with other sites (Fisher’s exact test: P=0.038). Appropriateness of CT examinations was also associated with a confirmation of the diagnostic hypothesis (χ²=87.41, 1 df, P<0.001). Results of the multiple logistic regression analysis partially confirmed those of the univariate analysis, except for length of stay and admission type that were not significantly associated with appropriateness of CT (table 4).
Multiple logistic regression analysis results examining inappropriateness of CT and MRI according to several explanatory variables
Appropriateness of MRI examinations, after univariate analysis, was associated with the diagnostic hypothesis confirmation (χ2=7.62, 1 df, P=0.006), and MRI scan region, with vascular and musculoskeletal system/spine/extremities being the least appropriate anatomical scan regions (Fisher’s exact test: P<0.001) (table 3). Appropriate MRI examinations were also significantly more likely among patients who received only one MRI examination during hospital stay (χ2=35.24, 1 df, P<0.001). These findings were completely confirmed after multivariate analysis (table 4).
Discussion
To the best of our knowledge, this study represents the first attempt to assess the appropriateness of inpatient CT and MRI examinations in Italy, using the ACR-AC as reference. Indeed, prior studies comprised only outpatient requests referred to diagnostic imaging departments, whereas our sample is the first study exploring examinations requested during hospital stay. Appropriate use of MRI and CT is very important both medically and economically. There have been suggestions of various factors influencing overutilisation in many countries, including defensive medicine.10 18
This study showed an overall higher appropriateness than previous studies,10 11 and it is not completely surprising since in Italy the provision of inpatient care, free of charge for all, is properly addressed by a specialist who clinically evaluates the patient. Nonetheless, a lower appropriateness rate compared with other hospital settings, such as the emergency department, was shown.14
However, comparisons with previous studies must be made with caution, since differences exist with respect to forms of care and methodology. First, as already stated, our data are from inpatient subjects. Moreover, in other studies, reference criteria were based on different recommendations or on the ACR-AC in combination with other guidelines.10 11 19 Rosenkrantz et al 20 who, similarly to us, used exclusively ACR-AC, showed a higher percentage of appropriate investigations (almost 90%) that is close to our results. Regarding inpatient imaging, Moriarity et al 12 examined the effect of electronic clinical decision support (CDS) using ACR-AC for imaging requests and focused on the average AC score before and after CDS use.
As reported in previous studies,10 21 inappropriate use of imaging services included head CT for chronic headache and cerebrovascular diseases, and lumbar spine MRI for acute back pain. These results also match with an analysis of utilisation trends among Medicare beneficiaries in the USA, showing that almost 30% of patients underwent imaging studies within the first 28 days of an episode of acute low back pain,22 although appropriateness guidelines from many specialties, including those of the ACR, do not recommend any imaging 6 weeks before any episode of acute back pain without ‘red flags’ suggesting serious disease. Another reason for inappropriate imaging scans included whole body CT for cancer screening and recent indications provided by the American College of Preventive Medicine have strongly advised against this practice.23
Inappropriateness of CT and MRI was associated with multiple factors that warrant careful attention. We found that inappropriate CT and MRI were less likely to confirm the diagnostic hypothesis. This observation helps validate the value of the AC in mitigating the use of those imaging procedures likely to provide a negative result. Moreover, as reported in previous studies,10 11 20 the correct orientation of the clinician and the use of an appropriate diagnostic technology contribute to confirm diagnostic hypothesis and, indeed, the AC were designed to ideally select for examinations expected to have maximal diagnostic yield, when balanced with cost and imaging-related risks.
We also found an association between inappropriateness of CT examinations and contrast agent use. This result has already been reported24 and highlights the importance of a careful use of contrast agent, because this can result in unnecessary exposure of patients to the risk of adverse reactions or nephropathy induced by these agents.25
As expected, there was a correlation between the anatomical scan region and inappropriateness of radiological procedures, particularly for scans of the muscoloskeletal system and the spine/estremities. In these sites, CT and MRI are generally used as second-line examinations to solve specific diagnostic problems, whereas aspecific clinical conditions such as low back pain should be managed through proper clinical observation and first-line radiological examinations.
Our study showed that a relevant percentage of patients received multiple CT or MRI examinations, and repeated examinations were more likely to be inappropriate.26–28 As reported in previous studies, repeated imaging is common, and an uncertain proportion of them likely represents an inappropriate use and overuse. For example, Ip et al 29 showed that the great majority of repeated abdominal imaging occurred contrary to radiologists’ follow-up recommendations. Given the frequency and volume of repeat testing and its potential impact on quality of care and costs, future studies should evaluate strategies to improve the appropriateness of repeat testing and follow-up imaging recommendations.
Several strategies have been evaluated to reduce the overuse and inappropriate use of CT and MRI. Request for a consultation with a radiology specialist before the examination and evaluation of the requests using a computerised preauthorisation system3 seems to have significantly reduced the number of inappropriate examinations. Moreover, early work on the impact of CDS in reducing redundant imaging is promising. O’Connor et al 30 showed that CDS led to the cancellation of 5% of repeat CT orders. Moriarity et al 12 reported a slight increase in the average AC score after CDS introduction. Findings reported in these studies suggest the possible use of these strategies for inpatient radiological examinations.
The results of our study should be interpreted in light of few potential limitations. First, retrospective data collection may have distorted the actual rate of appropriateness, since it is influenced by the quality of medical records. Accuracy and completeness are two main characteristics that may affect data quality. In the present study, we are more prone to affirm that lack of data, instead of incorrect data, could have led to an alteration in the evaluation of the clinical condition, with a relative underestimation of appropriateness. Nevertheless, retrospective data collection is a common and accepted method for the evaluation of appropriateness and also to estimate wasteful imaging.31 32 Second, collected data are referred to 2012. It is therefore possible that, in this 5-year period, the awareness campaigns, clinical decision support systems33 34 and legislation may have resulted in an increase in the appropriateness rate of radiological procedures. In this context, the Council Directive 2013/59 Euratom, which stipulates procedures, roles and responsibilities that need to be observed by hospitals and professionals involved in medical radiation exposures,35 has been introduced into Italian national legislation in 2015, with a possible positive impact on imaging examinations appropriateness.
Moreover, our cohort comprised imaging examinations performed in the inpatient setting at a teaching hospital and a non-teaching acute care hospital located in Southern Italy, and this cohort may not be representative of all Italian hospitals. However, we are confident that the findings of the study may be representative of at least the southern part of our country. Third, we evaluated the appropriateness exclusively through ACR-AC that is not commonly used in Italy. However, although European alternatives to ACR-AC, such as RCR iRefer, French or Italian guidelines for radiological examinations appropriateness evaluation were available,36–38 we decided to apply ACR-AC, as we were interested to gain an objective evaluation of appropriateness, with the assignment of a numeric score, and none of these other alternatives would allow this approach. In addition, Italian guidelines for diagnostic imaging were published in 2004 and never updated.38 Therefore, in light of our experience, it would be worth applying the ACR-AC to the Italian context. Finally, in those cases where multiple examinations were performed, we decided to classify a particular case as inappropriate if at least one examination was inappropriate, but we cannot exclude a possible bias in the findings, that could have reduced the extent of the gap between appropriate and inappropriate examinations. However, of the total 751 CT and 371 MRI performed, respectively 137 CT (18%) and 132 (35.6%) MRI were repeated imaging examinations. Thus, we may be confident that the impact of this bias on our results, if present, is probably marginal.
Conclusion
Our findings showed that there is a significant percentage of inpatient inappropriate imaging exams, and specific areas for improvement have been identified. The study shows that the tool used is reliable and has adequate validity to measure the extent of appropriateness of diagnostic imaging also in our context and for inpatient examinations. Further research is needed to expand appropriateness evaluation in this care setting, to investigate more thoroughly internal and external causes of inappropriate use of imaging examinations and also to evaluate the effectiveness of some strategies such as the use of a computerised preauthorisation system in order to reduce inappropriateness.
References
Footnotes
Contributors AB gave substantial contributions to the conception and design of the study and to the data analysis and interpretation and wrote the first draft of the paper. FL and RZ collected the data and contributed to the data analysis and drafting the paper. MP gave substantial contributions in the design of the study and was responsible for the data analysis and revising the paper critically for important intellectual content. All authors approved the final paper as submitted and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding This research received no specific grant from any funding agency in the public, commercial or not- for-profit sectors.
Competing interests None declared.
Patient consent Obtained.
Ethics approval The study protocol was approved by the Institutional Ethical Committee of “Mater Domini” Hospital in Catanzaro, Italy (7 May 2013).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Survey data were not included in the present article and are available from the authors.