Objective To validate an existing clinical decision support tool to risk-stratify patients with acute kidney injury (AKI) for hydronephrosis and compare the risk stratification framework with nephrology consultant recommendations.
Setting Cross-sectional study of hospitalised adults with AKI who had a renal ultrasound (RUS) ordered at a large, tertiary, academic medical centre.
Participants Two hundred and eighty-one patients were included in the study cohort. Based on the risk stratification framework, 111 (40%), 76 (27%) and 94 (33%) patients were in the high-risk, medium-risk and low-risk groups for hydronephrosis, respectively.
Outcomes Outcomes were the presence of unilateral or bilateral hydronephrosis on RUS.
Results Thirty-five patients (12%) were found to have hydronephrosis. The high-risk group had 86% sensitivity and 67% specificity for identifying hydronephrosis. A nephrology consult was involved in 168 (60%) patients and RUS was recommended by the nephrology service in 95 (57%) cases. Among patients with a nephrology consultation, 9 (56%) of the 16 total patients with hydronephrosis were recommended to obtain an RUS.
Conclusions We further externally validated a risk stratification framework for hydronephrosis. Clinical decision support systems may be useful to supplement clinical judgement in the evaluation of AKI.
- quality in health care
- acute renal failure
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Our study evaluates the utility of a clinical decision support tool to risk-stratify patients with acute kidney injury (AKI) for hydronephrosis.
We found that the clinical decision support tool predicted hydronephrosis cases with high sensitivity and moderate specificity.
Our results highlight that risk stratification frameworks may supplement clinical judgement in the evaluation of inpatient AKI.
Limitations include the study being a single-centre study that evaluated only patients who had a renal ultrasound and not all patients with AKI.
Our analysis was also limited to information present in the electronic health record.
Renal ultrasounds (RUS) are commonly ordered in hospitalised patients with acute kidney injury (AKI) to evaluate kidney size and parenchyma and to rule out obstructive pathology.1 2 Obstructive uropathy causing hydronephrosis is an uncommon cause of AKI, and clinical decision support systems (CDSS) could be used to inform which patients may benefit from RUS to rule out hydronephrosis.1 3–10
A risk stratification framework to identify patients who are at high risk of having hydronephrosis has been previously derived, but is currently underutilised.11 12 Several factors may account for this, including lack of knowledge of the framework and lack of workflow integration. Recommendations from consulting nephrologists also inform RUS ordering for hospitalised patients with AKI; however, the relationship between risk stratification frameworks and nephrologist recommendations is unclear.
In this study, we further externally validated a risk stratification framework for hydronephrosis at our institution. We add to the literature by comparing the risk stratification tool with nephrology consultation recommendations for RUS ordering. We also examined the reason for the examination, incidental findings, and additional work-up from RUS. Understanding the potential role of CDSS to risk-stratify patients for hydronephrosis may improve the appropriateness of RUS ordering in the clinical evaluation of AKI.13
Study design and population
We performed a cross-sectional study of hospitalised adults at an urban, 1134-bed academic medical centre in New York, New York. We included all patients who met the following three criteria: (1) admitted to the hospitalist service between 9 June 2013 and 24 October 2014; (2) experienced AKI (defined as creatinine rise >0.3); and (3) had a formal RUS completed. Imaging was performed by trained, experienced ultrasonographers.
Our primary predictor was high-risk, medium-risk, or low-risk category for hydronephrosis based on the risk stratification framework developed by Licurse et al (online supplemental table 1).12 The following are the seven criteria included in the framework: a history of hydronephrosis (4 points), and non-black race, history of recurrent urinary tract infections (UTIs), diagnosis consistent with possible obstruction (abdominal or pelvic mass, benign prostatic hypertrophy, pelvic surgery, or neurogenic bladder), absence of exposure to inpatient nephrotoxic agents (aspirin >81 mg, diuretics, ACE inhibitors or vancomycin), absence of congestive heart failure, and/or absence of prerenal AKI (pressor use or sepsis) (1 point each). A total score of 4 or more points was classified as high risk, 3 points as medium risk, and 2 or fewer points as low risk.12 We performed an additional sensitivity analysis defining prerenal AKI as pressor use, sepsis or history of hypotension, defined as at least two consecutive blood pressure measurements below 80 mm Hg systolic or below 60 mm Hg diastolic. Additional predictors were presence of nephrology consultation and RUS recommended by nephrology.
We performed a retrospective chart review of data in the electronic health record (EHR) to determine the presence or absence of each predictor, using only data available before the RUS was ordered. One author (JZ) performed the initial chart review, which was recorded in the REDCap web-based application (Vanderbilt University). Another author (CG) independently reviewed a random selection of 10% of medical records, with 96% agreement (Cohen’s κ=0.913, p<0.001).
Our primary outcome was the presence of unilateral or bilateral hydronephrosis. Severity of hydronephrosis (mild, moderate and severe) was ascertained by radiologists in the official RUS report. Secondary outcomes were incidental findings in the RUS radiology report, urological procedure after RUS, and further imaging as a result of RUS during the hospitalisation. Incidental findings were classified into the following categories: increased echogenicity, simple cysts, complex cysts, renal atrophy or cortical thinning, non-obstructive nephrolithiasis, renal enlargement, renal mass, absence of kidney, obstructive nephrolithiasis, or other.14 We also determined the reason for RUS ordering based on comments written in the ‘Reason for exam’ field in the RUS order.
We described patient characteristics according to risk group for hydronephrosis, presence of nephrology consultation, and RUS recommended by the consult service. We determined the prevalence of patients with hydronephrosis by risk group, nephrology consultation, and RUS recommendation. χ2 and analysis of variance tests were performed to compare patient characteristics by risk group for hydronephrosis and nephrology consultation status. We reported the test characteristics of the risk stratification framework for identifying hydronephrosis. We calculated Pearson correlation coefficients to assess the correlation of factors in the risk score with hydronephrosis in our population. Methods to account for multiple comparisons, such as Bonferroni correction, were not employed. Analyses were performed in Stata V.15 statistical software.15
Patient and public involvement
Patients were not involved in the design of the study, including the research question, outcome measures, or conduct. Results will be disseminated to patient groups via email listservs and via publication in the peer-reviewed literature.
Three hundred and twenty-two patients admitted to the hospitalist service received an RUS during the study period. Forty-one of these patients were not included because they did not have AKI. The final cohort included 281 patients. The mean age was 64 years, 57% of patients were male, and 63% were non-black (table 1).
Risk stratification framework
There were 111 (40%), 76 (27%), and 94 (33%) patients in the high-risk, medium-risk, and low-risk groups for hydronephrosis, respectively. Patients in the high-risk group were less likely to be black (14% vs 63%, p<0.001) and had higher creatinine levels at admission (4.2 mg/dL vs 3.0 mg/dL, p=0.048) compared with those in the low-risk group (table 1). Thirty-five patients (12%) were found to have hydronephrosis, of whom 30 (86%) were captured in the high-risk group (table 2). Thirty of the high-risk patients (27%), one medium-risk patient (1%), and four of the low-risk patients (4%) were found to have hydronephrosis on RUS. The high-risk group had 86% sensitivity and 67% specificity for identifying hydronephrosis (table 3A). The medium-to-high-risk group had 89% sensitivity and 37% specificity for identifying hydronephrosis (table 3). Among patients with hydronephrosis, 17 of 35 (49%) had unilateral hydronephrosis and 18 of 35 (51%) had bilateral hydronephrosis. Test characteristics did not meaningfully change after sensitivity analysis changing the definition of prerenal AKI (online supplemental table 2). Five of the seven criteria were found to be statistically significantly correlated with increased risk of hydronephrosis in our population (table 4).
A nephrology consult was involved in 168 patients (60%). Patients with nephrology consultation had fewer recurrent UTIs (2% vs 17%, p<0.001), higher creatinine levels at admission (4.7 mg/dL vs 2.2 mg/dL, p<0.001), and higher baseline creatinine (2.1 mg/dL vs 1.3 mg/dL, p<0.001) compared with those who did not have nephrology consultation (table 5). RUS was recommended by the nephrology service in 95 of 168 cases (57%). Similarly, in the low-risk patient group, of the 61 low-risk patients (65%) who had a nephrology consult, 35 (57%) were recommended to obtain an RUS. Of the 95 patients recommended for an RUS, 9 (9%) had hydronephrosis. Among patients with a nephrology consultation, 9 (56%) of the 16 total patients with hydronephrosis were recommended to obtain an RUS (table 2).
Reason for examination, incidental findings and additional work-up
The majority of RUS were ordered for AKI or renal failure (57%) or to rule out hydronephrosis or obstruction (24%). Additional reasons for examination were AKI indicated predominantly by oliguria or anuria, rule out pyelonephritis or abscess, evaluate for nephrolithiasis, and other (table 6). Twenty-four percent of RUS listed rule out hydronephrosis or obstruction as the reason for examination. Only three RUS specifically listed the reason for examination as evaluating kidney size, echogenicity, or chronicity of kidney disease.
Only 37 patients (13%) had no abnormal findings on RUS. Even within the low-risk group, only 16% of RUS were without abnormalities. The most common finding was increased echogenicity, which occurred in 195 patients (69%). One hundred and two (36%) had simple cysts and 34 (12%) had complex cysts (table 7).
Four patients (1.4%) in the entire cohort underwent a urological procedure, all of whom were in the high-risk group. Thirty-five patients (12%) received further imaging as a result of the RUS findings. Of the patients who underwent further imaging, 21 (60%), 11 (31%) and 3 (9%) were in the high-risk, medium-risk and low-risk groups, respectively.
We found that the high-risk group in a risk stratification framework identified 86% of episodes of hydronephrosis in hospitalised patients with AKI who obtained an RUS.11 12 The majority (60%) of RUS were ordered on medium-risk or low-risk patients, while only 3% of these patients were found to have hydronephrosis. Our findings highlight the potential additive role of CDSS for hydronephrosis in supplementing clinical judgement in the evaluation of AKI.
Prior analyses4 11 have externally validated the Licurse et al12 risk stratification framework. Ip et al11 found 93.4% sensitivity and 25.1% specificity for hydronephrosis, similar to our results, using a cut-off between low-risk and medium-risk/high-risk patients.4 Our results support the use of a higher cut-off between low-risk/medium-risk and high-risk patients to improve specificity with minimal reduction in sensitivity.
Given that this risk stratification framework has been externally validated and has promising test characteristics, the next step is to prospectively implement and evaluate it. The American College of Radiology supports the use of CDSS as a method to ensure appropriate imaging, and CDSS has been associated with decreased imaging utilisation.16 17 A pragmatic randomised trial could assess if the risk stratification framework built within an EHR-enabled CDSS versus usual care improves AKI diagnosis, management, and/or appropriateness of imaging.
The risk stratification framework studied was aimed specifically at identifying obstructive pathology on RUS in patients with AKI. To assess if consulting or primary teams requesting RUS were evaluating factors other than hydronephrosis, such as size or echogenicity of kidneys, we assessed the reasons for RUS ordering listed in the ‘Reason for exam’ field. The majority of RUS were ordered for AKI or renal failure (57%), without further explanation. We were thus unable to reliably assess the clinical rationale for RUS ordering.
We found that the majority (87%) of RUS had incidental findings, which was similar to prior estimates.18 Incidental findings pose challenges to physicians and patients, and can cause patient anxiety and additional interventions.19 Importantly, over 10% of patients had important incidental findings, such as complex cysts, which may require further imaging and evaluation. The high prevalence of incidental findings with RUS imaging raises questions about when disclosure of these findings is warranted.20 While the majority of RUS had abnormalities, there was a low prevalence of urological procedures and further imaging during the hospitalisation.
The major strength in our approach was the detailed chart review used in our analysis. There are also limitations to our results. We conducted a single-site study and did not assess all patients with AKI, only those who had an RUS ordered. Our analysis was limited to information present in the EHR, so we did not capture nephrology consult recommendations that may have been given verbally. In some cases where RUS was obtained prior to nephrology consultation, it was unclear based on the available documentation whether the consultant would have also recommended the study. Lastly, the definition used for prerenal AKI was use of pressors or history of sepsis, which may lack sensitivity for other clinically significant hypovolaemic states.12
In conclusion, we further validated a risk stratification framework for hydronephrosis at our institution. There was a very high prevalence of incidental findings on RUS, but additional imaging and procedures were low during the inpatient hospitalisation. Our findings highlight the potential utility of CDSS to supplement comprehensive clinical evaluation in AKI.
Data availability statement
Data are available upon reasonable request.
The Mount Sinai Institutional Review Board reviewed and approved the design of the present study.
The authors thank Dr Andrew S Dunn for editorial assistance.
Contributors Research idea and study design: SLT, JZ, HC, CG. Data acquisition: JZ, CG. Data analysis/interpretation: SLT, JZ. Statistical analysis: SLT, JZ. Supervision or mentorship: HC, CG. All authors contributed important intellectual content during manuscript drafting and revision and approved the final manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests SLT receives consulting fees from Bayer, outside of the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.