Article Text

PDF

Optimal cut points of plasma and urine neutrophil gelatinase-associated lipocalin for the prediction of acute kidney injury among critically ill adults: retrospective determination and clinical validation of a prospective multicentre study
  1. Kristen M Tecson1,2,3,
  2. Elisabeth Erhardtsen4,
  3. Peter M Eriksen4,
  4. A Osama Gaber5,
  5. Michael Germain6,
  6. Ladan Golestaneh7,
  7. Maria de los Angeles Lavoria4,
  8. Linda W Moore5,
  9. Peter A McCullough1,3,8,9
  1. 1 Baylor Heart and Vascular Institute, Dallas, Texas, USA
  2. 2 Baylor Scott & White Research Institute, Dallas, Texas, USA
  3. 3 Texas A&M College of Medicine Health Science Center, Dallas, Texas, USA
  4. 4 BiPorto Diagnostics A/S, Copenhagen, Denmark
  5. 5 Houston Methodist Hospital, Houston, Texas, USA
  6. 6 Baystate Medical Center, Springfield, Massachusetts, USA
  7. 7 Albert Einstein College of Medicine, Bronx, New York, USA
  8. 8 Baylor University Medical Center, Dallas, Texas, USA
  9. 9 Baylor Jack and Jane Hamilton Heart and Vascular Hospital, Dallas, Texas, USA
  1. Correspondence to Dr Peter A McCullough; peteramccullough{at}gmail.com

Abstract

Objectives To determine the optimal threshold of blood and urine neutrophil gelatinase-associated lipocalin (NGAL) to predict moderate to severe acute kidney injury (AKI) and persistent moderate to severe AKI lasting at least 48 consecutive hours, as defined by an adjudication panel.

Methods A multicentre prospective observational study enrolled intensive care unit (ICU) patients and recorded daily ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma and urine NGAL. We used natural log-transformed NGAL in a logistic regression model to predict stage 2/3 AKI (defined by Kidney Disease International Global Organization). We performed the same analysis using the NGAL value at the start of persistent stage 2/3 AKI.

Results Of 245 subjects, 33 (13.5%) developed stage 2/3 AKI and 25 (10.2%) developed persistent stage 2/3 AKI. Predicting stage 2/3 AKI revealed the optimal NGAL cutoffs in EDTA plasma (142.0 ng/mL), heparin plasma (148.3 ng/mL) and urine (78.0 ng/mL) and yielded the following decision statistics: sensitivity (SN)=78.8%, specificity (SP)=73.0%, positive predictive value (PPV)=31.3%, negative predictive value (NPV)=95.7%, diagnostic accuracy (DA)=73.8% (EDTA plasma); SN=72.7%, SP=73.8%, PPV=30.4%, NPV=94.5%, DA=73.7% (heparin plasma); SN=69.7%, SP=76.8%, PPV=32.9%, NPV=94%, DA=75.8% (urine). The optimal NGAL cutoffs to predict persistent stage 2/3 AKI were similar: 148.3 ng/mL (EDTA plasma), 169.6 ng/mL (heparin plasma) and 79.0 ng/mL (urine) yielding: SN=84.0%, SP=73.5%, PPV=26.6%, NPV=97.6, DA=74.6% (EDTA plasma), SN=84%, SP=76.1%, PPV=26.8%, NPV=96.5%, DA=76.1% (heparin plasma) and SN=75%, SP=75.8%, PPV=26.1, NPV=96.4%, DA=75.7% (urine).

Conclusion Blood and urine NGAL predicted stage 2/3 AKI, as well as persistent 2/3 AKI in the ICU with acceptable decision statistics using a single cut point in each type of specimen.

  • neutrophil gelatinase associated lipocalin
  • acute kidney injury
  • risk prediction
  • sensitivity
  • specificity

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Strengths and limitations of this study

  • This study used biomarker data from both plasma and urine samples.

  • Acute kidney injury diagnosis was adjudicated by an expert panel.

  • We used an unbiased, data-driven approach to identify a single cut point for each plasma and urinary NGAL sample to predict stage 2 or 3 acute kidney injury and persistent stage 2 or 3 acute kidney injury.

  • This small prospective cohort study had limited clinical follow-up.

Introduction

Serum creatinine and urine output are used to diagnose acute kidney injury (AKI) as defined by the 2012 Kidney Disease International Global Organization (KDIGO) guidelines (see online supplementary table 1); however, creatinine is an imperfect marker as it peaks anywhere from 2 to 7 days following an insult to the kidneys.1 Additionally, urine output may lead to an inaccurate diagnosis of AKI in the intensive care unit (ICU), as it is commonly manipulated in the ICU through use of intravenous fluids and diuretics. Neutrophil gelatinase-associated lipocalin (NGAL) has been shown to peak following tubular injury earlier than changes in creatinine and urine output, making it a desirable biomarker for predicting the development of AKI.2 Previous studies have shown the utility of this biomarker, but have suffered from heterogeneous AKI thresholds and lack of validation of parenchymal renal involvement in the injury. In those studies, AKI may have been due to a transient haemodynamically related reduction in renal filtration due to poor forward perfusion or impaired plasma refill, which represents a distinct AKI pathophysiology.3 4 Furthermore, the optimal body fluid for NGAL assay performance (blood or urine) is unknown, as is the optimal cut point for the detection of moderate to severe AKI (KDIGO stage 2 or 3). We set out to use a commercially available NGAL assay in a multicentre, blinded study to assess its performance in two types of plasma samples and in one urine sample for the prediction of moderate to severe AKI, as defined by KDIGO and adjudicated by clinicians blinded to the NGAL values.

Supplementary Material

Supplementary Table 1

Methods

Study details

This study prospectively enrolled consecutive adult patients admitted to an ICU or critical care setting after obtaining informed consent between 5 March 2014 and 15 April 2015 from four participating hospitals in Boston and Springfield, Massachusetts; Bronx, New York; and Houston, Texas; the protocol may be found at the following: http://www.bioporto.com/Products/Featured-areas/NGAL.aspx. Each study site received institutional review board (IRB) approval prior to enrolling patients (IRBs used for this study are as follows: Baystate Health IRB #1; Houston Methodist Research Institute IRB 1; Partners Human Research Committee; Biomedical Research Alliance of New York IRB). Those with history of nephrectomy, renal transplantation and/or renal replacement therapy initiated before admission were not eligible to participate. In addition to the standard clinical care and laboratory testing, one urine and two plasma samples were drawn and frozen per day in the ICU, up to 8 days. The samples were shipped to a central laboratory where NGAL was assayed by BioPorto Diagnostics A/S, Copenhagen, Denmark. This test has a measurable range of 25–3000 ng/mL.5 At mean NGAL concentrations of 97, 116 and 112 ng/mL in EDTA plasma, heparin plasma and urine, respectively, the coefficients of variation were 3.1%, 1.8% and 2.8%, respectively. If patients were discharged early from the ICU during the study period, serum creatinine levels were collected manually from the hospital data system for 48 hours provided the patient was still hospitalised. Reference standard AKI was determined daily, up to 8 days, by a three-physician adjudication panel using the KDIGO guidelines. AKI was confirmed by the adjudication panel, and disagreements were settled by conference calls and discussion of the cases. Adjudicators were blinded to the NGAL values and physician notes concerning the renal diagnosis.6

Patient involvement

Patient involvement began at the time of informed consent; patients and caregivers were not involved in the design of, recruitment for, or conduct of this study. The development of the outcome measures was not informed by patients’ priorities, experiences or preferences. The participants’ contributions to this study are deeply appreciated, and the results of the study will be publicly available to the patients via this publication.

Analyses

This manuscript was constructed via retrospective analyses of prospectively collected data and was developed under STAndards for the Reporting of Diagnostic accuracy studies guidelines; ethics committee approval was obtained prior to conducting these analyses. NGAL values were highly skewed in all three sample types so we used the natural-log transformed distributions (figure 1A–C). We performed logistic regression using the natural log-transformed baseline NGAL (i.e. ln(baseline NGAL)) value as a predictor for the outcome of moderate/severe (stage 2/3) AKI at any point during observation. Additionally, we defined persistent moderate/severe (stage 2/3) AKI as two or more days of consecutive stage 2/3 AKI, and we constructed another logistic regression model for the outcome of persistent 2/3 AKI using a wandering ln(baseline NGAL) value as the predictor variable. The wandering ln(baseline NGAL) value was taken on the morning of the first day of a 48-hour persistent AKI period. If AKI did not occur, the wandering ln(baseline NGAL) value was taken to be the ln(baseline NGAL). We found the optimal NGAL value for both outcomes by using the criterion of smallest distance to the ‘perfect point’ on the receiver operating characteristic (ROC) curve, (0,1). Baseline NGAL values <5 ng/mL were adjusted using a regression model, which affected approximately 2% of the data. We assessed differences in baseline characteristics between those who had at least 1 day of stage 2/3 AKI at any time during observation and those who did not via Wilcoxon rank sum and χ2 tests; we did the same for those who had persistent stage 2/3 AKI and those who did not have persistent stage 2/3 AKI at any point during observation. Continuous variables are reported as medians (quarter 1, quarter 3). Categorical variables are reported as frequencies and proportions.

Figure 1

Mean and standard error of natural log-transformed NGAL by day and acute kidney injury status ((A) EDTA plasma; (B) heparin plasma; (C) urine). AKI, acute kidney injury; EDTA, ethylenediaminetetraacetic acid; NGAL, neutrophil gelatinase-associated lipocalin.

Results

A total of 252 patients were screened for the study; two were screen failures, and five did not have sufficient information to be included in analyses. Hence, 245 patients were enrolled and included in analyses. Only two patients in this study had cardiac surgery during the observational period. No adverse events occurred during the observation period. There were 33 (13.5%) subjects who developed stage 2/3 AKI as determined by the adjudicators. With this event rate, we had 95% power to detect an area under the curve (AUC) as low as 0.69. Age and gender did not differ significantly between those with or without stage 2/3 AKI; however, those with stage 2/3 AKI had more comorbidities (sepsis: 18.2% vs 5.7%, p=0.01; diabetes 48.5% vs 27.8%, p=0.04; chronic kidney disease (CKD): 33.3% vs 10.4%, p<0.001), longer ICU stays (4 (2, 8) days vs 2 (1, 3) days, p=0.001) and higher baseline NGAL levels (EDTA plasma: 276 (155, 517) ng/mL vs 98 (70, 154) ng/mL, p<0.001; heparin plasma: 288 (150, 669) ng/mL vs 100 (68, 160) ng/mL, p<0.001; urine: 190 (61, 1133) ng/mL vs 31 (15, 67), p<0.001) than those without AKI (table 1). Using the ln(baseline EDTA plasma NGAL) as the independent variable in a logistic regression model for the outcome of stage 2/3 AKI at any time yielded an AUC=0.76, 95% CI 0.64 to 0.87, p<0.0001 (figure 2A). Optimising the ROC curve with respect to the distance from (0,1) resulted in a cut point of 142.0 ng/mL, which yielded a sensitivity (SN) of 78.8%, specificity (SP) of 73.0%, positive predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7% and diagnostic accuracy (DA) of 73.8%. Similar results are shown for heparin plasma and urine NGAL in figure 2B and C.

Figure 2

Receiver operating characteristic curves for natural log-transformed baseline NGAL as a predictor of stage 2/3 acute kidney injury with the optimal NGAL threshold identified ((A) EDTA plasma; (B) heparin plasma; (C) urine). Decision statistics are presented for the optimal cut point (bold) as well as at cut points approximately 20 ng/mL below and above the optimal value. LR+ indicates positive likelihood ratio, LR− indicates negative likelihood ratio, and DA indicates diagnostic accuracy. AUC, area under the curve; EDTA, ethylenediaminetetraacetic acid; NGAL, neutrophil gelatinase-associated lipocalin. 

Table 1

Patient characteristics by occurrence of moderate/severe acute kidney injury (AKI)

Twenty-five (10.2%) subjects had persistent stage 2/3 AKI. Demographics did not differ significantly between those who did and those who did not have persistent stage 2/3 AKI; however, those with persistent stage 2/3 AKI had a higher rate of CKD (36.0% vs 10.9%, p<0.001), longer ICU stays (6 (2, 8) days vs 2 (1, 3) days, p<0.001) and higher wandering baseline NGAL values (EDTA plasma: 287 (188, 517) ng/mL vs 98 (70, 156) ng/mL, p<0.001; heparin: 355 (178, 716) ng/mL vs 101 (68, 162) ng/mL, p<0.001; urine: 231.1 (84.3, 1203) ng/mL vs 32 (15, 75) ng/mL, p<0.001) (table 2). We used the wandering ln(baseline EDTA plasma NGAL) in a logistic regression model to predict persistent stage 2/3 AKI, which yielded an AUC=0.85, 95% CI 0.77 to 0.94, p<0.001. The optimal cut point for NGAL based on the distance to (0, 1) was 148.3 ng/mL (figure 3A). This value yielded SN=84.0%, SP=73.5%, PPV=26.6%, NPV=97.6%, DA=74.6%. Similar cut points and results are shown for heparin plasma and urine NGAL in figure 3B and C. Odds ratios from the regression models are shown in table 3.

Figure 3

Receiver operating characteristic curves for wandering natural log-transformed baseline NGAL as a predictor of persistent stage 2/3 acute kidney injury with the optimal NGAL threshold identified ((A) EDTA plasma; (B) heparin plasma; (C) urine). Decision statistics are presented for the optimal cut point (bold) as well as at cut points approximately 20 ng/mL below and above the optimal value. LR+ indicates positive likelihood ratio,LR− indicates negative likelihood ratio, and DA indicates diagnostic accuracy. AUC, area under the curve; EDTA, ethylenediaminetetraacetic acid; NGAL, neutrophil gelatinase-associated lipocalin.

Table 2

Sample characteristics for subjects with and without persistent stage 2/3 AKI

Table 3

ORs for stage 2/3 acute kidney injury and persistent stage 2/3 acute kidney injury using EDTA, heparin and urine NGAL values as predictors (odds are associated with an approximate 2.73-fold increase in NGAL value)

Discussion

We found that a single measured threshold for NGAL, either in blood or urine, was effective in risk prediction for the development of moderate to severe AKI and persistent AKI, as defined by AKI lasting 48 consecutive hours or longer. NGAL measured on a daily basis performed well in the prognostication of persistent AKI. Our results were internally consistent and add to the literature as they yielded unbiased cut points from prospectively collected samples and adjudicated AKI outcomes. We found that urine and plasma yielded different absolute NGAL concentrations; however, within each sample type the cut point for the prediction of stage 2/3 AKI or persistent AKI was similar, making daily measurements and utilisation of a single threshold feasible, provided there is consistency in sample source and handling procedures. Our results are concordant with a prior study that observed slightly better discriminatory power of plasma NGAL than urinary NGAL, with an AUC as high as 0.84. 7 Thus, we believe, for ease of use, plasma NGAL is more likely to be incorporated into routine care than urinary NGAL.

Our findings build on the work of others using a variety of NGAL assays demonstrating the concept that 25 kilodalton protein is rapidly and reliably upregulated after ischaemia/reperfusion injury to the kidney.8   9 4 The work by Nickolas et al revealed that NGAL even has the ability to distinguish AKI from non-progressive CKD, a feat that creatinine has failed to accomplish.10 Furthermore, the urine NGAL measured in the emergency department was associated with the development of end-stage renal disease requiring dialysis in the hospital. A 2010–2011 study in Western Australia, n=102, demonstrated the effectiveness of using NGAL to predict inpatient AKI (based on risk, injury, failure, loss of kidney function and end-stage kidney disease classification) in a tertiary care setting. In the Australian study, the Biosite Triage NGAL in whole blood (Alere, Inverness Medical, Australia) was found to have a single optimal cut-off of 89 ng/mL and had SN=68%, SP=70% and AUC=0.71, 95% CI 0.58 to 0.84.11 While the threshold was similar to that found in our EDTA and heparin samples, this study used all stages of AKI as the endpoint and did not have blinded adjudication by a committee. Nevertheless, these data are in line with our conclusion that NGAL measured in blood performs well as a prognostic aid for AKI. A meta-analysis of 1478 patients with sepsis demonstrated that NGAL was associated with the development of AKI, need for renal replacement therapy and mortality.12 Marino et al’s study of 101 emergency department patients with sepsis confirmed that NGAL was related to (the severity of) AKI; however, NGAL concentrations were also elevated in the absence of AKI in these patients.13 Numerous studies have demonstrated that NGAL rises after cardiac surgery and is congruent with AKI.14   15 Additionally, NGAL concentrations increase after contrast-induced AKI; however, it is to a much lesser extent than from cardiac surgery or critical illness.16 An extensive literature review and meta-analysis by Haase et al revealed a wide range of optimal reported NGAL cutoffs for AKI prediction (100–270 ng/mL).17 The optimal plasma cutoffs found in our study to detect stage 2/3 AKI (EDTA: 142.0 ng/mL, heparin: 149.5 ng/mL) were within the range reported in the meta-analysis. Haase et al suggested cutoffs between 150 ng/mL and 170 ng/mL for adults, which is slightly higher than our findings and may be attributable to the high rate of postoperative patients used in the meta-analysis, as well as the varied definitions of AKI. Our study is the first among these to demonstrate that NGAL can be used to risk predict stage 2/3 AKI and its persistence in a cohort of ICU patients. We further demonstrated that a single cut-off value performs well in terms of decision statistics for the outcomes of stage 2/3 AKI and persistent stage 2/3 AKI.

There have been large prospective studies examining novel biomarkers other than NGAL for the prediction of stage 2/3 AKI among the critically ill. Kashani and colleagues (n=728) measured the concentrations of urinary tissue inhibitor of metalloproteinases (TIMP)-2 and insulin-like growth factor binding protein 7 (IGFBP7), and combined the two measures into one predictor, referred to as (TIMP-2)*(IGFBP7), NephroCheck (Astute Medical, San Diego, California, USA) to identify imminent stage 2/3 AKI (ie, in the next 12 hours), which occurred in 14% of study participants. Baseline urinary (TIMP-2)*(IGFBP7) yielded an AUC=0.80, 97% CI 0.76 to 0.83; however, the full decision statistics of an optimal cut point were not disclosed. 18 A second study by Bihorac and colleagues (n=420) evaluated urinary (TIMP-2)*(IGFBP7) for the same endpoint. In this study, 71 (17.4%) patients had imminent stage 2/3 AKI and the model had an AUC=0.82, 95% CI 0.76 to 0.88. However, (TIMP-2)*(IGFBP7) did not appear amenable to a single threshold value. Two cut points were presented: one yielding high SN and low SP and the other yielding low SN and high SP. At the cut-off of 0.3 (ng/mL)2/1000, the SN=92, 95% CI 85% to 98%, but the SP=46, 95% CI 41% to 52%. At a sevenfold higher cut point of 2.0 (ng/mL)2/1000, the SN=37, 95% CI 26% to 47% and SP=95, 95% CI 93% to 97%.19 In comparison, an NGAL EDTA plasma level of 142.0 ng/mL in our study predicted stage 2/3 AKI and yielded SN=78.8, 95% CI 73.7% to 83.9% and SP=73.0, 95% CI 67.4% to 78.6%. Thus, for NGAL at a single cut point, the lower bounds of the 95% CIs for sensitivity and specificity were well above 60% and 40%, respectively. Accordingly, NGAL appears to have considerable advantages over (TIMP-2)*(IGFBP7) in forecasting AKI over a longer time window, at a single prognostic threshold, and avoids low sensitivity or specificity. Additionally, we demonstrated NGAL performed well when measured daily using similar cut points for the prediction of persistent stage 2/3 AKI. Daily use in risk prediction has not been demonstrated in any study with (TIMP2)*(IGFBP7).

Limitations

Our study has all the limitations of small prospective cohort studies of biomarkers including limited clinical follow-up and lack of precise follow-up information, including accurate urine output. We did not consider stage I AKI in the definition of the primary endpoint because prior studies suggested that many in this category could have transient haemodynamic elevations of creatinine without parenchymal AKI. Thus, our event rate is lower than other studies that included all KDIGO stages of AKI. Our study sample was non-homogeneous in that it included patients with sepsis, a diagnosis that has been shown to increase NGAL levels even for those without AKI.13 Furthermore, due to the small sample size, we could not analyse sepsis or CKD subjects separately, rendering it impossible to determine whether different cutoffs were needed. To examine confounders’ relations with (persistent) stage 2/3 AKI, we built multivariate models; however, neither sepsis nor CKD and DM were statistically significant when considered with NGAL simultaneously. Consequently, the adjusted OR estimates changed only slightly, and the corresponding p values were unchanged. Perhaps this is due, in part, to the fact that NGAL can distinguish AKI from non-progressive CKD. Additionally, the small sample size did not permit the use of training and/or validation sets, which prohibited further investigation of the models. However, all AUCs were at least 0.76, which indicates good predictive ability. Finally, we did not have information on long-term outcomes such as need for renal replacement therapy or mortality, which would have been useful in integrating NGAL with serum creatinine and urine output in an in-depth assessment of patient outcomes.20

Conclusion

NGAL concentrations of 142.0, 149.5 and 78.0 ng/mL in EDTA plasma, heparin plasma and urine, respectively, were found to be optimal in predicting stage 2/3 AKI in ICU patients participating in a prospective observational study. Similarly, NGAL concentration thresholds of 148.3, 169.6 and 79.2 ng/mL in EDTA plasma, heparin plasma and urine, respectively, were optimal in predicting persistent stage 2/3 AKI in the same cohort of ICU patients. A larger prospective study to provide external validation of these cut points is warranted.

Acknowledgments

The authors would like to thank the study participants who graciously shared their time and materials for the purposes of this research.

Acknowledgments

The authors would like to thank the study participants who graciously shared their time and materials for the purposes of this research.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
View Abstract

Footnotes

  • Contributors PAM conceived the study. PME funded the study. AOG, MG, LG and LWM initiated the study design and helped with implementation. KMT conducted the statistical analysis. KMT and PAM drafted the article. All authors contributed to the manuscript's refinement and approved the final version.

  • Funding The study was funded by BioPorto Diagnostics A/S.

  • Competing interests EE, PME, and ML are employees of BioPorto Diagnostics A/S. The remaining authors have nothing to disclose.

  • Patient consent Obtained.

  • Ethics approval Institutional Review Board (Baystate Health IRB #1; Houston Methodist Research Institute (HMRI) IRB 1; Pratners Human Research Committee; Biomedical Research Alliance of New York (BRANY) IRB).

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement The data from this study will not be made available.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.