Article Text

Download PDFPDF

Original research
Early prediction of in-hospital mortality in patients with congestive heart failure in intensive care unit: a retrospective observational cohort study
  1. Didi Han1,2,
  2. Fengshuo Xu1,2,
  3. Luming Zhang1,
  4. Rui Yang1,2,
  5. Shuai Zheng1,3,
  6. Tao Huang4,
  7. Haiyan Yin1,
  8. Jun Lyu4,5
  1. 1Intensive Care Unit, Jinan University First Affiliated Hospital, Guangzhou, China
  2. 2School of Public Health, Xi'an Jiaotong University, Xi'an, People's Republic of China
  3. 3School of Public Health, Shannxi University of Chinese Medicine, Xianyang, People's Republic of China
  4. 4Department of Clinical Research, The First Affiliated Hospital of Jinan University, Guangzhou, People's Republic of China
  5. 5Guangdong Provincial Key Laboratory of Traditional Chinese Medicine Informatization, Jinan University, Guangzhou, People's Republic of China
  1. Correspondence to Dr Jun Lyu; lyujun2020{at}jnu.edu.cn

Abstract

Objective Congestive heart failure (CHF) is a clinical syndrome in which the heart disease progresses to a severe stage. Early diagnosis and risk assessment of death of patients with CHF are critical to prognosis and treatment. The purpose of this study was to establish a nomogram that predicts the in-hospital death of patients with CHF in the intensive care unit (ICU).

Design A retrospective observational cohort study.

Setting and participants Data for the study were from 30 411 patients with CHF in the Medical Information Mart for Intensive Care database and the eICU Collaborative Research Database (eICU-CRD).

Primary outcome In-hospital mortality.

Methods Univariate logistic regression analysis was used to select risk factors associated with in-hospital mortality of patients with CHF, and multivariate logistic regression was used to build the prediction model. Discrimination, calibration and clinical validity of the model were evaluated by AUC, calibration curve, Hosmer-Lemeshow χ2 test and decision curve analysis, respectively. Finally, data from 15 503 patients with CHF in the multicentre eICU-CRD were used for external validation of the established nomogram.

Results The inclusion criteria were met by 15 983 subjects, whose in-hospital mortality rate was 12.4%. Multivariate analysis determined that the independent risk factors were age, race, norepinephrine, dopamine, phenylephrine, vasopressin, mechanical ventilation, intubation, hepatic failure (HepF), heart rate, respiratory rate, temperature, systolic blood pressure (SBP), anion gap (AG), blood urea nitrogen (BUN), creatinine, chloride, mean corpuscular volume (MCV), red blood cell distribution width (RDW) and white cell count (WCC). The C-index of the nomogram (0.767, 95% CI 0.759 to 0.779) was superior to that of the traditional Sequential Organ Failure Assessment, Acute Physiology Score III and Get With The Guidelines Heart Failure scores, indicating its discrimination power. Calibration plots demonstrated that the predicted results are in good agreement with the observed results. The decision curves of the derivation and validation sets both had net benefits.

Conclusion The 20 independent risk factors for in-hospital mortality of patients with CHF were age, race, norepinephrine, dopamine, phenylephrine, vasopressin, mechanical ventilation, intubation, HepF, heart rate, respiratory rate, temperature, SBP, AG, BUN, creatinine, chloride, MCV, RDW and WCC. The nomogram, which included these factors, accurately predicted the in-hospital mortality of patients with CHF. The novel nomogram has the potential for use in clinical practice as a tool to predict and assess mortality of patients with CHF in the ICU.

  • Adult intensive & critical care
  • Adult cardiology
  • Congenital heart disease
  • Heart failure

Data availability statement

Data are available in a public, open access repository. Publicly available data sets were analysed in this study and these data can be found at https://mimic-iv.mit.edu/ and https://eicu-crd.mit.edu/. Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi: 10.5061/dryad.tx95x6b18.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Logistic regression analysis was used to select variables associated with in-hospital mortality of patients with congestive heart failure (CHF).

  • The nomogram to predict mortality of patients with CHF in the intensive care unit was built based on the included risk factors.

  • The application of the nomogram can provide clinical decision-making opinions for clinicians to improve the prognosis of inpatients with CHF.

  • Discrimination, calibration and clinical validity of the nomogram were evaluated by AUC, calibration curve, Hosmer-Lemeshow χ2 test and decision curve analysis, respectively.

Introduction

Congestive heart failure (CHF) is a severe clinical syndrome in which the cardiac structure and/or function are abnormal. Inadequate pumping or filling capacity of the ventricles, inability of the cardiac blood output to meet the body’s metabolic demands, hypoperfusion of tissues and organs, and concomitant pulmonary and systemic congestion are common causes of CHF. Increased internal pressure causes typical symptoms such as dyspnoea and fatigue, and may also be accompanied by typical signs (eg, jugular vein distention, pulmonary rales and peripheral oedema).1 Heart failure can be classified into left heart failure, right heart failure or whole heart failure depending on its location. In addition, according to severity, heart failure can be divided into acute heart failure and chronic heart failure.2

CHF is not a separate disease but a complicated phenomenon that occurs after various other heart diseases develop to a more serious level, with relatively high morbidity and mortality. Patients are usually admitted to the intensive care unit (ICU) for treatment. In the USA, approximately 10%–15% of hospitalised patients with heart failure are admitted to the ICU.3 4 Despite major breakthroughs in the diagnosis and treatment in recent years, mortality in heart failure remains high and the 5-year survival rate is similar to that of malignant tumours.5

There have been many predictive models for the prognosis of patients with CHF, but due to various reasons these models have not been widely used in clinical practice.6–8 The Get With The Guidelines Heart Failure (GWTGHF) risk score, previously proposed by Peterson et al,9 used commonly used clinical variables to predict in-hospital mortality and provided clinicians with a valid risk stratification tool, but was proposed for a non-contemporary cohort,10 which may have limited its applicability to current practice. Traditional critical care scoring systems for patients in the ICU, including the Sequential Organ Failure Assessment (SOFA) and the Acute Physiology Score III (APSIII), are often not as accurate when applied to patients with CHF.

Due to the insufficiency of existing predictive models, a more simple, efficient and accurate clinical application tool is needed to predict mortality of hospitalised patients with CHF. Therefore, we aimed to establish a more authoritative, efficient and easy-to-follow in-hospital mortality prediction model for patients with CHF based on the data of patients with CHF in a large critical illness database (Medical Information Mart for Intensive Care, MIMIC-IV). This will provide some reference for medical staff to improve the poor prognosis of patients with CHF in the process of clinical practice (in the process of clinical practice, it can provide certain guiding value for medical staff to improve the poor prognosis of CHF patients).

Methods

Data source

MIMIC-IV is a single-centre, free-access database that contains data on more than 40 000 ICU patients covering the period from 2008 to 2019.11 Another intensive care database, the eICU Collaborative Research Database (eICU-CRD), was used to validate the findings. The eICU-CRD is a database of multiple, longitudinal, multicentre retrospective cohort studies of patients in 335 ICUs in the USA from 2014 to 2015.12 It includes demographic records, physiological indicators for bedside monitoring, diagnoses based on the International Classification of Disease-9 (ICD-9)code and other laboratory data.

Study population

In the MIMIC-IV database, the ICD-9 code for screening CHF is 4280. Similarly, the ICD-9 code for CHF was used to screen patients from the eICU-CRD. For patients with multiple ICU admissions, only data from the first admission were considered. Exclusion criteria were age <18 years and missing outcome data. After applying these criteria, the study included 15 983 and 14 428 patients with CHF in the derivation cohort and in the validation cohort, respectively. The selection process is demonstrated in figure 1.

Figure 1

Flow chart of patient selection. CHF, Congestive heart failure; eICU-CRD, eICU Collaborative Research Database; MIMIC-IV, Medical Information Mart for Intensive Care.

Data retrieval and outcomes

Based on previous studies, the demographic characteristics included age, race, sex, severity score (SOFA and APSIII scores), comorbidities (eg, chronic obstructive pulmonary disease (COPD), diabetes, hepatic failure (HepF) and acute myocardial infarction (AMI)), drug information (norepinephrine, dopamine, epinephrine, phenylephrine and vasopressin), mechanical ventilation and intubation, and vital signs first recorded on ICU admission (heart rate, respiratory rate, systolic blood pressure (SBP) and temperature). Laboratory test indexes on admission were also collected, including anion gap (AG), chloride, blood urea nitrogen (BUN), calcium, creatinine, potassium, sodium, mean corpuscular haemoglobin (MCH), mean corpuscular haemoglobin concentration (MCHC), mean corpuscular volume (MCV), platelet, haemoglobin, red blood cell distribution width (RDW), red blood cell (RBC) and white cell count (WCC). Demographic characteristics and vital signs in the first 24 hours after admission were recorded, and the results of the first measurement on admission were selected as laboratory examination indicators. To accurately compare data from the eICU-CRD for external validation, we attempted to extract the same variables as those from the MIMIC-IV.

The primary outcome was in-hospital mortality, which was defined based on survival status at discharge.

Statistical analyses

Continuous variables are expressed by mean±SD or median (IQR), and analysis was performed using Student’s t-test or Mann-Whitney U test. Categorical variables are presented as numbers (percentages), and comparisons were performed using χ2 test or Fisher’s exact test.

Univariate logistic regression analysis showed that all significant risk factors were entered into the multivariate logistic regression analysis. The nomogram was constructed based on the results of the multivariate logistic regression analysis. Calibration plots and the Hosmer-Lemeshow χ2 test are usually used to assess the degree of calibration of the model.13 The discriminative ability of a model is usually evaluated using the C-index or the Receiver Operating Characteristic (ROC) curve.14 Decision curve analysis (DCA) assessed the clinical usefulness of the model.15 In addition, the prediction accuracy of the model was assessed by computing the net reclassification improvement (NRI) and the integrated discriminant improvement (IDI).16

SPSS software (V.21.0) and R software (V.4.0.1) were used for statistical analysis. P<0.05 was considered statistically significant.

Patient and public involvement

Patients and the public were not directly involved in this study.

Results

Baseline characteristics

We selected 15 983 patients with CHF based on the MIMIC-IV. For nomogram construction and external validation, 14 428 patients with CHF were selected from the multicentre eICU-CRD. The following characteristics were, respectively, recorded in the derivation and validation sets: median age of patients (74 years and 72 years), male (54.6.0% and 52.4%), white (69.2% and 74.9%), and had comorbidities of COPD (60.5% and 20.9%), AMI (31.6% and 1.7%), diabetes (44.2% and 31.4%) and HepF (2.7% and 1.0%). The proportion of patients who received mechanical ventilation and intubation in the derivation and validation sets was 85.8% and 7.5%, respectively. Overall in-hospital mortality was 12.4% and 12.8% in the MIMIC-IV and eICU-CRD, respectively. The baseline data of the derivation and validation sets were similar. The baseline characteristics of the patients are shown in table 1.

Table 1

Baseline characteristics of patients with CHF on admission

Prognostic factors for in-hospital mortality

Prognostic factors significantly related to in-hospital mortality of patients with CHF were age, race, norepinephrine, dopamine, epinephrine, phenylephrine, vasopressin, mechanical ventilation, intubation, AMI, HepF, heart rate, respiratory rate, temperature, SBP, AG, BUN, creatinine, calcium, chloride, haemoglobin, potassium, MCHC, MCV, RDW, RBC and WCC. Multivariate logistic regression analysis was then performed on these variables to control for confounding effects as much as possible. Finally, age, race, norepinephrine, dopamine, phenylephrine, vasopressin, mechanical ventilation, intubation, HepF, heart rate, respiratory rate, temperature, SBP, AG, BUN, creatinine, chloride, MCV, RDW and WCC were determined as the prognostic factors for patients with CHF. Detailed data are shown in table 2.

Table 2

Logistic regression analysis for patients with CHF

A nomogram for predicting in-hospital mortality

A novel nomogram (figure 2) was developed using multivariate logistic regression analysis. Higher total scores for all risk factors indicated higher risk of in-hospital mortality. The nomogram indicates that age is the factor with the most significant effect on prognosis, followed by respiratory rate, RDW, temperature, AG, BUN, WCC, SBP, MCV, heart rate, creatinine, vasopressin, chloride, dopamine, HepF, norepinephrine, intubation, phenylephrine, mechanical ventilation and race. SOFA, APSIII and GWTGHF scores were compared with our novel nomogram.

Figure 2

Nomogram for predicting in-hospital mortality in patients with congestive heart failure. AG, anion gap; BUN, blood urea nitrogen; HepF, hepatic failure; MCV, mean corpuscular volume; RDW, red blood cell distribution width; SBP, systolic blood pressure; WCC, white cell count.

Performance of the nomogram

The C-index of the nomogram was 0.839 (95% CI 0.829 to 0.848), indicating good discrimination ability. The ROC curves of the SOFA, APSIII and GWTGHF scores and the novel nomogram used to predict in-hospital mortality are presented in figure 3. The Area Under Curve (AUC) values indicated that our novel nomogram model is better than the SOFA, APSIII and GWTGHF score models.

Figure 3

ROC curves of the SOFA, APSIII, GWTGHF scores and the nomogram in the derivation set (A) and the validation set (B). APSIII, Acute Physiology Score III; AUC, Area Under Curve; GWTGHF, Get With The Guidelines Heart Failure; ROC, Receiver Operating Characteristic; SOFA, Sequential Organ Failure Assessment.

The Hosmer-Lemeshow χ2 value of the novel nomogram was 6.6 (p=0.677), showing no significant difference. The calibration curves showed good consistency between the predicted and actual probability (figure 4). The net benefit from the DCA curve of the novel nomogram was greater than the SOFA, APSIII and GWTGHF scores in the derivation set (figure 5).

Figure 4

Calibration curves of the SOFA, APSIII, GWTGHF scores and the nomogram in the derivation set (A–D) and the validation set (E–H). APSIII, Acute Physiology Score III; GWTGHF, Get With The Guidelines Heart Failure; SOFA, Sequential Organ Failure Assessment.

Figure 5

Decision curve analysis of the SOFA, APSIII, GWTGHF scores and the nomogram in the derivation set (A) and the validation set (B). APSIII, Acute Physiology Score III; GWTGHF, Get With The Guidelines Heart Failure; SOFA, Sequential Organ Failure Assessment.

The novel nomogram model had an NRI and IDI of 0.601 (95% CI 0.522 to 0.651) and 0.176 (95% CI 0.165 to 0.187) in the derivation set (table 3). All values indicate that the nomogram’s prediction performance represented considerable improvement.

Table 3

Validation of the nomogram

External validation of the nomogram

This study also identified 14 428 eligible patients with CHF in the eICU-CRD. Applying the prediction nomogram to this validation set indicated similar prediction effects, with a C-index of 0.767 (95% CI 0.756 to 0.779). The ROC curves for predicting in-hospital mortality are presented in figure 3. The AUC values indicated that our novel nomogram model has better predictive ability than the SOFA, APSIII and GWTGHF score models.

The Hosmer-Lemeshow χ2 value of the novel nomogram was 12.1 (p=0.210), showing no significant difference. The calibration curves indicate that it is well calibrated for clinical application in the validation set (figure 4). Through data verification of eICU-CRD, the DCA also shows the high clinical application value of the nomogram (figure 5). The NRI and IDI of the validation set are presented in table 3.

Discussion

CHF is the final manifestation of various cardiovascular diseases. The structure or function of the myocardium becomes dysfunctional, leading to restricted ventricular ejection or filling. With acute exacerbation or progression of the disease, insufficient tissue and organ perfusion results and ultimately leads to death.17 The disease has a relatively rapid onset and affected patients are in a relatively serious condition. Although some scoring systems and models for prognostic research or risk factor analysis of ICU patients are available, there is none specifically for the prognosis of patients with CHF. We therefore planned to establish a more comprehensive and accurate prognostic nomogram for patients with CHF in the ICU.

A nomogram is an effective tool for risk assessment.18 19 It provides a simple graphical representation for complex statistical prediction models, and is also suitable for prognostic analysis of individual patients.20 The present multivariate logistic regression indicated that age, race, norepinephrine, dopamine, phenylephrine, vasopressin, mechanical ventilation, intubation, HepF, heart rate, respiratory rate, temperature, SBP, AG, BUN, creatinine, chloride, MCV, RDW and WCC are prognostic factors for survival of patients with CHF. The main advantages of the novel nomogram are its simplicity, efficiency and accuracy of application. The nomogram uses 20 clinical variables that are easy to obtain and do not require complex calculations, making it easy for clinicians to use.

Glasgow Coma Scale (GCS) and APSIII scores are commonly used severity scores for ICU patients, and patients with CHF in the ICU tend to be more severe. The GCS score is used to determine the state of consciousness of patients with CHF,21 and the SOFA and APSIII scores determine the physiological state of patients in the ICU.22 The GWTGHF risk score is a well-validated and widely accepted heart failure scoring system that is commonly used to risk-stratify in-hospital mortality.23–25 This predictive model was validated in our research group and compared with the model we developed. The performance of our nomogram is superior to other scores.

Several variables have been indicated to be associated with mortality in patients with heart failure, such as age, gender, body mass index, diabetes, COPD, low SBP and creatinine.26 The study showed that heart rate, respiratory rate, temperature, SBP, AG, BUN, creatinine, chloride, MCV, RDW and WCC were predictors of in-hospital mortality.

The number of elderly patients admitted to the ICU has increased rapidly in recent years. The median age of ICU patients in many Western countries has increased to over 65 years.27 Age is often considered an important indicator of prognosis of diseases. Many recognised prediction models have used age to predict the severity or mortality of CHF.28 29 Similarly, our results also indicate that age is a risk factor for patients with CHF, which may be due to older patients having more comorbidities and poor response to treatment. In addition, our finding that non-African American ethnicity has an increased CHF mortality risk is consistent with previous research.30

Clinically, patients with CHF are treated with cardiac strengthening, diuresis, oxygen and other treatment measures. Although these treatment methods can achieve certain effects, for patients with relatively rapid disease development, ventilation function cannot be improved and ultimately the treatment effect will be reduced. Our study shows that use of catecholamines is associated with an increased risk of poor outcome in patients with CHF, which is similar to previous studies.31–33 In addition, our study also demonstrated mechanical ventilation and intubation increased the risk of in-hospital death of patients with CHF. This is consistent with previous research.10 The reason may be that early mechanical ventilation has a significant inhibitory effect on cardiac function and may induce AMI or further enlarge the infarction area, resulting in poor prognosis.

Previous studies have also found high respiratory rate to be related to poor heart failure prognosis.4 Our results similarly indicated that increased respiratory rate was a risk factor for increased in-hospital mortality. Changes in body temperature and WCC reflect the body’s inflammatory status, and values that are abnormal may be risk factors for poor prognoses. Our study indicated that lower body temperature, elevated WCC, higher AG and elevated MCV increased the risk of mortality of patients with CHF, which is consistent with the findings of previous studies.34 35 RDW is a parameter that reflects the heterogeneity of RBC volume and is expressed by the coefficient of variation of RBC volume.36 Results from a previous study showed that higher RDW was an independent predictor of poor prognosis,37 which is similar to our findings.

An ideal clinical prognostic model should be reliable and easy to use. For example, some clinically accessible, reliable and non-invasive clinical and laboratory indicators are usually used to construct prognostic models. Moreover, the performance of novel nomogram was superior to GWTGHF and other severity scores.

Most previous studies on the risk and prognostic factors for CHF have been limited by their relatively small samples.38 This study is therefore the first that we know of to have simultaneously analysed patients with CHF in MIMIC-IV and eICU-CRD and to obtain a large sample to develop a prognostic nomogram. This retrospective clinical study involved multiple research centres, and the results provide significant reference information to guide clinical practice.

Inevitably, this study also had some limitations. First, the retrospective research design may reduce the validity of our findings, which could be validated in future prospective case–control studies. Second, although as many confounders as possible were adjusted for some covariates, residual confounders could not be completely removed and some key comorbidity data were missing from the data set, and the relationships between certain variables (ie, temperature and WCC) and the outcome were treated as linear when they were likely to be bimodal. Third, MIMIC-IV and eICU-CRD contain data from 2008 to 2019 and from 2014 to 2015, respectively. The sample is old and some characteristics of the disease may have changed over time, which may affect our findings. Finally, due to the limitations of public databases, many other variables that may affect the model and its results, such as smoking and alcohol consumption, and detailed information on patient treatment and medication were not collected.

Conclusion

We have determined that the important risk factors for in-hospital mortality of patients with CHF are age, race, norepinephrine, dopamine, phenylephrine, vasopressin, mechanical ventilation, intubation, HepF, heart rate, respiratory rate, temperature, SBP, AG, BUN, creatinine, chloride, MCV, RDW and WCC. We developed and validated a prognostic nomogram for CHF that is highly accurate. The nomogram constructed in this study provides a method and choice for evaluation of the survival prognosis of patients with CHF in the ICU, and may become an auxiliary predictive tool for use in clinical practice to predict and evaluate patients and understand their in-hospital survival status.

Data availability statement

Data are available in a public, open access repository. Publicly available data sets were analysed in this study and these data can be found at https://mimic-iv.mit.edu/ and https://eicu-crd.mit.edu/. Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi: 10.5061/dryad.tx95x6b18.

Ethics statements

Patient consent for publication

Ethics approval

The clinical data used to support the findings of this study were provided by the MIMIC- IV and the eICU database. Although the database is publicly and freely available, researchers must complete the National Institutes of Health’s web-based course known as Protecting Human Research Participants to apply for permission to access the database. The establishment of the MIMIC-IV database was approved by the Massachusetts Institute of Technology (Cambridge, Massachusetts) and Beth Israel Deaconess Medical Center (Boston, Massachusetts). eICU was made available by Philips Healthcare in partnership with the MIT Laboratory for Computational Physiology. Consent was obtained for the original data collection. Therefore, ethical approval and the need for informed consent were waived for the studies on this database. The study was an analysis of two third-party anonymised publicly available databases with pre-existing institutional review board (IRB) approval. Informed consent was unnecessary because the MIMIC-IV and eICU data are anonymous and publicly available.

Acknowledgments

We thank the staff members of the collaborative research team at the Laboratory for Computational Physiology, Massachusetts Institute of Technology.

References

Footnotes

  • Contributors DH and FX conceptualised the research aims, planned the analyses and guided the literature review. LZ and RY extracted the data from the MIMIC-IV database. DH, SZ and TH participated in data analysis and interpretation. DH and RY wrote the first draft of the paper and the other authors provided comments and approved the final manuscript. HY and JL participated in project administration, supervision and visualisation. JL was regarded as the guarantor. The interpretation and reporting of these data were the sole responsibility of the authors.

  • Funding The study was supported by the Guangdong Provincial Key Laboratory of Traditional Chinese Medicine Informatization (2021B1212040007).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.