Article Text
PDF
Abstract
Objectives We aimed to develop and validate a postoperative delirium (POD) prediction model for patients admitted to the intensive care unit (ICU).
Design A prospective study was conducted.
Setting The study was conducted in the surgical, cardiovascular surgical and trauma surgical ICUs of an affiliated hospital of a medical university in Heilongjiang Province, China.
Participants This study included 400 patients (≥18 years old) admitted to the ICU after surgery.
Primary and secondary outcome measures The primary outcome measure was POD assessment during ICU stay.
Results The model was developed using 300 consecutive ICU patients and was validated using 100 patients from the same ICUs. The model was based on five risk factors: Physiological and Operative Severity Score for the enumeration of Mortality and morbidity; acid–base disturbance and history of coma, diabetes or hypertension. The model had an area under the receiver operating characteristics curve of 0.852 (95% CI 0.802 to 0.902), Youden index of 0.5789, sensitivity of 70.73% and specificity of 87.16%. The Hosmer-Lemeshow goodness of fit was 5.203 (p=0.736). At a cutoff value of 24.5%, the sensitivity and specificity were 71% and 69%, respectively.
Conclusions The model, which used readily available data, exhibited high predictive value regarding risk of ICU-POD at admission. Use of this model may facilitate better implementation of preventive treatments and nursing measures.
- postoperative
- delirium
- models, statistical
- intensive care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first publication of a new research instrument to predict patients’ risk of intensive care unit postoperative delirium.
The risk factors included in the model are available in the information systems of hospitals at all levels.
The model facilitates immediate grading assessment after intensive care unit admission and advance drug or non-drug preventive measures.
The influence of drugs was not considered when developing the prediction model.
The suitability of the model in other hospitals needs further external validation.
Introduction
Delirium and postoperative delirium in the intensive care Unit
Delirium can be defined as a syndrome characterised by the acute onset of cerebral dysfunction with a change or fluctuation in baseline mental status, inattention and either disorganised thinking or an altered level of consciousness.1 Postoperative delirium (POD) is an acute central nervous system pathology that occurs in 27.6%–87.0% of patients after surgery.1–4 A number of predisposing and accelerating risk factors, including surgery, contribute to postoperative surgery delirium.1–3 Moreover, POD in the intensive care unit (ICU-POD) increases the risk of nosocomial infection as well as the mortality rate, prolongs the length of hospital stay and incurs higher inpatient cost.5 In addition, patients above 65 years of age with POD have poorer outcomes and are more likely to have prolonged hospitalisation and ICU stay6 and longer intubation times, particularly after cardiac surgery.7
The early identification of delirium and initiation of active interventions can reduce the influence of delirium on the patient. Delirium alone may simultaneously increase awareness of the early indications of acute and serious physiological problems, resulting in a timely diagnosis of the disorder, which may improve patients’ outcomes and quality of life. Additionally, its recognition contributes to reduction in distress among patients and their families.8
Relationship between surgery and delirium
Although surgery can provide successful treatment, it also causes psychological and physical harm.2 Furthermore, surgical trauma can result in blood loss; the subsequent rehydration treatment can render the patient susceptible to electrolyte disorder, changing the patient’s internal environment. Additionally, the stress of surgery may also result in secretion of interleukins and other inflammatory mediators, which makes patients prone to delirium.9 Moreover, postoperative pain also contributes to delirium.10
Operations vary widely by surgical specialty in the associated risk for POD.2 For example, the risk for POD with partial mastectomy is low (OR 0.2, 95% CI 0.1 to 0.4); the risk for POD with repair of a recurrent, incarcerated or strangulated inguinal hernia is moderate (OR 1.0, 95% CI 0.3 to 3.8) and the risk for POD with a Whipple operation is high (OR 4.0, 95% CI 2.7 to 6.1).2
Delirium guidelines
In 2013, the American Academy of Critical Care Medicine published the Clinical Practice Guidelines for the Management of Pain, agitation and delirium in adult patients in the intensive care unit (PAD).11 This guideline considers delirium to be related to coma, chronic diseases, diabetes mellitus and other factors after operation. PAD recommended that delirium assessment should be routinely performed in all ICU patients and mobilise ICU patients early when feasible to reduce the incidence and duration of delirium.1 Use of the Confusion Assessment Method (CAM) for the ICU or the Intensive Care Delirium Screening Checklist (ICDSC) was recommended as the most valid and reliable scales to assess delirium in ICU patients1; however, this allows only timely discovery of ICU-POD and does not help prevent and reduce the occurrence of ICU-POD. Nonetheless, if risk of ICU-POD is assessed immediately after admission to the ICU and related risk factors are prevented, risk of ICU-POD can be reduced. In addition, a previous study12 indicated that the effect of active prevention on high-risk patients is significant.
POD prediction models
At present, there are three delirium risk prediction models for ICU patients. The Prediction of DELIRium in ICu patients (PRE-DELIRIC), developed in 2012, includes 10 risk factors: age, APACHE II score, metabolic acidosis, infection, morphine use, sedative use, pre-admission coma, blood urea nitrogen level, type of hospitalisation and emergency admission.13 The Early PREdiction model for DELIRium in ICU patients (E-PRE-DELIRIC), developed in 2015, includes nine predictors: age, history of cognitive impairment, history of alcoholism, blood urea nitrogen, admission type, emergency admission, mean arterial blood pressure, corticosteroid use and respiratory failure.14 However, the risk factors included in these models are the same for all patients, regardless of whether or not they have undergone surgery; therefore, these factors fail to address the specific characteristics of ICU patients after surgery.
Only one risk prediction model for postoperative ICU patients is available. Kim et al developed the DELirium Prediction based on Hospital Information (Delphi) model in 2016.15 This model is based on age, low physical activity, hearing impairment, heavy alcoholism, history of prior delirium, ICU admission, emergency surgery, open surgery and increased preoperative C-reactive protein.15 Unfortunately, the Delphi model included several risk factors based only on a literature review and thus might have omitted many important factors related to delirium. Additionally, some risk factors are based on experience rather than on evidence.16–19 Before implementing this study, we completed a meta-analysis for ICU-POD risk factors, which has been published in China.20 The risk factors included in the prediction model were based on our previous meta-analysis and delirium clinical guidelines.1 20 Thus, this study aimed to develop a risk prediction model of POD according to ICU-POD-related risks determined during our previous meta-analysis. Furthermore, we hope this study can provide further evidence to improve understanding of delirium.
Methods
Patient and public involvement statement
This study included patients from our hospital.
Study design and population
In this prospective observational study, we collected all necessary data, developed and calibrated the risk prediction model and then used the model in patients admitted to the surgical, cardiovascular surgical and trauma surgical ICU (all trauma types in different organs requiring surgical treatment) of an affiliated hospital of a medical university in Heilongjiang Province, China.
Patients who were admitted in any of these three ICUs at the hospital from April 2017 to January 2018 after surgery with complete data were enrolled. Inclusion criteria were as follows: (1) patients aged 18 years or older, (2) patients admitted in the ICU after surgery to continue treatment instantly and (3) patients recovering from anaesthesia and who were able to communicate. Exclusion criteria were as follows: (1) patients who were comatose all the time during the ICU treatment, (2) patients with previous history of mental illness or psychosis before operation, (3) patients who stayed in the ICU for less than 24 hours and (4) patients who had interventional surgery. Findings are reported according to the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) guidelines.21
Sample size calculation
The model contained nine candidate predictors: Physiological and Operative Severity Score for the enumeration of Mortality and morbidity (POSSUM) score, sex, history of coma (history of transitional conscious disturbance before admission, discovered when doctors were collecting the patient’s medical history), history of hypertension (systolic blood pressure ≥140 mm Hg, with or without diastolic pressure ≥90 mm Hg; or simple hypertension with systolic blood pressure ≥140 mm Hg only diagnosed previously before admission by clinicians and had recorded in the medical record), history of diabetes (diagnosed before admission by clinicians and recorded in the medical record), mechanical ventilation, history of alcohol dependence (based on International Statistical Classification of Diseases and Related Health problems (10th Revision) (ICD-10)22 and diagnosed by clinicians), acid–base imbalance (including metabolic acidosis, metabolic alkalosis, respiratory acidosis and respiratory alkalosis, identified by the first blood gas analysis) and length of operation. For each candidate predictor, at least 5–10 patients with delirium were required based on the sample size formula for modeling23; as the minimum incidence of ICU-POD is 27.6%,3 we used this value. With an anticipated delirium incidence of 27.6% and an expected attrition of 10%–20%, we aimed to enrol at least 1960 patients (9×5 × (1+0.2) / 0.276). To ensure the reliability of the model, the research team consulted a statistician, and a total of 300 patients were finally enrolled for developing the model (figure 1).
Study flow chart.
Measurements
Intensive Care Delirium Screening Checklist
ICDSC was an eight-item checklist that was developed based on the Diagnosis and Statistical Manual of Mental Disorders (4th Edition) criteria features of delirium.24 The ICDSC routinely combines collected data (such as orientation with short observations) of obvious manifestations of described features. Obvious manifestations of a checklist item during the evaluation period were scored as one point.
The ICDSC exhibited a high sensitivity of 99%.24 A score of 4 points or more will detect 99% of patients who will go on to have a diagnosis of delirium; however, it would also falsely identify 36% of patients in whom a psychiatric assessment will not result in this diagnosis.25 The checklist allows the ICDSC scoring system to be a user-friendly tool, which is useful in early and systematic screening of delirium in the ICU. The Chinese version of ICDSC demonstrated acceptable reliability, with a Cronbach alpha of 0.74 and validity of 0.93.
POSSUM Score
The widely adopted POSSUM score is generally used by surgeons to evaluate postoperative complications and risk of mortality within 30 days after major operations such as macrovascular surgery, gastric cancer surgery, colorectal surgery, thoracic surgery and joint replacement.26 The POSSUM score has been evaluated in numerous studies, and its modifications have been recognised as highly effective for surgical audit purposes. It is composed of 18 variables, derived originally from the multivariate analysis of 48 physiological and 14 operative variables, and it has a four-level exponential score of severity, of which the first 12 are preoperative physiological factors that include age, cardiac signs, respiratory history, blood pressure, pulse, Glasgow coma score, haemoglobin, white cell count, urea, sodium, potassium and ECG. The remaining six factors are operative variables, including operative severity, multiple procedures, total blood loss, peritoneal soiling, presence of malignancy and mode of surgery.26
Other risk factors
Other risk factors were collected by self-report and included history of coma, hypertension, diabetes, mechanical ventilation, alcohol dependence and acid–base imbalance, as well as length of operation.
Data collection
In this study, decisions about admitting the patients to the ICU for treatment were made at the doctors’ discretion, depending on the severity of the patients’ illness. The risk factors involved in this study were collected by systemically trained researchers as soon as the patients awoke after admission in the ICU; the patients who met the inclusion criteria were assessed using the ICDSC scale every 12 hours. Considering the need for ongoing observation for ICDSC, patients were required to stay in the ICU for at least 24 hours.
During this time, the researchers and bedside nurses would pay special attention to the patient’s change of consciousness. If the patient exhibited a significant change of consciousness, evaluation of delirium would be re-initiated to determine whether the patient had delirium. The evaluation was stopped if the patient developed ICU-POD. If not, the evaluation would be continued until the end of treatment in the ICU.
If some data were missing when collecting the information, the patient was excluded from the study. So in this study, no data were missing, and all data were available in the hospital’s patient information system.
Statistical analysis
The general data of patients are expressed as the frequency and percentage; normally distributed data are expressed by mean±SD and non-normally distributed data are expressed as results of the rank sum test. Continuous data were compared using the t-test, and non-continuous data were compared using the chi-squared test. We used univariate logistic regression to develop the prediction model by assessing the association between each potential risk factor and ICU-POD, and ORs for developing ICU-POD were calculated for each item. If a p value was less than 0.05 in the univariate analysis, a binary logistic regression analysis was conducted (excluding risk factors with p values≥0.10) to evaluate the independent association of each risk factor with the occurrence of delirium. We estimated the prognostic ability of the model using the area under the receiver operating characteristic curve (AUROC). Data were analysed using IBM SPSS Statistics V.22.0.
Results
A total of 400 patients were included in the study, of whom 300 were used for the development of the model and 100 for the validation. The model group consisted of 163 (54.33%) men and 142 (45.67%) women, aged 53.91±12.50 years. About 83 (27.67%) patients underwent cardiac surgery, 68 (22.67%) underwent abdominal surgery, 106 (35.33%) underwent thoracic surgery and 43 (14.33%) underwent orthopaedic surgery. Among them, 81 (27.00%) developed ICU-POD. The validation group comprised 58 (58.00%) men and 42 (42.00%) women, aged 52.53±12.79 years. A total of 20 (20.00%) patients underwent cardiac surgery, 35 (35.00%) underwent abdominal surgery, 27 (27.00%) underwent thoracic surgery and 18 (18.00%) underwent orthopaedic surgery. Of them, 31 (31.00%) developed ICU-POD. The demographic characteristics of the two groups are shown in table 1.
Patient characteristics
Discrimination and calibration of the ICU-POD prediction model
Statistically significant factors in univariate analysis (table 2) were included as independent variables, and delirium was included as the dependent variable in the binary logistic regression analysis. The risk factors included were assigned as shown in table 1. The developed ICU-POD prediction model comprised five predictors: POSSUM score, acid–base imbalance, coma, diabetes and hypertension (table 3). The performance of the model was evaluated using receiver operating characteristic curve (ROC) analysis. The AUROC was 0.852 (95% CI 0.802 to 0.902) with a sensitivity and specificity of 0.707 and 0.872, respectively, with a Youden index of 0.579, indicating a good prediction effect. The results of the Hosmer-Lemeshow goodness-of-fit test (5.203, p=0.736) indicated that the model had a satisfactory prediction effect requiring no further calibration (figure 2).
Delirium risk factors in the single factor analysis
Postoperative delirium prediction model for critically ill patients
ROC curves for the ICU-POD prediction model. ROC, receiver operator characteristic; ICU-POD, intensive care unit postoperative delirium.
Clinical validation of the model
ICU-POD risk was divided into four groups based on a method introduced in a study by van den Boogaard et al.19: very low (0%–10%), low (10%–20%), moderate (20%–40%) and high risk for ICU-POD (>40%). According to the previous literature, at a cutoff value of 64.5%, the sensitivity and specificity were 74.2% and 98.6%, respectively (table 4).
Prediction effect of the model at different cutoff points
To determine risk factors for ICU-POD, factors were compared between patients with and without ICU-POD. There was no significant difference in sex and operation duration between the two groups (p>0.05). By contrast, a significant difference in POSSUM score, coma, hypertension, diabetes, acid–base imbalance, alcohol dependence and mechanical ventilation was observed between the two groups (p>0.05) (table 5). After conducting a binary logistic regression analysis of the risk factors that demonstrated statistical significance, we constructed the following ICU-POD prediction model: In[R / (1 − R)]=0.310 for acid–base imbalance +1.228 for diabetes history +1.308 for hypertension history +3.428 for coma history +0.113 for POSSUM score − 6.963. The model has an AUROC of 0.852, Youden index of 0.5789, sensitivity of 70.73% and specificity of 87.16% (figure 2). Nurses can derive a patient’s score using this formula. The closer the patient’s score is to the cutoff value (Youden index), the higher the risk of delirium. When the Youden index is used as the cutoff value, sensitivity and specificity of the model are the highest and thus the model exhibits the strongest ability to predict delirium.
Assigned value of risk factors
Discussion
The pathogenesis and pathophysiology of ICU-POD is not yet entirely understood; nonetheless, it is obvious that multiple risk conditions and precipitating factors are related to ICU-POD. In this study, we found that acid–base imbalance, diabetes, hypertension, coma and POSSUM score were associated with the development of ICU-POD in a large cohort of patients. This is partly consistent with a study concluding that diabetic patients are more prone to POD.27 There are some other studies28 29 that have shown associations of ICU-POD with diabetes, acid–base imbalance, coma and hypertension. Nevertheless, to our knowledge, this is the first analysis to report on associations among all of those five risk elements together, including the POSSUM score for the first time ever, in the development of ICU-POD.
In view of the high prevalence of POD in ICU patients and its serious consequences, prediction of ICU-POD is clearly of major significance for both medical staff and patients.3 16 Therefore, in this prospective study, we developed and validated a model for predicting ICU-POD. Basically, almost all delirium prediction models currently available are for patients with all kinds of diseases and conditions, including medical, surgical, gynaecological and paediatric. Before development of the current model, the Delphi model was the only one that was specialised for predicting POD. Nevertheless, our model is aimed at postoperative patients only. A previous study reported that surgery is an independent risk factor that dominates the development of ICU-POD.2 Our model plays a significant role in better understanding of delirium and can be used as a reference for future research. Compared with previous prediction models,14 16 30 31 the ICU-POD model has a high predictive value and can readily be calculated. The risk factors included in the model are available in the information systems of hospitals at all levels, rendering it possible for the model to be widely adopted. Moreover, a previous study32 pointed out that both the predictive effect and the preferences of the users should be considered when developing risk forecasting models. The ICU-POD model consists of only five risk factors and is more concise in contrast with other delirium prediction models, making it more favourable to clinical workers. In addition, in the Delphi model, a nursing delirium screening checklist (Nu-DESC)33 and the CAM34 were used for the evaluation of delirium. In contrast, we applied ICDSC for the current model, as recommended by PAD guideline concerning delirium assessment. Therefore, we thought that the application of different diagnostic tools for delirium would affect the evaluation of delirium, so we preferred the ICDSC as a more sensitive and more convenient tool for delirium assessment. Furthermore, we intended to explore the relationship between the severity of different operations and the occurrence of delirium, and we used the POSSUM score as a risk factor, as it can quantify the impact of the surgical procedure on the patient. Nevertheless, in the Delphi model, only the type of operation (emergency surgery and open surgery) was included; as this does not address the problem explored in this study, we did not adopt this model.
A previous study reported that surgery is a risk factor that contributes to the development of ICU-POD.2 Moreover, the POSSUM scoring system includes not only the evaluation of the patients’ physical condition but also the evaluation of operative severity. Moreover, this scoring system has a high predictive value and wide scope of application.26 Worse preoperative physical condition and more severe surgery are reflected in a higher score. The POSSUM score assesses the physiological status (circulatory system, respiratory system, state of consciousness and biochemical indexes) of the patients. The operative severity indicator of the system could reflect the severity of the trauma acquired by the patient after undergoing surgery.35 Thus, we decided to apply POSSUM in our study.
In the study, to make the model align better with the actual situation of postoperative patients, we explored the risk factors of ICU-POD by conducting a preliminary meta-analysis.20 Previous studies1 36 have demonstrated that age, hypertension, coma and severity of illness at admission are risk factors of delirium, all of which are embodied in the POSSUM score and other risk factors of the ICU-POD prediction model, thus ensuring the credibility of the results of the study as well as pertinence of the model to postoperative patients.
Furthermore, the ICU-POD prediction model can also classify the risk of ICU-POD, facilitating initiation of preventive measures for patients with different risk levels. It also suggests that nurses should pay close attention to patients with higher risk of ICU-POD. In addition, the model facilitates immediate grading assessment after ICU admission and advance drug or non-drug preventive measures. Previous studies1 12 37 showed that the prevention of delirium decreased the morbidity and shortened the duration of delirium, further indicating that the results of the study are of great significance.
The study found that the higher the risk of delirium, the better the performance of the model. However, in daily practice, with constant changes in patients’ conditions, a predicted low risk does not exclude the possibility of development of ICU-POD. Besides, patients who had a history of coma before admission were 30.821 times more susceptible to ICU-POD, which was the highest among the risk factors in this study (table 3). This could be attributed to the concussion that happened at the moment of injury, during which the patient would suffer a transitional conscious disturbance for no more than half an hour and would recover soon after that. There might not be any apparent changes in the patient’s brain; however, there could be potential injury in the central nervous system, causing dysphoria, restlessness and hypomnesia in the patient, which may lead to ICU-POD. Therefore, medical staff should pay close attention to patients with a history of coma before admission and employ active protection measures for the head and brain. In addition, medical staff should also re-evaluate the patient’s condition in a timely manner to adjust the preventive treatment and nursing strategies for ICU-POD.
When patients enter the room, the medical staff can complete the collection of patient data according to the latest physiological test results, operation records and the first blood gas analysis results after admission to the ICU. When applying this model, the medical staff should pay attention and actively intervene with patients whose score is more than 0.5789, as this indicates higher risk of having ICU-POD. When the patient’s score is close to the critical value, the medical staff should also pay enough attention to avoid ICU-POD. When the patient’s condition changes significantly, it is recommended that the medical staff predict ICU-POD to help avoid delirium to the extent possible.
Limitations
This study has some limitations. For instance, the influence of medicine use was not considered when developing the ICU-POD prediction model. However, as suggested in the PAD guidelines,1 non-benzodiazepine sedatives should be administered to prevent delirium. However, we did not include the impact of sedatives, because different hospitals tend to use different medicines. Additionally, in our ICUs, where the research was conducted, the doctors use only non-benzodiazepines according to the recommendations of PAD guidelines, which are universally regarded as references when treating ICU patients. It was therefore difficult to distinguish the effects of different drugs in the study. In future studies, multicenter studies can be conducted to explore the effects of different drugs. In addition, because of limited resources, we were unable to perform a multicenter study, contributing to the limited generalisability of the model. Furthermore, since the variables in our study were mainly binary variables, the regression methods we used were all for binary variables. Moreover, there are dedicated scoring systems used in cardiac surgery, such as the Logistic European System for Cardiac Operative Risk Evaluation (EuroSCORE)38 and the EuroSCORE II,39 whereas the POSSUM score is a morbidity and mortality scale for general surgery. However, we searched the background of the EuroSCORE and EuroSCORE II carefully but found that neither had a Chinese version. Since the reliability and validity of these scoring systems in China remain unknown, we were unable to apply these scoring systems in this study. Therefore, we applied the POSSUM scoring system, which has a Chinese version with demonstrated reliability and validity, is also obtainable and has been applied by many Chinese researchers.
Additionally, although the validation was initiated after developing the ICU-POD prediction model, because the severity of illness was similar since the data were collected in the same hospital, the model had a satisfactory predictive value. Nevertheless, the suitability of the model for use in other hospitals needs further external validation.
Conclusions
Our study developed and validated the ICU-POD prediction model for postsurgical ICU patients. The model has a high predictive value, uses readily available data, is simple and easy to apply and enables timely identification of the risk of ICU-POD. Use of this model will facilitate targeted initiation of preventive measures early at admission, decrease incidence of ICU-POD and reduce the influence of delirium on patients.
Acknowledgments
We would like to thank Editage (www.editage.cn) for English language editing.
References
Footnotes
HX, WZ and XW are joint first authors.
Contributors HX, WZ and XW contributed equally to this work and should be considered co-first authors. HX contributed to the design of the study, data analysis, interpretation of the results and writing the manuscript. WZ contributed to writing the manuscript. YF critically reviewed the manuscript. TW and XW contributed to the acquisition of and maintaining the data. GC contributed to the analysis of the data. All authors finally approved the final version of manuscript.
Funding This study was funded by the National Natural Science Foundation of China, grant number 71704040.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethical approval for this study (protocol no. KY2017-026) was provided by the Second Affiliated Hospital of Harbin Medical University, Harbin, China on 7 December 2015. Additionally, all participants or guardians provided written informed consent, and the institutional ethical review board approved the study.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.