You are here

  • Home
  • Archive
  • Volume 8, Issue 3
  • Development of the Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator for prediction of postsurgical mortality and need for intensive care unit admission risk: a single-center retrospective study

Article Text

Article menu

Download PDFPDF

XML
Health services research
Research
Development of the Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator for prediction of postsurgical mortality and need for intensive care unit admission risk: a single-center retrospective study
  1. Diana Xin Hui Chan1,
  2. Yilin Eileen Sim1,
  3. Yiong Huak Chan2,
  4. Ruban Poopalalingam1,
  5. Hairil Rizal Abdullah1,3
  1. 1 Division of Anaesthesiology, Singapore General Hospital, Singapore
  2. 2 Biostatistics Unit, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
  3. 3 Duke-NUS Medical School, Singapore
  1. Correspondence to Dr Hairil Rizal Abdullah; Hairil.rizal.abdullah{at}singhealth.com.sg

Abstract

Introduction Accurate surgical risk prediction is paramount in clinical shared decision making. Existing risk calculators have limited value in local practice due to lack of validation, complexities and inclusion of non-routine variables.

Objective We aim to develop a simple, locally derived and validated surgical risk calculator predicting 30-day postsurgical mortality and need for intensive care unit (ICU) stay (>24 hours) based on routinely collected preoperative variables. We postulate that accuracy of a clinical history-based scoring tool could be improved by including readily available investigations, such as haemoglobin level and red cell distribution width.

Methodology Electronic medical records of 90 785 patients, who underwent non-cardiac and non-neuro surgery between 1 January 2012 and 31 October 2016 in Singapore General Hospital, were retrospectively analysed. Patient demographics, comorbidities, laboratory results, surgical priority and surgical risk were collected. Outcome measures were death within 30 days after surgery and ICU admission. After excluding patients with missing data, the final data set consisted of 79 914 cases, which was divided randomly into derivation (70%) and validation cohort (30%). Multivariable logistic regression analysis was used to construct a single model predicting both outcomes using Odds Ratio (OR) of the risk variables. The ORs were then assigned ranks, which were subsequently used to construct the calculator.

Results Observed mortality was 0.6%. The Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator, consisting of nine variables, was constructed. The area under the receiver operating curve (AUROC) in the derivation and validation cohorts for mortality were 0.934 (0.917–0.950) and 0.934 (0.912–0.956), respectively, while the AUROC for ICU admission was 0.863 (0.848–0.878) and 0.837 (0.808–0.868), respectively. CARES also performed better than the American Society of Anaesthesiologists-Physical Status classification in terms of AUROC comparison.

Conclusion The development of the CARES surgical risk calculator allows for a simplified yet accurate prediction of both postoperative mortality and need for ICU admission after surgery.

  • surgical risk
  • postoperative mortality
  • icu stay
  • risk prediction
  • risk calculator

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-019427

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The Combined Assessment of Risk Encountered in Surgery (CARES) model predicts risk for both 30-day mortality and need for intensive care unit (ICU) admission postoperatively with good accuracy. Prediction of the risk for postoperative ICU admission is novel and not currently available for most risk stratification tools. This adds to the utility of the model and aids in decision-making process and health resource planning.

    • The CARES surgical risk calculator comprises of simple and easily accessible variables available from routine preoperative evaluation for most surgeries.

    • It is the first risk stratification tool to incorporate the use of red cell distribution width, a novel haematological biomarker which has been shown to be of value in predicting mortality risk.

    • This is a retrospective study design.

    • CARES has only been validated in a single centre, hence, there is a need for further prospective studies to externally validate the tool in other institutions across the region. 

    Introduction and background 

    About 250 million surgeries are performed worldwide each year, and this number is increasing rapidly.1 As access to surgery improves, the number of patients with postoperative complications will also increase.2 3 Previous studies demonstrated that a large proportion of postoperative mortality occurs in a smaller, distinct group of patients with high-risk characteristics, and <15% from this group were admitted to intensive care units (ICU) postoperatively.4 5 In the preoperative assessment of a surgical patient, it is prudent to counsel the patient on the risks of postoperative mortality and need for critical care monitoring after surgery. Therefore, accurate preoperative prediction and stratification of surgical risks is becoming even more important for perioperative shared decision-making process, guiding allocation of resources and improving patient outcomes.

    However, predicting postoperative risks and identifying patients at a higher risk of adverse events have traditionally been based on individual surgeon experience and augmented by published rates in the literature, either from single institution studies or clinical trials.6 Unfortunately, these estimates are typically not specific to an individual patient’s risk factors. Moreover, existing risk stratification tools have their own limitations. The currently available risk stratification tools, for example, American Society of Anaesthesiologists-Physical Status (ASA-PS), Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity (POSSUM), Surgical Outcome Risk Tool (SORT) and American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) were all derived from the Western population and healthcare systems, from which Asians would serve as outliers in view of the their different socioeconomic, cultural, genetic makeup and healthcare systems. There is a paucity of surgical risk stratification models which is derived from or has been validated in the Asian population. This limits the uptake and applicability of these models in the region.

    Furthermore, the individual risk stratification tools suffers from wide inter-user variability (ASA-PS),7 need for data which are not available during the preoperative period (POSSUM),8 9 lack of validation outside the derived population’s region (SORT, ACS-NSQIP) and the complexity of the model itself (ACS-NSQIP). Hence, the inertia to use them may be due to concerns over the accuracy, complexity and also the requirement for a large number of variables. To improve the performance of predictive models, there are recent interests in the use biomarkers such as B-type natriuretic peptides10 11 and cardiac troponin12 to predict mortality. However, these markers may not be easily available across all laboratories and most often are not part of the routine preoperative investigations. More recently, readily available haematological biomarkers such as red cell distribution width (RDW) and degree of anaemia (if present) have been shown to be associated with postoperative mortality risk.13–15 Incorporation of these biomarkers with preoperative clinical factors may improve the accuracy of a surgical risk model.

    We aim to develop a locally derived, simple and accurate surgical risk calculator that consisted of readily available preoperative clinical and laboratory variables, which can predict both mortality and ICU admission with just a single set of variables. We hypothesise that the development of this risk stratification tool could help us accurately predict the risk of (1) 30-day postsurgical mortality and (2) requiring admission to ICU for >24 hours during the surgical admission, which may serve as a surrogate for major postoperative complications in the Singapore population.

    Methodology

    Data source and patients 

    Institutional Review Board approval was obtained (Singhealth CIRB 2014/651/D) prior to the start of the study, which waived the requirement of individual informed consent. Retrospective data were collected and analysed from the electronic medical records of 90 785 patients aged 18 and older who underwent surgery under general or regional anaesthesia between 1 January 2012 and 31 October 2016 in Singapore General Hospital, a 1700-bedded tertiary academic hospital in Singapore. These clinical records were sourced from our institution’s clinical information system (Sunrise Clinical Manager, Allscripts, Illinois, USA) and stored in our enterprise data repository and analytics system (SingHealth-IHiS Electronic Health Intelligence System (eHINTS)). eHINTS collects reliable data on patient demographics, laboratories, comorbidities and 30-day postoperative outcomes for patients undergoing surgeries from all surgical subspecialties. It integrates information from multiple healthcare transactional systems including administration, clinical and ancillary systems. Mortality data on the system were synchronised with the National Electronic Health Records, ensuring a near complete follow-up. We excluded patients who underwent cardiac surgery, neurosurgery, transplant and burns surgery, and evaluated only the outcomes of the index surgery for patients who underwent multiple surgeries during the study period. After excluding patients with missing data, the final data set consisted of 79 914 cases (figure 1).

    Figure 1
    Figure 1

    Flow chart of the patient cohort. In total, 100 873 index cases were identified from operating theatre listing. We excluded patients who underwent cardiac surgery, neurosurgery, transplant and burns surgery, and evaluated only the outcomes of the index surgery for patients who underwent multiple surgeries during the study period. After excluding the above cases, 90 785 cases remained for consideration. Of these, 10 871 cases had missing variables and the final number of cases included in our patient cohort for statistical analysis was 79 914. LA, Local Anaesthesia; NES, Neurosurgery.

    Procedures and definitions

    Data collected include patient demographics as well as preoperative comorbidities and laboratory data (table 1). These are routine clinical and laboratory data that were electronically collected during the preoperative anaesthesia assessment visit. Priority of surgery (emergency or elective) and surgical risk classification were based on the 2014 European Society of Cardiology (ESC) and the European Society of Anaesthesiology (ESA) guidelines16 17 . Emergency cases in our hospital are classified as cases requiring operation within 24 hours, and they are further subcategorised into their degree of urgency. Missing data were excluded and complete case analysis was done.

    View this table:
    Table 1

    Descriptive data for the study, including 30-day mortality, any intensive care unit (ICU) admission and ICU admission >24 hours

    Preoperative laboratory results including full blood count (FBC) and renal panel (RP) were taken as the latest blood results within 90 days before the surgery, and up to the day of surgery but before the start time of surgery. These results include preoperative haemoglobin, red cell distribution width levels and serum creatinine levels. The presence of anaemia was defined by WHO’s gender-based classification of anaemia severity.18 Mild anaemia was defined as haemoglobin concentration of 11–12.9 g/dL in males and 11–11.9 g/dL in females; moderate anaemia was defined for both genders to be haemoglobin concentration between 8–10.9 g/dL and severe anaemia defined as haemoglobin concentration <8.0 g/dL. Pre-existing chronic kidney disease, if present, is graded based on the estimated glomerular filtration rate by the Modification of Diet in Renal Disease equation according to 2012 Kidney Disease: Improving Global Outcomes (KDIGO) guidelines.19 Red cell distribution width (RDW) is reported as a coefficient of variation (percentage) of red blood cell volume with the normal reference range for RDW in this hospital laboratory to be 10.9%–15.7%. Levels >15.7% were defined a priori as high RDW, and this corresponded to the 89th centile of RDW values in our study population. A high RDW has been shown to be associated with an increased risk of mortality that is independent of the severity of anaemia.20 The chosen cut-off value of 15.7% was shown to have a sensitivity of 39.5%, specificity of 89.3%, positive predictive value of 5.3% and negative predictive value of 99.0%.13 The individual components of revised cardiac risk index were defined as per the original study by Lee et al,21 and the ASA status follows that of the ASA-PS definitions.22

    Statistics

    All analyses were performed using IBM SPSS V.21.0. The data set (90 785 subjects) was randomly divided to 70%:30%, with the former used as the derivation cohort and the latter as validation cohort.

    The Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator was developed using ORs of the risk variables obtained from the logistic regression for postsurgical 30-day mortality or ICU >24 hours within the admission. The ORs were assigned rank scores. The model was then validated on the 30% cohort. The Hosmer-Lemeshow (H-L) test for calibration was used to show the goodness of fit for the models developed.

    In the initial development of the model, we looked at each outcome individually before combining the significant variables to create a combined risk prediction model for both mortality and need for postoperative ICU stay of >24 hours (ICU >24 hours). For the initial analysis, we included potentially significant variables for each outcome and performed stepwise multivariate logistic regression to obtain the minimum number of variables that retained the accuracy of prediction. The accuracy of prediction was estimated using the receiver operating curve (ROC) and the area under the receiver operating curve (AUROC).

    The significant variables for each outcome were then combined to construct a single risk prediction model. We tested the accuracy of the model in predicting either mortality or ICU >24 hours using the ROC and AUROC. H-L tests were performed to show goodness of fit for the model. The performance of the CARES surgical risk calculator was then compared with that of the ASA-PS and the ASA-PS with propensity scoring to adjust for possible differences in the development of the models. Statistical significance was set at P<0.05.

    Results

    From a total of 90 785 cases, 63 715 (70%) patients were randomly selected into the derivation cohort and the remaining 27 070 (30%) into the validation cohort. Observed mortality in the derivation and validation cohorts were similar, at 0.6%. 1.2% (770 patients) in the derivation cohort were admitted to ICU for more than 24 hours, while that in the validation cohort is 1.4% (375) patients. Descriptive data for these cohorts are summarised in table 1.

    Model development and derivation

    Mortality outcome

    For mortality, 12 variables which were found to be significant in univariate analysis (age, surgical risk, race, anaemia, chronic kidney disease, RDW, presence of cerebrovascular accident, ischaemic heart disease, congestive heart failure, insulin-requiring diabetes mellitus, ASA status and gender) were included. The AUROC (the highest ever AUC to be obtained for this set of data) for this 12-variable black-box model was 0.931 (0.916–0.946). Stepwise logistic regression retained only seven significant variables (table 2) which produced the same AUROC of 0.931 (0.915–0.947) of the 12-variable black-box model.

    View this table:
    Table 2

    Seven significant variables following stepwise logistic regression for mortality outcome

    Both the ORs and rank scores are presented in table 2. While rounded ORs are commonly used for risk prediction models, some of our ORs are too high and may skew the final score. Hence, to handle these ‘extreme scores’, a ranking system was developed for scoring in the calculator. Additionally, this final rank-based model facilitates utilisation of the risk calculator by keeping it simple enough for regular clinical consult use. This mortality model with an AUROC of 0.928 (0.912–0.945), compared with the original AUROC of 0.931 (0.915–0.947) from the OR-based model, does not compromise accuracy of the risk prediction and at the same time offers increased usability (even in health systems without regular electronic medical records use). The AUROC are shown in online supplementary appendix figure 1.

    Supplementary file 1

    The H-L test for calibration demonstrated good fit for the final model (P=0.79) predicting postoperative mortality for both the derivation and validation cohort (online supplementary appendix figures 2 and 3).

    ICU>24-hour outcome

    For the ICU >24- hour outcome, 13 variables which were significant in the univariate analysis (age, surgical risk, race, anaemia, RDW, presence of cerebrovascular accident, ischaemic heart disease, congestive heart failure, chronic kidney disease, insulin-requiring diabetes mellitus, ASA status, surgical priority and gender) were included. The AUROC (the highest ever AUC to be obtained for this set of data) for this 13-variable black-box model was 0.876 (0.861–0.890). Stepwise logistic regression retained seven significant variables (age, surgical risk, anaemia, congestive heart failure, ASA status) with AUROC of 0.873 (0.858–0.888). Table 3 shows the seven significant variables and their corresponding OR and rank scores.

    View this table:
    Table 3

    Seven significant variables following stepwise logistic regression for intensive care unit >24-hour outcome

    For the similar reasons as with the mortality model above, a risk score model using the rank of the ORs was developed, with an AUROC of 0.867 (0.852–0.882), retaining accuracy of the risk prediction. The AUROC are shown in the appendix (online supplementary appendix figure 4).

    The H-L test for calibration showed good fit for the final model (P=0.81) predicting need for ICU stay >24 hours in both the derivation and validation cohorts (online supplementary appendix figures 5 and 6).

    Combined modelling for both mortality and ICU

    To further increase the ease of use for CARES surgical risk calculator, we explored a combined model that is accurate in predicting both mortality and morbidity with just a single set of variables. The above results of separate modelling for each outcome predictions demonstrated robust and accurate individual model in predicting respective outcomes. We now combine the predictors to create a single model. This model consists of nine variables: age, surgical risk, anaemia, RDW, ischaemic heart disease, ASA, surgical priority, gender and presence of congestive heart failure.

    The combined predictors are shown in table 4.

    View this table:
    Table 4

    Final Combined Assessment of Risk Encountered in Surgery model, combining predictors for both mortality and intensive care unit (ICU) admission

    These OR-based nine variables yielded an AUROC of 0.936 (0.920–0.953) for mortality and 0.874 (0.859–0.889) for ICU. Using the rank scores, the AUROC are 0.934 (0.917–0.950) and 0.863 (0.848–0.878) for mortality and ICU, respectively, which again show that accuracy was not compromised. The ROCs are shown in figure 2. The corresponding H-L plots are available in the appendix (online supplementary appendix figures 7 and 8).

    Figure 2
    Figure 2

    Receiver operative curves (ROCs) for mortality and intensive care unit (ICU) >24-hour outcomes in the derivation cohort when the combined model was used to predict the above outcomes. These combined OR model yielded an area under the ROC (AUROC) of 0.936 (0.920–0.953) for mortality and 0.874 (0.859–0.889) for ICU. Using the rank scores, the AUROC are 0.934 (0.917–0.950) and 0.863 (0.848–0.878) for mortality and ICU, respectively, which again show that accuracy was not compromised.

    We tested the rank score model on the validation cohort. The AUROC was 0.934 (0.912–0.956) for mortality and 0.837 (0.808–0.868) for ICU >24 hours. The ROCs are shown in figure 3. The corresponding H-L plots are also shown in the appendix (online supplementary appendix figures 9 and 10).

    Figure 3
    Figure 3

    Receiver operative curves (ROCs) for mortality and intensive care unit (ICU)>24-hour outcomes in the validation cohort when the combined model was used to predict the above outcomes. The area under the ROC (AUROC) was 0.934 (0.912–0.956) for mortality and 0.837 (0.808–0.868) for ICU>24 hours.

    The cumulative rank scores were subsequently categorised into different bands classifying the risk of mortality as low, low–moderate, moderate–high or high, and a corresponding mortality probability was assigned to each band (table 5). Clinical decision-making suggestions are included in the table.

    View this table:
    Table 5

    Risk categories and clinical decision making

    Applicability of the CARES model for combined mortality and ICU prediction

    We provide a hypothetical example of a patient to illustrate the application of the model. A 76-year-old Chinese  man with history of hyperlipidaemia, hypertension, previous ischaemic heart disease with stent inserted with no evidence of congestive heart failure is scheduled for an elective right hemicolectomy for colorectal cancer. FBC shows a starting haemoglobin of 12 and an RDW of 12.1.

    Using the scoring method, the patient would have a combined rank score of 8+5+2+3+7+2=28, which places him in the moderate–high-risk category predicting a 1.9% risk of 30-day postsurgical mortality and 4.9% risk of need for postoperative ICU stay for >24 hours.

    Using these figures, we would be able to counsel the patient appropriately and plan for optimisation of modifiable risk factors preoperatively and appropriate perioperative monitoring and surveillance.

    Comparison with ASA-PS

    When the performance of combined model CARES was compared with both the ASA-PS and its propensity model, CARES performed better than both with an AUROC of 0.934 (0.917–0.950) for mortality and 0.863 (0.848–0.878) for ICU >24 hours (see table 6).

    View this table:
    Table 6

    Area under the receiver operating curve (AUROC) comparison between Combined Assessment of Risk Encountered in Surgery (CARES) and American Society of Anaesthesiologists (ASA) for mortality and intensive care unit (ICU) >24 hours

    Discussion

    We developed and internally validated a simple, locally derived CARES surgical risk calculator comprising nine preoperative variables to predict both 30-day mortality and need for ICU (ICU admission for >24 hours), respectively, in adults undergoing non-cardiac and non-neurological surgery. The nine variables are age, gender, ASA status, surgical priority, surgical risk, presence of anaemia, RDW, presence of ischaemic heart disease and congestive heart failure. The web-based version of this risk calculator is currently under construction.

    Presently, the use of preoperative risk stratification systems is not routinely done due to combination of factors such as the complexity of the system, the need for additional non-routine preoperative tests, use of intraoperative and postoperative variables and the inability to calculate an individual percentage mortality and morbidity risk.23 The CARES surgical risk calculator comprises simple and easily accessible variables available from routine preoperative evaluation for most surgeries, with the exception of healthy patients coming for the lowest risk surgeries, which may not even need an FBC.24 The inclusion of more variables would greatly increase the time taken to collect data and thereby reduce the usability of the tool. Ease of use and face validity are two important factors that may encourage widespread, routine use of risk prediction tools,25 both of which are true for the CARES surgical risk calculator. The CARES surgical risk calculator stood out from most other risk stratification tools not only in that it was derived from the intended Southeast Asian population but it also predicts both postoperative mortality and the need for postoperative ICU care using a single set of easily accessible variables. This provides convenience and improves efficiency to the clinician in a busy outpatient setting.

    The prediction of postsurgical 30-day mortality risk is an important clinical outcome that is of interest to both surgeons and patients and therefore aids in shared decision making.26 Thirty-day all-cause postoperative mortality is a widely accepted, valid and relevant outcome measure of surgical care.27 For this study, the postoperative mortality data were synced with the National Registry of Death data, ensuring the integrity and completeness of the data.

    The prediction of the risk of ICU admission for >24 hours postoperatively is novel and not available for most current risk stratification tools. The capability to predict ICU admission for >24 hours after surgery would aid physicians to determine the postoperative patient disposition plan before surgery and therefore healthcare resource allocation. This could improve patients’ outcome by reducing failure to rescue events28 and improve the efficiency of the valuable ICU bed allocation. The disposition of a patient immediately after surgery usually depends on both objective and subjective factors. Objective variables include patient comorbidities, preoperative evaluation, surgical risk and intraoperative haemodynamic. Subjective factors are usually operator-dependent and involve the clinical judgement and comfort level of the anaesthetist and surgeon involved.29 30 While an ICU admission by itself would not be a useful measure of morbidity, length of stay in ICU may be seen as an indirect measure of morbidity-related outcome.31 We defined ICU admission for >24 hours as a significant outcome on observation that patients who were discharged from ICU within the first 24 hours may have been safely monitored postoperatively in a lower intensity unit.

    CARES reflects the critical significance of age as a predictor of risk. Age is a significant independent predictor, and this should be reflected in the risk counselling. This will help to increase the awareness of the impact of ageing on mortality among both the clinicians and patients and their careers.

    Despite being a single-centre, locally derived risk stratification tool, CARES has a number of advantages. It is the largest study to develop and validate a surgical risk stratification tool in the heterogeneous, multiracial cohort of patients undergoing non-cardiac surgery in the Southeast Asia region. The CARES tool is a parsimonious model, consisting of only 9 preoperative variables, of which includes 7 clinical variables and 2 simple preoperative blood tests which are used to calculate both mortality and ICU admission risk, compared with 18 preoperative, intraoperative and postoperative variables for POSSUM and 22 preoperative patient risk factors for the ACS-NSQIP model.8 32 Despite the small number of variables used to compute surgical mortality and morbidity risk, CARES demonstrated high performance in our population cohort for both outcomes. This may be due to the use of biomarkers, which could increase the performance of risk prediction tools. Our CARES model incorporated the use of RDW which is obtained from the almost routine preoperative FBC test. Recent studies have shown the value of RDW in predicting postoperative mortality.13 14 20 33

    In the development of our risk prediction tool, we focused on the utility and usability of the tool. We computed the ORs of each variable category and assigned a rank score to each of them. The cumulative rank scores were matched to corresponding risks for the outcomes. This resulted in a more user-friendly calculator, without compromising on the accuracy. This could lead to better physician and patient acceptance in using the tool for shared decision making as well as healthcare resource planning (see tables 2 and 5).

    The ASA-PS is one of the most commonly used risk prediction tools currently in Singapore. We compared the AUROC of the CARES surgical risk calculator with that of ASA-PS and found that CARES showed considerable superiority in performance in both the mortality and need for ICU prediction. With easily available preoperative data, good accuracy and the availability of a web-based calculator, it is hoped that adoption of the CARES tool in calculating preoperative surgical risk may exceed that of other models.

    The limitations for this study include the inherent nature of retrospective data.

    In our data collection, we also did not include the presence of chronic obstructive pulmonary disease (COPD) as a variable for the development of our risk stratification tool. COPD itself has been not been conclusively shown to be strongly associated with increased postoperative mortality in non-cardiac or non-thoracic surgeries.34 35 One major weakness of our study is the lack of proper definition in our classification on the priority of surgery. As this is a retrospective study, we were unable to give an a priori definition of emergency surgery. We classified the priority of surgery according to that recorded in surgical records. However, in our centre, emergency surgery is further subdivided into four categories, with category A being surgeries that require operation within 6 hours of admission and category D being surgeries that can wait up to 24–48 hours. While the Surgical Risk Tool36 uses the National Confidential Enquiry into Perioperative Deaths classification of surgical priority which differentiates between scheduled, elective, urgent and emergency surgeries,6 the lack of discrimination in the priorities of our emergency surgery data contributes to possible classification bias.

    Furthermore, while the CARES tool is based on a large data set, it has only been validated in a single centre, hence, there is a need for further prospective studies to validate the tool in other institutions across the region. Despite this limitation, the sample analysed was representative of the local population and the demographics of hospital admissions in Singapore37 38 The results are generalisable owing to the broad representation of the range of surgical specialties. It predicts both postoperative mortality risk and need for ICU in our local population and has the potential to be a specific risk prediction tool in the Southeast Asian population.

    The CARES surgical risk calculator has many firsts. Besides being the first locally derived preoperative risk prediction tool and the first calculator to predict both mortality and ICU need after surgery, it is also the first risk stratification tool to incorporate the use of RDW. The generalisability of the model in international cohorts remains unknown and needs to be explored. External validation of the CARES tool is integral to test its validity further, as is the periodic recalibration and re-evaluation of the model to maintain validity with healthcare advancements, and demonstrate both utility and accuracy in predicting mortality and morbidity. The CARES tool has a potential to be expanded to be used for prediction for other postoperative morbidities, and further prospective studies should be focused on this. Likewise, studies evaluating the impact of risk stratification on improving patient outcomes through individual care planning should be a research priority as there is an opportunity to improve outcomes substantially.

    Conclusion

    The CARES tool provides an accurate prediction for mortality and need for postoperative ICU among surgical patients in Singapore. It is easily accessible and should be used in conjunction with clinical judgement to aid shared decision making and plan for ICU resource allocation. The development of the web-based calculator further facilitates user accessibility and utility of the tool.

    Acknowledgments

    The authors thank Mr Koh Yee Jin (principal systems specialist, Department of Health Insights, Integrated Health Information Systems Pte Ltd, Singapore) for his invaluable help in the data extraction process , Ms Sudha Harikrishnan, from the Department of Anaesthesiology, Singapore General Hospital for her help in data extraction and Associate Professor Ong Biauw Chi, Chair of Medical Board, Sengkang General Hospital, Singapore for her insightful mentorship.

    References

    1. 1.↵
      2. Weiser TG ,
      3. Haynes AB ,
      4. Molina G , et al
      . Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet 2015;385:S11.doi:10.1016/S0140-6736(15)60806-6
      OpenUrl
    2. 2.↵
      2. Alkire BC ,
      3. Raykar NP ,
      4. Shrime MG , et al
      . Global access to surgical care: a modelling study. Lancet Glob Health 2015;3:e316–e323.doi:10.1016/S2214-109X(15)70115-4
      OpenUrlCrossRef
    3. 3.↵
      2. Tevis SE ,
      3. Kennedy GD
      . Postoperative complications and implications on patient-centered outcomes. J Surg Res 2013;181:106–13.doi:10.1016/j.jss.2013.01.032
      OpenUrl
    4. 4.↵
      2. Pearse RM ,
      3. Harrison DA ,
      4. James P , et al
      . Identification and characterisation of the high-risk surgical population in the United Kingdom. Crit Care 2006;10:R81.doi:10.1186/cc4928
      OpenUrlCrossRefPubMed
    5. 5.↵
      2. Pearse RM ,
      3. Moreno RP ,
      4. Bauer P , et al
      . Mortality after surgery in Europe: a 7 day cohort study. Lancet 2012;380:1059–65.doi:10.1016/S0140-6736(12)61148-9
      OpenUrlCrossRefPubMedWeb of Science
    6. 6.↵
      2. Hoile RW
      . The National Confidential Enquiry into Peri-operative Deaths (NCEPOD). Aust Clin Rev 1993;13:11–15.
      OpenUrlPubMed
    7. 7.↵
      2. Cohen ME ,
      3. Bilimoria KY ,
      4. Ko CY , et al
      . Effect of subjective preoperative variables on risk-adjusted assessment of hospital morbidity and mortality. Ann Surg 2009;249:682–9.doi:10.1097/SLA.0b013e31819eda21
      OpenUrlCrossRefPubMedWeb of Science
    8. 8.↵
      2. Copeland GP ,
      3. Jones D ,
      4. Walters M
      . POSSUM: a scoring system for surgical audit. Br J Surg 1991;78:355–60.doi:10.1002/bjs.1800780327
      OpenUrlCrossRefPubMed
    9. 9.↵
      2. Brooks MJ ,
      3. Sutton R ,
      4. Sarin S
      . Comparison of Surgical Risk Score, POSSUM and p-POSSUM in higher-risk surgical patients. Br J Surg 2005;92:1288–92.doi:10.1002/bjs.5058
      OpenUrlCrossRefPubMedWeb of Science
    10. 10.↵
      2. Fox AA ,
      3. Shernan SK ,
      4. Collard CD , et al
      . Preoperative B-type natriuretic peptide is as independent predictor of ventricular dysfunction and mortality after primary coronary artery bypass grafting. J Thorac Cardiovasc Surg 2008;136:452–61.doi:10.1016/j.jtcvs.2007.12.036
      OpenUrlCrossRefPubMedWeb of Science
    11. 11.↵
      2. Biccard BM ,
      3. Rodseth RN
      . Utility of clinical risk predictors for preoperative cardiovascular risk prediction. Br J Anaesth 2011;107:133–43.doi:10.1093/bja/aer194
      OpenUrlCrossRefPubMed
    12. 12.↵
      2. Biccard BM ,
      3. Devereaux PJ ,
      4. Rodseth RN
      . Cardiac biomarkers in the prediction of risk in the non-cardiac surgery setting. Anaesthesia 2014;69:484–93.doi:10.1111/anae.12635
      OpenUrl
    13. 13.↵
      2. Sim YE ,
      3. Wee HE ,
      4. Ang AL , et al
      . Prevalence of preoperative anemia, abnormal mean corpuscular volume and red cell distribution width among surgical patients in Singapore, and their influence on one year mortality. PLoS One 2017;12:e0182543.doi:10.1371/journal.pone.0182543
    14. 14.↵
      2. Chen PC ,
      3. Sung FC ,
      4. Chien KL , et al
      . Red blood cell distribution width and risk of cardiovascular events and mortality in a community cohort in Taiwan. Am J Epidemiol 2010;171:214–20.doi:10.1093/aje/kwp360
      OpenUrlCrossRefPubMedWeb of Science
    15. 15.↵
      2. Borné Y ,
      3. Smith JG ,
      4. Melander O , et al
      . Red cell distribution width in relation to incidence of coronary events and case fatality rates: a population-based cohort study. Heart 2014;100:1119–24.doi:10.1136/heartjnl-2013-305028
      OpenUrlAbstract/FREE Full Text
    16. 16.↵
      2. Glance LG ,
      3. Lustik SJ ,
      4. Hannan EL , et al
      . The Surgical Mortality Probability Model: derivation and validation of a simple risk prediction rule for noncardiac surgery. Ann Surg 2012;255:696–702.doi:10.1097/SLA.0b013e31824b45af
      OpenUrlCrossRefPubMed
    17. 17.↵
      2. Kristensen SD ,
      3. Knuuti J ,
      4. Saraste A , et al
      . ESC/ESA Guidelines on non-cardiac surgery: cardiovascular assessment and management: The Joint Task Force on non-cardiac surgery: cardiovascular assessment and management of the European Society of Cardiology (ESC) and the European Society of Anaesthesiology (ESA). Eur Heart J 2014;2014:2383–431.
      OpenUrl
    18. 18.↵
      Haemoglobin concentrations for the diagnosis of anaemia and assessment of severity. http://www.who.int/vmnis/indicators/haemoglobin.pdf.
    19. 19.↵
      Kidney Disease: Improving Global Outcomes (KDIGO) CKD-MBD Work Group. KDIGO clinical practice guideline for the diagnosis, evaluation, prevention, and treatment of Chronic Kidney Disease-Mineral and Bone Disorder (CKD-MBD). Official j the int society of nephrology 2009;76:1–140.
      OpenUrl
    20. 20.↵
      2. Bazick HS ,
      3. Chang D ,
      4. Mahadevappa K , et al
      . Red cell distribution width and all-cause mortality in critically ill patients. Crit Care Med 2011;39:1913–21.doi:10.1097/CCM.0b013e31821b85c6
      OpenUrlCrossRefPubMedWeb of Science
    21. 21.↵
      2. Lee TH ,
      3. Marcantonio ER ,
      4. Mangione CM , et al
      . Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation 1999;100:1043–9.doi:10.1161/01.CIR.100.10.1043
      OpenUrlAbstract/FREE Full Text
    22. 22.↵
      2. Hackett NJ ,
      3. De Oliveira GS ,
      4. Jain UK , et al
      . ASA class is a reliable independent predictor of medical complications and mortality following surgery. Int J Surg 2015;18:184–90.doi:10.1016/j.ijsu.2015.04.079
      OpenUrl
    23. 23.↵
      2. Minto G ,
      3. Biccard B
      . Assessment of the high-risk perioperative patient. Continuing Education in Anaesthesia Critical Care & Pain 2014;14:12–17.doi:10.1093/bjaceaccp/mkt020
      OpenUrlCrossRef
    24. 24.↵
      2. Fleisher LA ,
      3. Fleischmann KE ,
      4. Auerbach AD , et al
      . 2014 ACC/AHA guideline on perioperative cardiovascular evaluation and management of patients undergoing noncardiac surgery: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines. Developed in collaboration with the American College of Surgeons, American Society of Anesthesiologists, American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Anesthesiologists, and Society of Vascular Medicine Endorsed by the Society of Hospital Medicine. J Nucl Cardiol 2015;22:162–215.doi:10.1007/s12350-014-0025-z
      OpenUrlCrossRefPubMed
    25. 25.↵
      2. Moonesinghe SR ,
      3. Mythen MG ,
      4. Das P , et al
      . Risk stratification tools for predicting morbidity and mortality in adult patients undergoing major surgery: qualitative systematic review. Anesthesiology 2013;119:959–81.doi:10.1097/ALN.0b013e3182a4e94d
      OpenUrlCrossRefPubMedWeb of Science
    26. 26.↵
      2. Yek JL ,
      3. Lee AK ,
      4. Tan JA , et al
      . Defining reasonable patient standard and preference for shared decision making among patients undergoing anaesthesia in Singapore. BMC Med Ethics 2017;18:6.doi:10.1186/s12910-017-0172-2
      OpenUrl
    27. 27.↵
      2. Birkmeyer JD ,
      3. Dimick JB ,
      4. Birkmeyer NJ
      . Measuring the quality of surgical care: structure, process, or outcomes? J Am Coll Surg 2004;198:626–32.doi:10.1016/j.jamcollsurg.2003.11.017
      OpenUrlCrossRefPubMedWeb of Science
    28. 28.↵
      2. Ghaferi AA ,
      3. Birkmeyer JD ,
      4. Dimick JB
      . Complications, failure to rescue, and mortality with major inpatient surgery in medicare patients. Ann Surg 2009;250:1029–34.doi:10.1097/SLA.0b013e3181bef697
      OpenUrlCrossRefPubMedWeb of Science
    29. 29.↵
      2. Memtsoudis S ,
      3. Besculides M ,
      4. Swamidoss C
      . Factors influencing patient disposition after ambulatory herniorrhaphy. Ambul Surg 2005;12:67–73.doi:10.1016/j.ambsur.2005.06.004
      OpenUrl
    30. 30.↵
      2. Gillies MA ,
      3. Pearse RM
      . Intensive Care after High-risk SurgeryWhat’s in a Name? Anaesthesiology 2016;124:761–2.
      OpenUrlCrossRefPubMed
    31. 31.↵
      2. Abelha FJ ,
      3. Castro MA ,
      4. Landeiro NM , et al
      . [Mortality and length of stay in a surgical intensive care unit.]. Rev Bras Anestesiol 2006;56:34–45.
      OpenUrlPubMed
    32. 32.↵
      2. Bilimoria KY ,
      3. Liu Y ,
      4. Paruch JL , et al
      . Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg 2013;217:833–42.doi:10.1016/j.jamcollsurg.2013.07.385
      OpenUrlCrossRefPubMed
    33. 33.↵
      2. Horne BD ,
      3. Anderson JL ,
      4. Muhlestein JB , et al
      . Complete blood count risk score and its components, including RDW, are associated with mortality in the JUPITER trial. Eur J Prev Cardiol 2015;22:519–26.doi:10.1177/2047487313519347
      OpenUrlCrossRefPubMed
    34. 34.↵
      2. de Albuquerque Medeiros R ,
      3. Faresin S ,
      4. Jardim J
      . [Postoperative lung complications and mortality in patients with mild-to-moderate COPD undergoing elective general surgery]. Arch Bronconeumol 2001;37:227–34.
      OpenUrlCrossRefPubMed
    35. 35.↵
      2. Kroenke K ,
      3. Lawrence VA ,
      4. Theroux JF , et al
      . Postoperative complications after thoracic and major abdominal surgery in patients with and without obstructive lung disease. Chest 1993;104:1445–51.doi:10.1378/chest.104.5.1445
      OpenUrlCrossRefPubMedWeb of Science
    36. 36.↵
      2. Sutton R ,
      3. Bann S ,
      4. Brooks M , et al
      . The Surgical Risk Scale as an improved tool for risk-adjusted analysis in comparative surgical audit. Br J Surg 2002;89:763–8.doi:10.1046/j.1365-2168.2002.02080.x
      OpenUrlCrossRefPubMedWeb of Science
    37. 37.↵
      2. Liu YE ,
      3. Hu S ,
      4. Chan SP , et al
      . The epidemiology and surgical outcomes of patients undergoing primary total hip replacement: an Asian perspective. Singapore Med J 2009;50:15–19.
      OpenUrlPubMedWeb of Science
    38. 38.↵
      Statistics Ministry of Health. https://www.moh.gov.sg/content/moh_web/home/statistics.html (accessed 21 Aug 2017).

    Footnotes

    • Contributors DXHC: data analysis and interpretation, prepared draft manuscript, revised draft manuscript and approved final manuscript for submission. YES: data acquisition and curation, data analysis and interpretation, revised draft manuscript and approved final manuscript for submission. YHC: data analysis and interpretation, revised draft manuscript and approved final manuscript for submission. RP: funding acquisition and approved final manuscript for submission. HRA: designed and conceptualised study, funding acquisition, data acquisition and curation, data analysis and interpretation, revised draft manuscript and approved final manuscript for submission. All the authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    • Funding This work was supported by the department fund of Department of Anaesthesiology, Singapore General Hospital. The funding sources had no role in the design of this study and the analysis and interpretation of the results.

    • Disclaimer The authors affirm that this manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

    • Competing interests HRA is a recipient of SingHealth Duke-NUS Nurturing Clinician Scientists Scheme Award, project number 12/FY2017/P1/15-A29.

    • Patient consent Not required.

    • Ethics approval This study has been approved by the SingHealth Centralised Institutional Review Board (CIRB).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi: 10.5061/dryad.v142481.