Objectives This study aimed to validate the performance of the Modified Early Warning Score (MEWS) in a Chinese emergency department and to determine the best cut-off value for in-hospital mortality prediction.
Design A prospective, single-centred observational cohort study.
Setting This study was conducted at a tertiary hospital in South China.
Participants A total of 383 patients aged 18 years or older who presented to the emergency department from 17 May 2017 through 27 September 2017, triaged as category 1, 2 or 3, were enrolled.
Outcomes The primary outcome was a composite of in-hospital mortality and admission to the intensive care unit. The secondary outcome was using MEWS to predict hospitalised and discharged patients.
Results A total of 383 patients were included in this study. In-hospital mortality was 13.6% (52/383), and transfer to the intensive care unit was 21.7% (83/383). The area under the receiver operating characteristic curve of MEWS for in-hospital mortality prediction was 0.83 (95% CI 0.786 to 0.881). When predicting in-hospital mortality with the cut-off point defined as 3.5, 158 patients had MEWS >3.5, with a specificity of 66%, a sensitivity of 87%, an accuracy of 69%, a positive predictive value of 28% and a negative predictive value of 97%, respectively.
Conclusion Our findings support the use of MEWS for in-hospital mortality prediction in patients who were triaged category 1, 2 or 3 in a Chinese emergency department. The cut-off value for in-hospital mortality prediction defined in this study was different from that seen in many other studies.
- ("modified early warning score") or "mews"
- ("emergency") and "triage"
- emergency department
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
This prospective observational study was carried out according to workflow, which is the most cost-effective option and reduces difficulty in data collection.
This study used a prospective study design and provided a new cut-off point for the Modified Early Warning Score (MEWS) using the receiver operating characteristic curve analysis to increase sensitivity in predicting in-hospital mortality.
This study evaluated the MEWS only once, on patient admission, which means that dynamic changes in the score could not be observed during patient hospitalisation.
This prospective cohort study recruited participants at a single medical centre, which could limit the generalisability of the study findings.
Different kinds of triage systems have been developed worldwide to assess the illness severity of patients presenting to emergency departments (EDs) who are assigned treatment priorities.1 2 In China, there is a lack of a unified triage standard to manage patients when they present to the ED.3 The triage standard used in hospitals in Shenzhen is a new four-level Chinese emergency triage criteria, published by the Public Hospital Administration of Shenzhen Municipality in August 2013.3 It categorises patients as near death (level 1), critically ill (level 2), acute (level 3) and not acute (level 4), requiring treatment immediately, in 10 min, in 30 min and in 4 hours, respectively. This is mainly decided according to patients’ presenting complaints and questions about potentially aggravating factors. According to acuity, level 1, level 2 and level 3 are urgent patients with a higher risk of serious adverse events, such as hospital admission and mortality, compared with level 4, which describes non-urgent patients.4 5
Therefore, an excellent scoring system is urgently required for mortality predictions in patients admitted to the ED. Today, there are a number of scoring systems designed to predict the chances of hospitalisation, intensive care unit (ICU) admission or in-hospital mortality in ED patients.6 7 The VitalPac Early Warning Score, Modified Early Warning Score (MEWS), Rapid Emergency Medicine Score, Emergency Department Sepsis Score and Rapid Acute Physiology Score are the most commonly employed systems for bedside evaluation.8–12
MEWS was introduced in 2001 by Subbe and colleagues,13 who modified it from the Early Warning Score (EWS). The MEWS is a simple physiological scoring system which includes five physiological parameters—systolic blood pressure, pulse rate, respiratory rate, temperature and level of consciousness—that can easily be collected at the moment of presentation. The MEWS is widely used in wards, ICU and EDs to detect the clinical deterioration of patients or to predict clinical outcomes.6 7 13
A large number of studies have reported that MEWS is an effective tool for in-hospital mortality prediction.14–17 However, there have also been studies conducted on different populations or in different areas reporting that MEWS is not an adequate scoring system for predicting in-hospital mortality.18 19 Moreover, the MEWS cut-off value (for in-hospital mortality prediction reported in studies) varies.9 10 15 20–22 A study conducted on 518 patients in ICU indicated that patients with MEWS ≥6 had significantly higher mortality than those with MEWS <6.22 However, another study, which examined the performance of MEWS in assessing non-traumatic critical patients in an ED, showed the MEWS cut-off value was 3.15 Therefore, this study hypothesises that MEWS performance and cut-off value may differ according to the specific population.
The MEWS is also used to evaluate patient conditions in Chinese EDs, including focusing on the relationship between factors and clinical outcomes, using prehospital MEWS to identify non-trauma patients requiring life-saving intervention and risk stratification of patients before interfacility transport.23–25 However, information on MEWS validation is limited to in-hospital mortality predictions in patients triaged as level 1, 2 or 3 in Chinese EDs. Hence, the aim of this study was to evaluate MEWS performance in predicting in-hospital mortality of the population in a Chinese emergency treatment room and to find the best cut-off value.
A prospective, single-centred observational cohort study was conducted in the ED of a tertiary hospital in Shenzhen, China to evaluate the ability of the MEWS to predict in-hospital mortality in patients presenting to the emergency treatment room who were categorised level 1, 2 or 3.
The study was carried out at a tertiary hospital, the First Affiliated Hospital of Shenzhen University, which saw 173 000 ED presentations in 2017. Of the 173 000 ED presentations, approximately 6600 patients were admitted to the emergency treatment room. Data of patients presenting to the emergency treatment room between 17 May 2017 and 27 September 2017 were collected. Patients aged 18 years or older triaged as categories 1, 2 and 3 were included in the study. Patients who had died prior to arrival in the ED and patients who needed ward admission, ICU admission or rescue according to the doctor’s judgement, or who ignored the doctor’s advice and left the hospital due to a variety of reasons, were excluded from the study. Patients with insufficient information were also excluded.
Sample size calculation
This study calculated the sample size using G*Power V.22.214.171.124 (http://www.softpedia.com/get/Science-CAD/G-Power.shtml). The estimated sample size was 319, with an accuracy index of 0.95 and a marginal error of 0.05 with 95% confidence level and 80% power.
Participant involvement and data collection
Patients who presented to our ED were evaluated and triaged by the triage nurse, who had more than 5 years of experience. Patients were triaged to near death (level 1), critically ill (level 2), acute (level 3) and not acute (level 4). This is decided according to the triage guidelines and the judgement of the triage nurse. According to acuity, patients triaged to levels 1 and 2 were sent to the emergency treatment room; patients triaged to level 3 were given priority in the consulting room or sent to the emergency treatment room if the triage nurse judged the patient’s condition to be serious; and patients triaged to level 4 were sent to wait outside the consulting room.
Physiological parameters were measured by nurses and researchers at the time of admittance to the emergency treatment room. Respiratory rate was counted manually for more than a full minute; heart rate and blood pressure were measured using an automatic electronic sphygmomanometer (HBP-9020) or multifunctional ECG monitor (Philips Jin Kewei, G30). Body temperature was measured using an infrared ear thermometer (PRO 4000). The level of consciousness was recorded as the best response to the AVPU score (A for alert, V for reacting to vocal stimulus, P for reacting to pain and U for unresponsive). Patient information was recorded using a questionnaire designed by the researchers. The following information was included: age, gender, nationality, educational background, mode of transportation to hospital, disease type, main diagnosis, body temperature, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, peripheral oxygen saturation, AVPU score, triage level, MEWS score (online supplementary appendix 1) and mortality.
The patients were followed up by the researcher until discharge, death or for a maximum of 90 days. The researchers calculated the MEWS using patients’ five recorded physiological parameters. In-hospital mortality was the main outcome. The predictive accuracy of the MEWS was evaluated by the receiver operating characteristic (ROC) curve. Sensitivity, specificity, accuracy, and positive and negative predictive values (PPV and NPV) were analysed to indicate the predictive power of the scoring system. The patients were divided into two groups: MEWS <4 and MEWS ≥4. The intergroup differences in the baseline characteristic physiological parameters and the scores between the two groups were also evaluated.
The primary outcome was a composite of in-hospital mortality and admission to the ICU. The secondary outcome of this study was using MEWS to predict admission to the general ward unit or discharge from hospital.
Descriptive statistics were tabulated for the overall sample. Mean and SD were calculated for continuous variables, and frequencies and percentages for all other categorical variables. Data distribution of each variable between the MEWS <4 and the MEWS ≥4 groups was compared. In addition, the area under the ROC curve (AUC) was measured to evaluate the predictive ability of the MEWS. Finally, sensitivity, specificity, accuracy, PPV and NPV were also analysed. Regression analysis was used to address confounding variables of age and gender. P<0.05 was regarded as statistically significant. EpiData V.3.1 was used for data entry, and then exported to tab-delimited text files. All analyses were performed using R (http://www.R-project.org) and EmpowerStats (www.empowerstats.com, X&Y Solutions, Boston, Massachusetts) software.
A total of 516 patients met the eligibility criteria, with 133 patients excluded. Among the patients who were excluded from the study, 10 had already died when they were sent to the ED, while 46 patients in the ED ignored the advice of doctors and left the hospital due to a variety of reasons. Another 65 patients left the hospital after being admitted to the ward or ICU. Twelve patients were excluded due to insufficient information (figure 1). Ultimately, 383 patients were enrolled in the study. Of this total, 255 (66.6%) patients were male; the mean age of all patients was 59.6±18.3 years, and the ethnicity of the majority of patients was Han Chinese (98.2%). Among the 383 patients, 52.5% and 21.7% were admitted to the ward and ICU from the ED, respectively. Nervous system, cardiovascular and respiratory diseases were the three most common disease types seen in these patients, consisting of more than half of the population. In the baseline characteristics between the MEWS <4 and MEWS ≥4 groups, a number of baseline characteristics showed significant differences, with p<0.05. Detailed patient baseline characteristics are shown in table 1.
The patients were divided into two groups: MEWS ≥4 and MEWS <4. Physiological parameters include body temperature, systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, percutaneous oxygen saturation and mental status, which were different between the two groups, and the difference was statistically significant. However, between the two groups, there were no differences in terms of blood sugar and length of stay. In addition, a total of 277 critically ill patients were triaged as level 1 and level 2, requiring treatment within 10 min. There were more critically ill patients in the MEWS ≥4 group than in the MEWS <4 group (150/158 vs 127/225, p<0.001). The proportion of in-hospital mortality was 13.6% (52/383), with most of these patients in the MEWS ≥4 group (7/52 vs 45/52, p<0.001). Detailed physiological parameters of the two groups are outlined in table 2.
The MEWS in-hospital mortality predictive ability is shown by AUC, at 0.83 (95% CI 0.786 to 0.881) (figure 2). Logistic regression analysis was used to examine the association between MEWS and the primary and secondary outcome measures. As seen in table 3, the MEWS was significantly associated with in-hospital mortality (OR, 1.65, 95% CI 1.44 to 1.89, p<0.001), admission to ICU (OR, 1.54, 95% CI 1.39 to 1.72, p<0.001), and predicting admission to a general ward unit or discharge from hospital (OR, 1.55, 95% CI 1.28 to 1.89, p<0.001) (table 3). The MEWS in-hospital mortality, predictive ability of admission to ICU and predictive ability of admission to a general ward unit or discharge from hospital are shown in figure 2–4, respectively.
In this observational cohort study, the MEWS showed good performance for in-hospital mortality prediction, with AUC values at 0.83. The higher the score, the higher the ratio of in-hospital mortality, indicating that MEWS was significantly correlated with patient mortality. In patients with MEWS ≥4, compared with MEWS <4, a number of variables, such as age, triage level, vital signs, means of arrival and disease type, are influencing factors of death in ED patients. The study demonstrated that MEWS is an effective tool for in-hospital mortality prediction for ED patients triaged to levels 1, 2 and 3.
MEWS is a widely used scoring system in many countries, but differences between these studies, including study setting, population and disease type, have led to differences in the predictive ability of the MEWS. The AUC, specificity and sensitivity were the most common indexes reported in studies on MEWS performance.17 20 26 27 A large proportion of studies have reported that MEWS is an effective tool for mortality prediction, with AUC ranging from approximately 0.70 to 0.89 for the most frequently used threshold (MEWS=5), and specificity and sensitivity ranging from 0.67 to 0.72, and from 0.65 to 0.71, respectively.9 17 20 26 However, less information was provided on the accuracy, PPV and NPV of the score. In a single-centre, observational cohort study conducted at an urban tertiary care medical centre in Chicago, adult patients who were suspected of contracting an infection in a hospital ward or ED were included.20 Discrimination for in-hospital mortality was moderate, with MEWS AUC of 0.73 (95% CI 0.71 to 0.74).
Furthermore, there are also studies demonstrating that MEWS is not an efficient system for mortality prediction, with an approximate AUC of less than 0.6, with study populations that included patients with sepsis admitted to medical wards, surgical patients presenting to EDs and adults admitted to medical wards, respectively.9 17 22 This study showed that disease and population differences seem to strongly determine MEWS performance. However, MEWS performance in ED patients who were triaged as level 1, 2 or 3 had previously not been validated. Our study found that mortality prediction for the MEWS is good (AUC, 0.83, 95% CI 0.79 to 0.88).
When combined with sensitivity and specificity, the maximum was defined as the best threshold. In this study, in order to increase the proportion of in-hospital mortality prediction and reduce missed diagnoses, sensitivity is more important than specificity. When the threshold was 4, the specificity, accuracy and NPV improved at the cost of sensitivity and PPV, and the number of deaths due to missed diagnosis increased from 6 to 16. Hence, this study defined the MEWS cut-off point as 4, which was different from a previous prospective study, whose MEWS cut-off point was defined as 3.15 However, the MEWS cut-off point defined as 3 in this study was different from that of many other studies, whose MEWS cut-off point was defined as 5 or higher.10 20–22 For the baseline characteristics of patients in this study, respiratory system diseases, digestive system diseases, circulatory system diseases and nervous system diseases were found in 70.7% of the population and 67.3% of non-survivors, with the median (IQR) MEWS at 3 (3). Different kinds of diseases and populations may have contributed to the difference. In general, our study provides evidence that the MEWS is an efficient system for in-hospital mortality prediction in an ED.
Limitations and implications for future research
There are several limitations in our study. First, this was a single-centre, observational cohort study at a tertiary hospital in Shenzhen. Patient outcomes may have been affected by the level of care provided by the hospital, and may therefore have also affected the performance of the MEWS for in-hospital mortality prediction. Second, the population included in this study was selected according to triage criteria that were only published in Shenzhen. Therefore, our study results may not be generalisable to other settings. Third, we evaluated the MEWS only once, on patient admission. Dynamic changes in the score could not be observed during patient hospitalisation. Hence, we could not exclude the possibility that re-evaluation of this clinical score during hospitalisation may have improved or reduced the MEWS performance in this setting. In future, a multicentre study should be conducted to reduce the effect of the sample size not being representative. In addition, due to the varied performance of MEWS in other studies, research on specific diseases is also required in order for the use of MEWS to be more accurate. While the actual number of enrolled subjects was 383 (higher than the required sample size of 319), we excluded 133 patients in the analysis due to missing data, resulting in potential selection bias. Thus, future research should implement strategies to minimise missing data in patient report forms.
This study found that MEWS was an accurate score for predicting in-hospital mortality and admission to ICU in a Chinese ED. Future multicentric prospective cohort studies are needed to validate the study findings. As patients with MEWS equal to or higher than 4 had higher rates of in-hospital mortality and ICU admission, calculating MEWS may be an important indicator for closely monitoring patients, making an immediate request to contact the doctor in charge and establishing a rapid response intervention team. In this hospital, the ED triage system has already added MEWS as one of the vital parameter monitors and designed an algorithm in the triage system that can automatically calculate MEWS.
We sincerely thank Xinglin Chen for her excellent assistance with the statistical analyses. We also thank all patients for participating in this study.
Patient consent for publication Obtained.
Contributors Study design: XX, WH and QL. Data acquisition: WH. Data analysis and interpretation: WH, LP, LW, JZ, YW and YZ. Project administration: XX and WT. Manuscript first draft: XX and WH. Statistical analysis: WH. Manuscript revision: WT and YZ.
Funding This work was supported by the Science and Technology Department of Guangdong Province, China (grant number 2014A020212574).
Competing interests None declared.
Ethics approval The study was approved by the ethical committee of the Second People’s Hospital of Shenzhen (no 20141201005).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement We are glad to share data collected in this study upon request from the corresponding author.