You are here

Article Text

Article menu
XML
Evidence based practice
Original research
Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review
  1. Baneen Alhmoud1,
  2. Timothy Bonnici1,2,
  3. Riyaz Patel2,3,4,
  4. Daniel Melley4,
  5. Bryan Williams2,3,
  6. Amitava Banerjee1,2,4
  1. 1 Institute of Health Informatics, University College London, London, UK
  2. 2 University College London Hospitals NHS Trust, London, UK
  3. 3 Institute of Cardiovascular Science, University College London, London, UK
  4. 4 Barts Health NHS Trust, London, UK
  1. Correspondence to Dr Amitava Banerjee; ami.banerjee{at}ucl.ac.uk

Abstract

Objective To assess predictive performance of universal early warning scores (EWS) in disease subgroups and clinical settings.

Design Systematic review.

Data sources Medline, CINAHL, Embase and Cochrane database of systematic reviews from 1997 to 2019.

Inclusion criteria Randomised trials and observational studies of internal or external validation of EWS to predict deterioration (mortality, intensive care unit (ICU) transfer and cardiac arrest) in disease subgroups or clinical settings.

Results We identified 770 studies, of which 103 were included. Study designs and methods were inconsistent, with significant risk of bias (high: n=16 and unclear: n=64 and low risk: n=28). There were only two randomised trials. There was a high degree of heterogeneity in all subgroups and in national early warning score (I2=72%–99%). Predictive accuracy (mean area under the curve; 95% CI) was highest in medical (0.74; 0.74 to 0.75) and surgical (0.77; 0.75 to 0.80) settings and respiratory diseases (0.77; 0.75 to 0.80). Few studies evaluated EWS in specific diseases, for example, cardiology (n=1) and respiratory (n=7). Mortality and ICU transfer were most frequently studied outcomes, and cardiac arrest was least examined (n=8). Integration with electronic health records was uncommon (n=9).

Conclusion Methodology and quality of validation studies of EWS are insufficient to recommend their use in all diseases and all clinical settings despite good performance of EWS in some subgroups. There is urgent need for consistency in methods and study design, following consensus guidelines for predictive risk scores. Further research should consider specific diseases and settings, using electronic health record data, prior to large-scale implementation.

PROSPERO registration number PROSPERO CRD42019143141.

  • adult intensive & critical care
  • risk management
  • clinical governance
  • epidemiology

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2020-045849

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • The first systematic review to investigate the performance of early warning scores (EWS) in different patient disease subgroups and clinical settings.

    • Meta-analysis was performed for different EWS and national EWS validation studies in different disease and clinical setting subgroups.

    • This study is limited to specific diseases and settings and does not consider the use of EWS in the general population.

    • Analysis of predictive accuracy of EWS is based on area under the curve, not other validation measures.

    • During the study period 1997–2019, approaches to EWS and their validation have changed.

    Introduction

    Across diseases, patient deterioration can range from critical care review and sepsis to cardiorespiratory arrest and death.1 2 Delays or failures in timely detection of deterioration adversely affect prognosis, morbidity, mortality and healthcare utilisation.3 For example, the 20 000 in-hospital cardiac arrests per year in England are associated with costs of £50 million for resuscitation and postarrest care.4 Around the world, earlier recognition and prevention of deterioration in unwell patients has far-reaching implications for reduction in mortality and morbidity, reduction in the cost of healthcare and allocation of scarce high dependency and critical care resources. Preventive interventions are needed to overcome these challenges.5

    Specific characteristics have long been known to be associated with deteriorating patient health,2 5–8 including physiological parameters, such as heart rate and blood pressure.5 9–12 Early warning scores (EWS), widely used in high-income countries, were borne out of the need for early detection of patient deterioration. EWS are tools derived from prediction models that assess patient characteristics and physiological parameters to stratify the risk of developing a worsening event or need for medical attention.13 The algorithms underlying EWS can be ‘aggregate-weighted’ to sum up a set of parameters to produce a score or use more advanced statistical modelling.14 EWS inform clinical decision making, enabling escalation of attention and care when required. Universal tools, such as the modified early warning score (MEWS),15 were developed for use across different hospital settings, but specialised, non-standard EWS are also designed for particular subgroups, for example, Rapid Emergency Medicine Score16 and Quick Sequential Organ Failure Assessment (qSOFA)17 for patients with infections. In recognising different settings, EWS may have compromised simplicity and timeliness of assessment.13 For example, a number of EWS rely on parameters that do not exist in the first hours of assessment, such as blood investigations and imaging.1 18 19

    From fragmented implementation and inadequate early assessment via specialised tools, EWS have shifted back to universal prediction models, particularly, the national early warning score (NEWS),20 followed by NEWS2.21 NEWS was designed to produce a universal assessment of acute illness severity across the National Health Service (NHS).22 While showing good discrimination compared with other EWS, especially in predicting mortality, there was a need to accommodate additional clinical parameters in the score. The updated NEWS2, emphasising appropriate scoring for type 2 respiratory failure, confusion and severe sepsis,21 was formally endorsed by NHS England23 to be the EWS used in acute care. However, there have been concerns regarding excessive calls to clinicians, administrative workload and variable symptoms across diseases and settings.24 The effectiveness of the universal EWS (box 1) with standardised use across all settings is not clear in specific disease populations25 and requires validation to estimate discrimination and calibration, like other clinical prediction models.26 While internal validation is useful, generalisability and reproducibility needs external validation.27

    Box 1

    Definitions

    • Universal early warning scores (EWS): EWS that are globally adopted and applicable in every setting and for any disease subgroup.

    • Standardised EWS: EWS model with a set of parameters used in a unified approach to predict deterioration in any patient subgroup.8 23

    • External validation: evaluation of the model’s predictive accuracy with data different than the one sued for model development.27

    • Internal validation: evaluation of a model’s predictive accuracy with the same data set used for the development or in a population in which the model is intended for use.27

    • Discrimination: the ability of a model to distinguish between the patients who will develop an outcome of interest and the ones who will not.26

    • Calibration: the accuracy of risk estimates in relation to the observed number of events.73

    Systematic reviews have evaluated EWS in prehospital, intensive care unit (ICU) and general settings,3 28 29 and sepsis,15 with narrow inclusion criteria and inadequate assessment of study quality. A recent systematic review evaluated development and validation of EWS in general patients but did not include studies in specific disease subgroups or settings.30

    Objective

    In a systematic review, we will assess performance of universal EWS in particular diseases and clinical settings in predicting mortality, transfer to ICU and cardiac arrest.

    Methods

    Search strategy

    The protocol adhered to Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines.31 Published articles were identified in MEDLINE, CINHAL and Embase, between 1997 (initial development of EWS) and 2019. The Cochrane database was searched for systematic reviews (CDSR) and trials (CENTRAL). For grey literature, Google Scholar was searched. During the screening procedure, studies were added from references in review articles and studies. Search strategies were developed by two authors (BA and AB) and reviewed by a third author (TB). Terms used for searching databases include terms for early warning or track and trigger scores and acronyms, identified subgroups and settings (eg, Medical Subject Heading (MeSH)) and free-text search terms (figure 1; online supplemental methods).

    Supplemental material

    Figure 1
    Figure 1

    Search strategy and included studies regarding universal early warning scores (EWS) in different disease subgroups and clinical settings.

    Inclusion and exclusion criteria

    Patient subgroups were identified according to disease categories and clinical settings (online supplemental methods). Studies were included if: (1) validation of a universal EWS with standardised prediction model in adult patients; (2) EWS validation was in a specific setting or disease; (3) the performance of the EWS, or the impact on mortality, transfer to ITU and cardiac arrest, was examined; and (4) they were prospective or retrospective cohort, cross-sectional, case–control design or trials.

    Studies were excluded if: (1) patients were less than 16 years of age; (2) EWS performance was only examined in derivation, not validation; (3) non-universal EWS was developed for a specific subgroup, for example, obstetric early warning score for obstetric patients or qSOFA for patients with infections; or (4) EWS validation was performed in a general patient dataset or setting, for example, validation in a general hospital without consideration of hospital subgroups.

    Data extraction

    Articles were screened by title and abstract by one author (BA), then full-text screening was by two reviewers (BA and AB). Data were extracted independently by two reviewers (BA and AB) using a standardised and piloted data form. A third reviewer (TB) resolved any disagreements. Items for extraction for studies examining predictive accuracy were based on the CHARMS32 checklist, except for tool derivation, which was excluded.

    Quality assessment

    Risk of biases in validation studies was assessed using Prediction model Risk Of Bias ASsessment Tool (PROBAST),33 which classifies studies as low, unclear or high risk of bias in four aspects: participant selection, predictors, outcomes and analysis within the overall risk of bias and the study applicability domains.

    Evidence synthesis

    We conducted the analysis using MS Excel and R programs. We summarised the results using descriptive statistics and graphical plots. Meta-analysis was performed, in different subgroups, using area under the curve (AUC) for identified universal EWS and for NEWS in studies. Fisher-Z transformation for correlation coefficients was conducted for AUC into normally distributed Z with 95% CI to evaluate the effect size and test for the heterogeneity. Where applicable, narrative synthesis was conducted.

    Patient and public involvement

    Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

    Results

    Study characteristics

    Of the 16 181 articles identified by our search, we screened 1355 articles by title and abstract, assessing 770 articles in full for eligibility. We included 103 studies, published between 2006 and 2019, in the final stage. These studies were predominantly observational (retrospective=65, prospective=36 and RCT=2). Emergency department (ED) (n=48) was the most common clinical setting, followed by medical (n=12), ICU (n=12) and surgical (n=9) settings. Sepsis (n=33) was the most common disease subgroup. Other subgroups ranged from respiratory (n=8) to renal (n=1)(figures 1 and 2). Mortality was the main studied outcome. Cardiac arrest was infrequently studied (n=8).

    Figure 2
    Figure 2

    Number of studies regarding performance of early warning scores in different disease subgroups and clinical settings. Each bubble represents the disease subgroup and/or setting where different early warning scores were examined. The size of the bubble represents the number of studies (n), and overlapping bubbles show studies where disease subgroup and settings overlap. CVD, cardiovascular diseases; ED, emergency department; GiI, gastrointestinal diseases; ICU, intensive care unit.

    Quality assessment

    There was a significant risk of bias found in majority of studies (high risk=16; unclear risk=64) and low risk in only 28 studies. In terms of applicability, narrow inclusion of conditions in a certain disease group was commonly related to risk of bias, while in general settings, biases were often due to low sample size or unspecified timing of EWS assessment. There was a wide variation in sample size (median: 551 and range: 43–920 029). There was variation in defining study population by number of patients, hospital admissions or not specifying the particular study sample. Almost half of the studies (n=49; 48%) validated in <500 patients with either multiple observations or a single observation set (tables 1–4). External validation was more common (n=83) than internal validation (n=18), and two studies included internal and external validations (online supplemental table S1).

    View this table:
    Table 1

    Studysetting and design for included studies of predictive performance for early warning scores in patient subgroups and settings

    View this table:
    Table 2

    Early warning scores, predictive measures and outcomes for included studies in patient subgroups and settings

    View this table:
    Table 3

    Study design and setting forincluded studies of predictive performance for early warning scores in clinical settings

    View this table:
    Table 4

    Early warning scores, predictive measures and outcomes for included studies of predictive performance for early warning scores in clinical settings

    EWS validation in patient subgroups

    Subgroups and EWS

    In the studies validating EWS, there was heterogeneity in subgroup definitions, models and methods of predictive accuracy. There was overlap between diseases and settings commonly between studies of patients with infections receiving care in ED34–36 and patients with sepsis admitted to ICU.37 38 EWS models that were integrated with electronic health records (EHRs) were examined in recent studies (n=9). Research on datasets using EWS-embedded EHRs had larger sample sizes, ranging from 50439 to 13 014 patients40 (tables 1–4), with moderate to high predictive ability (AUC: 0.65–0.85). Several studies included comparison between different EWS in the same cohort (n=21)35 38 41 (online supplemental table S2).

    Methodology

    There was significant heterogeneity in methods across studies. The majority of studies were observational. Evaluation of predictive accuracy of different EWS in the same study was common.22 42–44 To measure accuracy of EWS, AUC was most commonly used (n=94), especially when comparing different EWS in the same study.22 45 Presentation of results was variable; for example, confidence intervals were missing in many studies. Other measures, such as analysing sensitivity and specificity, prognostic index and ORs, were found in only eight studies (tables 1–4). Consequently, it was only feasible to analyse predictive accuracy in studies where AUC was the selected measure.

    Timing from EWS assessment to endpoints was variable. Many studies included (n=43) AUC within 24–48 hours, while 11 studies had endpoints more than 48 hours after EWS. However, the majority (n=65; 63%) did not specify time horizon or in-hospital outcome.

    Predictive performance of EWS

    Outcomes were most commonly mortality, transfer to ICU, developing sepsis (in patients with infections) and cardiac arrest. Few studies examined other outcomes, for example, respiratory arrest (n=1) and organ failure (n=4). Mortality, ICU admission and cardiac arrest were best predicted in medical (AUC mean: 0.74, 0.75 and 0.74)46–48 and surgical settings (0.80, 0.79 and 0.75),49 50 and respiratory diseases (0.75, 0.80 and 0.75), respectively. EWS prediction of sepsis had reasonable predictive performance in all subgroups (AUC: 0.71–0.79) and infectious diseases in particular (AUC: 0.79). Certain outcomes related to specific disease groups were not studied, for example, cardiac arrest was not studied in cardiac patients22; respiratory arrest was not tested in respiratory patients.46–52

    The best predictive performance was found in studies examining cardiac,46 stroke46 53 and renal46 diseases (AUC: 0.93, 0.88 and 0.87, respectively). In emergency settings, predictive accuracy was variable (AUC: 0.56–0.91).54–58 In haematology and oncology diseases, EWS predictive accuracy was suboptimal in mortality (online supplemental figure S1), cardiac arrest and ICU transfer (AUC: 0.52–0.69; figures 3 and 4).59–61 EWS prediction of ICU transfer was reasonable in ED,57 62 infectious diseases,63 64 and where both groups overlap,42 65 but not in gastroenterology and haematology (AUC: 0.64 and 0.60)60 66 (online supplemental figure S2) Cardiac arrest was the least examined outcome among the three endpoints (n=8) and unstudied in cardiac diseases (figures 3 and 4, online supplemental figure S3).

    Figure 3
    Figure 3

    Early warning score performance in different disease subgroups. Each bubble represents critical events predicted by early warning scores for each disease subgroup with average AUC of studies beside each event type. The size of the bubble represents the number of studies in each subgroup. CA, cardiac arrest; CVD, cardiovascular diseases; GI, gastrointestinal diseases; ICU, intensive care unit; OF, organ failure; RA, respiratory arrest.

    Figure 4
    Figure 4

    Early warning score performance in different clinical settings. Each bubble represents critical events predicted by early warning scores for each disease subgroup with average AUC of studies beside each event type. The size of the bubble represents the number of studies in each subgroup. CA, cardiac arrest; ED, emergency department; ICU, intensive care units; OF, organ failure; RA, respiratory arrest.

    For mortality prediction, meta-analysis of included EWS showed high degree of statistical heterogeneity across all subgroups (I2=72%–99%) (figure 5). In validation studies of NEWS in different disease subgroups, there was also significant heterogeneity (I2=99%; figure 6).

    Figure 5
    Figure 5

    Forest plot of predictive accuracy of universal early warning scores (EWS) for mortality in different disease subgroups and clinical settings. CVD, cardiovascular diseases; ED, emergency department; GI, gastrointestinal diseases; Hem, haematological diseases; ICU, intensive care units; Infec, infectious diseases; Med, medical settings; Onco, oncology diseases; Renal, renal diseases; renal diseases; Resp, respiratory diseases; stroke, patients who had a stroke; Surg, surgical settings . Note: number following author(s) and year indicate more than one EWS evaluated in the study.

    Figure 6
    Figure 6

    Forest plot of predictive accuracy of NEWS for mortality. AUC, area under the curve; NEWS, national early warning score.

    Discussion

    In this comprehensive review of universal EWS across all diseases and settings, we had three main findings. First, EWS studies in different diseases and clinical settings were heterogeneous in methodology, predictive performance measures and number of studies in each subgroup. Second, validation of EWS is limited in specialised settings, including cardiac disease. Third, despite widespread EHR and EWS integration, few studies have explored EHR-based EWS.

    Inconsistency in evaluation and the lack of high-quality validation make the evidence of validity questionable, ultimately affects how EWS can and should be used in clinical practice as a risk score for deterioration prediction. Heterogeneity across studies in all subgroups challenges implementation of EWS in all diseases and all settings. In methodology, observations selections method, time horizon between EWS score and event and the metric used in assessment were inconsistent. Choosing multiple observations or a single observation prior the outcome may not significantly affect the ranking of EWS.67 Yet, selecting a single observation is generally associated with high AUC compared with multiple observations,46 67 supporting the use of multiple observations for each episode. Moreover, AUC, the most commonly used measure of predictive performance, has limitations and other metrics, including positive predictive value, should also be assessed.68 Recording observations at an agreed threshold point before events in a standardised method is necessary to evaluate EWS effectively.

    The universal EWS with standardised models were primarily designed for general patient populations in wards and EDs and remain underevaluated in specific diseases and settings. In medical and ED contexts, EWS perform well, suggesting the role of EWS in general settings, or at the early stage of clinical assessment. Our positive findings in respiratory disease may indicate the emphasis of several EWS, such as NEWS2, on respiratory changes when patients are deteriorating. Specific disease areas may show unique alarm signs when critical events are anticipated, which may not be captured by universal EWS, such as NEWS2, where prediction of deterioration is based on predefined thresholds in all patients.23 Critical events are commonly associated with CVD. With CVD being a leading cause of mortality globally, and the significant impact of morbidity on health and social care, early detection of deterioration is necessary.69 However, EWS are poorly validated in CVD, some of the parameters may not be applicable and EWS may be unrepresentative.25 A recent study of NEWS2 in patients with COVID-19 infection found poor performance in severity prediction,70 despite pre-existing conditions being common and predictive in patients with severe outcomes. EWS may need to take account of disease-specific risk factors and comorbidities.

    Widespread uptake of EHR and digitisation of patient observations are expected to contribute to efficient use of EWS by reducing human errors in documentation and calculation, as well as delays in escalation of care. However, relatively few studies have considered EHR-based EWS, and those studies have not analysed whether predictive performance of EWS is related to EHR use, diseases or settings. Investigating implementation and adoption of EWS is necessary to understand the application and performance of EWS. Predictive algorithms derived by machine learning have been successfully used in developing and validating EWS41 71 but will require robust evaluation. Studying the implementation process of EWS within EHR will provide opportunities for qualitative and quantitative insights into escalation of care, as well as facilitators and barriers to use of EWS in routine practice.

    There are several limitations in this review and in included studies. We aimed for a comprehensive investigation of all EWS developing since 1997, but this long study period may lead to bias in comparing studies with old and new validation approaches statistically and technically. We excluded EWS specifically derived and validated for particular disease populations or settings and excluded studies considering a general patient population. Meta-analysis was only done for studies using AUC, excluding other methods for assessing performance of EWS. The distinction between general patient settings and specific disease or patient subgroups is dependent on hospital, healthcare system and country, and there is inevitably overlap between patients and settings at different stages in patient pathways. It was only feasible to include studies with a clear disease or setting identified to avoid confusion.

    Validation of EWS in disease subgroups should consider similarities and differences across diseases, sample size and include measures of model discrimination and calibration. Further research should adhere to established guidelines on clinical outcomes and predictive clinical scoring for decision making, such as the PROGRESS framework.72

    Conclusion

    Universal EWS in specific disease subgroups and settings require further validation of their performance in detecting worsening outcomes. Despite good performance in respiratory patients and medical and surgical settings in studies to date, the predictive accuracy of EWS in all disease subgroups and all clinical settings remains unknown. The current evidence base does not necessarily support use of standard EWS in all patients in all settings. Future research should include validation of EWS in particular patient subgroups and settings, with standardised methodology following established guidelines. Going towards the utilisation of EHR for EWS development, validation and implementation within EHR should be considered for improved EWS systems.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as supplementary information.

    References

    1. ↵
      2. Cetınkaya HB ,
      3. Koksal O ,
      4. Sigirli D , et al
      . The predictive value of the modified early warning score with rapid lactate level (ViEWS-L) for mortality in patients of age 65 or older visiting the emergency department. Intern Emerg Med 2017;12:1253–7.doi:10.1007/s11739-016-1559-7 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27804076
      OpenUrlPubMed
    2. ↵
      2. Cei M ,
      3. Bartolomei C ,
      4. Mumoli N
      . In-hospital mortality and morbidity of elderly medical patients can be predicted at admission by the modified early warning score: a prospective study. Int J Clin Pract 2009;63:591–5.doi:10.1111/j.1742-1241.2008.01986.x pmid:http://www.ncbi.nlm.nih.gov/pubmed/19220521
      OpenUrlCrossRefPubMed
    3. ↵
      2. Alam N ,
      3. Hobbelink EL ,
      4. van Tienhoven AJ , et al
      . The impact of the use of the early warning score (EWS) on patient outcomes: a systematic review. Resuscitation 2014;85:587–94.doi:10.1016/j.resuscitation.2014.01.013 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24467882
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      2. Hogan H ,
      3. Hutchings A ,
      4. Wulff J , et al
      . Interventions to reduce mortality from in-hospital cardiac arrest: a mixed-methods study. Health Serv Deliv Res 2019;7:1–110.doi:10.3310/hsdr07020
      OpenUrl
    5. ↵
      2. Adhikari NKJ ,
      3. Fowler RA ,
      4. Bhagwanjee S , et al
      . Critical care and the global burden of critical illness in adults. Lancet 2010;376:1339–46.doi:10.1016/S0140-6736(10)60446-1 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20934212
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      2. Hogan H ,
      3. Healey F ,
      4. Neale G , et al
      . Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual Saf 2012;21:737–45.doi:10.1136/bmjqs-2011-001159 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22927487
      OpenUrlAbstract/FREE Full Text
    7. ↵
      2. De Meester K ,
      3. Das T ,
      4. Hellemans K , et al
      . Impact of a standardized nurse observation protocol including MEWS after intensive care unit discharge. Resuscitation 2013;84:184–8.doi:10.1016/j.resuscitation.2012.06.017 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22796310
      OpenUrlCrossRefPubMed
    8. ↵
      2. Paterson R ,
      3. MacLeod DC ,
      4. Thetford D , et al
      . Prediction of in-hospital mortality and length of stay using an early warning scoring system: clinical audit. Clin Med 2006;6:281–4.doi:10.7861/clinmedicine.6-3-281 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16826863
      OpenUrlAbstract/FREE Full Text
    9. ↵
      2. Moon A ,
      3. Cosgrove JF ,
      4. Lea D , et al
      . An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR. Resuscitation 2011;82:150–4.doi:10.1016/j.resuscitation.2010.09.480 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21056524
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      2. Kause J ,
      3. Smith G ,
      4. Prytherch D , et al
      . A comparison of antecedents to cardiac arrests, deaths and emergency intensive care admissions in Australia and New Zealand, and the United Kingdom--the ACADEMIA study. Resuscitation 2004;62:275–82.doi:10.1016/j.resuscitation.2004.05.016 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15325446
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      2. Hillman KM ,
      3. Bristow PJ ,
      4. Chey T , et al
      . Duration of life-threatening antecedents prior to intensive care admission. Intensive Care Med 2002;28:1629–34.doi:10.1007/s00134-002-1496-y pmid:http://www.ncbi.nlm.nih.gov/pubmed/12415452
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      2. Wilkinson K ,
      3. Martin IC ,
      4. Gough MJ
      . National confidential enquiry into patient outcome and death. An age old problem. In: A review of the care received by elderly patients undergoing surgery. London: NCEPOD, 2011.
    13. ↵
      2. Morgan RJM ,
      3. Williams F ,
      4. Wright MM
      . An early warning scoring system for detecting developing critical illness. Clin Intensive Care 1997;8:100.
      OpenUrl
    14. ↵
      2. Linnen DT ,
      3. Escobar GJ ,
      4. Hu X , et al
      . Statistical modeling and aggregate-weighted scoring systems in prediction of mortality and ICU transfer: a systematic review. J Hosp Med 2019;14:161–9.doi:10.12788/jhm.3151 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30811322
      OpenUrlCrossRefPubMed
    15. ↵
      2. Hamilton F ,
      3. Arnold D ,
      4. Baird A , et al
      . Early warning scores do not accurately predict mortality in sepsis: a meta-analysis and systematic review of the literature. J Infect 2018;76:241–8.doi:10.1016/j.jinf.2018.01.002 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29337035
      OpenUrlPubMed
    16. ↵
      2. Wuytack F ,
      3. Meskell P ,
      4. Conway A , et al
      . The effectiveness of physiologically based early warning or track and trigger systems after triage in adult patients presenting to emergency departments: a systematic review. BMC Emerg Med 2017;17:38. doi:10.1186/s12873-017-0148-z pmid:http://www.ncbi.nlm.nih.gov/pubmed/29212452
      OpenUrlPubMed
    17. ↵
      2. Plevin R ,
      3. Callcut R
      . Update in sepsis guidelines: what is really new? Trauma Surg Acute Care Open 2017;2:e000088. doi:10.1136/tsaco-2017-000088 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29766091
      OpenUrlPubMed
    18. ↵
      2. Mohammed MA ,
      3. Rudge G ,
      4. Watson D , et al
      . Index blood tests and national early warning scores within 24 hours of emergency admission can predict the risk of in-hospital mortality: a model development and validation study. PLoS One 2013;8:e64340. doi:10.1371/journal.pone.0064340 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23734195
      OpenUrlPubMed
    19. ↵
      2. Vorwerk C ,
      3. Loryman B ,
      4. Coats TJ , et al
      . Prediction of mortality in adult emergency department patients with sepsis. Emerg Med J 2009;26:254–8.doi:10.1136/emj.2007.053298 pmid:http://www.ncbi.nlm.nih.gov/pubmed/19307384
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Royal College of Physicians of London
      . National early warning score (news): standardising the assessment of acute-illness severity in the NHS. R Coll Physician 2012.
    21. ↵
      1. Royal College of Physicians of London
      . Nhs England approves use of national early warning score (news) 2 to improve detection of acutely ill patients. R Coll Physician 2017.
    22. ↵
      2. Smith GB ,
      3. Prytherch DR ,
      4. Meredith P , et al
      . The ability of the National early warning score (news) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 2013;84:465–70.doi:10.1016/j.resuscitation.2012.12.016 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23295778
      OpenUrlCrossRefPubMed
    23. ↵
      2. Inada-Kim M ,
      3. Nsutebu E
      . News 2: an opportunity to standardise the management of deterioration and sepsis. BMJ 2018;360:k1260.doi:10.1136/bmj.k1260 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29559439
      OpenUrlFREE Full Text
    24. ↵
      2. Direkze S ,
      3. Jain S
      . Time to intervene? lessons from the NCEPOD cardiopulmonary resuscitation report 2012. Br J Hosp Med 2012;73:585–7.doi:10.12968/hmed.2012.73.10.585
      OpenUrlCrossRef
    25. ↵
      2. Badreldin AMA ,
      3. Doerr F ,
      4. Bender EM , et al
      . Rapid clinical evaluation: an early warning cardiac surgical scoring system for hand-held digital devices. Eur J Cardiothorac Surg 2013;44:992–8.doi:10.1093/ejcts/ezt232 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23756348
      OpenUrlCrossRefPubMed
    26. ↵
      2. Altman DG ,
      3. Royston P
      . What do we mean by validating a prognostic model? Stat Med 2000;19:453–73.doi:10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/10694730
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      2. Debray TPA ,
      3. Vergouwe Y ,
      4. Koffijberg H , et al
      . A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol 2015;68:279–89.doi:10.1016/j.jclinepi.2014.06.018 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25179855
      OpenUrlCrossRefPubMed
    28. ↵
      2. Smith MEB ,
      3. Chiovaro JC ,
      4. O'Neil M ,
      5. O’Neil M , et al
      . Early warning system scores for clinical deterioration in hospitalized patients: a systematic review. Ann Am Thorac Soc 2014;11:1454–65.doi:10.1513/AnnalsATS.201403-102OC pmid:http://www.ncbi.nlm.nih.gov/pubmed/25296111
      OpenUrlCrossRefPubMed
    29. ↵
      2. Williams TA ,
      3. Tohira H ,
      4. Finn J , et al
      . The ability of early warning scores (EWS) to detect critical illness in the prehospital setting: a systematic review. Resuscitation 2016;102:35–43.doi:10.1016/j.resuscitation.2016.02.011 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26905389
      OpenUrlPubMed
    30. ↵
      2. Gerry S ,
      3. Bonnici T ,
      4. Birks J , et al
      . Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology. BMJ 2020;369:m1501.doi:10.1136/bmj.m1501 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32434791
      OpenUrlAbstract/FREE Full Text
    31. ↵
      2. Moher D ,
      3. Liberati A ,
      4. Tetzlaff J , et al
      . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. doi:10.1371/journal.pmed.1000097 pmid:http://www.ncbi.nlm.nih.gov/pubmed/19621072
      OpenUrlPubMed
    32. ↵
      2. Moons KGM ,
      3. de Groot JAH ,
      4. Bouwmeester W , et al
      . Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the charms checklist. PLoS Med 2014;11:e1001744. doi:10.1371/journal.pmed.1001744 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25314315
      OpenUrlPubMed
    33. ↵
      2. Wolff RF ,
      3. Moons KGM ,
      4. Riley RD , et al
      . PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019;170:51. doi:10.7326/M18-1376 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30596875
      OpenUrlPubMed
    34. ↵
      2. Brink A ,
      3. Alsma J ,
      4. Verdonschot RJCG , et al
      . Predicting mortality in patients with suspected sepsis at the emergency department; a retrospective cohort study comparing qSOFA, SIRS and national early warning score. PLoS One 2019;14:e0211133. doi:10.1371/journal.pone.0211133 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30682104
      OpenUrlPubMed
    35. ↵
      2. Churpek MM ,
      3. Snyder A ,
      4. Sokol S , et al
      . Investigating the impact of different suspicion of infection criteria on the accuracy of quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores. Crit Care Med 2017;45:1805–12.doi:10.1097/CCM.0000000000002648 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28737573
      OpenUrlPubMed
    36. ↵
      2. Çıldır E ,
      3. Bulut M ,
      4. Akalın H , et al
      . Evaluation of the modified MEDS, MEWS score and Charlson comorbidity index in patients with community acquired sepsis in the emergency department. Intern Emerg Med 2013;8:255–60.doi:10.1007/s11739-012-0890-x pmid:http://www.ncbi.nlm.nih.gov/pubmed/23250543
      OpenUrlCrossRefPubMed
    37. ↵
      2. Siddiqui S ,
      3. Chua M ,
      4. Kumaresh V , et al
      . A comparison of pre ICU admission SIRS, EWS and Q SOFA scores for predicting mortality and length of stay in ICU. J Crit Care 2017;41:191–3.doi:10.1016/j.jcrc.2017.05.017 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28575814
      OpenUrlPubMed
    38. ↵
      2. Khwannimit B ,
      3. Bhurayanontachai R ,
      4. Vattanavanit V
      . Comparison of the accuracy of three early warning scores with SOFA score for predicting mortality in adult sepsis and septic shock patients admitted to intensive care unit. Heart Lung 2019;48:240–4.doi:10.1016/j.hrtlng.2019.02.005 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30902348
      OpenUrlPubMed
    39. ↵
      2. Vaughn JL ,
      3. Kline D ,
      4. Denlinger NM , et al
      . Predictive performance of early warning scores in acute leukemia patients receiving induction chemotherapy. Leuk Lymphoma 2018;59:1498–500.doi:10.1080/10428194.2017.1376744 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28901797
      OpenUrlCrossRefPubMed
    40. ↵
      2. Henry KE ,
      3. Hager DN ,
      4. Pronovost PJ , et al
      . A targeted real-time early warning score (TREWScore) for septic shock. Sci Transl Med 2015;7:299ra122–299.doi:10.1126/scitranslmed.aab3719 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26246167
      OpenUrlAbstract/FREE Full Text
    41. ↵
      2. Pimentel MAF ,
      3. Redfern OC ,
      4. Gerry S , et al
      . A comparison of the ability of the National early warning score and the National early warning score 2 to identify patients at risk of in-hospital mortality: a multi-centre database study. Resuscitation 2019;134:147–56.doi:10.1016/j.resuscitation.2018.09.026 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30287355
      OpenUrlCrossRefPubMed
    42. ↵
      2. Churpek MM ,
      3. Snyder A ,
      4. Han X , et al
      . Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med 2017;195:906–11.doi:10.1164/rccm.201604-0854OC pmid:http://www.ncbi.nlm.nih.gov/pubmed/27649072
      OpenUrlCrossRefPubMed
    43. ↵
      2. de Groot B ,
      3. Stolwijk F ,
      4. Warmerdam M , et al
      . The most commonly used disease severity scores are inappropriate for risk stratification of older emergency department sepsis patients: an observational multi-centre study. Scand J Trauma Resusc Emerg Med 2017;25:91. doi:10.1186/s13049-017-0436-3 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28893325
      OpenUrlPubMed
    44. ↵
      2. Smith GB ,
      3. Prytherch DR ,
      4. Schmidt PE , et al
      . Review and performance evaluation of aggregate weighted 'track and trigger' systems. Resuscitation 2008;77:170–9.doi:10.1016/j.resuscitation.2007.12.004 pmid:http://www.ncbi.nlm.nih.gov/pubmed/18249483
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      2. Smith GB ,
      3. Prytherch DR ,
      4. Schmidt PE , et al
      . A review, and performance evaluation, of single-parameter "track and trigger" systems. Resuscitation 2008;79:11–21.doi:10.1016/j.resuscitation.2008.05.004 pmid:http://www.ncbi.nlm.nih.gov/pubmed/18620794
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      2. Kellett J ,
      3. Kim A
      . Validation of an abbreviated Vitalpac™ early warning score (views) in 75,419 consecutive admissions to a Canadian regional hospital. Resuscitation 2012;83:297–302.doi:10.1016/j.resuscitation.2011.08.022 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21907689
      OpenUrlCrossRefPubMed
    47. ↵
      2. Duckitt RW ,
      3. Buxton-Thomas R ,
      4. Walker J , et al
      . Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. Br J Anaesth 2007;98:769–74.doi:10.1093/bja/aem097 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17470844
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      2. Abbott TEF ,
      3. Torrance HDT ,
      4. Cron N , et al
      . A single-centre cohort study of national early warning score (news) and near patient testing in acute medical admissions. Eur J Intern Med 2016;35:78–82.doi:10.1016/j.ejim.2016.06.014 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27346295
      OpenUrlCrossRefPubMed
    49. ↵
      2. Cuthbertson BH ,
      3. Boroujerdi M ,
      4. McKie L , et al
      . Can physiological variables and early warning scoring systems allow early recognition of the deteriorating surgical patient? Crit Care Med 2007;35:402–9.doi:10.1097/01.CCM.0000254826.10520.87 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17205002
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      2. Bartkowiak B ,
      3. Snyder AM ,
      4. Benjamin A , et al
      . Validating the electronic cardiac arrest risk triage (eCART) score for risk stratification of surgical inpatients in the postoperative setting: retrospective cohort study. Ann Surg 2019;269:1059–63.doi:10.1097/SLA.0000000000002665 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31082902
      OpenUrlCrossRefPubMed
    51. ↵
      2. Qin Q ,
      3. Xia Y ,
      4. Cao Y
      . Clinical study of a new modified early warning system scoring system for rapidly evaluating shock in adults. J Crit Care 2017;37:50–5.doi:10.1016/j.jcrc.2016.08.025 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27626832
      OpenUrlPubMed
    52. ↵
      2. Sbiti-Rohr D ,
      3. Kutz A ,
      4. Christ-Crain M , et al
      . The National early warning score (news) for outcome prediction in emergency department patients with community-acquired pneumonia: results from a 6-year prospective cohort study. BMJ Open 2016;6:e011021. doi:10.1136/bmjopen-2015-011021 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27683509
      OpenUrlPubMed
    53. ↵
      2. Liljehult J ,
      3. Christensen T
      . Early warning score predicts acute mortality in stroke patients. Acta Neurol Scand 2016;133:261–7.doi:10.1111/ane.12452 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26104048
      OpenUrlCrossRefPubMed
    54. ↵
      2. Chiew CJ ,
      3. Liu N ,
      4. Tagami T , et al
      . Heart rate variability based machine learning models for risk prediction of suspected sepsis patients in the emergency department. Medicine 2019;98:e14197. doi:10.1097/MD.0000000000014197 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30732136
      OpenUrlPubMed
    55. ↵
      2. Bilben B ,
      3. Grandal L ,
      4. Søvik S
      . National Early Warning Score (NEWS) as an emergency department predictor of disease severity and 90-day survival in the acutely dyspneic patient - a prospective observational study. Scand J Trauma Resusc Emerg Med 2016;24:80. doi:10.1186/s13049-016-0273-9 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27250249
      OpenUrlPubMed
    56. ↵
      2. Goulden R ,
      3. Hoyle M-C ,
      4. Monis J , et al
      . qSOFA, SIRS and news for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg Med J 2018;35:345–9.doi:10.1136/emermed-2017-207120 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29467173
      OpenUrlAbstract/FREE Full Text
    57. ↵
      2. Dundar ZD ,
      3. Ergin M ,
      4. Karamercan MA , et al
      . Modified early warning score and VitalPac early warning score in geriatric patients admitted to emergency department. Eur J Emerg Med 2016;23:406–12.doi:10.1097/MEJ.0000000000000274 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25919485
      OpenUrlPubMed
    58. ↵
      2. Bulut M ,
      3. Cebicci H ,
      4. Sigirli D , et al
      . The comparison of modified early warning score with rapid emergency medicine score: a prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department. Emerg Med J 2014;31:476–81.doi:10.1136/emermed-2013-202444 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23562988
      OpenUrlAbstract/FREE Full Text
    59. ↵
      2. Smith GB ,
      3. Prytherch DR ,
      4. Schmidt PE , et al
      . Review and performance evaluation of aggregate weighted 'track and trigger' system. Resuscitation 2008;77:170–9.doi:10.1016/j.resuscitation.2007.12.004 pmid:http://www.ncbi.nlm.nih.gov/pubmed/18249483
      OpenUrlCrossRefPubMedWeb of Science
    60. ↵
      2. Mulligan A
      . Validation of a physiological track and trigger score to identify developing critical illness in haematology patients. Intensive Crit Care Nurs 2010;26:196–206.doi:10.1016/j.iccn.2010.03.002 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20493707
      OpenUrlCrossRefPubMed
    61. ↵
      2. Cooksley T ,
      3. Kitlowski E ,
      4. Haji-Michael P
      . Effectiveness of modified early warning score in predicting outcomes in oncology patients. QJM 2012;105:1083–8.doi:10.1093/qjmed/hcs138 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22855285
      OpenUrlCrossRefPubMedWeb of Science
    62. ↵
      2. Eckart A ,
      3. Hauser SI ,
      4. Kutz A , et al
      . Combination of the National early warning score (news) and inflammatory biomarkers for early risk stratification in emergency department patients: results of a multinational, observational study. BMJ Open 2019;9:e024636. doi:10.1136/bmjopen-2018-024636 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30782737
      OpenUrlPubMed
    63. ↵
      2. Ghanem-Zoubi NO ,
      3. Vardi M ,
      4. Laor A , et al
      . Assessment of disease-severity scoring systems for patients with sepsis in general internal medicine departments. Crit Care 2011;15:R95. doi:10.1186/cc10102 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21401927
      OpenUrlPubMed
    64. ↵
      2. Albur M ,
      3. Hamilton F ,
      4. MacGowan AP
      . Early warning score: a dynamic marker of severity and prognosis in patients with gram-negative bacteraemia and sepsis. Ann Clin Microbiol Antimicrob 2016;15:23. doi:10.1186/s12941-016-0139-z pmid:http://www.ncbi.nlm.nih.gov/pubmed/27071911
      OpenUrlPubMed
    65. ↵
      2. Innocenti F ,
      3. Tozzi C ,
      4. Donnini C , et al
      . SOFA score in septic patients: incremental prognostic value over age, comorbidities, and parameters of sepsis severity. Intern Emerg Med 2018;13:405–12.doi:10.1007/s11739-017-1629-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28188577
      OpenUrlPubMed
    66. ↵
      2. Hu SB ,
      3. Wong DJL ,
      4. Correa A , et al
      . Prediction of clinical deterioration in hospitalized adult patients with hematologic malignancies using a neural network model. PLoS One 2016;11:e0161401. doi:10.1371/journal.pone.0161401 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27532679
      OpenUrlPubMed
    67. ↵
      2. Jarvis SW ,
      3. Kovacs C ,
      4. Briggs J , et al
      . Are observation selection methods important when comparing early warning score performance? Resuscitation 2015;90:1–6.doi:10.1016/j.resuscitation.2015.01.033 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25668311
      OpenUrlPubMed
    68. ↵
      2. Romero-Brufau S ,
      3. Huddleston JM ,
      4. Naessens JM , et al
      . Widely used track and trigger scores: are they ready for automation in practice? Resuscitation 2014;85:549–52.doi:10.1016/j.resuscitation.2013.12.017 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24412159
      OpenUrlPubMed
    69. ↵
      2. Mozaffarian D
      . Global scourge of cardiovascular disease. J Am Coll Cardiol 2017;70:26–8.doi:10.1016/j.jacc.2017.05.007
      OpenUrlFREE Full Text
    70. ↵
      2. Carr E ,
      3. Bendayan R ,
      4. Bean D
      . Supplementing the National early warning score (NEWS2) for Anticipating early deterioration among patients with COVID-19 infection. medRxiv 2020.
    71. ↵
      2. Churpek MM ,
      3. Yuen TC ,
      4. Park SY , et al
      . Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards*. Crit Care Med 2014;42:841–8.doi:10.1097/CCM.0000000000000038 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24247472
      OpenUrlCrossRefPubMed
    72. ↵
      2. Hemingway H ,
      3. Croft P ,
      4. Perel P , et al
      . Prognosis research strategy (progress) 1: a framework for researching clinical outcomes. BMJ 2013;346:e5595. doi:10.1136/bmj.e5595 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23386360
      OpenUrlPubMed
    73. ↵
      2. Van Calster B ,
      3. McLernon DJ ,
      4. van Smeden M , et al
      . Calibration: the Achilles heel of predictive analytics. BMC Med 2019;17:230. doi:10.1186/s12916-019-1466-7 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31842878
      OpenUrlPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @BaneenAlhmoud, @DrRiyazPatel, @amibanerjee1

    • Contributors AB conceived the study. BA, AB and TB conducted the search, data extraction and data analysis. BA wrote the initial draft of the manuscript. All authors contributed to interpretation of findings, critical review and revisions of the manuscript.

    • Funding BA has received PhD funding from the Saudi Arabian Cultural Bureau.

    • Competing interests AB has received research grants from AstraZeneca.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.