You are here

Article Text

Article menu

Download PDFPDF

Review
Five-level emergency triage systems: variation in assessment of validity
  1. Akira Kuriyama1,2,
  2. Seigo Urushidani3,
  3. Takeo Nakayama1
  1. 1 Department of Health Informatics, Kyoto University School of Public Health, Kyoto, Japan
  2. 2 Department of General Medicine, Kurashiki Central Hospital, Kurashiki, Okayama, Japan
  3. 3 Department of Emergency Medicine, Kurashiki Central Hospital, Kurashiki, Okayama, Japan
  1. Correspondence to Dr Akira Kuriyama, Department of Health Informatics, Kyoto University School of Public Health, Yoshida-konoe-cho Sakyo-ku Kyoto 606-8501 Japan; akira.kuriyama.jpn{at}gmail.com

Abstract

Introduction Triage systems are scales developed to rate the degree of urgency among patients who arrive at EDs. A number of different scales are in use; however, the way in which they have been validated is inconsistent. Also, it is difficult to define a surrogate that accurately predicts urgency. This systematic review described reference standards and measures used in previous validation studies of five-level triage systems.

Methods We searched PubMed, EMBASE and CINAHL to identify studies that had assessed the validity of five-level triage systems and described the reference standards and measures applied in these studies. Studies were divided into those using criterion validity (reference standards developed by expert panels or triage systems already in use) and those using construct validity (prognosis, costs and resource use).

Results A total of 57 studies examined criterion and construct validity of 14 five-level triage systems. Criterion validity was examined by evaluating (1) agreement between the assigned degree of urgency with objective standard criteria (12 studies), (2) overtriage and undertriage (9 studies) and (3) sensitivity and specificity of triage systems (7 studies). Construct validity was examined by looking at (4) the associations between the assigned degree of urgency and measures gauged in EDs (48 studies) and (5) the associations between the assigned degree of urgency and measures gauged after hospitalisation (13 studies). Particularly, among 46 validation studies of the most commonly used triages (Canadian Triage and Acuity Scale, Emergency Severity Index and Manchester Triage System), 13 and 39 studies examined criterion and construct validity, respectively.

Conclusion Previous studies applied various reference standards and measures to validate five-level triage systems. They either created their own reference standard or used a combination of severity/resource measures.

  • triage
  • triage systems
  • emergency departments
  • reference standards
  • measures
  • validity
  • criterion validity
  • construct validity
  • severity
  • urgency
  • systematic review.

https://doi.org/10.1136/emermed-2016-206295

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Triage at EDs is a decision-making process that is applied to identify patients who require immediate attention to achieve optimal outcomes.1 Triage systems are scales developed primarily to categorise patients who do and who do not need immediate intervention by urgency, and optimise resources in the ED to apply them to those who need immediate care.

    Validating triage systems is essential because they can impact the outcomes of patients in need of immediate care. A challenge is to determine the appropriate reference standard and measure for validation that discriminate patients into a ‘true’ category of urgency. In validation studies of triage systems, criterion and construct validities have been mainly discussed. Criterion validity looks at the correlation of a scale with some external criterion of the disorder under study (reference standard).2 This is often the ‘gold standard’. Construct validity looks for the correlation in assessments obtained from several scales purported to measure the same construct (measures).2 3 In validating triage systems, criterion validity would be ‘true’ urgency of patients, most likely determined by experts, while construct validity represents severity-related variables such as mortality, admission and resource and time spent on patients. Criterion validity should be preferred in validation of scales, but given the lack of a convenient gold standard for ‘urgency’, Moll has suggested that the best proxy reference standard comprises prognostic markers, disease severity and case complexity.4 Furthermore, there is no consensus as to what are acceptable reference standards or measures in validating triage systems.

    Therefore, we systematically reviewed the reference standards and measures used in published validation studies of triage systems to provide an understanding of the basis on which triage scales have been validated.

    Methods

    This study proceeded according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement for reporting systematic reviews.5 The protocol for this systematic review is registered in PROSPERO (CRD42015027653).

    Search and selection of studies

    The American College of Emergency Physicians /Emergency Nurses Association Five-Level Triage Task Force recommended the use of five-level triage scales as they generally showed better reliability compared with three-level or four-level ones.6 Travers et al showed that a five-level triage system was more reliable and discriminative than a three-level one.7 A growing trend to use the five-level triage systems was also noted.8 Among them, the Canadian Triage and Acuity Scale (CTAS), the Emergency Severity Index (ESI), the Manchester Triage System (MTS) and the Australasian Triage System (ATS) are the five-level triage systems that were most frequently studied, and we first focused on these four. We also examined other five-level triage systems separately. We required these studies to meet the following conditions: (1) they included patients who presented at EDs from all categories of triage scales, unless a pragmatic or ethical need was required to exclude some from certain urgency categories; (2) they were of any design and (3) were published in peer-reviewed journals, and (4) they were explicitly testing the validity of one or more triage scales. Studies from any age groups as well as ambulatory and transferred patients were included. Studies were excluded if they used these scales as characteristics of the participants or explanatory variables, or if they focused on a limited spectrum of diseases and symptoms or populations classified in certain urgency categories without clear rationale. We also excluded reviews, editorials, letters, conference proceedings or abstracts and studies that focused solely on inter-rater reliability.

    We searched PubMed, EMBASE and CINAHL for potentially eligible studies. We designed a sensitive search strategy as follows: ‘Canadian Triage and Acuity Scale’ OR ‘Emergency Severity Index’ OR ‘Manchester Triage Scale’ OR ‘Australasian Triage Scale’. Next, we searched PubMed for studies on other five-level triage systems with the following search terms: ‘Emergency Service, Hospital’[Mesh] AND ‘Triage’[Mesh]. There were no language restrictions. The last search date was 29 February 2016.

    Two authors (AK and SU) independently screened abstracts to identify potentially eligible studies. The same authors then retrieved the full texts, independently assessed the eligibility of these studies, and screened their included reference lists. Any uncertainty about the eligibility of a study was resolved through discussion with the third author (TN).

    Data extraction

    Two authors (AK and SU) independently extracted the study characteristics (study design, country, number of study sites and triage scale applied), participant demographics (patient category by age and sample size) and the reference standards and measures that were used to evaluate the validity of the triage scales.

    Analysis

    The reference standards and measures to validate the triage systems and how they were used in evaluating the validity are described. As the goal was to describe the range of standards and measures used to validate triage systems, and not evaluate the accuracy of that validation, assessment of risk for bias in each original study was waived.

    Results

    Description of studies

    Our search for studies on the four most studied triage systems yielded 998 articles (figure 1), of which 46 met inclusion criteria9–54 (table 1). Among them, 21 assessed ESI, 14 CTAS and 14 MTS. None evaluated the validity of ATS. Seven studies compared two more scales: MTS and ESI,38 MTS, ESI and an informally structured triage system (ISS),34 CTAS and the Taiwan Triage and Acuity Scale (TTS),28 ESI and TTS,48 CTAS and ESI,41 and MTS V.1 and 2,33 MTS and a modified version of MTS.49 Twelve studies were from the USA,9 11 12 15 23 31 35–37 42 43 53 nine from Canada,10 13 18 20–22 41 47 seven from the Netherlands,32 34 38–40 49 54 three from Switzerland,24 25 50 two from Brazil,30 52 Portugal44 45 and Taiwan,28 48 respectively, and one each was from Andorra, Germany, Kuwait, Norway, Saudi Arabia, South Korea, Sweden and Spain. One was an international multicentre study that examined MTS.33 The median sample size was 1042 (range 50–550 940) patients, and a median of 1 site (range 1–12 sites) was studied. Adult, paediatric and geriatric populations were examined in 17, 20 and 4 articles, respectively. Six studies evaluated validity in both adult and paediatric populations, and 11 studies did not specify the populations. Eighteen studies were retrospective and 28 prospective observational.

    Figure 1
    Figure 1

    Study selection.

    View this table:
    Table 1

    Characteristics of included studies on Australasian Triage System, Canadian Acuity and Triage Scale (CTAS), Emergency Severity Index (ESI) and Manchester Triage System (MTS)

    Our search for other five-level triage systems yielded 3227 articles. Ten triage systems from 11 articles were finally included for analysis: Clinical GPS,55 Echelle Liégeoise d’Index de Sévérité à l’Admission,56 FRench Emergency Nurses Classification in Hospital scale,57 Hacettepe Emergency Triage System,58 Medical Emergency Triage and Treatment System,59 Netherlands Triage System,60 Pediatric Triage and Acuity Scale,61 Rapid Triage Score,62 Rapid Emergency Triage and Treatment System-Hospital Unit West63 and Soterion Rapid Triage System.64 65 All these systems were examined in single-centre studies, and 4 out of 11 were prospectively conducted55 58 60 62 (table 2).

    View this table:
    Table 2

    Characteristics of other five-level triage systems

    Reference standards and measures

    For studies using criterion validity, reference standards included degree of urgency with objective standard criteria developed by expert panels for their studies a priori, and other triage systems that had already been in use as a means of validating a triage system. For studies of construct validity, measures included patient prognosis, costs and resource use that were gauged during the ED stay and after hospitalisation (box).

    Box

    Reference standards and measures used in the validation studies of five-level triage systems

    Reference standards (criterion validity)

    • Objective standard criteria/urgency set by expert panels for their studies a priori

    • Existing emergency triage systems

    • Immediate life-saving interventions

    Measures (construct validity)

    • Overall admissions

    • Admissions to intensive care or monitored units

    • ED length of stay

    • Costs in EDs

    • Number of resource used in EDs

    • Mortality in EDs

    • Leaving without being seen

    • Waiting times before examinations by physicians in EDs

    • Referrals to outpatients after the discharge from EDs

    • In-hospital mortality

    • Hospital length of stay

    • Costs after hospitalisation

    • Six-month survival

    • Sixty-day mortality

    • Thirty-day mortality

    • Ninety-day mortality

    We identified five main outcomes for the validation of triage systems (table 3). For criterion validity, outcomes included (1) agreement of assigned degree of urgency with objective standard criteria set by expert panels for their studies a priori, (2) overtriage and undertriage and (3) sensitivity and specificity for defined reference standards. Outcomes for studies using construct validity were (1) associations between assigned degree of patient urgency, their prognosis, cost and resource use in EDs; and (2) associations between assigned degree of urgency and patient prognosis and cost after hospitalisation were examined.

    View this table:
    Table 3

    Approaches with reference standards and measures in the validation of Australasian Triage System, Canadian Triage and Acuity System (CTAS), Emergency Severity Index (ESI) and Manchester Triage Systems (MTS)

    For studies on criterion validity, nine studies used a reference standard that was developed by investigators.2 14 29 32 34 39 40 46 49 Six of these were studies of MTS, and one each was of CTAS and ESI, respectively. One study evaluated MTS, ESI and ISS. The details of reference standards are shown in table 4.

    View this table:
    Table 4

    Reference standards developed by investigators for validating triage systems

    Overtriage and undertriage of patients in ED were measured in eight studies (three examining MTS, two ESI, one CTAS, one comparing MTS and a modified version of MTS, and one comparing MTS, ESI and ISS).29 34 37 39 46 47 49 50 Overtriage and undertriage were defined in four simulation studies of vignettes and three prospective studies as the degree of urgency assigned by nurses being above or below that assigned by experts, respectively.29 34 39 46 47 49 50 Travers et al defined overtriage as patients who were rated ESI 1, 2 or 3 (acuity) but required <2 resources or those who were assessed as ESI 1 but who were not hospitalised, and undertriage as those who were assessed as ESI 4 or 5 (non-acuity) but received ≥2 resources or were hospitalised.37

    The sensitivity and specificity of some reference standards were measured in seven studies (three respectively examining MTS and ESI, and one comparing MTS, ESI and ISS).9 31 32 34 39 40 46 47 The reference standards comprised patients receiving immediate life-saving intervention,9 31 each of five degrees of urgency,34 and high and low degrees of urgency32 39 40 46 that were predefined by investigators.

    For studies on construct validity, association between assigned degree of patient urgency and one or more measures gauged during the ED stay and after hospitalisation was examined. Admissions from the ED were most commonly measured, with 33 studies using this outcome.9 11–15 17–28 32–34 36–38 42–46 48 51 Other measures gauged during the ED stay included admissions to intensive care or monitored units,19 20 22 24 26 27 35 50 mortality in the ED (n=9),26 27 36 38 42–45 50 patients leaving without being attended,16 20 46 duration of wait before being examined by a physician, referrals to outpatients after discharge from the ED, Worthing Physiological Scoring System scores,54 length of stay (LOS) in the ED11–13 15 16 19 20 22–26 28 35 37 42 43 46 48 and costs consumed in EDs. The measure in 17 studies was the amount of resources used in EDs.9 11 13 15 17 21–25 32 34 35 37 41 42 46 51 A few of these studies specifically counted specialist consultations, necessity for monitoring, diagnostic procedures (electrocardiography, laboratory examinations, diagnostic imaging, blood cultures and invasive diagnostic tests)44 and treatment (intravenous fluids, transfusions, mechanical ventilation, inhalers and life-saving interventions). Measures gauged after admission included in-hospital mortality rates,19 24 25 27 36 41 52 hospital LOS,19 24 27 35 50 52 30-day50 and 6-month survival,53 and costs consumed after admission.

    Other five-level triage systems

    Reference standards and measures used to validate triage systems were similar to those for the four triage systems (table 5). Three and nine studies used criterion and construct validity, respectively. Of note, some studies used other triage systems (CTAS and ESI) that had already been in use, as the reference standards.55 58 62

    View this table:
    Table 5

    Approaches with reference standards and measures in the validation of other five-level triage systems

    Discussion

    We found that, of 57 validation studies, a variety of reference standards and measures were used. Overall, construct validity (51 studies) was more frequently examined than criterion validity (14 studies). Particularly, validation studies of ESI (5 of 21 studies) and MTS (7 of 14 studies) more frequently used a form of criterion validity compared with CTAS (2 of 14 studies). Validation studies of these three triages commonly used some construct validity; CTAS (13 studies), ESI (20 studies) and MTS (10 studies).

    Criterion validity should be preferred in validation of scales. In validating triage systems, true ‘urgency’ of patients should serve as reference standards, because triage systems rank the speed of care for a patient, or namely, urgency. However, our study found that many validation studies focused on severity or construct validity, likely due to the lack of established criterion validity in triage system research.66 For criterion validity, reference standards were mostly the studies’ own criteria, which were either urgency alone or a combination of urgency and severity, determined by investigators. Other triage systems that are already in use could also serve as reference standards for a newly introduced triage system, which reduces the variability of reference standards.28 34

    Triage scales could be considered a type of decision rule, in which the goal of the rule is to predict the need for immediate care. However, unlike decision rules, they are consensus-based and lack the typical multistep data gathering and statistical processes resulting in derivation, and then prospective validation on a unique population. Moll suggested a four-step approach to validate a triage system4: consensus-based derivation of decision rules for different degrees of urgency, validation of a system with a reference standard as the best proxy for prognosis in a single setting (internal validation), modification of triage rules and validation in various emergency care settings (external validation). For studies evaluating criterion validity, most reference standards or criteria developed by the investigators for their validation studies followed Moll’s framework of reference standards or the best ‘proxy’ based on the information of urgency and severity. We speculate that the investigators of studies assessing criterion validity needed to establish reference standards based on both urgency and severity because urgency is hard to determine or predict based on a limited amount of information gained during the triage.

    All validation studies are currently subject to the limitations described above due to the absence of perfect standard references of urgency. Clinicians need to know that the validity of triage systems has not been perfectly determined and their weaknesses remain obscure. Bearing this in mind, clinicians need to triage patients using the available triage systems.

    The present study has some limitations. First, it was designed simply to review and describe the methodologies used in validation studies of emergency triage systems without the intent to suggest the most appropriate reference standard or measure for a validation study. Second, our study focused on five-level triage systems. We might have missed other methodology as well as reference standards and measures used to examine other triage systems. Despite these limitations, we summarised the reference standards and measures used in validation studies of triage systems, and described their drawbacks and advantages. This information should provide an important rationale for future validation studies of triage systems.

    Conclusions

    The most commonly used triage systems have been validated using both criterion and construct validity of emergency triage systems. The difficulty in defining a surrogate for urgency means that studies must either create their own reference standard (often an expert panel) or use a combination of severity/resource measures which approximate but are not the same as urgency. Given that the limitations of validation studies are not completely understood and given the potential flaws of triage systems, future studies should attempt to elucidate the weaknesses of triage systems in terms of presenting signs and symptoms and the characteristics of patients.

    Supplementary Material

    Supplementary Figure 1

    References

    1. ↵
      2. FitzGerald G ,
      3. Jelinek GA ,
      4. Scott D , et al
      . Emergency department triage revisited. Emerg Med J 2010;27:86–92.doi:10.1136/emj.2009.077081
      OpenUrlAbstract/FREE Full Text
    2. ↵
      2. Streiner DL ,
      3. Norman GR
      . Health measurement scales: a practical guide to their development and use. Oxford; New York: Oxford University Press, 2003.
    3. ↵
      2. Strauss ME ,
      3. Smith GT
      . Construct validity: advances in theory and methodology. Annu Rev Clin Psychol 2009;5:1–25.doi:10.1146/annurev.clinpsy.032408.153639
      OpenUrlCrossRefPubMed
    4. ↵
      2. Moll HA
      . Challenges in the validation of triage systems at emergency departments. J Clin Epidemiol 2010;63:384–8.doi:10.1016/j.jclinepi.2009.07.009
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      2. Liberati A ,
      3. Altman DG ,
      4. Tetzlaff J , et al
      . The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med 2009;151:W–94.doi:10.7326/0003-4819-151-4-200908180-00136
      OpenUrlCrossRef
    6. ↵
      2. Fernandes CM ,
      3. Tanabe P ,
      4. Gilboy N , et al
      . Five-level triage: a report from the ACEP/ENA five-level triage task force. J Emerg Nurs 2005;31 39–50.doi:10.1016/j.jen.2004.11.002
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      2. Travers DA ,
      3. Waller AE ,
      4. Bowling JM , et al
      . Five-level triage system more effective than three-level in tertiary emergency department. J Emerg Nurs 2002;28:395–400.doi:10.1067/men.2002.127184
      OpenUrlCrossRefPubMed
    8. ↵
      2. McHugh M ,
      3. Tanabe P ,
      4. McClelland M , et al
      . More patients are triaged using the emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med 2012;19:106–9.doi:10.1111/j.1553-2712.2011.01240.x
      OpenUrlPubMed
    9. ↵
      2. Hong R ,
      3. Sexton R ,
      4. Sweet B , et al
      . Comparison of START triage categories to emergency department triage levels to determine need for urgent care and to predict hospitalization. Am J Disaster Med 2015;10:13–21.doi:10.5055/ajdm.2015.0184
      OpenUrl
    10. ↵
      2. Ma W ,
      3. Gafni A ,
      4. Goldman RD
      . Correlation of the Canadian pediatric emergency triage and acuity scale to ED resource utilization. Am J Emerg Med 2008;26:893–7.doi:10.1016/j.ajem.2008.02.024
      OpenUrlCrossRefPubMed
    11. ↵
      2. Baumann MR ,
      3. Strout TD
      . Evaluation of the emergency severity Index (version 3) triage algorithm in pediatric patients. Acad Emerg Med 2005;12:219–24.doi:10.1111/j.1553-2712.2005.tb00872.x
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      2. Baumann MR ,
      3. Strout TD
      . Triage of geriatric patients in the emergency department: validity and survival with the emergency severity Index. Ann Emerg Med 2007;49:234–40.doi:10.1016/j.annemergmed.2006.04.011
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      2. Dong SL ,
      3. Bullard MJ ,
      4. Meurer DP , et al
      . Predictive validity of a computerized emergency triage tool. Acad Emerg Med 2007;14:16–21.doi:10.1111/j.1553-2712.2007.tb00362.x
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      2. Dong SL ,
      3. Bullard MJ ,
      4. Meurer DP , et al
      . Emergency triage: comparing a novel computer triage program with standard triage. Acad Emerg Med 2005;12:502–7.doi:10.1111/j.1553-2712.2005.tb00889.x
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      2. Eitel DR ,
      3. Travers DA ,
      4. Rosenau AM , et al
      . The emergency severity index triage algorithm version 2 is reliable and valid. Acad Emerg Med 2003;10:1070–80.doi:10.1111/j.1553-2712.2003.tb00577.x
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      2. Elkum NB ,
      3. Barrett C ,
      4. Al-Omran H
      . Canadian emergency department triage and acuity scale: implementation in a tertiary care center in Saudi Arabia. BMC Emerg Med 2011;11:3.doi:10.1186/1471-227X-11-3
      OpenUrlPubMed
    17. ↵
      2. Elshove-Bolk J ,
      3. Mencl F ,
      4. van Rijswijck BT , et al
      . Validation of the emergency severity Index (ESI) in self-referred patients in a european emergency department. Emerg Med J 2007;24:170–4.doi:10.1136/emj.2006.039883
      OpenUrlAbstract/FREE Full Text
    18. ↵
      2. Gouin S ,
      3. Gravel J ,
      4. Amre DK , et al
      . Evaluation of the paediatric canadian triage and acuity Scale in a pediatric ED. Am J Emerg Med 2005;23:243–7.doi:10.1016/j.ajem.2004.02.046
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      2. Gräff I ,
      3. Goldschmidt B ,
      4. Glien P , et al
      . The german version of the Manchester Triage System and its quality criteria--first assessment of validity and reliability. PLoS One 2014;9:e88995.doi:10.1371/journal.pone.0088995
    20. ↵
      2. Gravel J ,
      3. Fitzpatrick E ,
      4. Gouin S , et al
      . Performance of the canadian triage and acuity Scale for children: a multicenter database study. Ann Emerg Med 2013;61:27–32.doi:10.1016/j.annemergmed.2012.05.024
      OpenUrlCrossRefPubMed
    21. ↵
      2. Gravel J ,
      3. Gouin S ,
      4. Goldman RD , et al
      . The canadian triage and acuity Scale for children: a prospective multicenter evaluation. Ann Emerg Med 2012;60:71–7.doi:10.1016/j.annemergmed.2011.12.004
      OpenUrlCrossRefPubMed
    22. ↵
      2. Gravel J ,
      3. Manzano S ,
      4. Arsenault M
      . Validity of the Canadian paediatric triage and acuity scale in a tertiary care hospital. CJEM 2009;11:23–8.doi:10.1017/S1481803500010885
      OpenUrlCrossRefPubMed
    23. ↵
      2. Green NA ,
      3. Durani Y ,
      4. Brecher D , et al
      . Emergency severity index version 4: a valid and reliable tool in pediatric emergency department triage. Pediatr Emerg Care 2012;28:753–7.doi:10.1097/PEC.0b013e3182621813
      OpenUrlPubMed
    24. ↵
      2. Grossmann FF ,
      3. Nickel CH ,
      4. Christ M , et al
      . Transporting clinical tools to new settings: cultural adaptation and validation of the emergency severity Index in german. Ann Emerg Med 2011;57:257–64.doi:10.1016/j.annemergmed.2010.07.021
      OpenUrlPubMed
    25. ↵
      2. Grossmann FF ,
      3. Zumbrunn T ,
      4. Frauchiger A , et al
      . At risk of undertriage? testing the performance and accuracy of the emergency severity index in older emergency department patients. Ann Emerg Med 2012;60:317–25.doi:10.1016/j.annemergmed.2011.12.013
      OpenUrlCrossRefPubMed
    26. ↵
      2. Jiménez JG ,
      3. Murray MJ ,
      4. Beveridge R , et al
      . Implementation of the Canadian emergency department triage and acuity Scale (CTAS) in the principality of Andorra: can triage parameters serve as emergency department quality indicators? CJEM 2003;5:315–22.
      OpenUrlPubMed
    27. ↵
      2. Lee JY ,
      3. Oh SH ,
      4. Peck EH , et al
      . The validity of the canadian triage and acuity Scale in predicting resource utilization and the need for immediate life-saving interventions in elderly emergency department patients. Scand J Trauma Resusc Emerg Med 2011;19:68.doi:10.1186/1757-7241-19-68
      OpenUrlCrossRefPubMed
    28. ↵
      2. Ng CJ ,
      3. Hsu KH ,
      4. Kuan JT , et al
      . Comparison between canadian triage and acuity Scale and Taiwan Triage System in emergency departments. J Formos Med Assoc 2010;109:828–37.doi:10.1016/S0929-6646(10)60128-3
      OpenUrlCrossRefPubMed
    29. ↵
      2. Olofsson P ,
      3. Gellerstedt M ,
      4. Carlström ED
      . Manchester triage in Sweden-interrater reliability and accuracy. Int Emerg Nurs 2009;17:143–8.doi:10.1016/j.ienj.2008.11.008
      OpenUrlCrossRefPubMed
    30. ↵
      2. Pinto D ,
      3. Salgado PO ,
      4. Chianca TC
      . Predictive validity of the manchester triage system: evaluation of outcomes of patients admitted to an emergency department. Rev Lat Am Enfermagem 2012;20:1041–7.doi:10.1590/S0104-11692012000600005
      OpenUrl
    31. ↵
      2. Platts-Mills TF ,
      3. Travers D ,
      4. Biese K , et al
      . Accuracy of the emergency severity Index triage instrument for identifying elder emergency department patients receiving an immediate life-saving intervention. Acad Emerg Med 2010;17:238–43.doi:10.1111/j.1553-2712.2010.00670.x
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      2. Roukema J ,
      3. Steyerberg EW ,
      4. van Meurs A , et al
      . Validity of the manchester triage system in paediatric emergency care. Emerg Med J 2006;23:906–10.doi:10.1136/emj.2006.038877
      OpenUrlAbstract/FREE Full Text
    33. ↵
      2. Seiger N ,
      3. van Veen M ,
      4. Almeida H , et al
      . Improving the manchester triage system for pediatric emergency care: an international multicenter study. PLoS One 2014;9:e83267.doi:10.1371/journal.pone.0083267
    34. ↵
      2. Storm-Versloot MN ,
      3. Ubbink DT ,
      4. Kappelhof J , et al
      . Comparison of an informally structured triage system, the emergency severity index, and the manchester triage system to distinguish patient priority in the emergency department. Acad Emerg Med 2011;18:822–9.doi:10.1111/j.1553-2712.2011.01122.x
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      2. Tanabe P ,
      3. Gimbel R ,
      4. Yarnold PR , et al
      . The emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs 2004;30:22–9.doi:10.1016/j.jen.2003.11.004
      OpenUrlCrossRefPubMed
    36. ↵
      2. Tanabe P ,
      3. Gimbel R ,
      4. Yarnold PR , et al
      . Reliability and validity of scores on the emergency Severity Index version 3. Acad Emerg Med 2004;11:59–65.doi:10.1197/j.aem.2003.06.013
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      2. Travers DA ,
      3. Waller AE ,
      4. Katznelson J , et al
      . Reliability and validity of the emergency severity index for pediatric triage. Acad Emerg Med 2009;16:843–9.doi:10.1111/j.1553-2712.2009.00494.x
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      2. van der Wulp I ,
      3. Schrijvers AJ ,
      4. van Stel HF
      . Predicting admission and mortality with the emergency severity index and the manchester triage system: a retrospective observational study. Emerg Med J 2009;26:506–9.doi:10.1136/emj.2008.063768
      OpenUrlAbstract/FREE Full Text
    39. ↵
      2. van der Wulp I ,
      3. van Baar ME ,
      4. Schrijvers AJ
      . Reliability and validity of the Manchester Triage System in a general emergency department patient population in the Netherlands: results of a simulation study. Emerg Med J 2008;25:431–4.doi:10.1136/emj.2007.055228
      OpenUrlAbstract/FREE Full Text
    40. ↵
      2. van Veen M ,
      3. Steyerberg EW ,
      4. Ruige M , et al
      . Manchester triage system in paediatric emergency care: prospective observational study. BMJ 2008;337:a1501.doi:10.1136/bmj.a1501
      OpenUrlAbstract/FREE Full Text
    41. ↵
      2. Worster A ,
      3. Fernandes CM ,
      4. Eva K , et al
      . Predictive validity comparison of two five-level triage acuity scales. Eur J Emerg Med 2007;14:188–92.doi:10.1097/MEJ.0b013e3280adc956
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      2. Wuerz RC ,
      3. Milne LW ,
      4. Eitel DR , et al
      . Reliability and validity of a new five-level triage instrument. Acad Emerg Med 2000;7:236–42.doi:10.1111/j.1553-2712.2000.tb01066.x
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      2. Wuerz RC ,
      3. Travers D ,
      4. Gilboy N , et al
      . Implementation and refinement of the emergency severity index. Acad Emerg Med 2001;8:170–6.doi:10.1111/j.1553-2712.2001.tb01283.x
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      2. Santos AP ,
      3. Freitas P ,
      4. Martins HM
      . Manchester Triage System version II and resource utilisation in the emergency department. Emerg Med J 2014;31:148–52.doi:10.1136/emermed-2012-201782
      OpenUrlAbstract/FREE Full Text
    45. ↵
      2. Martins HM ,
      3. Cuña LM ,
      4. Freitas P ,
      5. Manchester I
      . Is Manchester (MTS) more than a triage system? A study of its association with mortality and admission to a large portuguese hospital. Emerg Med J 2009;26:183–6.doi:10.1136/emj.2008.060780
      OpenUrlAbstract/FREE Full Text
    46. ↵
      2. Ruipérez TH ,
      3. Costa CL ,
      4. Martínez MGA , et al
      . Evidencias de validez del sistema de triaje emergency Severity Index en un servicio de urgencias de un hospital general. Emergencias 2015;27:301–6.
      OpenUrl
    47. ↵
      2. Bergeron S ,
      3. Gouin S ,
      4. Bailey B , et al
      . Agreement among pediatric health care professionals with the pediatric canadian triage and acuity scale guidelines. Pediatr Emerg Care 2004;20:514–8.doi:10.1097/01.pec.0000136067.07081.ae
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      2. Chi CH ,
      3. Huang CM
      . Comparison of the emergency severity index (ESI) and the Taiwan triage system in predicting resource utilization. J Formos Med Assoc 2006;105:617–25.doi:10.1016/S0929-6646(09)60160-1
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      2. van Veen M ,
      3. Steyerberg EW ,
      4. Van’t Klooster M , et al
      . The manchester triage system: improvements for paediatric emergency care. Emerg Med J 2012;29:654–9.doi:10.1136/emermed-2011-200562
      OpenUrlAbstract/FREE Full Text
    50. ↵
      2. Steiner D ,
      3. Renetseder F ,
      4. Kutz A , et al
      . Performance of the manchester triage system in adult medical emergency patients: a prospective cohort study. J Emerg Med 2016;50:678–89.doi:10.1016/j.jemermed.2015.09.008
      OpenUrl
    51. ↵
      2. Al-Hindi AA ,
      3. Al-Akhfash AA ,
      4. Fareed AM , et al
      . Efficacy of implementation of a 5 scale pediatric triage and acuity scale in pediatric emergency, Saudi Arabia. Saudi Med J 2014;35:999–1004.
      OpenUrl
    52. ↵
      2. Guedes HM ,
      3. Martins JC ,
      4. Chianca TC
      . Predictive value of the Manchester Triage System: evaluation of patients' clinical outcomes. Rev Bras Enferm 2015;68:4045–51.doi:10.1590/0034-7167.2015680107p
      OpenUrl
    53. ↵
      2. Wuerz R
      . Emergency severity index triage category is associated with six-month survival. ESI Triage Study Group. Acad Emerg Med 2001;8:61–4.doi:10.1111/j.1553-2712.2001.tb00554.x
      OpenUrlPubMedWeb of Science
    54. ↵
      2. van der Wulp I ,
      3. Rullmann HA ,
      4. Leenen LP , et al
      . Associations of the emergency severity index triage categories with patients' vital signs at triage: a prospective observational study. Emerg Med J 2011;28:1032–5.doi:10.1136/emj.2010.096172
      OpenUrlAbstract/FREE Full Text
    55. ↵
      2. Elias P ,
      3. Damle A ,
      4. Casale M , et al
      . A web-based tool for patient triage in emergency department settings: validation using the emergency severity index. JMIR Med Inform 2015;3:e23.doi:10.2196/medinform.3508
      OpenUrl
    56. ↵
      2. Jobé J ,
      3. Ghuysen A ,
      4. Gérard P , et al
      . Reliability and validity of a new French-language triage algorithm: the ELISA scale. Emerg Med J 2014;31:115–20.doi:10.1136/emermed-2012-201927
      OpenUrlAbstract/FREE Full Text
    57. ↵
      2. Taboulet P ,
      3. Moreira V ,
      4. Haas L , et al
      . Triage with the french emergency nurses classification in Hospital scale: reliability and validity. Eur J Emerg Med 2009;16:61–7.doi:10.1097/MEJ.0b013e328304ae57
      OpenUrlCrossRefPubMed
    58. ↵
      2. Ozüçelik DN ,
      3. Kunt MM ,
      4. Karaca MA , et al
      . A model of complaint based for overcrowding emergency department: five-level Hacettepe emergency triage system. Ulus Travma Acil Cerrahi Derg 2013;19:205–14.doi:10.5505/tjtes.2013.44789
      OpenUrl
    59. ↵
      2. Widgren BR ,
      3. Jourak M
      . Medical emergency triage and treatment system (METTS): a new protocol in primary triage and secondary priority decision in emergency medicine. J Emerg Med 2011;40:623–8.doi:10.1016/j.jemermed.2008.04.003
      OpenUrlCrossRefPubMed
    60. ↵
      2. van Ierland Y ,
      3. van Veen M ,
      4. Huibers L , et al
      . Validity of telephone and physical triage in emergency care: the Netherlands triage system. Fam Pract 2011;28:334–41.doi:10.1093/fampra/cmq097
      OpenUrlCrossRefPubMedWeb of Science
    61. ↵
      2. Chang YC ,
      3. Ng CJ ,
      4. Wu CT ,
      5. Cj N ,
      6. Ct W , et al
      . Effectiveness of a five-level iaediatric triage system: an analysis of resource utilisation in the emergency department in Taiwan. Emerg Med J 2013;30:735–9.doi:10.1136/emermed-2012-201362
      OpenUrlAbstract/FREE Full Text
    62. ↵
      2. Betz M ,
      3. Stempien J ,
      4. Wilde A , et al
      . A comparison of a formal triage scoring system and a quick-look triage approach. Eur J Emerg Med 2016;23:185–9.doi:10.1097/MEJ.0000000000000239
      OpenUrl
    63. ↵
      2. Pérez N ,
      3. Nissen L ,
      4. Nielsen RF , et al
      . The predictive validity of RETTS-HEV as an acuity triage tool in the emergency department of a danish Regional Hospital. Eur J Emerg Med 2016;23:33–7.doi:10.1097/MEJ.0000000000000173
      OpenUrl
    64. ↵
      2. Maningas PA ,
      3. Hime DA ,
      4. Parker DE , et al
      . The soterion rapid triage system: evaluation of inter-rater reliability and validity. J Emerg Med 2006;30:461–9.doi:10.1016/j.jemermed.2005.05.037
      OpenUrlCrossRefPubMedWeb of Science
    65. ↵
      2. Maningas PA ,
      3. Hime DA ,
      4. Parker DE
      . The use of the soterion rapid triage system in children presenting to the emergency department. J Emerg Med 2006;31:353–9.doi:10.1016/j.jemermed.2006.01.011
      OpenUrlCrossRefPubMedWeb of Science
    66. ↵
      2. Twomey M ,
      3. Wallis LA ,
      4. Myers JE
      . Limitations in validating emergency department triage scales. Emerg Med J 2007;24:477–9.doi:10.1136/emj.2007.046383
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Contributors AK and TN conceived the study design and interpreted the data. AK and SU acquired the data. AK analysed the data and drafted the manuscript. All authors critically revised and approved the submission of the manuscript.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    Linked Articles