Design and validation of a clinical scale for prehospital stroke recognition, severity grading and prediction of large vessel occlusion: the shortened NIH Stroke Scale for emergency medical services
======================================================================================================================================================================================================

* Jan Christoph Purrucker
* Florian Härtig
* Hardy Richter
* Andreas Engelbrecht
* Johannes Hartmann
* Jonas Auer
* Christian Hametner
* Erik Popp
* Peter Arthur Ringleb
* Simon Nagel
* Sven Poli

## Abstract

**Objective** To develop an NIH Stroke Scale (NIHSS)-compatible, all-in-one scale for rapid and comprehensive prehospital stroke assessment including stroke recognition, severity grading and progression monitoring as well as prediction of large vessel occlusion (LVO).

**Methods** Emergency medical services (EMS) personnel and stroke physicians (n=326) rated each item of the NIHSS regarding suitability for prehospital use; best rated items were included. Stroke recognition was evaluated retrospectively in 689 consecutive patients with acute stroke or stroke mimics, prediction of LVO in 741 consecutive patients with ischaemic stroke with acute vessel imaging independent of admission NIHSS score.

**Results** Nine of the NIHSS items were rated as ‘suitable for prehospital use.’ After excluding two items in order to increase specificity, the final scale (termed shortened NIHSS for EMS, sNIHSS-EMS) consists of ‘level of consciousness’, ‘facial palsy’, ‘motor arm/leg’, ‘sensory’, ‘language’ and ‘dysarthria’. Sensitivity for stroke recognition of the sNIHSS-EMS is 91% (95% CI 86 to 94), specificity 52% (95% CI 47 to 56). Receiver operating curve analysis revealed an optimal cut-off point for LVO prediction of ≥6 (sensitivity 70% (95% CI 65 to 76), specificity 81% (95% CI 76 to 84), positive predictive value 70 (95% CI 65 to 75), area under the curve 0.81 (95% CI 0.78 to 0.84)). Test characteristics were non-inferior to non-comprehensive scales.

**Conclusions** The sNIHSS-EMS may overcome the sequential use of multiple emergency stroke scales by permitting parallel stroke recognition, severity grading and LVO prediction. Full NIHSS-item compatibility allows for evaluation of stroke progression starting at the prehospital phase.

*   stroke severity grading
*   triage
*   large vessel occlusion
*   emergency medicine

### Strengths and limitations of this study

*   Prehospital stroke assessment is increasingly gaining relevance in the era of endovascular interventions for large vessel occlusions (LVO). Sound triage decisions will have a major impact on patients’ outcomes. As those are left entirely to emergency medical services (EMS) personnel, it is essential to equip them with an effective tool to guide prehospital triage.

*   The new clinical scale (shortened NIHSS for EMS, sNIHSS-EMS), developed and validated in this study, is the first scale assessed for parallel stroke recognition, severity grading and LVO prediction. Sequential use of multiple emergency stroke scales may thus be avoided.

*   A multinational survey among different emergency medical systems and professions was performed to identify items suitable for use in prehospital emergency situations.

*   The sNIHSS-EMS shares full compatibility with the in-hospital gold-standard NIH Stroke Scale, but remains simple and easy to use.

*   The scale will be incorporated into a prehospital stroke triage algorithm in a large regional stroke network, but no prospective data are available yet, which is acknowledged as a limitation.

## Introduction

A considerable number of stroke scales for prehospital use have been published over recent years.1 2 However, most of these scales only focus on single aspects of acute stroke care, that is, either stroke recognition,1 2 early prediction of outcome,3 prediction of thrombolysis,4 5 or severity grading and large vessel occlusion (LVO).3 6–18 Consequently, to provide a comprehensive prehospital stroke assessment, emergency medical services (EMS) personnel must apply at least two scales. Furthermore, the majority of existing scales lack compatibility with the NIH Stroke Scale (NIHSS), the in-hospital ‘gold-standard’ for stroke severity grading.2 This impedes the seamless evaluation of stroke progression from pre to in-hospital care. In the era of endovascular treatment of LVO, decisions regarding direct emergency referrals to specialised comprehensive stroke centres will have a major impact on patients’ outcomes.19 20 As those are left entirely to EMS personnel, it is essential to equip them with an effective tool to guide prehospital triage.

We present the development and validation of a novel comprehensive stroke scale, specifically designed for prehospital use with input from EMS. Our aim was to allow for parallel stroke recognition, severity grading and—owing to full NIHSS compatibility—progression monitoring as well as LVO prediction.

## Methods

### International online survey

We invited non-neurological EMS personnel (paramedics and emergency physicians) and stroke physicians from Austria, Germany and Switzerland to rate each individual NIHSS item regarding its applicability in a prehospital emergency setting. Invitations were sent out via the German Stroke Society, the German Society for Neuro-Intensive Care and Emergency Medicine as well as EMS providers. Participation was voluntary, no financial incentive was offered and participation was only allowed once. Non-neurological EMS personnel did not use the NIHSS routinely and did not receive specific NIHSS training before the survey. For each NIHSS item, we created and provided a short video demonstrating in-hospital bedside assessment according to the NIHSS training instructions (a screenshot is shown as figure 1A in the online supplementary appendix). Having watched the video, participants were asked to rate each NIHSS item regarding its suitability for prehospital use on a 6-item scale, ranging from 0 (most suitable) to 5 (most unsuitable). Ratings were automatically entered into a database together with name (optional), profession, professional experience and place of work. Participation was possible from 19 November 2015 until 15 April 2016, the prespecified closing date.

### Supplementary file 1

[[SP1.pdf]](pending:yes) 

### Patient cohorts

Test characteristics of the newly designed scale were calculated with regard to performance in stroke recognition and prediction of acute LVO using two distinct clinical cohorts described below.

For stroke recognition, we used a prospectively collected cohort of consecutive patients with acute ischaemic or haemorrhagic stroke and stroke mimics, which had already served as a validation cohort in a previous comparison of existing stroke scales.2 In summary, the database consists of pseudonymised data of consecutive patients (including comatose) with preclinical ‘suspected acute CNS disorder’ admitted to the Emergency Room of the Department of Neurology, Heidelberg University Hospital, Germany, by EMS between November 2007 and August 2010. For all patients, a full-length NIHSS score assessed by certified raters was available at admission. The diagnostic reference standard was the diagnosis at hospital discharge. Cases were dichotomised (by the authors AE and CH) in stroke and non-stroke, that is, stroke mimics. AE and CH were blinded for the admission NIHSS and sNIHSS-EMS scores.

Test characteristics regarding the prediction of LVO were calculated in a prospectively collected second cohort consisting of consecutive patients with acute ischaemic stroke, admitted to the Department of Neurology, Tuebingen University Hospital, Germany, between January 2013 and July 2015. In accordance with local standard operating procedures, all received acute vessel imaging on admission independent of stroke severity. Neuroradiological reports and original images were reviewed by the authors HR and SP for presence of acute LVO. HR and SP were blinded to patients’ NIHSS scores. Cases were considered as LVO positive if an acute symptomatic occlusion was present in one of the following arteries: common carotid artery, internal carotid artery, carotid T, middle cerebral artery (including M1/M2 segments), anterior cerebral artery, basilar artery or posterior cerebral artery.

### Statistics

To determine suitable items for use in the prehospital phase, we analysed the online survey response data set; median and IQRs were calculated. NIHSS items receiving median scores of 0 and 1 were—as predefined—regarded eligible for further consideration. Rating differences between the professional groups (ie, non-neurological EMS personnel and stroke physicians) were determined using the Mann-Whitney U test. For the calculation of test performance regarding stroke recognition, the sNIHSS-EMS score was dichotomised as indicative of stroke (score ≥1), or not (score=0). Sensitivity (the proportion of patients with stroke who had a positive test, ie, indicative of stroke) and specificity (the proportion of non-stroke patients who had a negative test), positive predictive value (PPV) and negative predictive value (NPV) were calculated with 95% CIs. Details of the sample size calculation are described in the extended methods in the online supplementary appendix. To determine the predictive power for LVO detection, we calculated sensitivity, specificity, PPV and NPV, with 95% CI for each scale score ranging from 0 to 29 for the sNIHSS-EMS, and from 0 to 42 for the original NIHSS. Accuracy is reported additionally. Receiver operating curve (ROC) analysis was performed, area under the curve (AUC) and Youden index were calculated. For comparison of the sNIHSS-EMS with existing dedicated LVO prediction scales,7 10–12 14 15 we calculated the corresponding scores using the NIHSS equivalents and cut-offs as stated in the original publications. Statistical comparison of AUCs was performed according to DeLong *et al*.21 Calculation of the Los Angeles Motor Scale (LAMS) for our LVO cohort was not possible since the item ‘grip-strength’ was not routinely documented. p Values were two sided with values less than .05 considered statistically significant. SPSS (V.23.0.0.2, IBM), MedCalc (V.16.8.4, Ostend, Belgium) and GraphPad Prism (V.6.0b, San Diego, California, USA) were used for data handling and analysis, and graphic presentation. This study was performed in accordance with the STARD guidelines for studies on diagnostic tests.

## Results

### Scale development

Three hundred twenty-six (13%) of 2562 recipients responded to our international online survey (Austria, Germany and Switzerland), with the majority (57%) representing non-neurological EMS personnel (33% paramedics and 24% prehospital emergency physicians); 33% stroke physicians and 10% not specified. Participants reported a high level of professional experience (>10 years, 45%; <5 years, 20%).

Nine of the NIHSS items received a median score of 0 or 1 (equivalent to most suitable and suitable for prehospital use), whereas the items ‘best gaze’, ‘visual’, ‘limb ataxia’ and ‘extinction’ were rated as less suitable and thus removed from further analyses (table 1A in the online supplementary appendix). Although rating by stroke physicians was more rigorous, item selection based on median ratings of 0 or 1 was not shifted by the professional vote (table 1A).

We decided to exclude item 1b (level of consciousness (LOC) questions) and item 1c (LOC commands). Despite being easily assessable and thus rated suitable for prehospital use, these two items are either present in the absence of stroke as frequent features of non-stroke conditions (eg, dementia, infection or dehydration)22 or heavily influenced by aphasia23 and thus redundant for stroke recognition. The new 7-item scale was termed ‘shortened NIHSS for emergency medical services’ (sNIHSS-EMS; table 1).

View this table:
[Table 1](http://bmjopen.bmj.com/content/7/9/e016893/T1)

Table 1 
The sNIHSS-EMS

### Stroke recognition and severity grading

In our stroke recognition validation cohort of 689 consecutive patients with ‘suspected acute CNS disorder,’ 29% received ‘stroke’ as discharge diagnosis. Patients with ischaemic stroke (n=200) had an admission NIHSS of 9 (IQR 4–17), patients with haemorrhagic stroke (n=55) of 17 (IQR 5–35). Non-stroke patients (n=489) had a median admission NIHSS of 1 (IQR 0–6). The sNIHSS-EMS was found to have 90.5% (95% CI 85.6 to 94.2) sensitivity and 51.5% (95% CI 47.0 to 56.1) specificity for stroke recognition (PPV 43.3% (95% CI 38.5 to 48.2), NPV 93.0% (95% CI 89.3 to 95.6)). Cross tabulations are shown in table 2A in the online supplementary appendix. Excluding patients in a coma (n=49), sensitivity was 89.1% (95% CI 83.6 to 93.3) and specificity was 54.2% (95% CI 49.5 to 58.8).

### LVO prediction

In the distinct LVO validation cohort of consecutive 741 patients with ischaemic stroke with acute vessel imaging independent of their admission NIHSS score (86.9% CT-angiography (CT-A); see table 3A in the online supplementary appendix for patient characteristics), an ROC analysis of the sNIHSS-EMS regarding LVO prediction revealed a maximal Youden index at the cut-point of ≥6 (sensitivity 70.3% (95% CI 64.7 to 75.5), specificity 80.7% (95% CI 76.8 to 84.3); figure 1, table 2). For comparison, in the original NIHSS, the maximal Youden index was calculated for a cut-point of ≥9 (table 2). Combined reinclusion of the NIHSS items ‘visual’, ‘gaze’ and ‘extinction’ improved test characteristics (AUC 0.826 vs 0.808, p<0.001). Reinclusion of singular items did not improve test characteristics. Exclusion of patients in a coma (n=5) did not change the optimal cut-off and test characteristics (sensitivity 70.0% (64.4–75.3), specificity 81.1% (77.1–84.6)).

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/7/9/e016893/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/7/9/e016893/F1)

Figure 1 
Receiver operating curves for prediction of acute large vessel occlusion. 3I-SS, 3-item Stroke Scale; CPSSS, Cincinnati Prehospital Stroke Severity Scale; FAST-ED, Field Assessment Stroke Triage for Emergency Destination; NIHSS, NIH Stroke Scale; PASS, Prehospital Acute Stroke Severity Scale; RACE, Rapid Arterial Occlusion Evaluation scale; ref, reference; sNIHSS-EMS, shortened NIHSS for emergency medical services.

View this table:
[Table 2](http://bmjopen.bmj.com/content/7/9/e016893/T2)

Table 2 
Cut-off points for prediction of acute large vessel occlusion

We validated the sNIHSS-EMS against existing LVO prediction scales through applying them to our cohort and calculation of ROC and Youden indices (table 3, figure 1). No statistically significant differences compared with existing scales were found, except for the full-length NIHSS, and the sNIHSS-8. Notably, due to characteristics of our cohort, external validation based on maximal Youden indices led to cut-points different from those reported in the respective original publications (table 3).

View this table:
[Table 3](http://bmjopen.bmj.com/content/7/9/e016893/T3)

Table 3 
Comparison of clinical scales for prehospital prediction of LVO

## Discussion

The sNIHSS-EMS is the first comprehensive stroke scale assessed for parallel stroke recognition, severity grading and LVO prediction. Test characteristics regarding identification of patients with LVO are non-inferior to existing LVO prediction scales. Furthermore, compatibility with the item assessment in the full-length NIHSS allows for continuous evaluation of the clinical course from pre to in-hospital care. It may thus represent the ideal stroke scale for routine use in prehospital emergency medical care.

As previously shown by our work group,2 some of the available stroke severity scales3 6 may be used for stroke recognition with similar sensitivity and specificity when compared with scales developed for stroke recognition alone. Existing scales, however, either include items requiring complex assessment (such as extinction11 15) or exclude items highly relevant for evaluation of stroke progression (such as level of consciousness, arm or leg motor function3 7 11).

Sensitivity of the sNIHSS-EMS regarding stroke recognition (91%) was superior to previously published results for the simpler Cincinnati Prehospital Stroke Scale (CPSS; 85%) and Field Assessment Stroke Triage (FAST; 87%) evaluated in the same cohort of patients.2 In contrast, specificity (52%) was lower compared with the CPSS (65%) and FAST (64%).2 As the overall burden of a missed stroke outweighs the potentially increased workload of emergency departments, higher sensitivity may be considered more relevant. Simpler stroke scales may provide a slightly faster initial assessment, but subsequently require the use of at least one additional scale to determine stroke severity or predict LVO. The use of multiple scales, however, may be error prone and complicates communication with receiving hospitals.

According to recent European and American recommendations, clinical screening tools may be considered in order to facilitate direct transport of patients with suspected LVO to Comprehensive Stroke Centers (CSC) with endovascular facility.20 24 For LVO prediction, our analysis revealed a maximum Youden index for the cut-point of ≥6 for the sNIHSS-EMS and, in accordance with previous findings, 9 for the original NIHSS.25 Importantly, to adjust for hospital capacities and local stroke network requirements, this threshold can be adapted: higher cut-points result in an increased specificity (table 2) leading to reduced numbers of patients bypassing Acute Stroke Ready Hospitals (ASRH) or Primary Stroke Centers (PSC) without endovascular facility.

The NIHSS items ‘visual’, ‘gaze’ and ‘extinction’ are part of some dedicated LVO prediction scales,10 11 14 15 but were not included in the sNIHSS-EMS due to unfavourable ratings regarding prehospital assessability. Reinclusion of each separate item did not result in the presumed higher predictive value for LVO detection. Only combined reinclusion of all three rejected items led to marginally enhanced test characteristics, but would result in a significantly increased number of complex-to-assess items and thus an inconvenient scale.

For comparison with existing scales, we externally validated dedicated LVO prediction scales in our cohort by using the cut-points as provided in the original publications and found the sNIHSS-EMS to offer comparable sensitivity and specificity (table 3). Better test characteristics reported in the original publications for some scales may be due to differences in the definition of LVO (eg, the 3I-SS (3-item stroke scale) focused on carotid T and M1 occlusions only,10 while the LAMS also included M3/4 occlusions16). The sNIHSS-8, which had a higher AUC in the ROC analysis than the sNIHSS-EMS, was not developed for LVO prediction and includes items rejected by EMS personnel in our survey due to the complexity of correct assessment.

LVO prediction by clinical scales has recently been criticised due to the high false-negative rate compared with vessel imaging.17 26 The sNIHSS-EMS is not intended to substitute in-hospital acute vessel imaging,17 and prehospital acute vessel imaging is still an exception.27 Currently, mainly due to the narrow time window for effective intravenous thrombolysis, patients are transferred to the closest stroke centre regardless of LVO suspicion. In the era of interventional thrombectomy however, ASRH or PSC may have to be bypassed in favour of CSC with endovascular facility in sensibly selected cases.

Based on clinical criteria alone, the sNIHSS-EMS identifies the majority of patients with acute LVO, that is, those patients who might benefit from a direct transfer to CSC with endovascular facility. In addition, the minority of patients with LVO not bypassed to endovascular ready CSC (ie, total score <6 despite LVO) are not lost to endovascular therapy since secondary transportation to an endovascular ready CSC is still possible.

The sNIHSS-EMS is designed to permit the monitoring of stroke progression from pre to in-hospital care on the item level, a feature that has been neglected in other scales. Clinical implications include the earlier recognition of symptom fluctuation with consequences, for example, for blood pressure management or selection of imaging modality. In practice, if a ‘2’ is scored for ‘Motor Leg left’ on the sNIHSS-EMS, a ‘4’ on the same item during routine NIHSS evaluation in the Emergency Room points to early clinical deterioration. Clinical scores using merged items (eg, ‘hemiparesis’10 or ‘language/dysarthria’)4 or modified item scoring (eg, motor function scoring from 0 to 2 instead of 0 to 4)11 12 14–16 impede seamless monitoring of symptom progression.

Despite the positive aspects of the sNIHSS-EMS, some limitations of the present study require further discussion. Test characteristics regarding LVO prediction were calculated in a cohort of patients with confirmed ischaemic stroke because determination of the ‘true’ LVO prediction threshold is only possible in a cohort without stroke mimics or haemorrhagic stroke. However, although this approach is in concordance with methods used in the past in the design of dedicated LVO prediction scales,12 14 16 future prospective validation in the prehospital target population will be necessary to determine prevalence-dependent test characteristics. We were not able to assess LVO prediction of the LAMS because the item ‘grip strength’ is not part of the NIHSS, and thus was not routinely documented in our cohort. According to a retrospective validation study in anterior circulation stroke, the sensitivity of the LAMS for LVO prediction was reported as 81% (at a threshold of 4).16 As patients with stroke mimics (and thus no LVO) exhibit low NIHSS scores, inclusion of these cases into the analyses would lead to an increased specificity of our cut-points. The sNIHSS-EMS is not able to differentiate between ischaemic and haemorrhagic strokes. This might not be a disadvantage as patients severely affected with haemorrhagic stroke benefit from direct admission to a CSC with neurological intensive care capacity.28 Despite involvement of EMS systems from three European countries, generalisability to further EMS systems around the world cannot be concluded. The low response rate of our online survey makes a non-response bias likely. Due to the participants’ high professional experience, one might have expected a shift of the suitability assessment towards more complex items. However, this was not observed. As a strength of this study, LVO was evaluated by CT-A or MR-angiography, and not with less accurate duplex sonography as done in previous studies evaluating LVO prediction scales.11 16 The sNIHSS-EMS was primarily designed to fulfil requirements for prehospital use. Although kept simple, additional training on the new scale is recommended. Moreover, the sNIHSS-EMS may also serve in telemedicine with usually non-neurological physicians performing the initial patient examination.

## Conclusion

The sNIHSS-EMS may overcome the need for sequential use of multiple emergency stroke scales by enabling parallel stroke recognition, severity grading and LVO prediction. Full NIHSS-item compatibility permits evaluation of stroke progression starting from the prehospital phase. Offering comparable test characteristics as dedicated scales, the sNIHSS-EMS may be a promising tool for rapid and comprehensive prehospital stroke assessment and triage.

## Acknowledgments

We thank all survey participants for their contribution. Those who gave consent to the publication of their names are listed in the online supplementary appendix. We thank Louise Alice Härtig for language revision of the manuscript. We acknowledge support by Deutsche Forschungsgemeinschaft and Open Access Publishing Fund of University of Tübingen.

## Footnotes

*   Contributors SP and JCP conceived and designed the study. EP provided the EMS data. HR and SP created and validated the LVO prediction database. SP, JCP, AE and CH collected and analysed the data. JH and JA developed and maintained the online survey. JCP, FH and SP drafted the article. JCP, HR, FH, CH, PAR, SN and SP revised the manuscript.

*   Competing interests Personal fees, travel support, speaker honoraria or research grants were received from Bayer Healthcare (FH, HR, PAR, SN, SP), BeneChill (SP), BMS Pfizer (PAR, SP), Boehringer 8 Ingelheim (JCP, PAR, SN, SP), Brainomix Ltd. (SN), Covidien (SP), C.R. Bard (SP), EMCOOLS (SP), Helena Laboratories (FH, SP), HVM Medical (SP), Medtronic (SN), Pfizer (JCP, SN), Raumedic (SP) and ZOLL (SP). The other authors report no conflicts of interest.

*   Ethics approval Ethical approvals were obtained from the ethics committee of the Medical Faculty Heidelberg and the ethics committee of the University Hospital Tuebingen, Germany (protocol 8 numbers S8109/2013 and 648/2015BO2, respectively). Written informed consent was waived.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement No additional data are available.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) 

## References

1.  1. Brandler ES , Sharma M , Sinert RH , et al . Prehospital stroke scales in urban environments: a systematic review. Neurology 2014;82:2241–9.[doi:10.1212/WNL.0000000000000523](http://dx.doi.org/10.1212/WNL.0000000000000523) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1212/WNL.0000000000000523&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24850487&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

2.  2. Purrucker JC , Hametner C , Engelbrecht A , et al . Comparison of stroke recognition and stroke severity scores for stroke detection in a single cohort. J Neurol Neurosurg Psychiatry 2015;86:1021–8.[doi:10.1136/jnnp-2014-309260](http://dx.doi.org/10.1136/jnnp-2014-309260) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiam5ucCI7czo1OiJyZXNpZCI7czo5OiI4Ni85LzEwMjEiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNy85L2UwMTY4OTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  3. Tirschwell DL , Longstreth WT , Becker KJ , et al . Shortening the NIH Stroke scale for use in the prehospital setting. Stroke 2002;33:2801–6.[doi:10.1161/01.STR.0000044166.28481.BC](http://dx.doi.org/10.1161/01.STR.0000044166.28481.BC) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjEwOiIzMy8xMi8yODAxIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpvcGVuLzcvOS9lMDE2ODkzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  4. Iguchi Y , Kimura K , Watanabe M , et al . Utility of the Kurashiki Prehospital Stroke Scale for hyperacute stroke. Cerebrovasc Dis 2011;31:51–6.[doi:10.1159/000320854](http://dx.doi.org/10.1159/000320854) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1159/000320854&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20980754&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

5.  5. Hasegawa Y , Sasaki N , Yamada K , et al . Prediction of thrombolytic therapy after stroke-bypass transportation: the Maria Prehospital Stroke Scale score. J Stroke Cerebrovasc Dis 2013;22:514–9.[doi:10.1016/j.jstrokecerebrovasdis.2013.02.007](http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2013.02.007) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jstrokecerebrovasdis.2013.02.007&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23489953&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

6.  6. Kimura K , Inoue T , Iguchi Y , et al . Kurashiki prehospital stroke scale. Cerebrovasc Dis 2008;25:189–91.[doi:10.1159/000113739](http://dx.doi.org/10.1159/000113739) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18219200&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

7.  7. Llanes JN , Kidwell CS , Starkman S , et al . The Los Angeles Motor Scale (LAMS): a new measure to characterize stroke severity in the field. Prehosp Emerg Care 2004;8:46–50.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1080/312703002806&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14691787&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

8.  8. Meyer BC , Hemmen TM , Jackson CM , et al . Modified National Institutes of Health Stroke Scale for use in stroke clinical trials: prospective reliability and validity. Stroke 2002;33:1261–6.[doi:10.1161/01.STR.0000015625.87603.A7](http://dx.doi.org/10.1161/01.STR.0000015625.87603.A7) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjMzLzUvMTI2MSI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzkvZTAxNjg5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

9.  9. Whelley-Wilson CM , Newman GC . A stroke scale for emergency triage. J Stroke Cerebrovasc Dis 2004;13:247–53.[doi:10.1016/j.jstrokecerebrovasdis.2004.04.005](http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2004.04.005) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jstrokecerebrovasdis.2004.04.005&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17903982&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

10. 10. Singer OC , Dvorak F , du Mesnil de Rochemont R , et al . A simple 3-item stroke scale: comparison with the National Institutes of Health Stroke Scale and prediction of middle cerebral artery occlusion. Stroke 2005;36:773–6.[doi:10.1161/01.STR.0000157591.61322.df](http://dx.doi.org/10.1161/01.STR.0000157591.61322.df) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjg6IjM2LzQvNzczIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpvcGVuLzcvOS9lMDE2ODkzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

11. 11. Pérez de la Ossa N , Carrera D , Gorchs M , et al . Design and validation of a prehospital stroke scale to predict large arterial occlusion: the rapid arterial occlusion evaluation scale. Stroke 2014;45:87–91.[doi:10.1161/STROKEAHA.113.003071](http://dx.doi.org/10.1161/STROKEAHA.113.003071) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjc6IjQ1LzEvODciO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNy85L2UwMTY4OTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

12. 12. Katz BS , McMullan JT , Sucharew H , et al . Design and validation of a prehospital scale to predict stroke severity: cincinnati prehospital stroke severity scale. Stroke 2015;46:1508–12.[doi:10.1161/STROKEAHA.115.008804](http://dx.doi.org/10.1161/STROKEAHA.115.008804) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ2LzYvMTUwOCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzkvZTAxNjg5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

13. 13. Teleb MS , Ver Hage A , Carter J , et al . Stroke vision, aphasia, neglect (VAN) assessment-a novel emergent large vessel occlusion screening tool: pilot study and comparison with current clinical severity indices. J Neurointerv Surg 2017;9:122–6.[doi:10.1136/neurintsurg-2015-012131](http://dx.doi.org/10.1136/neurintsurg-2015-012131) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTE6Im5ldXJpbnRzdXJnIjtzOjU6InJlc2lkIjtzOjc6IjkvMi8xMjIiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNy85L2UwMTY4OTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

14. 14. Hastrup S , Damgaard D , Johnsen SP , et al . Prehospital Acute Stroke Severity Scale to predict Large artery occlusion: design and comparison with Other scales. Stroke 2016;47:1772–6.[doi:10.1161/STROKEAHA.115.012482](http://dx.doi.org/10.1161/STROKEAHA.115.012482) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ3LzcvMTc3MiI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzkvZTAxNjg5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

15. 15. Lima FO , Silva GS , Furie KL , et al . Field Assessment Stroke Triage for Emergency Destination: a simple and Accurate Prehospital Scale to detect large vessel Occlusion strokes. Stroke 2016;47:1997–2002.[doi:10.1161/STROKEAHA.116.013301](http://dx.doi.org/10.1161/STROKEAHA.116.013301) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ3LzgvMTk5NyI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzkvZTAxNjg5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

16. 16. Nazliel B , Starkman S , Liebeskind DS , et al . A brief prehospital stroke severity scale identifies ischemic stroke patients harboring persisting large arterial occlusions. Stroke 2008;39:2264–7.[doi:10.1161/STROKEAHA.107.508127](http://dx.doi.org/10.1161/STROKEAHA.107.508127) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjM5LzgvMjI2NCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzkvZTAxNjg5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

17. 17. Heldner MR , Hsieh K , Broeg-Morvay A , et al . Clinical prediction of large vessel occlusion in anterior circulation stroke: mission impossible? J Neurol 2016;263:1633–40.[doi:10.1007/s00415-016-8180-6](http://dx.doi.org/10.1007/s00415-016-8180-6) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1007/s00415-016-8180-6&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=27272907&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

18. 18. Vanacker P , Heldner MR , Amiguet M , et al . Prediction of large vessel occlusions in acute Stroke: national Institute of Health Stroke Scale is hard to beat. Crit Care Med 2016;44:e336–43.[doi:10.1097/CCM.0000000000001630](http://dx.doi.org/10.1097/CCM.0000000000001630) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/CCM.0000000000001630&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=26958750&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

19. 19. Rinaldo L , Brinjikji W , McCutcheon BA , et al . Hospital transfer associated with increased mortality after endovascular revascularization for acute ischemic stroke. J Neurointerv Surg 2016:neurintsurg-2016-012824.[doi:10.1136/neurintsurg-2016-012824](http://dx.doi.org/10.1136/neurintsurg-2016-012824) 
    
    

20. 20. Pride GL , Fraser JF , Gupta R , et al . Prehospital care delivery and triage of stroke with emergent large vessel occlusion (ELVO): report of the Standards and guidelines Committee of the Society of Neurointerventional Surgery. J Neurointerv Surg 2017;9.[doi:10.1136/neurintsurg-2016-012699](http://dx.doi.org/10.1136/neurintsurg-2016-012699) 
    
    

21. 21. DeLong ER , DeLong DM , Clarke-Pearson DL . Comparing the Areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45.[doi:10.2307/2531595](http://dx.doi.org/10.2307/2531595) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2307/2531595&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=3203132&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1988Q069100016&link_type=ISI) 

22. 22. Nor AM , Davis J , Sen B , et al . The recognition of Stroke in the emergency room (ROSIER) scale: development and validation of a stroke recognition instrument. Lancet Neurol 2005;4:727–34.[doi:10.1016/S1474-4422(05)70201-5](http://dx.doi.org/10.1016/S1474-4422(05)70201-5) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S1474-4422(05)70201-5&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16239179&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000233002400017&link_type=ISI) 

23. 23. Majerus S , Bruno MA , Schnakers C , et al . The problem of aphasia in the assessment of consciousness in brain-damaged patients. Prog Brain Res 2009;177:49–61.[doi:10.1016/S0079-6123(09)17705-1](http://dx.doi.org/10.1016/S0079-6123(09)17705-1) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0079-6123(09)17705-1&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19818894&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000280613100008&link_type=ISI) 

24. 24. Fiehler J , Cognard C , Gallitelli M , et al . European recommendations on Organisation of Interventional Care in acute stroke (EROICAS). Int J Stroke 2016;11:701–16.[doi:10.1177/1747493016647735](http://dx.doi.org/10.1177/1747493016647735) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/1747493016647735&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=27462090&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 

25. 25. Heldner MR , Zubler C , Mattle HP , et al . National Institutes of Health stroke scale score and vessel occlusion in 2152 patients with acute ischemic stroke. Stroke 2013;44:1153–7.[doi:10.1161/STROKEAHA.111.000604](http://dx.doi.org/10.1161/STROKEAHA.111.000604) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ0LzQvMTE1MyI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzkvZTAxNjg5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

26. 26. Turc G , Maïer B , Naggara O , et al . Clinical scales do not reliably identify acute ischemic stroke patients with Large-Artery occlusion. Stroke 2016;47:1466–72.[doi:10.1161/STROKEAHA.116.013144](http://dx.doi.org/10.1161/STROKEAHA.116.013144) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToic3Ryb2tlYWhhIjtzOjU6InJlc2lkIjtzOjk6IjQ3LzYvMTQ2NiI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzkvZTAxNjg5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

27. 27. John S , Stock S , Masaryk T , et al . Performance of CT Angiography on a Mobile Stroke treatment unit: implications for triage. J Neuroimaging 2016;26:391–4.[doi:10.1111/jon.12346](http://dx.doi.org/10.1111/jon.12346) 
    
    

28. 28. Diringer MN , Edwards DF . Admission to a neurologic/neurosurgical intensive care unit is associated with reduced mortality rate after intracerebral hemorrhage. Crit Care Med 2001;29:635–40.[doi:10.1097/00003246-200103000-00031](http://dx.doi.org/10.1097/00003246-200103000-00031) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/00003246-200103000-00031&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11373434&link_type=MED&atom=%2Fbmjopen%2F7%2F9%2Fe016893.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000167522400024&link_type=ISI)