Objectives The National Early Warning Score (NEWS) helps to estimate mortality risk in emergency department (ED) patients. This study aimed to investigate whether the prognostic value of the NEWS at ED admission could be further improved by adding inflammatory blood markers (ie, white cell count (WCC), procalcitonin (PCT) and midregional-proadrenomedullin (MR-proADM).
Design Secondary analysis of a multinational, observational study (TRIAGE study, March 2013–October 2014).
Setting Three tertiary care centres in France, Switzerland and the USA.
Participants A total of 1303 adult medical patients with complete NEWS data seeking ED care were included in the final analysis. NEWS was calculated retrospectively based on admission data.
Main outcome measures The primary outcome was all-cause 30-day mortality. Secondary outcome was intensive care unit (ICU) admission. We used multivariate regression analyses to investigate associations of NEWS and blood markers with outcomes and area under the receiver operating curve (AUC) as a measure of discrimination.
Results Of the 1303 included patients, 54 (4.1%) died within 30 days. The NEWS alone showed fair prognostic accuracy for all-cause 30-day mortality (AUC 0.73), with a multivariate adjusted OR of 1.26 (95% CI 1.13 to 1.40, p<0.001). The AUCs for the prediction of mortality using the inflammatory markers WCC, PCT and MR-proADM were 0.64, 0.71 and 0.78, respectively. Combining NEWS with all three blood markers or only with MR-proADM clearly improved discrimination with an AUC of 0.82 (p=0.002). Combining the three inflammatory markers with NEWS improved prediction of ICU admission (AUC 0.70vs0.65 when using NEWS alone, p=0.006).
Conclusion NEWS is helpful in risk stratification of ED patients and can be further improved by the addition of inflammatory blood markers. Future studies should investigate whether risk stratification by NEWS in addition to biomarkers improve site-of-care decision in this patient population.
Trial registration number NCT01768494; Post-results.
- risk management
- internal medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
This is the first multinational study investigating the association of the National Early Warning Score (NEWS) and adverse outcomes at emergency department admission.
This is the first study evaluating additional impact of inflammatory biomarkers on NEWS.
Due to its design as a secondary analysis of an observational study, results are at best hypothesis generating.
There is the possibility of a selection bias due to exclusion of patients with missing vital status data.
With increasingly overwhelmed emergency departments (EDs), it is vital that well-validated risk stratification systems are implemented to rapidly identify and efficiently respond to patients at risk since delays in treatment may result in poor outcomes.1 2 At the same time, risk stratification systems should ensure reliable identification of patients who may not need urgent care. Risk scores such as the pneumonia severity index for patients presenting with pneumonia3 or the Global Registry of Acute Coronary Events risk score for patients with myocardial infarction are applicable mainly to specific patient populations.4
Emerging data suggest that in unselected medical patients presenting to the ED, early warning scores (EWS) (ie, track-and-trigger systems) are gaining importance for risk stratification and that their usage for this purpose may exceed that of classical triage tools such as the Manchester Triage System.5 6 Of the numerous EWS that have been investigated for this purpose, the National Early Warning Score (NEWS) may be the best evaluated and most widely used.6 It was originally developed in the UK by the Royal College of Physicians to standardise and improve detection of patients at risk for deterioration, with the routine recording of a minimum of physiological parameters.7 8 Multiple studies have revealed its superiority over other risk stratification tools with regard to prediction of mortality.6 9 Since its introduction in 2012, NEWS has been validated and widely implemented in different patient populations5 10–12 worldwide.5 11 13–16 It has also been used recently for early risk stratification in unselected ED patients and has shown promise in the prediction of mortality.6 The Royal College of Physicians now recommends its use in EDs in its updated NEWS 2 report issued in 2017.7 However, risk stratification at this early stage in patient care immediately following presentation to the ED is challenging due to lack of clinical information. Moreover, NEWS and other triage scores are based on vital signs and represent a patient’s clinical state at a single time point when vital signs are frequently still unremarkable, as a consequence of which they may miss patients at risk.
The use of blood markers assessed at the time of ED presentation may add prognostic information17 and even improve the prognostic value of EWSs. Although investigations trying to improve the discriminative value of NEWS by combining different markers such as serum lactate16 18 or D-dimer19 with NEWS have been undertaken, their results are contradictory. This may be due to inadequate selection of blood markers. The inflammatory blood marker procalcitonin (PCT) is a marker for bacterial infections. It has been employed for stewardship of antibiotic therapy20 and has been shown to improve risk assessment in different patient populations such as those with sepsis21 or malignant disease,22 and in patients at cardiovascular risk.23 Adrenomedullin (ADM) is a potent vasodilator peptide hormone widely expressed by many tissues that acts both as an autocrine and paracrine mediator.24 25 It is released in higher quantities in infectious and inflammatory states.26 27 Midregional proadrenomedullin (MR-proADM) is generated during the processing of the prohormone of ADM, is stable in human plasma and may directly reflect the release of ADM.28 Several studies have revealed MR-proADM to be highly predictive of adverse outcomes in specific patient populations such as those with chronic heart failure,29 sepsis,25 lower respiratory tract infections,30 31 myocardial infarction32 and urinary tract infections.33 Moreover, MR-proADM has been shown to predict adverse outcomes in patients presenting to the ED with non-specific complaints.34
The first aim of this study was to investigate the prognostic accuracy of NEWS in an unselected multinational cohort of medical patients presenting to the ED. In a second step, we aimed to investigate whether adding the inflammatory blood markers white cell count (WCC), PCT and MR-proADM would improve the predictive value of NEWS at the time of ED presentation.
This is a retrospective analysis of data from a multinational, observational cohort study that enrolled patients from March 2013 to October 2014 at three tertiary care hospitals in Aarau (Switzerland), Paris (France) and Clearwater (Florida, USA) with the aim of determining whether the addition of biomarkers from distinct biological pathways would improve early risk stratification and initial triage of patients on ED admission. The study protocol and details regarding the study design have been published previously.35 36
Patient and public involvement
Patients were not involved in the development of the research question or the design of the study.
All adult medical patients seeking ED care at one of the participating hospitals were consecutively enrolled. They were included in this analysis if an initial blood draw had been performed as part of the routine assessment in the ED. Surgical and paediatric patients were excluded. There were no further exclusion criteria so as to reflect patient diversity and challenges of ‘real-life’.
Data collection and score assignment
At time of admission to the ED, all patients were assessed by a triage nurse and assigned an initial triage priority based on the routine hospital algorithm. All participants provided a thorough medical history and underwent a physical examination, including measurement of vital signs and laboratory assessment with collection of leftover blood samples for later analysis. Additionally, we recorded the main presenting clinical symptoms and complaints, sociodemographic characteristics and comorbidities. All information was initially entered in a case report form and subsequently stored in a centralised, password-secured databank (SecuTrial; interActive Systems GmbH, Berlin, Germany).
Throughout the hospital stay, the patients were managed by physicians, nurses and social workers independent of the research team. All patients were contacted by telephone interview 30 days after admission using a predefined questionnaire to assess vital and functional status, unplanned hospital readmission and other clinical outcomes.
The NEWS was calculated retrospectively based on available admission data pertaining to the following six parameters: respiratory rate, oxygen saturation, temperature, blood pressure, pulse rate and level of consciousness. Corresponding to the NEWS 2 score chart of the Royal College of Physicians of the UK,7 continuous variables were awarded a range of 0–3 points, while the level of consciousness was binary coded with 0 points if absent and 3 points if altered. Since data on whether supplemental oxygen was given were not available, the additional 2 points that would have been assigned if supplemental oxygen was given were not included in the calculation. Using the resulting aggregate score, patients were classified into three NEWS categories representing low (0–4 points), moderate (5–6 points), or high (≥7 points) risk. Additionally, patients in the low-risk group scoring 3 points for a single physiological parameter were reclassified in the moderate risk group, as recommended by the Royal College of Physicians.7 As data on supplemental oxygen was not available, results in this paper correspond to a NEWS—potentially minus 2 points.
Study endpoints, overall hypothesis and research aim
The primary endpoint of this analysis was all-cause mortality within 30 days of ED admission. The secondary endpoint was admission to the intensive care unit (ICU) during hospital stay. The decision regarding ICU admission was left to the discretion of the treating physicians. To assess the endpoints, patients were followed throughout their hospital stay, and phone interviews were conducted 30 days after admission. If the patient could not be reached, the patient’s family or general practitioner was contacted. The aim of this secondary analysis was to investigate the association of a single calculation of NEWS at time of admission with the respective outcomes. In a second step, we aimed to assess whether the predictive value of NEWS could be improved by combining these prognostic biomarkers with the NEWS.
Blood draws and biomarkers
We decided to investigate associations of three inflammatory biomarkers namely WCC, MR-proADM and PCT. The WCC was part of the routine laboratory measurement at ED admission. Based on the normal range of WCC (4.0–10.0 g/L), we defined the following cut-offs: <4.0 g/L: representing levels lower than normal, 4.0–10.0 g/L: representing the normal range, 10.01–15.0 g/L: representing low-to-moderate inflammatory response and >15.0 G/L: representing marked inflammation. Both PCT and MR-proADM were batch-measured later from leftover blood samples. The samples were routinely collected at admission, immediately centrifuged, aliquoted and frozen at −20°C. The results of these analyses were not available at time of hospitalisation. Thus, physicians and patients were blinded to their results. The PCT levels were measured with a highly sensitive time-resolved amplified cryptate emission technology assay (PCT Kryptor, B.R.A.H.M.S. AG, Hennigsdorf, Germany) with a lower detection limit of 0.02 µg/L and assay sensitivity of 0.06 µg/L.37 The MR-proADM levels were measured in plasma with a sandwich immunoassay as described elsewhere.28 The assay has an analytical detection limit of 0.08 nmol/L. We defined cut-offs of PCT and MR-proADM corresponding to quartiles.
For statistical analyses, we used STATA V.12.1. Two-tailed tests were used, and p values of <0.05 were considered significant. We used descriptive statistics such as median with quartiles, and frequencies to describe the population, as appropriate. To assess group differences, we used Kruskal-Wallis test for continuous, skew variables, and Pearson’s χ2 test for categorial and binary variables. We performed logistic regression analyses to investigate associations of NEWS and biomarkers with primary and secondary outcomes, respectively. We developed different models with stepwise adjustment for potential important confounders (ie, age, gender, main diagnoses leading to ED admission and comorbidities). Age was used as a linear covariate. According to the main admission diagnosis, the following diagnostic groups were generated: infectious disease, cardiovascular disease, metabolic disorder, malignant disease, neurological disease, gastrointestinal disease, pulmonary disease and other disease. Comorbidities were assigned using patients’ medical history and International Statistical Classification of Diseases and Related Health Problems (ICD-10) diagnostic codes and include chronic obstructive lung disease, heart failure, coronary heart disease, diabetes, hypertension, stroke, malignant disease and renal failure. In statistical models, comorbidities were coded as binary variables. Results of the regression analyses are presented as ORs with 95% CIs. The raw distribution of the biomarker data was skewed. After log transformation with a base of 10, the distribution of the biomarker data approximated a normal distribution. Therefore, ORs correspond to a 10-fold increase in log-transformed values. Regression analyses were repeated in predefined subgroups stratified by diagnoses leading to ED admission. Discriminative performance was determined by means of area under the receiver operating characteristics (AUC), where AUC 0.6–0.7 is considered poor, 0.7–0.8 fair, 0.8–0.9 good, and >0.9 excellent. The AUCs were systematically calculated for univariate models including NEWS and/or inflammatory markers and not for models adjusted for the aforementioned confounders. We used Pearson’s χ2 test to compare AUCs. For further illustration, we generated Kaplan-Meier survival plots by NEWS category and for each of the three biomarkers stratified by quartiles.
Of a total of 7132 patients presenting to the EDs of the participating hospitals (1000 Clearwater, 1553 Paris and 4579 Aarau), 1303 (940 Clearwater, 355 Paris and 8 Aarau) patients had complete information for calculation of NEWS (excluding data on supplemental oxygen) and were included in the final analysis. A comparison of the total cohort and the cohort selected for this analysis can be found in the online supplementary materials table A1. The median age of included patients was 66 years and 50.5% were male. The most frequent main complaints were respiratory symptoms (27.3%), thoracic pain (18.4%) and non-thoracic pain (9.5%). The most prevalent diagnoses at ED admission were cardiovascular disease (37.3%), infections (14.6%), neurological disease (13.5%) and gastrointestinal disease (12.3%). Patients had a high burden of comorbidities including hypertension (43.6%), diabetes (20.6%), coronary heart disease (12.6%) and congestive heart failure (11.8%). Additional baseline characteristics of the general population and stratified by NEWS categories are listed in table 1.
NEWS and mortality
The 30-day mortality rate following admission to the ED was 4.1%. In unadjusted logistic regression analyses, we found a significant association between 30-day mortality and NEWS overall (OR 1.35, 95% CI 1.23 to 1.48, p<0.001) and stratified into risk groups with corresponding ORs of 2.45 (95% CI 1.29 to 4.66, p=0.006) for the moderate risk NEWS category and 7.89 (95% CI 3.85 to 16.18, p<0.001) for the high risk NEWS category, respectively, compared with the low risk NEWS category. These associations remained robust after stepwise adjustment for confounders (see table 2 for univariate and fully adjusted ORs. Data on all of the remaining models are presented in the online supplementary materials table A2). Receiver operating statistics showed fair discriminative performance with regard to 30-day all-cause mortality, with an AUC of 0.73 (95% CI 0.66 to 0.80).
Incremental impact of inflammatory blood markers on prediction of mortality
The inflammatory blood markers WCC, PCT and MR-proADM showed low to fair prognostic accuracy for prediction of 30 day mortality with AUCs of 0.64 (95% CI 0.56 to 0.72), 0.71 (95% CI 0.64 to 0.79) and 0.78 (95% CI 0.73 to 0.84), respectively. Corresponding regression analyses for continuous values and stratified by cut-offs are shown in table 2. Adding all three markers to NEWS significantly improved the predictive value to an AUC of 0.82 (95% CI 0.77 to 0.88, p=0.002). Interestingly, adding only MR-proADM to a model with NEWS showed a similar AUC of 0.82 (95% CI 0.77 to 0.87, p=0.002) for prediction of mortality (table 3, figure 1). We further calculated Kaplan-Meier survival estimates (figure 2).
NEWS and ICU admission
During their hospital stay, 13.1% of patients were admitted to the ICU. Similar to findings with the primary endpoint, unadjusted regression analyses showed significant associations of NEWS with ICU admission (OR 1.25, 95% CI 1.17 to 1.33, p<0.001). Stratification of NEWS by risk categories showed respective ORs of 2.74 (95% CI 1.90 to 3.95, p<0.001) for the moderate risk category and 4.93 (95% CI 2.91 to 8.35, p<0.001) for the high risk category, compared with the low risk category. As before, the results stayed robust after adjusting for important confounders (see table 2 for univariate and fully adjusted ORs. All of the remaining models are presented in the online supplementary materials table A3).
When receiver operating statistics were determined, NEWS showed low discriminative performance with regard to ICU admission (AUC 0.65, 95% CI 0.61 to 0.70).
Incremental impact of inflammatory blood markers on prediction of ICU admission
The predictive value of the blood markers WCC (AUC 0.54, 95% CI 0.49 to 0.59), PCT (AUC 0.62, 95% CI 0.57 to 0.67) and MR-proADM (AUC 0.67, 95% CI 0.62 to 0.72) for ICU admission was low, with only MR-proADM showing slightly better prognostic accuracy than NEWS. For univariate and multivariate regression analyses for associations of blood markers with ICU admission, see table 2. A combined model of NEWS with all three blood markers again improved discriminative performance (AUC 0.70, 95% CI 0.65 to 0.75, p=0.006). Similar to the association with mortality, a model including only NEWS and MR-proADM showed equal results for prediction of ICU admission (AUC 0.70, 95% CI 0.65 to 0.74, p=0.009) (table 3, figure 1).
Analyses of subgroups showed similar association of NEWS with 30-day mortality among different diagnoses leading to ED admission (online supplementary materials figure A1).
This multinational study of heterogenous medical ED patients found a fair performance of the NEWS for prediction of 30-day mortality when calculated at a single time point at ED admission. Results remained robust after adjustment for potential confounders and among different subgroups. Additionally, we found that the predictive value of NEWS was improved by adding inflammatory blood markers, in particular MR-proADM. We also found the discriminative value of NEWS for prediction of our secondary outcome, ICU admission, was less strong but was also improved by adding PCT and MR-proADM to the model.
The 30-day mortality in our study (4.1%) was in line with the 4.0%–5.7% reported in other investigations that included similar patient populations and examined similar outcome measures.19 38 Prevalence of ICU admission during hospital stay in our study (13.1%) was within ICU admission rates in several other studies that reported a range between 1% and 17.4%.5 10 13 16 38 39 This wide range may be a reflection of different healthcare systems studied and particularly of different follow-up periods, as the different investigations examined short-term outcomes within 24 or 48 hours of admission, respectively. In this respect, our results are most notably consistent with the findings of several Scandinavian studies.5 13 15
Regarding predictive performance, other studies investigating discriminative power of NEWS documented at the time of ED arrival found similar AUCs for prediction of in-hospital mortality or 30-day mortality, with reported AUCs of 0.65–0.77 in patients with suspected infection/sepsis40–42 and AUCs of 0.77 to 0.84 in general ED patients, respectively.16 38
Predictive performance for ICU admission in our study (AUC 0.65, 95% CI 0.61 to 0.70) was slightly lower than the AUCs in other publications that report AUCs of 0.67–0.857 for prediction of ICU admission in different patient populations.9 16 41 43
Our finding of MR-proADM as a solitary predictor of 30-day mortality (AUC 0.78, 95% CI 0.73 to 0.84) is in line with the result of another Swiss study reporting an AUC of 0.732 for the same purpose in a cohort of patients with non-specific complaints presenting to the ED.34
To the best of our knowledge, this is the first study investigating NEWS in a multinational cohort of medical ED patients. Moreover, this is the first study investigating the potential additional impact of promising inflammatory markers, namely PCT and MR-proADM, on NEWS for the prediction of adverse outcome.
One could argue that adding blood markers to a clinical score might complicate its calculation, but in EDs, initial blood draws are part of routine care, which is why the additional determination of inflammatory marker levels do not change existing processes. In contrast, WCC results are available rapidly, and indeed PCT point-of-care tests that provide results within minutes are being developed.44 Measurements of additional blood markers thus might partially overcome user-dependency of EWSs and might therefore improve early risk stratification. Our study reveals that MR-proADM could be a particularly suitable and promising blood marker for early risk stratification when combined with a clinical score such as NEWS. As a result, identification of patients needing urgent care could be improved. However, right now, there is no point-of-care test available that would allow rapid measurement of MR-proADM. Clinicians would likely evaluate results from biomarker assays in the light of the patient’s condition at the time the test results become available, not at time the blood sample was drawn. As ED patients are in a dynamic situation, NEWS and the patient’s condition might have changed already when results become available.
This study has some limitations. First, since it is a study where NEWS was calculated retrospectively, associations between NEWS and biomarkers and outcomes are likely confounded. We addressed this limitation at least partially by adjusting for important confounders. However, with residual confounding being likely, our results are at best hypothesis generating. Second, NEWS was calculated retrospectively, and vital signs were not measured more frequently than standard clinical practice. However, we addressed possible bias regarding a treatment paradox since observed mortality may be lower in studies where NEWS is acted on. Third, due to missing vital sign data—particularly respiratory rate—a large number of patients were not eligible for final analysis. This is in line with other studies reporting that EWS were often incomplete45 and that among others, respiratory rate was documented in only 30%–60% of cases.45 46 However, our sample is still relatively large and represents a multinational cohort of unselected medical ED patients. Moreover, given that health system organisation strongly influences populations at EDs, the multinational nature of our study provides external validity. However, there is the possibility of a selection bias as the cohort included in this analysis differed to the initial total cohort in age, some of the main symptoms and main diagnoses at ED admission and comorbidities. This was addressed in the regression models by adjusting for the aforementioned confounders. The results remained robust. Fourth, information on the use of oxygen support was not available retrospectively and therefore did not contribute to NEWS calculation, which reduced the maximum score from 20 to 18. This might have led to misclassification of patients and might have diminished the discriminatory power of NEWS. It is likely that the fraction of patients that were under-rated by 2 points due to missing information on oxygen supplements would increase from the low via the moderate to the high-risk NEWS group. This could represent at systematical bias where sicker patients (probably with worse outcomes) would potentially be misclassified with fewer points in the NEWS score, possibly resulting in statistical inflation of the effect of the lower NEWS values. However, the Royal College of Physicians recommends addition of a weighting score of 2 not by default for all patients with supplemental oxygen but only for patients requiring supplemental oxygen to maintain their optimal oxygen saturation.7 As the optimal oxygen saturation may be different in varying patient groups, most notably in patients with hypercapnic respiratory failure, the actual requirement of supplemental oxygen is hard to determine and requires evaluation of a qualified and experienced physician. As a result, certain patients might mistakenly score 2 additional points for supplemental oxygen, which can again result in misclassification of the NEWS risk category. However, as mentioned before, the discriminative power of NEWS in our study is comparable with other similar investigations. Fifth, we only had a few events that could limit reliability and is reflected in the rather broad 95% CIs. This was addressed at least in part by hard endpoints and a structured follow-up with phone interviews 30 days after admission or through contacting the patient’s family or general practitioner. Sixth, the decision regarding ICU admission was left to the discretion of treating physicians. This reflects procedures followed in the included centres and may be different from those in other hospitals. Last, NEWS was not designed to be a single time point tool but rather a ‘track-and-trigger’ system in individual patients. Accuracy of NEWS may thus be different if multiple measurements at different time points are considered.
Combining NEWS calculated on admission to the ED with markers of inflammation such as MR-proADM and PCT improves the predictive value of NEWS in unselected medical patients. The combination of NEWS and MR-proADM might prove to be a particularly promising tool for early risk stratification. Whether these theoretical benefits of improved risk stratification at ED admission can be translated into improved outcomes has to be examined in future interventional studies.
This multidisciplinary and interprofessional trial was only possible in close collaboration with personnel from multiple departments within the Kantonsspital Aarau, including social services (Anja Keller and Regina Schmid), nursing (Susanne Schirlo and Petra Tobias), central laboratory (Martha Kaeslin and Renate Hunziker), medical control (Juergen Froehlich, Thomas Holler and Christoph Reemts) and information technology (Roger Wohler, Kurt Amstad, Ralph Dahnke and Sabine Storost), the Clinical Trial Unit at the University Hospital of Basel (Stefan Felder and Timo Tondelli) and all participating patients, nurses and physicians. Editorial assistance was provided by Prasad Kulkarni, PhD, CMPP, of Asclepius Medical Communications LLC, Ridgewood, New Jersey, USA, and was funded by the authors.
Patient consent for publication Not required.
AE and SIH contributed equally.
Contributors PS had complete access to all study data and takes full responsibility for the integrity of the data and the accuracy of the analyses. BM and PS were involved in the conceptualisation and design of the study. PH, DA, AA, AH and PS were responsible for the acquisition, analysis or interpretation of the data. AE, SIH and PS performed the statistical analyses and drafted the manuscript. AK and SH reviewed the draft and revised the manuscript for important intellectual content. All authors approved the final version of the manuscript and the decision to submit the manuscript for publication. AE and SIH contributed equally to this work.
Funding ThermoFisher provided an unrestricted research grant for the TRIAGE study. PS is supported by the Swiss National Science Foundation (SNSF Professorship, PP00P3_150531/1) and the Research Council of the Kantonsspital Aarau (1410.000.044).
Competing interests AK, BM, PH and PS have received research grants and support from BRAHMS AG (now ThermoFisher Scientific Biomarkers) and bioMérieux for attending meetings and fulfilling speaking engagements. BM has served as a consultant to both companies
Ethics approval Since it was an observational quality control study, the Institutional Review Boards (IRB) of the three hospitals approved the study and waived the need for individual informed consent (main Swiss IRB: Ethikkommission Kanton Aargau (EK 2012/059); French IRB: CCTIRS—Le Comité consultatif sur le traitement de l’information en matière de recherche (C.C.T.I.R.S.; CPP ID RCB: 2013-A00129-36); US IRB MPM-SAH Institutional Review Board, Clearwater Florida (IRB number 2013_005)).
Provenance and peer review Not commissioned; externally peer reviewed.