Improving Recognition of Pediatric Severe Sepsis in the Emergency Department: Contributions of a Vital Sign–Based Electronic Alert and Bedside Clinician Identification
Recognition of pediatric sepsis is a key clinical challenge. We evaluate the performance of a sepsis recognition process including an electronic sepsis alert and bedside assessment in a pediatric emergency department (ED).
Methods
This was a cohort study with quality improvement intervention in a pediatric ED. Exposure was a positive electronic sepsis alert, defined as elevated pulse rate or hypotension, concern for infection, and at least one of the following: abnormal capillary refill, abnormal mental status, or high-risk condition. A positive electronic sepsis alert prompted team assessment or huddle to determine need for sepsis protocol. Clinicians could initiate team assessment or huddle according to clinical concern without positive electronic sepsis alert. Severe sepsis outcome defined as activation of the sepsis protocol in the ED or development of severe sepsis requiring ICU admission within 24 hours.
Results
There were 182,509 ED visits during the study period, with 86,037 before electronic sepsis alert implementation and 96,472 afterward, and 1,112 (1.2%) positive electronic sepsis alerts. Overall, 326 patients (0.3%) were treated for severe sepsis within 24 hours. Test characteristics of the electronic sepsis alert alone to detect severe sepsis were sensitivity 86.2% (95% confidence interval [CI] 82.0% to 89.5%), specificity 99.1% (95% CI 99.0% to 99.2%), positive predictive value 25.4% (95% CI 22.8% to 28.0%), and negative predictive value 100% (95% CI 99.9% to 100%). Inclusion of the clinician screen identified 43 additional electronic sepsis alert–negative children, with severe sepsis sensitivity 99.4% (95% CI 97.8% to 99.8%) and specificity 99.1% (95% CI 99.1% to 99.2%). Electronic sepsis alert implementation increased ED sepsis detection from 83% to 96%.
Conclusion
Electronic sepsis alert for severe sepsis demonstrated good sensitivity and high specificity. Addition of clinician identification of electronic sepsis alert–negative patients further improved sensitivity. Implementation of the electronic sepsis alert was associated with improved recognition of severe sepsis.
Introduction
Severe sepsis is a complex clinical syndrome resulting from the systemic inflammatory response to infection. Each year, more than 75,000 children are treated for severe sepsis in the United States, resulting in substantial morbidity, up to 20% mortality, and more than $4.8 billion in US health care expenditures.1, 2, 3 Early and accurate recognition of pediatric severe sepsis is challenging because many children present initially with compensated shock without apparent hypotension.4, 5, 6 Consequently, identifying the rare child with severe sepsis or septic shock from among many nonseptic patients with fever and tachycardia7 who present to a pediatric emergency department (ED) is truly akin to finding the proverbial needle in a haystack.
Editor’s Capsule Summary
What is already known on this topic
Identification of children with septic shock at emergency department triage is challenging.
What question this study addressed
Will a 2-step sepsis trigger tool, an electronic alert, and a clinician bedside assessment (“sepsis huddle”) accurately identify children with severe sepsis?
What this study adds to our knowledge
In this single-center quality improvement effort, implementation of the 2-stage trigger tool had a high sensitivity for severe sepsis (324 of 326 patients; 99%), although 18 patients were not initially clinically treated.
How this is relevant to clinical practice
A 2-stage trigger tool helped to identify children with severe sepsis but could not completely replace frequent clinical reassessments.
Although recent reports demonstrate improved timeliness of severe sepsis therapy,8, 9 decreased sepsis-related organ dysfunction, decreased hospital and ICU length of stay, and decreased mortality with protocol-guided care for pediatric severe sepsis or septic shock,10, 11, 12 determining which patients may most benefit remains problematic. Alerts based on clinical physiologic data embedded in an electronic health record system have been studied as potential methods to facilitate sepsis recognition in adults.13, 14, 15 Several investigators have evaluated candidate alerts based on systemic inflammatory response criteria and signs of shock (eg, hypotension, elevated lactate level) implemented in adult clinical settings, including the ED, ICU, and general inpatient ward, with various results on processes of care.16, 17, 18, 19 A recent inpatient pediatric study used vital sign–based screening, but not an electronic health record–based alert, for identification of sepsis and demonstrated increased screening adherence and protocol use.20 Each of these alerts relies on vital sign or laboratory result abnormalities, without the addition of physical examination elements and clinician judgment about perfusion adequacy, which are crucial to identify the pediatric patient with severe sepsis among the many with systemic inflammatory response who may rapidly improve with conservative therapy such as antipyretics and oral hydration alone. However, clinical judgment alone seems insufficient: a previous pediatric ED–based study indicated that physician judgment of sepsis, without an electronic health alert, identified only 72% of children presenting with severe sepsis.21
We have previously studied the potential for electronic health record–based alerts to improve recognition of severe sepsis in children. We retrospectively applied an electronic alert based on criteria developed by the American Academy of Pediatrics Septic Shock Collaborative (Figure E1, available online at http://www.annemergmed.com) and compared this to a prospectively applied physician identification screen for severe sepsis in children in the ED. The retrospectively applied alert increased sensitivity while reducing specificity compared with the physician identification screen alone.21 However, there has not been prospective application and study of a pediatric ED–based electronic alert, to our knowledge. In this study, we prospectively implemented an electronic health record sepsis alert in a pediatric ED and assessed both the test characteristics of the alert and the effect of alert implementation on severe sepsis recognition. We hypothesized that implementation of the electronic sepsis alert would improve recognition of pediatric sepsis in the ED.
Section snippets
Setting and Selection of Participants
The study was conducted in a freestanding academic children’s hospital ED with more than 90,000 annual visits. The study period was June 1, 2013, to May 31, 2015, with June 1, 2013, to May 31, 2014, providing preimplementation data and June 1, 2014, to May 31, 2015, providing postimplementation data. All patients presenting to the ED during the study period were included. Patients transferred from another institution to our ED were included if they received the initial dose of intravenous
Results
This study occurred in the context of an existing sepsis quality improvement program in our ED. The team planned the intervention according to previous data suggesting gaps in appropriate sepsis recognition. Before electronic sepsis alert implementation, the team presented data to the division about these gaps in care and associated patient outcomes.11 There were ongoing educational updates to the ED division throughout the implementation period in regard to successes and failures of electronic
Limitations
There are several limitations to this study: First, it included a single pediatric academic center, and we do not know how it will perform in other settings, thus limiting generalizability. Larger-scale efforts, including several ongoing multicenter quality improvement collaboratives, will help to address this concern in the coming years.
Second, the vital sign cutoffs used in this study were not empirically derived and were chosen according to existing institutional practice. It is possible
Discussion
We have implemented a process that includes an electronic sepsis alert and bedside clinician assessment to facilitate identification of pediatric patients who developed severe sepsis or septic shock requiring pediatric ICU care within 24 hours of their ED stay. The electronic sepsis alert required bedside evaluation of just 1% of the ED census and had 85% sensitivity and 99% specificity. Sensitivity was optimized to 99% when clinician identification was used to augment sepsis recognition of
To determine the effect of implementation of an automated sepsis screening tool on the median cost of affected patient encounters.
This retrospective cohort study used propensity score–matched comparison groups to assess the difference in median cost for comparable affected patient encounters before and after the implementation of an automated sepsis screening tool in a large US children's hospital emergency department (ED) with >90 000 annual visits. All patient encounters in 2018 impacted by the automated sepsis screening tool were included and compared with a propensity score–matched comparison group drawn from patient encounters in 2012 that might have been affected by the screening tool had it been active at that time. The main outcome was the change in the median cost for comparable affected patient encounters.
The overall median cost for those affected by an automated sepsis screening tool decreased by 21.2%, from $6454 (IQR, $968-$21 697) to $5084 (IQR, $802-$16 618). The median cost for encounters with an associated International Classification of Diseases sepsis code decreased by 51.1%, from $58 685 (IQR, $32 224-$134 895) to $28 672 (IQR, $16 796-$60 657).
The median cost for comparable patient encounters decreased with implementation of an automated sepsis screening tool in the pediatric ED. Costs were decreased even more substantially for patients with sepsis. In addition to improving outcomes, an automated sepsis screening tool appears to be at least cost-effective and may be cost-saving, an incentive for more widespread use of this technology.
Please see page 760 for the Editor’s Capsule Summary of this article.
Supervising editor: Lise E. Nigrovic, MD, MPH
Author contributions: FB, EA, and JZ conceived and designed the study, and obtained research funding. MA, AS, and JL aided in study implementation. KH managed the data. FB, EA, AS, and JZ provided statistical advice on study design and analyzed the data. FB drafted the manuscript, and all authors contributed substantially to its revision. FB takes responsibility for the paper as a whole.
All authors attest to meeting the four ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
The funders were not involved in design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the article.
Readers: click on the link to go directly to a survey in which you can provide feedback to Annals on this particular article.
