Objective To systematically review the available evidence on paediatric early warning systems (PEWS) for use in acute paediatric healthcare settings for the detection of, and timely response to, clinical deterioration in children.
Method The electronic databases PubMed, MEDLINE, CINAHL, EMBASE and Cochrane were searched systematically from inception up to August 2016. Eligible studies had to refer to PEWS, inclusive of rapid response systems and teams. Outcomes had to be specific to the identification of and/or response to clinical deterioration in children (including neonates) in paediatric hospital settings (including emergency departments). 2 review authors independently completed the screening and selection process, the quality appraisal of the retrieved evidence and data extraction; with a third reviewer resolving any discrepancies, as required. Results were narratively synthesised.
Results From a total screening of 2742 papers, 90 papers, of varied designs, were identified as eligible for inclusion in the review. Findings revealed that PEWS are extensively used internationally in paediatric inpatient hospital settings. However, robust empirical evidence on which PEWS is most effective was limited. The studies examined did however highlight some evidence of positive directional trends in improving clinical and process-based outcomes for clinically deteriorating children. Favourable outcomes were also identified for enhanced multidisciplinary team work, communication and confidence in recognising, reporting and making decisions about child clinical deterioration.
Conclusions Despite many studies reporting on the complexity and multifaceted nature of PEWS, no evidence was sourced which examined PEWS as a complex healthcare intervention. Future research needs to investigate PEWS as a complex multifaceted sociotechnical system that is embedded in a wider safety culture influenced by many organisational and human factors. PEWS should be embraced as a part of a larger multifaceted safety framework that will develop and grow over time with strong governance and leadership, targeted training, ongoing support and continuous improvement.
Statistics from Altmetric.com
Strengths and limitations of this study
This review systematically and collectively synthesises the available evidence on the multiple components of paediatric early warning systems (PEWS).
The review highlights that PEWS should be embraced as a part of a larger multifaceted safety framework.
Future research needs to investigate PEWS as a complex multifaceted sociotechnical system embedded in a wider safety culture.
Owing to heterogeneous research designs, assessing quality across eligible studies was limited.
While no strong evidence underpinning any one PEWS was available, emerging work should contribute to this evidence base.
It is known that children who die or deteriorate unexpectedly in the hospital setting will often have observable features in the period before the seriousness of their condition is recognised. A seminal study of paediatric mortality in the UK estimated that approximately one in five children who die in hospital have avoidable factors leading to death and up to half of children have potentially avoidable factors.1 The report concluded that ‘there should be ways of telling if something is wrong with a child as early as possible, for example, an early warning scoring system’.1 Other studies have examined the signs (physiological and behavioural) of deterioration that may be present in the period preceding a cardiopulmonary arrest,2 ,3 and the fact that these features are often not recognised or acted on in a timely fashion by hospital staff.4 ,5 Recent years have also witnessed an increased risk of paediatric cardiopulmonary arrest, and its associated mortality, in acute healthcare settings largely as a consequence of increased acuity of care and higher dependency on technology.2 Although the percentage of paediatric cardiopulmonary arrests for inpatient admissions has been reported as low (eg, 0.7–3%),6 ,7 survival to discharge for children that experience inhospital cardiopulmonary arrest is poor (11–37%).3 ,6
Early warning scores are generally defined as bedside ‘track and trigger’ tools to help alert staff to clinically deteriorating children by periodic observation of physiological parameters, generation of a numeric score and predetermined criteria for escalating urgent assistance with a clear framework for communication. In using these physiological track and trigger systems, the goal is to ensure timely recognition of patients with potential or established critical illness and to ensure a timely and appropriate response from skilled staff. Critical to early warning scores are four integrated components which work together to provide a comprehensive safety system for clinically deteriorating patients and those that are most likely to identify and manage patients at highest risk for cardiac or respiratory arrest; (1) the afferent component which detects clinical deterioration and triggers an appropriate response; (2) the efferent component which consists of the personnel and resources providing the response (eg, medical emergency team (MET)); (3) the process improvement component containing elements such as auditing/monitoring/evaluation to enhance patient care and safety and (4) the governance/administrative component focusing on the organisational leadership, safety culture, education and processes required to implement and sustain the system.8 This highlights the need to view early warning tools as more than just a ‘score’, rather, they are part of a multifaceted ‘system’ approach based on the implementation of several complementary safety interventions to improve child patient safety and clinical outcomes.
In Ireland, a 2013 patient safety review by the Health Information and Quality Authority (HIQA) into the unexpected death of a young woman in a maternity setting identified several care failures.9 These included a lack of provision of basic fundamental care, failure to recognise risk of clinical deterioration, failure to act or escalate concerns about deterioration to appropriately qualified clinicians and lack of detail in medical record documentation about clinical status and potential risk of clinical deterioration. This led to a request from the Minister for Health that the Department of Health's National Clinical Effectiveness Committee commission and quality assure a number of National Clinical Guidelines; including early warning scores for adult, maternity and paediatric healthcare settings.
For paediatrics, this request presented several design challenges, including the need for an observation tool that would work in all paediatric care settings (secondary and specialist care) and a requirement to align with the Adult and Maternity scores. Additionally, the application of early warning scores to paediatric patients is more complex than in adults. There are several reasons for this: variation in age-specific thresholds for normal and abnormal physiology; children's inability or difficulty in articulating how or what they feel; children's physiological compensatory mechanisms; staff training issues and the need for more focused attention on respiratory deterioration.10 Finally, although many paediatric early warning systems (PEWS) have been developed and tested, uncertainty remains as to which system, or system feature, is most useful for paediatric patients. Even the concept of PEWS as a system (ie, the application of all four components in parallel as described above) is poorly developed.
The aim of this review was to systematically identify and synthesise available evidence on PEWS in acute paediatric healthcare settings for the detection of, and timely response to, clinical deterioration in children. The review questions were set by the Irish Department of Health who commissioned this review:
What is the available evidence on the effectiveness of different PEW detection systems?
What evidence exists on the effectiveness of PEW response mechanisms, and what interventions are used?
What evidence exists on PEWS implementation strategies/interventions?
This review was conducted and reported in accordance with the Centre for Reviews and Dissemination guidance for undertaking systematic reviews in healthcare,11 the National Clinical Effectiveness Committee Guideline Development Manual12 and the Preferred Reporting in Systematic Reviews and Meta-Analysis (PRISMA) criteria.13
Data sources and search strategy
The following electronic databases PubMed, MEDLINE, CINAHL, EMBASE and Cochrane (inclusive of Cochrane Database of Systematic Review; Database of Abstracts of Review Effects and CENTRAL—Cochrane Central Register of Controlled Trials) were systematically searched from database inception up to August 2016 using various combinations of controlled vocabulary (eg, MeSH) and free text words guided by our PICOS parameters (see online supplementary appendix 1). The search was limited by language (English). For unpublished research reports, grey literature databases, trial registers and national/international professional organisations and association websites were searched. To retrieve evidence-based clinical guidelines, electronic guideline clearinghouses were searched, scoping searches of Google and Bing were performed and a consultation process was conducted with key paediatric experts and paediatric hospitals internationally. Additional literature was sourced by contacting reference study authors and experts in the field and scanning bibliographies of all included papers.
Screening and selection process
Eligible papers had to refer to PEWS, inclusive of rapid response systems (RRS) and rapid response teams (RRT). Outcomes had to be specific to the identification of and/or response to clinical deterioration in child patients (including neonates) in paediatric hospital settings (including emergency departments). No study design restrictions were applied. We excluded papers that focused on paediatric community health settings; PEWS specific to intrahospital and/or interhospital transfer and/or transportation of critically ill children; trigger tools for identification of adverse events and/or harm caused by medical interventions; severity of illness scales and patient classification systems specifically for identifying illness acuity and mortality (except in cases where such studies included PEWS as comparative interventions) and studies which included child and adult populations when child-specific data could not be exclusively extracted.
For stage 1 screening, two reviewers independently assessed each title and abstract retrieved from the electronic searches for relevance. Any discrepancies were resolved by discussion and consensus with a third reviewer. If no abstract was available, the full-text paper was sourced and assessed. For studies deemed to meet the inclusion criteria, full texts of the studies were obtained. Full-text papers were independently assessed by two reviewers against the inclusion criteria before a final decision regarding inclusion/exclusion was confirmed. Any discrepancies were resolved by discussion and consensus with a third reviewer. Reasons for excluding studies from the review were noted (see figure 1).
Appraisal of the level of evidence
In an attempt to conduct a comprehensive review, all studies which met the inclusion criteria were included regardless of quality. Two reviewers appraised and classified the level of evidence of the included studies in accordance with the Scottish Intercollegiate Guidelines Network (SIGN) criteria for assessment of studies based on the type of study design. Assessing comparative quality across eligible studies proved difficult due to the heterogeneous nature of the research methodologies employed; including disparate research designs, different ranges for collecting data over time periods (from months to years), localised small case and comparative group selections, and diverse clinical contexts ranging from general medical and surgical units to specialised settings such as oncology, cardiac, endocrine and rehabilitation units.
Data extraction and synthesis
Two reviewers independently extracted and managed data from the included studies. Any discrepancies were resolved through consultation with a third reviewer. A data extraction table was developed to retrieve information pertaining to each study setting, aim, design, sample, intervention and main outcomes/findings. In line with the review research questions, the studies were segregrated by PEW detection systems, response mechanisms and implementation processes. All data were narratively synthesised as it was not possible to conduct a meta-analysis and/or a metasynthesis because of the heterogeneity of evidence retrieved including non-comparative research designs and diversity of systems, approaches and methods adopted in developing and implementing PEWS in paediatric contexts.
Overall search and selection results
A total of 2742 papers were identified as potentially eligible for inclusion in the review. Following first screening of titles and abstracts, 2616 papers were excluded because they were adult-focused, discussion papers, commentaries, conference abstracts and/or duplicate papers. Full texts of the remaining 126 papers were obtained. On second screening of these 126 full-text papers, a further 57 papers were excluded because they were adult-focused, both child and adult-focused in which it was not possible to segregate child and adult data, not specifically focused on the outcome of clinical deterioration, wrong setting (ie, not inpatient), concentrated on clinical deterioration at point of transportation, examined illness severity or acuity or were discussion papers, commentaries or conference abstracts. This left 69 papers that met the inclusion criteria. An additional 21 papers were sourced through secondary citations, personal communications with reference authors/experts in the field and web-resources. Subsequently, 90 papers fulfilled the eligibility criteria. Figure 1, an adapted PRISMA flow chart, visually displays the search and selection process.
Characteristics of included studies
The studies emanated from the USA (n=46), the UK (n=19), Canada (n=10), Canada and the UK (n=1), Australia (n=5), the Netherlands (n=2), Ireland (n=2), Norway (n=1), Pakistan (n=1), Sweden (n=1), Thailand (n=1) and South America (n=1). The majority of the studies were observational in design, and included 13 cohort studies, 11 case–control, 8 before and after and 6 cross-sectional surveys. There were eight review papers and three interrupted time series quasi-experimental studies. The remainder were chart/database reviews (n=23), quality improvement initiatives (n=9), qualitative studies (n=4) or case reports (n=1). There was one feasibility and reliability testing study, one cost-analysis exercise, one protocol and one course evaluation survey. Of the 90 included papers, 45 focused on PEW detection systems,2 ,3 ,6 ,7 ,10 ,14–53 29 examined PEW response mechanisms8 ,54–81 and 16 reported on PEW implementation strategies82–97 (see online supplementary appendix 2 for a summary of these studies including the level of evidence and rationale for judgement).
Review question 1: What is the available evidence on the effectiveness of different PEW detection systems?
Thirty-eight primary studies reported on original3 ,10 ,23 ,25 ,27 ,31 ,35 ,39 ,44 ,46 (see online supplementary appendix 3) and/or adopted/modified2 ,6 ,14–21 ,24 ,26 ,28–30 ,33–38 ,40–43 ,47 ,48 ,50–53 PEW detection systems for use in paediatric inpatient settings. Twenty-three of these 38 studies reported on the effectiveness of PEW detection systems using the performance criteria of sensitivity, specificity, receiver operating characteristic curve, positive predictive value and/or negative predictive value.2 ,3 ,6 ,10 ,14 ,15 ,18 ,20 ,21 ,23 ,25–27 ,29–31 ,33 ,37 ,39 ,40 ,48 ,50 ,53 Diversity in PEW physiological (and other) parameters and differences in age-dependent vital sign reference ranges made it difficult to compare and contrast performance criteria. To enable some comparisons to be made, further studies were excluded if they; were from specialist units if only one study was published, only reported on inter-rater and intrarater agreement, had <100 cases and did not report data on sensitivity and specificity. Figure 2 shows the diagnostic predictive accuracy of PEW detection systems from 11 studies.6 ,10 ,14 ,18 ,20 ,21 ,25 ,26 ,30 ,40 ,50 This illustrates that the effectiveness of PEW detection systems demonstrated wide-ranging sensitivity and specificity largely as a consequence of different settings adopting and self-regulating varying end point or surrogate markers for clinical deterioration (ie, cardiopulmonary arrest, PICU admission, mortality and interventions) and different standards for cut-off/threshold scores.
Review question 2: What evidence exists on the effectiveness of PEW response mechanisms, and what interventions are used?
Table 1 provides an overview of the evidence on PEW response interventions. Across 18 primary studies, the main PEW response intervention in use was health professional-activated RRS incorporating paediatric RRTs or METs.54–56 ,60 ,61 ,64–68 ,70 ,73–76 ,78 ,79 ,81 Where reported, RRS were available to be activated by any staff member 24 hours/day, 7 days a week. The staffing composition of the majority of RRT/METs included a critical care nurse, a physician and a respiratory therapist. The most common RRT/MET activation criteria were cardiovascular, respiratory and neurological status, alongside staff and family concern. Studies examining the effectiveness of RRSs reported on a number of clinical and process outcome data, for example, cardio/respiratory arrest (CPA) rates, mortality rates unplanned PICU transfers/admissions interventions required (ie, intubation, mechanical ventilation, inotropes) and MET/code blue activations. Collectively, findings revealed mixed evidence on the effectiveness of RRSs. For instance, although four studies reported a significant reduction in CPA rates and five studies found a significant reduction in mortality, there were an equal number of studies reporting non-significant findings.
Five papers reported on quality improvement initiatives for families to activate the RRS.59 ,62 ,63 ,69 ,71 Findings revealed that families infrequently activate the RRS, but when they do, the reason is largely as a consequence of communication failures rather than critical care deterioration. While physicians value family input and depend on families to explain their child's baseline condition and identify subtle changes in their child, physicians are apprehensive towards family-activated RRS because of potential misuse of resources, undermining of the clinician–family therapeutic relationship, increased family anxiety/burden and a need to provide knowledge/training to families.
Review question 3: What evidence exists on PEWS implementation strategies/interventions?
Table 2 provides an overview of evidence from 16 studies reporting on PEW implementation strategies/interventions. The evidence was diverse in approach, ranging from the adoption of social marketing principles to quality/performance improvement initiatives to chart reviews, qualitative studies and pre–post implementation surveys. Comparative evaluations were therefore difficult and no conclusions were drawn on an optimal implementation strategy to influence change in clinical/process outcomes (or indeed what the best clinical/process outcomes are to measure). Despite the limited evidence, valuable insights were gleaned into cultural, sociotechnical, education/training and organisational issues impacting, either positively or negatively, on the effective implementation of PEWS. For example, a number of qualitative and quality improvement studies highlighted the importance of creating an empowering culture that fosters trusting relationships, opens communication and supportive teamwork.83 ,85 ,87 ,90 ,96 Working through real-life cases and using a multiprofessional approach to PEWS education/training were positively evaluated for improving doctor–nurse communication, enhanced team-work and better use of the SBAR (Situation, Background, Assessment, Recommendations) communication technique.97 Significant improvements were also found in documented vital signs, communication episodes and intern hand-offs after ABC-SBAR (communication technique) training.92 ,93 The integration of situation awareness interventions into EWS was also recommended to recognise experienced clinicians tacit knowledge (ie, watcher/clinician gut feeling) and the incorporation of structures, such as huddles, to proactively identify risk and communicate concerns at bedside, unit and organisational level.85 ,86
No published evidence for the resource implications of complete PEWS (detection, response and implementation) was found. Bonafide et al84 prepared the cost of a MET component of PEWS and found three clinical deterioration events would offset MET costs (compared with pre-MET). After this, any clinical deterioration events averted (by MET) would represent cost savings. These findings relate to one element of PEWS and may not translate directly to PEW scoring systems or additional safety structures that enhance PEWS implementation.
This review systematically examined and synthesised evidence on PEWS as a comprehensive system comprised of detection, response and implementation components. For all three review questions, no conclusive answers on the effectiveness and impact of PEWS on clinical practice were identified. The review revealed the absence of a standard PEW scoring system across paediatric inpatient settings internationally, limited standardisation of outcomes to enable comparison of published PEWS studies and uncertainty regarding PEWS education and implementation processes at different institutions. This highlights the need for more organised multisite coordination and study around PEW scoring, systems usage, implementation and outcome measures. While the review revealed mixed outcomes, it is promising to see evidence suggesting positive directional trends in clinical outcomes, for example, reduced cardiopulmonary arrests, earlier intervention and transition to PICU with accompanying potential improvements in patient safety culture through enhanced multidisciplinary communication and team-work, for example.
The review draws attention to the fact that multiple distinct PEWS scoring systems are in use internationally, yet empirical evidence on which system is most effective is limited. Perhaps this is due to the heterogeneity in how the detection tools were developed, modified and investigated across included studies. Diversity in the composition of PEW detection systems (ie, physiological parameters, reference range values, trigger threshold points and clinical deterioration outcome markers) makes it difficult to compare and contrast performance criteria. It was rare, however, for any PEW detection system to have a high specificity and sensitivity. While some systems showed promising performance criteria, many were unable to be fully validated due to low sensitivity. Many contexts chose simplicity and clinical utility as a priority in deciding which PEW detection system to implement. The variety of PEW parameters used by local units is perhaps reflective of the desire to have locally derived systems.45 This presents difficulty for development of a national, and/or international, standard to guide clinical practice. Challenges exist in standardising a common scoring tool and in establishing a common language among healthcare professionals for recognising and responding to clinically deteriorating children. Indeed, the majority of PEW detection systems were evaluated at one point in time, and in single-site paediatric hospital settings, limiting the transferability of results. One multicentre case–control study40 was identified which validated the Canadian Bedside PEWS across inpatient units in four children's hospitals. Results are eagerly awaited from the first multicentre cluster randomised controlled trial evaluating the impact of Bedside PEWS across 22 hospitals internationally.42
The review identified that the main PEW response intervention in use internationally was health professional-activated RRSs, incorporating RRTs and METs. It was difficult to make comparisons, however, because of variations in how RRT/METs were operationalised in terms of team membership, activation criteria and determination of effectiveness. With limited uniformity on how clinical and process outcomes were defined and measured across studies, uncertainty remains around the impact of RRS on the timely intervention for children with clinical deterioration. Further evidence is also needed on family-activated response mechanisms to demonstrate improved patient outcomes.
Despite many anecdotal accounts emphasising the importance of the process of PEWS implementation, a dearth of published literature was sourced in this area. The review did identify, however, the need for cognisance to be given to the multifaceted nature of PEWS (ie, communication, multidisciplinary team-work and education, parent involvement), including the healthcare cultural context in which PEWS would be implemented. There is a need to move beyond reactive responses to include proactive assessment of children at risk of clinical deterioration (eg, concepts such as the watcher, huddles, roving teams).85 ,86 ,98 Healthcare professionals can benefit from improved situational awareness to proactively assess all relevant context around the child, family, tasks required, staff/team and environment.99 ,100
Despite its limitations, this review contributes important learning because no evidence was sourced that collectively examined the multiple components of PEWS as a complex healthcare intervention in a single study. Rather, the evidence examined PEWS in a piece-meal manner, focusing on one particular aspect (eg, detection, response or implementation) each time. The findings support Chapman et al's22 recently updated review which revealed low evidence to support paediatric track and trigger system (PTTS) implementation as a single intervention. There was, however, some moderate evidence to support the delivery of PTTS as part of a package of interventions or ‘care bundles’. Chapman et al22 contended that this may be reflective of the complexities of healthcare delivery. The multiple challenges inherent in the delivery of effective high-quality safe healthcare are increasingly recognised with the call for more proactive defence layers that focus on system, rather than human, resilience.100 One avenue to potentially assist with addressing the complexity of PEWS, and advancing this field of knowledge, is the integration of quality improvement, science and human factors. This is important because human factors are not independent issues that can be tackled in isolation or on a piece-meal basis but need to be integrated into the life cycle of the systems development.100 This could potentially lead to improvements in better outcomes and experiences for children and their families and also better system performance (ie, care) and professional development (ie, learning).101
Strengths and limitations
This manuscript systematically collated and synthesised evidence on the multiple components (detection, response and implementation) of PEWS collectively in one review. While a comprehensive search strategy was employed, and the recommended practices for the conduct and reporting of systematic reviews were adhered to, it is possible that some relevant papers may have been missed. Additionally, with the exclusion of non-English papers, there is the potential risk of publication bias. Although beyond the scope of this review, there is potentially other literature likely to be of relevance to informing the effectiveness of PEWS; most specifically to examine sociocontextual factors (eg, situation awareness and human factor) that may, or may not, work as active ingredients in the successful implementation of PEWS. There is some work emerging in this area.102
Recommendations for clinical practice
Clinicians working in inpatient paediatric units, and management at unit and organisational levels, need to recognise that the early detection of a deteriorating child is much more than identifying and responding to a score. Instead, through creation of a common language, PEWS should stimulate a heightened sense of situation awareness and open communication among clinicians about children at risk of clinical deterioration, thereby supporting, not replacing, clinical judgement. PEWS should be embraced as a part of a larger multifaceted safety framework that will develop and grow over time with strong governance and leadership, targeted training, ongoing support and continuous improvement.
Directions for future research
Future research needs to investigate PEWS as a complex multifaceted sociotechnical system that is embedded in a wider safety culture influenced by many organisational and human factors such as, but not limited to, clinician knowledge, experience and confidence; effective multidisciplinary communication and team-work; family engagement; situation awareness; decision-making; unit and hospital management and leadership; working conditions and the environment; and stress and fatigue. There is evidence of some potential emerging work in this area in the UK.103
This review identified that PEWS are widely used internationally. However, empirical evidence revealed a lack of consensus on which PEWS is most effective or useful. Notwithstanding the limited consensual evidence, positive trends in improved clinical outcomes, such as reduced cardiopulmonary arrest or earlier intervention and transfer to PICU, were reported. Additionally, the implementation of PEWS as one part of a wider safety culture has the potential to enhance multidisciplinary team working, communication and confidence in recognising and making clinical decisions about clinically deteriorating children. The lack of multicentre studies, no national guidelines, no research evaluating PEWS as a complex healthcare intervention and limited development of any underlying theory all impact on the consistency with which PEWS are defined, implemented and measured for effectiveness. Consequently, further research is required to establish what the true ‘active ingredients’ of PEWS interventions are in contributing to the detection and/or timely identification of, and response to, deterioration in improving clinical outcomes for children in inpatient hospital settings.
This work originated from a Department of Health commissioned review lead by the corresponding author in 2014. In August 2016, the authors revised and updated this review. The authors are grateful to the other project team members (Marie T O'Shea, Catherine Walshe, Melissa Corbally, Donal O'Mathuna, Anthony Staines, Caroline O'Connor) for their help in the original review process. The authors would also like to acknowledge members of our national paediatric expert advisory group for the original review, including Cormac Breatnach, Sharon Condon, Orla Franklin, Amanda Halpin, Una McAree and Ann Moran.
Contributors VL was the lead investigator for the original and revised review. VL and AM designed the review protocol, developed and ran the updated search searches, selected and appraised the papers, extracted data and drafted the initial manuscript. RM and JF commented on the protocol, searches, evidence appraisal and revised the manuscript for important intellectual content. All authors approved the final manuscript.
Funding This work was supported by the Irish Department of Health and overseen by the National Clinical Effectiveness Committee and Health Service Executive PEWS Guideline Development Group.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.