Statistics from Altmetric.com
Strengths and limitations of this study
This study estimates that one in seven children experience harm during admission to hospital in the UK. Most of this harm is temporary, but a significant minority is serious. This should generate discussion about patient safety in paediatrics.
The study used the UK Paediatric Trigger Tool that can be used by any hospital to learn about harm in order to prevent or reduce it.
The study does not distinguish between preventable and non-preventable harm; however, we believe that there is an opportunity to learn from any harm event.
The provision of care that is safe and reliable is a fundamental goal of modern healthcare. Patient safety is the prevention, reduction and amelioration of medical harm.1 ,2 Medical harm (synonymous with the terms patient harm and adverse event, AE) is defined as unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalisation or that results in death.3 Efforts to improve patient safety have been hampered by a lack of reliable data on the prevalence and nature of harm in all areas of practice. Patients and healthcare professionals need to understand the burden of harm in healthcare in order to develop effective interventions.4
Development of Trigger Tool methodology
The Global Trigger Tool (GTT) for measuring harm was developed by the Institute for Healthcare Improvement for use in adult care settings.3 Trigger tools have also been developed for specific populations and settings, including acute hospitals, surgery,5 critical care6 and primary care.7 One study used the GTT to measure harm at a large academic children's hospital in the USA and recommended the development of a paediatric specific tool.8 Paediatric-specific trigger tools have been developed for neonatology,9 paediatric critical care,10 ,11 medications12 ,13 and a general trigger tool for harm in hospitalised children.14 A UK version of the acute adult GTT had already been developed, but this was not applicable to a paediatric population. In 2008, the National Health Service (NHS) Institute for Innovation and Improvement undertook to develop a UK Paediatric Trigger Tool (UKPTT) that could be applied to all levels of acute paediatric care.
Trigger tool method for reviewing case notes
The trigger tool method is a retrospective review of 20 sets of healthcare records each month, using a standardised methodology. A random sample of 20 inpatient case notes is selected using a randomisation matrix on a monthly basis. The healthcare record is examined in a structured process for 20 min to search for ‘triggers’. A ‘trigger’ is a predefined event that alerts the reviewer to the possibility of patient harm. Once a trigger is identified, the reviewer uses clinical expertise to examine the records in more detail to understand the circumstances around the event. If harm is suspected, a second reviewer (usually a physician) is consulted to confirm and grade the AE using the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) grading system (table 4).15
An example of a trigger is the administration of the antidote medication naloxone (a trigger) to reverse the effects of opiates. This alerts the reviewer to a possible overdose of opioids. The reviewer examines the relevant parts of the healthcare record to assess whether the use of naloxone was for this reason or not. If this is the reason, then the harm is graded (see online supplementary file).
Development of the UKPTT
In 2008, the NHS Institute sponsored the development of a Paediatric Trigger Tool because at the time there was no tool available for hospitalised children. The tool design was informed by the early (and prepublication) findings of a Canadian Paediatric Trigger Tool (CPTT) study,14 and the UK Acute Trigger Tool for adults.16 The development was a coproduction involving the collaboration of patient safety experts from the NHS Institute, international leaders in Paediatric Trigger Tool development, and clinical experts from nine UK hospitals including children's hospitals and district general hospitals. Following discussion and testing, the group agreed on 40 paediatric oriented triggers to be included in the UKPTT. These were based on the triggers used in other tools, UK evidence of AEs and the experience of the reference group in harm and AEs (a subset of the coproduction group). Production of a UK tool was intended to enhance ownership by the clinicians, who would use it in practice and to modify triggers that were not appropriate for the UK setting. We also added a category for ‘other harm’ to capture harm that was not detected by one of the listed triggers.
The UKPTT advocates a working definition of patient harm as ‘anything, which you would not like to happen to yourself or a member of your family as a result of, or contributed to by, medical care’. The decision to aim for a broad definition was to focus on the patient rather than on the medical system—a less defensive approach. This is a broader definition than that given by Griffin and Resar3 or the Canadian tool and aimed to encourage clinicians to explore a holistic concept of harm than that traditionally reported. It allows the inclusion of acts of omission as well as commission. The definition includes missed or delayed diagnosis along with physical and psychological harm.
Through the coproduction, support was developed for UKPTT users such as face-to-face training, online and printed guidance and standardised data collection forms.
As part of the UKPTT development, the NHS Institute created a web-based trigger tool portal into which participating hospitals entered anonymised data. The portal calculated harm rates and produced run charts that hospitals could download. Contributing hospitals consented to their data being collated and published to further the understanding of harm in hospitalised children in the UK. Participating hospitals developed local administrative and governance arrangements for PTT reviews following the standard guidance (see online supplementary file).
Aims and methodology
The aims of this study are to:
Describe the rate and severity of harm occurring in hospitalised children from UK centres submitting data to the NHS Institute's Trigger Tool portal.
Report the frequency and positive predictive value (PPV) of triggers to detect harm.
Make recommendations for further application and development of the tool.
Participating hospitals, which voluntarily decided to use the PTT, included secondary, tertiary and quaternary centres. Reviewers were trained in trigger tool methodology either by experts at the NHS Institute or by using online resources with telephone support. Data were collected through the online trigger tool portal that opened in February 2008.
Data from 3992 case note reviews from 25 hospitals submitted to the trigger tool portal between February 2008 and November 2011 were analysed. Nine of the hospitals were children's hospitals; the remainder was classed as district general hospitals. Data from four additional hospitals that used the portal were excluded because they each submitted less than 10 case entries. Harm was recorded as occurring for 567 patients (14.2%) while the majority (85.8%) of patients experienced no evidence of harm. Reviewers identified 1001 AEs, an average of 1.8 events per patient experiencing harm.
There was considerable variation between hospitals in the number of case notes reviewed (12–622), the number of triggers detected (17–1877) and the overall harm rate reported (0—73.3%). Results from each hospital are reported in table 1. Of the 567 children who suffered an AE, the majority (n=356, 63%) experienced a single event. However, 211 (37%) patients suffered more than one AE within the same admission. One patient was reported to have suffered 10 AEs in a single admission. A summary of the number of AEs per case is presented in table 2.
Individual triggers varied in their ability to lead to detection of harm. The trigger Complications of procedure or treatment yielded the greatest amount of harm (182 AEs). The PPV varied from 80.0% for surgical site infection to 2.62% for missing observations/early warning scores. Also, the PPV was generally low in frequently identified triggers, such as missing observations/early warning scores (PPV 2.6%) and unplanned admission (PPV 4%). The positive triggers, AEs and PPV for each trigger are displayed in table 3.
The majority of AEs (n=923, 92.2%) resulted in temporary harm to the patient (grades E and F—see table 515); 43 AEs required life-sustaining interventions and 18 resulted in permanent harm. In 17 cases, the AE was believed to have contributed to the child's death (table 4).
The complexity of uncovering harm is reflected in the numerous ways that one has to measure it.17 Traditional methods such as incident reporting have limitations, especially that of under-reporting, due to the reliance on individual clinicians to recognise and report AEs, as well as a tendency to focus on error rather than on harm. The measurement and examination of harm, rather than that of error, recognises that efforts to improve patient safety benefit from focusing on incidents that result in actual harm, identifying high-risk situations, considering preventability and looking for means of early detection and harm limitation.18 This deeper understanding of the harm to which patients are exposed is a recent phenomenon and in paediatrics the potential risks to safety are multiple.19 The call for zero harm and the focus on safety in recent reports20 ,21 reflect the importance of the identification and understanding of harm as an essential part of patient care. Clinicians have not known the actual levels of harm caused and have relied on a reporting system for clinical incidents.
Harm rates vary widely because of multiple factors, such as the definition of harm used, the methodology employed and the population studied. Until recently, most studies sought to establish harm rates for benchmarking purposes over large populations with suggestions that between 3% and 17% of patients experience an AE during a hospital admission.22 Most of these studies used retrospective, unstructured case note reviews which are labour-intensive, costly and impractical for the routine monitoring of harm.23 Trigger tool methodology is accepted as one of the ways to measure harm in a way that allows local learning,1 ,3 ,2 Trigger tools have also been reported to provide consistent, reliable and relevant data at low cost,2 ,3 although the cost may vary between different hospitals.
This paper represents the largest study of paediatric harm using trigger tool methodology in the UK. While the primary purpose of a trigger tool is to gain local data for understanding harm in order to improve patient safety locally, we can learn about harm by examining the pooled experiences of the contributing hospitals.
The overall harm rate (the percentage of individual children experiencing one or more AEs during an admission) identified in this study was 14.2%. Previous studies focusing on hospitalised children have identified harm rates between 1% and 25.8% per admission for the general paediatric hospital population,12 ,13 ,18 and higher rates within the paediatric intensive care population of 26.1% to 62%.10 ,11 ,24 Two recent studies have examined harm in paediatric hospital populations using trigger tool methodology. The CPTT found a physician who reported a harm rate of 15.1% of admissions during a validation study across six paediatric hospitals in Canada.25 Like the UKPTT, the CPTT has been adapted to make triggers more sensitive and specific to paediatric settings. The second study, at a single paediatric academic medical centre in the USA using the adult GTT, found an overall harm rate of 25.8%.8
Variation in harm rates may reflect a number of factors. Different methodologies yield different rates of AE identification. Trigger tools are reported to yield higher rates of AE identification than traditional methods such as self-reporting and unstructured case note review.1 ,24 Definitions of harm vary, as do their interpretation. Professional groups may interpret AEs differently.26 Assessments of inter-rater reliability have reported high levels of agreement between review team members,8 ,26 ,27 but there is variability between different hospital department teams.28 In addition, some organisations or teams set a lower threshold for what they see as harm and they may change this over time. Finally, different populations are exposed to different levels of harm depending on the complexity of their illness and the intensity and duration of their care.2 Most studies report a harm rate per admission, meaning that longer admissions are more exposed to opportunity for harm.
The same reasons that explain the variation between international studies also explain much of the variation between hospitals in this study. Training was provided, but no independent assessment was made of the reviewer's interpretations or competence. The extremes of harm reported or its absence were seen in hospitals uploading low volumes of reviews and may be interpreted as the relative inexperience of the reviewers.29 There is also a wide variety across the level of hospital represented with the corresponding impact on risk due to patient complexity, need for surgery or critical care and length of stay. While we had no means of adjusting for acuity because of the random selection of notes from within hospitals, we believe that the overall group is broadly representative of the population of hospitalised children in the UK.
One could ask whether this level of variation diminishes the findings of the study. On the contrary, we believe it represents a real portrayal of complex issues. It is also a taste of what individual organisations can expect if they start to use the PTT to help understand and reduce the harm in their institution. They will need to consider all of these issues as they interpret their own findings.
The majority (92.3%) of AEs identified in this study represented temporary harm resulting in the child requiring an intervention, admission to hospital or prolongation of their hospital stay. While severe harm (permanent harm or harm that required life-sustaining measures or contributed to death) was rare, it still constituted 7.8% of the harm identified. Similar findings with respect to severity have been reported with 10% of AEs classified as severe in one study of harm in a paediatric intensive care unit.10 A study of AEs in hospitalised children reported that clinicians do not always recognise harm, even when the consequences to the child are severe.30 In this study, multiple AEs were relatively common, with 37% of those experiencing harm suffering two or more AEs in the same admission, far higher than in previous studies.8 ,25
Triggers varied in their PPV for AEs. Screening for triggers is the key task of the trigger tool method. Triggers that infrequently identify harm could be removed to increase the efficiency of the tool. Some triggers may be important markers of care quality, such as the missing/incomplete early warning score or baseline observations, despite the inability of the trigger to identify specific patient harm.31 This will influence the next iteration of the trigger tool as we refine the triggers and consider taking out some of those that had a low PPV.
A number of studies have examined the possibility of automated trigger detection from electronic medical records, which may make the process easier.13 ,32 We believe that there is value in the manual approach, and that it will be some time before paper-based medical records in hospitals in the UK are converted to electronic medical records. Users of the UKPTT have expressed to us the benefits of having an opportunity to examine the quality of medical and nursing note keeping and observations, which in some centres has resulted in initiatives to improve these elements.
There are a number of limitations to this study. The validity of trigger tool methodology is well established and we did not attempt to revalidate it again against another form of medical notes review for harm, as we did not believe this was necessary. The UKPTT differs from other trigger tools only in the constituent triggers. This study has provided PPVs for the 40 triggers included in the UKPTT. This validates the choice of high performing triggers and raises questions about the continued inclusion of low performing ones, which may be used to consider changes to the trigger profile. New triggers may also be suggested and could be tested for future versions of the tool.
The determination of inter-rater reliability may be important within departments but not necessarily between hospitals. The UKPTT is not recommended for benchmarking as the focus is on developing data for improvement rather than data for judgement.33 The methodology recommends consistency in the reviewing teams so that intrareliability is not an issue.34 We did not attempt to standardise the method of PTT data collection outside of the support provided and the recommendation on randomisation. Individual institutions made their own arrangements in terms of choosing and training reviewers. There were no checks of competence of reviewers or inter-rater reliability or of the accuracy of the data entered via the portal.
Parry et al35 note that the approach should be to look at all harm, not only preventable harm. It is our belief that the ability to measure harm and examine case notes using the UKPTT on a regular basis is an effective method of data capture and analysis, which provides hospitals with valuable insights into their quality of care, as every AE provides insight for improvement, whether deemed preventable or not.10
We did not attempt to find out how many of these AEs were detected through other methods, such as incident reporting (we expect that many were). The purpose of the UKPTT is to extend the ability to detect harm rather than to replace other approaches.
Conclusions and recommendations
In the context of the increasing demands to improve quality and safety for patients, the UKPTT provides a framework for paediatric clinicians to assess the rate of harm for their individual units, and the quality of their record keeping. This study highlights that currently there is a significant, measurable level of harm, which is sometimes severe, experienced by children admitted to hospitals in the UK. There is a range of predictive values for the triggers and some may be more useful than others. These findings will inform future modifications of the PTT including modifying or removing triggers. It will be important to test any new or augmented triggers with paediatric teams to assess their usefulness.
The recognition and examination of AEs through methods such as the UKPTT offers the potential to improve paediatric patient safety by concentrating efforts on strategies that reduce patient harm, rather than errors. The key is to produce information that promotes learning and improvement, with clinicians accepting their role to decrease harm from the perspective of the patient, rather than that of the healthcare provider.
We recommend that the UKPTT be used routinely in hospitals to assess harm and to help develop improvement interventions to reduce it. Although the PTT has been mainly used in children's hospitals, it can be used in district general or community hospitals, with a different spectrum of harm being detected. The UKPTT does not replace other reporting mechanisms, but is a useful addition to the methods already used to understand the harm caused to children in hospital care. Harm needs to be detected and assessed through a number of lenses and this lens allows clinicians to further understand what they do and how harm impacts on children. It provides a way to move from a reactive approach to safety to one that is more proactive and founded on harm free care.
The authors would like to thank Matt Tite at the National Health Service (NHS) Institute for Innovation and Improvement for extracting and processing the data. The authors would also like to thank all the participating institutions whose unidentifiable data have been made available for the analysis, and those who participated in the development of the UKPTT.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.