Article Text
PDF
Abstract
Background Retrospective record review using trigger tools remains the most widely used method for measuring adverse events (AEs) to identify targets for improvement and measure temporal trends. However, medical records often contain limited information about factors contributing to AEs. We implemented an augmented trigger tool that supplemented record review with debriefing front-line staff to obtain details not included in the medical record. We hypothesised that this would foster the identification of factors contributing to AEs that could inform improvement initiatives.
Method A trained observer prospectively identified events in consecutive patients admitted to a general medical ward in a tertiary care academic medical centre (November 2010 to February 2011 inclusive), gathering information from record review and debriefing front-line staff in near real time. An interprofessional team reviewed events to identify preventable and potential AEs and characterised contributing factors using a previously published taxonomy.
Results Among 141 patients, 14 (10%; 95% CI 5% to 15%) experienced at least one preventable AE; 32 patients (23%; 95% CI 16% to 30%) experienced at least one potential AE. The most common contributing factors included policy and procedural problems (eg, routine protocol violations, conflicting policies; 37%), communication and teamwork problems (34%), and medication process problems (23%). However, these broad categories each included distinct subcategories that seemed to require different interventions. For instance, the 32 identified communication and teamwork problems comprised 7 distinct subcategories (eg, ineffective intraprofessional handovers, poor interprofessional communication, lacking a shared patient care, paging problems). Thus, even the major categories of contributing factors consisted of subcategories that individually related to a much smaller subset of AEs.
Conclusions Prospective application of an augmented trigger tool identified a wide range of factors contributing to AEs. However, the majority of contributing factors accounted for a small number of AEs, and more general categories were too heterogeneous to inform specific interventions. Successfully using trigger tools to stimulate quality improvement activities may require development of a framework that better classifies events that share contributing factors amenable to the same intervention.
- Trigger tools
- Adverse events, epidemiology and detection
- Hospital medicine
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Introduction
Despite substantial investments in patient safety over the past decade, it is often concluded that the adverse event (AE) rate in the acute care setting has not improved over time.1–3 But arguments exist for why we need to treat this claim with caution. On the one hand, unchanged AE rates may reflect a paucity of effective interventions to improve patient safety or limited dissemination of such interventions.4 On the other hand, lack of improvement may also reflect the limitations of methods used to detect AEs,5 the most widely recognised being the retrospective record review method used in major AE studies.6–12 Retrospective record review has several inherent limitations because the ability to determine what occurred depends solely on the documentation in the clinical record. As a result, we may not capture all events, lack details needed to assess the preventability of some AEs and have difficulty ascertaining the causal factors that inform improvement efforts.
Major AE studies6–12 use a set of triggers to flag medical records with a higher likelihood of identifying an AE (eg, a patient with hypoglycaemia might have received an inappropriate dose of insulin causing an adverse drug event). Trigger tool methodology operationalises the use of a list of triggers to allow institutions to employ an AE identification strategy previously only available in research settings.13 ,14 For example, the Institute for Healthcare Improvement (IHI) developed the IHI Global Trigger Tool, which institutions apply to a random selection of medical records each month to measure and track their AE rate over time.13 Despite their broad use in a variety of inpatient and ambulatory care settings,15–26 retrospective record review and trigger tools suffer from the same types of limitations because both approaches rely primarily on what providers document in the clinical record.
To overcome limitations associated with retrospective AE detection methods, we hypothesised that an augmented trigger tool methodology called ‘prospective clinical surveillance’27–29 would foster the identification of factors contributing to AEs that could inform improvement initiatives. This method supplements near-real-time medical record review with discussions with front-line staff to identify a larger number of preventable AEs and provides a richer characterisation of the key latent contributing factors that underlie multiple types of AEs. In this study, we evaluate how far this augmented approach might allow us to identify targets for specific quality improvement (QI) initiatives.
Methods
Study setting and participants
We conducted prospective clinical surveillance on one 20-bed general medical ward in our tertiary care academic medical centre between November 2010 and February 2011 inclusive. The general medical service has a total of 125 beds and admits over 5000 patients per year. At this hospital, only cardiology, nephrology and oncology have separate inpatient services. However, the general medicine service still takes care of many patients with admitting diagnoses related to these three subspecialties. We focused on general medical patients because of the nature of the patients (elderly patients with multiple comorbidities), and the care that they receive (acute multidisciplinary complex inpatient care) places them at risk of exposure to AEs and might allow us to detect and analyse a larger number of events. We limited our surveillance to all patients admitted to a single general medicine unit rather than a sampling of different units to enrich the identification of local, unit-specific contributing factors to target with QI initiatives as opposed to identifying a list of generic causes of AEs (eg, falls, infections, medication errors).
Prospective clinical surveillance methodology
Prospective clinical surveillance differs from and enhances the traditional trigger tool approach in several key ways.27–29 First, although case finding still relies on the use of triggers, it occurs in near-real time (eg, usually within 48 h) as opposed to through retrospective record review. Second, a trained observer, fully integrated in the clinical environment, gathers data prospectively, supplementing near-real-time medical record review with debriefs of front-line staff involved in the case. Third, rather than relying on 1–2 reviewers external to the cases to adjudicate AEs, this method engages an interprofessional team that specifically includes front-line staff directly involved in the cases to review them. This intensified surveillance strategy optimally applies trigger tool methodology in order to increase detection of AEs, improve judgements of preventability and provide a richer understanding of contributing factors. Also, since this method involves front-line staff reviewing events for patients that are often still under their care, it might encourage them to bring forward additional events that otherwise might be difficult to detect or even galvanise them to take actions to address some of these problems.
Previous studies of prospective clinical surveillance provide a detailed description of the methodology, including case finding, peer review and event classification.27–29 We provide a brief description of our use of the methodology and its use to characterise contributing factors.
Case finding and peer review
A trained observer (an advanced practice nurse) reviewed patient records and the electronic patient record, attended daily multidisciplinary rounds and interacted with front-line staff on weekdays between 8:00 and 17:00 to screen for prespecified triggers (see online supplementary eTable 1). We used relevant triggers listed in the IHI Global Trigger Tool,13 supplemented by additional triggers particularly relevant to elderly medical patients, such as functional decline, in-hospital malnutrition and hospital-acquired delirium that previous AE studies did not emphasise. We piloted these new triggers and adjusted them to balance feasibility of detection with likelihood of uncovering actual events.
For example, when developing the new trigger to detect in-hospital malnutrition, we initially used 'suboptimal oral intake documented in the medical record for >72 h' and 'team concern about patient's oral intake' as triggers, but quickly discovered that front-line staff inconsistently documented dietary intake in the medical record. However, we noticed that physicians often 'ordered nutritional supplements' and made 'referrals to the dietician' for patients with poor oral intake, and so we eventually used these triggers in our study.
Medical record review occurred within 48 h of trigger detection. The advanced practice nurse summarised details related to the event, the healthcare team's response, the eventual patient outcome and the various factors that contributed to the event. As in prior trigger tool studies, our trained observer did not limit case review to the trigger itself. Instead, identification of a trigger led to a systematic process of reviewing specific documentation sources, such as the medication record and nursing assessments, in order to look for all types of patient safety and quality of care problems.
We purposely used an advanced practice nurse who had a pre-existing relationship with front-line staff on the study ward. In addition to carrying out structured debriefs of staff to enrich the characterisation of details surrounding AEs with information missing from the medical record, she fully integrated herself on the unit and had frequent interactions with front-line staff. This integration encouraged case finding since front-line staff felt comfortable bringing forward additional concerns that could represent AEs missed by the trigger tool. It also facilitated exploration of contributing factors through an open-ended debriefing approach to elicit staff perspectives and clarify details not available in the patient record, thus enriching the case summaries and allowing the review team to make more informed judgements about contributing factors.
Event classification
We convened weekly meetings to review and classify events. The principal investigator and advanced practice nurse, along with at least one of the two pharmacists and at least one of the two geriatric medicine physician study investigators, attended each meeting. The group reviewed each case using the same definitions and classifications as those used in major AE studies.6–12 We first assessed whether the event resulted in harm to the patient. If yes, the group then rated whether the harm was caused by medical care (ie, an ‘AE’) and whether it was preventable (ie, caused by an error). The review team rated the degree of harm, causation and preventability on a seven-point Likert scale (1=very unlikely; 4=close call, but favours; 7=very likely to indicate harm, causation or preventability) using a cut-off score of ≥4 to indicate harm, causation or preventability.
As in other studies, we defined potential AEs as cases involving an error with a reasonable chance that harm could result (eg, missed lab testing to measure serum levels of a renally cleared antibiotic in a patient with chronic kidney disease). We also identified cases with errors or substandard care where the likely consequence is unknown or non-specific (eg, absence of daily progress notes, medication orders without date or time stamp). The main reason for including these cases with substandard care was to ensure that we did not miss key latent contributing factors that cut across multiple AEs. We arrived at a final rating for all events based on the majority opinion of the reviewers and resolved disagreements in ratings through consensus.
Characterisation of contributing factors
Beyond classifying AEs by their severity (ranging from a critical lab value to death or permanent disability) and type (eg, adverse drug event, hospital-acquired infection, procedural complication), we further characterised all events, including AEs, potential AEs and additional errors by one or more categories of factors that contributed to the event. We based these categories of contributing factors initially on a previously published framework of patient safety problems,30 which underwent iterative modifications as new categories of factors were identified during our study. The key driver for defining categories was the potential for identifying a specific intervention (eg, preventing hospital-acquired deep venous thrombosis by improving administration of venous thromboembolism (VTE) prophylaxis31). We also identified causally distinct subcategories within some of the larger categories (eg, communication problems between staff are distinct from communication problems arising between staff and patients).
The members of the review team classified contributing factors for each event by consensus. To ensure that we applied the various categories and subcategories of contributing factors consistently throughout the study, we referred to written definitions for the various contributing factors.30 Two investigators (BMW and SD) also kept detailed notes during each meeting documenting how the team arrived at decisions regarding various categories of contributing factors, which they could refer to when review team members disagreed on categorisation of contributing factors, further improving the consistency of our process.
Analysis
We use descriptive statistics, including mean and SD or median and IQR for continuous variables and counts, percentages and 95% CIs for categorical variables, to summarise study results. We report event rates using two commonly accepted formats, namely the proportion of patients with ≥1 AE and the number of AEs per 1000 patient days. We also report the frequency of the different categories of contributing factors that led to AEs, potential AEs and errors or substandard care. We used Microsoft Excel and Access 2007 for all data management and analyses. The Sunnybrook Health Sciences Centre research ethics office approved this study.
Results
We carried out prospective clinical surveillance on 141 patients over 703 patient days (table 1). The median age was 79 years (IQR 64–85); 86 (61%) of the patients were women. The median length of stay was 9 days (IQR 5–17). Thirty-five (25%, 95% CI 17% to 32%) patients had significant comorbidities (ie, Charlson comorbidity index score ≥3 points), most commonly diabetes (23%, 95% CI 16% to 30%), prior stroke (20%, 95% CI 13% to 26%) and dementia (18%, 95% CI 11% to 24%). The median duration of surveillance per patient was 4 days (IQR 2–7).
Patient characteristics
Prospective surveillance detected at least one trigger in 73 (52%) patients and identified 22 AEs, including 15 (68%) that were preventable, 41 potential AEs and an additional 31 errors or cases with substandard care (table 2). Front-line staff spontaneously brought forward concerns to our observer that led to the discovery of 3 (14%) of the 22 AEs. At the individual patient level, 17 (12%, 95% CI 6% to 17%) patients experienced at least one AE, 14 patients (10%, 95% CI 5% to 15%) had at least one preventable AE and 32 (23%, 95% CI 16% to 30%) patients (28 additional patients) had at least one potential AE. We observed at least one error or instance of substandard care in an additional 26 (18%, 95% CI 12% to 25%) patients.
Adverse event (AE) risk and rate
The severity of harm associated with the 22 identified AEs ranged from transfer to the intensive care unit (8%) and suffering permanent harm (4%) to critically abnormal lab values without any overt symptoms (4%). More commonly, AEs led to temporary harm (41%), need for medical treatment (32%), increased monitoring and testing (32%) and psychological distress (32%). Non-infectious complications of hospitalisation (n=14, 25%), treatment problems (n=13, 23%) and medication problems (n=9, 16%) constituted the most common types of preventable and potential AEs (table 3).
Adverse event type
We identified numerous distinct categories of contributing factors associated with the events detected in our study (table 4). For the majority of events, we identified multiple contributing factors. Together, preventable and potential AEs had a median of three contributing factors (IQR 2–4); only six (11%) had a single identified contributing factor. The most common contributing factors were policy and procedural problems (eg, routine protocol violation, conflicting policies; 37%), communication and teamwork problems (34%), and medication process problems (23%).
Categories of contributing factors for preventable and potential adverse events identified through prospective clinical surveillance
The most commonly occurring categories of contributing factors in fact consisted of multiple distinct subcategories. To illustrate this observation, we include a more detailed description of one of the most frequently occurring categories of contributing factors, namely communication and teamwork problems (table 5). The 32 identified communication and teamwork problems exhibited considerable heterogeneity (eg, failure to handover care effectively, lacking a shared care plan for a patient, difficulty eliciting input from specialty services in a timely manner), which meant that even the major categories of contributing factors identified in our study consisted of subcategories that individually related to a much smaller subset of AEs.
Subcategories of communication problems contributing to adverse events identified by the trigger tool
Discussion
We achieved our goal of implementing an augmented approach to detect AEs using a trigger tool in near real time and supplementing record review with front-line staff debriefs, uncovering details not available in the patient record and enriching our assessment and classification of contributing factors. We detected AEs at a rate comparable to prior major AE studies: 12% of our patients had one or more AEs compared with 2.9–16.6% reported in prior retrospective record review studies.6–12 (A previous prospective clinical surveillance study observed an AE rate of 13.7%.29) Consistent with our goal of identifying AEs amenable to improvement activities, the proportion of preventable AE rate (68%) exceeded that in most other studies (37–51%6 ,11 ,12).
As in other studies, commonly occurring major categories of AEs involved medications, complications of hospitalisation (eg, falls) and treatment or management problems. We also encountered challenges with respect to AE heterogeneity within each of these major categories similar to those recently highlighted in a commentary32 revisiting the landmark Utah and Colorado Medical Practice Study.10 Reflecting on operative AEs, the most common category of AEs identified in that study,10 the lead author found that it “contained 20 types of AEs, each of which comprised additional subtypes and were caused by a large variety of errors.”32
While our study did not include operative AEs, the same heterogeneity within major categories applied. For instance, medication ordering errors have little to do with administration and dispensing errors. Effective interventions directed at the former include computerised order entry with effective decision support33 and involvement of clinical pharmacists.34 Reducing medication administration errors, by contrast, requires different types of interventions, such as bar coding.35 The non-infectious complications of hospitalisation present an even starker example since interventions to reduce falls will not affect pressure ulcers, VTE or delirium.
This heterogeneity within major AE categories presents a substantial challenge for improvement efforts as each category demands multiple interventions, each of which requires intense effort (eg, implementing computerised order entry, bar-coded medication administration, an effective falls prevention programme and various infection prevention and control strategies). Even when effective, implementation of several such interventions would only affect a small proportion of events and probably not achieve the goal of reducing the overall burden of patient harm within the organisation.
We anticipated this problem of heterogeneity within major AE categories such that even a suite of interventions directed at specific AE subtypes might not appreciably reduce the overall AE rate. This concern motivated our focus on characterising factors contributing to each event as comprehensively as possible. Consistent with the systems approach to patient safety and its emphasis on latent errors as well as active ones,36 we hypothesised that apparently distinct AE categories might share common contributing factors. By identifying common latent errors (eg, policy and procedures problems, poor teamwork and communication), we hoped to inform efforts to develop interventions likely to reduce multiple different types of AEs.
At first glance, we achieved this goal. Our prospective approach allowed us to debrief staff around the time of each event in order to identify and characterise contributing factors in ways not possible with medical record review alone. We identified a median of three contributing factors for each preventable or potential AE. Furthermore, a short list of contributing factors applied to numerous events. Problems related to policies and procedures contributed to 37% of events, and communication and teamwork problems contributed to 34%. One might surmise, therefore, that we could decrease the overall burden of AEs on our clinical service by directing our QI activities towards addressing these two general categories of patient safety problems with targeted interventions.
Though appealing, our results suggest that organisations may still face challenges with this approach. Similar to the problem of heterogeneity within major categories of AEs, we observed substantial heterogeneity within categories of contributing factors. Consider, for example, ‘communication and teamwork’, which prior studies have also identified as an important root cause of many patient safety problems.37 We identified seven subcategories of ‘communication and teamwork’ problems (table 5). If one took the approach of implementing interventions to directly target a particular subtype of communication problem, such as situation-background-assessment-recommendation training for all staff38 to improve the effectiveness of nurse-to-physician communication, one might still only affect a small subset of the overall AEs on our service that resulted from communication failures. Thus, just as an institution would need to implement multiple distinct interventions to demonstrably reduce the various subtypes of medication safety problems or non-infectious complications of hospitalisation, it seems that an institution attempting a more cross-cutting approach (eg, reducing teamwork and communication problems) would also need to implement several distinct interventions for just this one category.
It is possible that a single ‘teamwork’ or ‘communication’ intervention could by itself reduce multiple types of AEs or even impact a broad outcome such as hospital mortality. For example, a number of Veterans Health Administration hospitals instituted mandatory medical team training for all surgical teams, which required briefing and debriefing and included checklists as an integral part of the intervention,39 and reported significant reductions in surgical mortality compared with control hospitals. More recently, a handoff bundle consisting of team training, electronic medical record configuration, faculty development and structural changes to reduce interruptions produced a significant reduction in preventable AEs in nine paediatric hospitals.40
However, the intensity of these two interventions, developed and refined over several years at multiple collaborating institutions, speaks to the scope of effort likely required to reduce multiple types of AEs in order to detect an impact with a trigger tool. A single institution using a trigger tool as part of its routine safety monitoring and improvement efforts will likely struggle to develop de novo interventions that effectively address cross-cutting factors such as teamwork and communication problems since simple ‘off-the-shelf’ solutions do not exist in many cases to solve these complex, multifaceted problems.
Limitations
Our study has several potential limitations. Possibly we identified too few AEs and thus missed the opportunity to find sufficient numbers of AEs within any given category. However, our AE rate was comparable to prior patient safety studies as was the broad distribution of event types. We carried out the prospective clinical surveillance at a single centre that has invested heavily in safety, which may limit generalisability. Again, however, we detected similar types of AEs as in previous studies, so we believe our case finding reflects the usual types of patient safety problems that affect general medicine patients in most institutions.
We modified the way we applied the trigger tool with continuous surveillance rather than periodic random sampling of medical records, use of a trained observer with a pre-existing relationship with front-line staff and an expanded list of triggers. It is thus possible that our experience does not extend to the more traditional use of the global trigger tool. However, we made these modifications to the trigger tool methodology precisely to increase the likelihood of learning from AEs. The traditional global trigger tools and retrospective record review would likely suffer from the same challenges we encountered. We also implemented the prospective clinical surveillance methodology on a general medicine ward and included elderly patients with complex medical problems, so our findings and associated challenges may not be generalisable to other clinical settings, such as surgical or obstetrical wards, where the patient population, AEs and contributing factors may be more homogeneous. There are also concerns about the degree to which rates of harm detected using a trigger tool vary by reviewer.41 ,42 We tried to address this by convening weekly team meetings that included a core group of interprofessional members that used common criteria to rate harm by consensus. Variation in judgements of AEs can still exist across teams of reviewers,43 but this issue mostly pertains to studies primarily aimed at accurately measuring AE prevalence. Our main focus, by contrast, lay in identifying and classifying contributing factors.
Conclusions
Our findings suggest that an augmented trigger tool can identify a sample of AEs enriched for preventable events and characterise cross-cutting contributing factors that affect a meaningful proportion of these preventable and potential AEs. This approach has the potential to stimulate QI activities and track improvements over time under specific circumstances. However, the majority of contributing factors accounted for a small number of AEs, and more general categories were too heterogeneous to inform specific interventions. Successfully using trigger tools to stimulate QI activities may require the development of a framework that better classifies events that share contributing factors amenable to the same intervention.
Acknowledgments
We would like to thank core members of the team that met regularly to review and adjudicate events as a part of this study, including Deborah Brown-Farrell, Romina Marchesano, Grace Rzyczniak and Natalie Zur Nedden. We would also like to thank Lisha Lo for providing feedback on an earlier draft of the manuscript.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors All authors have made a substantial, direct, intellectual contribution to this study. BMW had full access to all of the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. All authors met the criteria for authorship. KS is the Editor-in-Chief for BMJ Quality and Safety. He remained at arm's length throughout the review process.
-
Funding Canadian Patient Safety Institute (RFA09-1194-ON). Sunnybrook Health Sciences Centre Foundation.
-
Competing interests None.
-
Ethics approval Sunnybrook Health Sciences Centre research ethics office.
-
Provenance and peer review Not commissioned; externally peer reviewed.