Article Text
PDF
Abstract
Introduction Trauma resuscitation is a complex and time-sensitive endeavour with significant risk for error. These errors can manifest from sequential system, team and knowledge-based failures, defined as latent safety threats (LSTs). In situ simulation (ISS) provides a novel prospective approach to recreate clinical situations that may manifest LSTs. Using ISS coupled with a human factors-based video review and modified framework analysis, we sought to identify and quantify LSTs within trauma resuscitation scenarios.
Methods At a level 1 trauma centre, we video recorded 12 monthly unannounced ISS to prospectively identify trauma-related LSTs. The on-call multidisciplinary trauma team participated in the study. Using a modified framework analysis, human factors experts transcribed and coded the videos. We identified LST events, categorised them into themes and subthemes and used a hazard matrix to prioritise subthemes requiring intervention.
Results We identified 843 LST events during 12 simulations, categorised into seven themes and 38 subthemes, of which 23 are considered critical. The seven themes relate to physical workspace, mental model formation, equipment, unclear accountability, demands exceeding individuals’ capacity, infection control and task-specific issues. The physical workspace theme accounted for the largest number of critical LST events (n=152). We observed differences in LST events across the four scenarios; complex scenarios had more LST events.
Conclusions We identified a diverse set of critical LSTs during trauma resuscitations using ISS coupled with video-based framework analysis. The hazard matrix scoring, in combination with detailed LST subthemes, supported identification of critical LSTs requiring intervention and enhanced efforts intended to improve patient safety. This approach may be useful to others who seek to understand the contributing factors to common LSTs and design interventions to mitigate them.
- simulation
- emergency department
- human factors
- quality improvement
Data availability statement
Data are available upon reasonable request. All deidentified participant data are available upon reasonable request from the study’s lead author (AP) who can be contacted at petro82@gmail.com.
Statistics from Altmetric.com
Introduction
The resuscitation of a critically injured patient is a high-stakes and dynamic process, and medical error remains common.1 2 The inherent complexity and unpredictability of trauma care challenges even the most experienced teams.3 Errors in trauma care often occur as a result of latent safety threats (LSTs): previously unrecognised system-based conditions that under certain circumstances manifest and threaten patient safety.4 5
Current methods used to study and improve trauma care are often limited by recall bias or an inability to verify critical details needed to understand how errors develop, and fail to capture system-based factors.6 7 Without a thorough understanding of the nature of errors in trauma care, corrective measures remain challenging. A reliable evaluation of all activities during trauma resuscitations is needed followed by a comprehensive analysis of how teams and their environments converge to produce errors.
In situ simulation (ISS), a technique where clinical teams engage in simulated clinical scenarios within the clinical workspace, may provide a superior mechanism to identify LSTs compared with centre-based simulation.5 8 9 The use of ISS allows for the study of workflows, processes and systems within the exact environment that reported critical events occurred under controlled and reproducible conditions.10–13
Human factors (HF)-based video review is a promising approach that seeks to prospectively detect LSTs. Video review allows for multiple viewings of an objective record of the simulation (ie, reducing recall issues), and facilitates a holistic understanding of the perspective of all trauma team members.14 15 In addition, the integration of HF, a discipline concerned with the interactions among humans and the systems in which they work, ensures that multiple system factors are considered, including but not limited to tools and technology, physical environment and layout, individual skills/knowledge and team interaction, task characteristics and organisational characteristics.16
The Trauma Resuscitation Using in situ Simulation Team Training (TRUST) study is an exploratory investigation that sought to:
Identify and prioritise LSTs during trauma ISS, using a prospective, video-based framework analysis.
Evaluate the feasibility of regular, ISS sessions at a level 1 trauma centre.
Methods
Setting and participants
We conducted the TRUST study at a Canadian level 1 trauma centre, a centre with approximately 800 trauma team activations annually (33% severely injured with Injury Severity Score >16) and 75 000 emergency department (ED) visits. The hospital’s two-bed trauma bay served as the location for study. All of the clinical equipment, systems and processes used during real trauma resuscitations were available to the trauma team during the simulations. Each simulation was video recorded by wall and ceiling-mounted cameras (GoPro Hero4, Sony Handycam Exmor R) capturing all areas of clinical care within the room. A dedicated overhead microphone (Aputure V-Mic D1 Directional Condenser Shotgun) was installed on the ceiling and handheld voice recorders were placed at workstations to capture conversations away from the patient. Additional methodological details can also be found in the study protocol published previously.17
Study participants consisted of the on-call trauma team on the day of the simulation. The trauma team members rotate daily, led by a staff physician in emergency medicine or trauma surgery (trauma team leader, TTL). The team typically consists of two ED nurses, two general surgery residents, one to two orthopaedic residents, an anaesthesia resident, a respiratory therapist, an X-ray technician and a clinical assistant. Consent was obtained in advance or on arrival in the trauma bay.
Study design
TRUST was an ISS study using scenarios developed from real patient cases identified by an institutional morbidity and mortality process. We conducted a chart review of adverse events, deviations from protocol adherence and unexpected deaths occurring between January 2013 and December 2014. A total of 132 cases (9%) from 1473 trauma team activations were identified. Twenty-six cases (26/132, 19.7%) were flagged and reviewed in greater detail. Two of the authors, both board-certified emergency physicians with >5 years each of experience in trauma care and as simulation educators, reviewed each case. Five recurrent clinical situations were identified that posed recurring threats to patient safety: difficult airway management, challenges with massive transfusion protocol (MTP) activation, injuries preceded or complicated by medical events, penetrating trauma and agitated patients. Using these situations, we developed four risk-informed simulation scenarios (table 1) described further in the published study protocol.17 We did not specifically script LSTs to occur but rather recreated clinical scenarios that aligned with previous adverse events. Scenario modifications were made to ensure they differed sufficiently from actual cases and could be conducted within 15 min. Each scenario was piloted and revised. We tested the scenarios for clarity (do participants respond appropriately to simulated clinical cues), flow (does the case progression make sense) and feasibility (can the main objectives be completed within 15 min).
Summary of simulation scenarios
ISS procedure
We conducted monthly, unannounced ISS during the study period (July 2015 to June 2016). Scenarios 1, 2 and 3 used SimMan 3G Human Patient Simulation (Laerdal Medical, Stavanger, Norway). Scenario 4 used a trained actor to perform as a standardised patient.
As a measure of feasibility, we collected several metrics regarding the ED status around the time we began each ISS session and determined whether to cancel or delay the session. Specifically, our study team recorded the number of patients in the ED, the number of critical patients who arrived within 2 hours of our planned ISS start time, the number of patients waiting to be seen in our acute area of the ED and the duration of both the scenario and the debriefing. ‘No-go’ considerations included department capacity >90%, patient acuity and the expected arrival of a trauma patient.18 We planned sessions to occur in the morning given our ED volumes are typically lowest during this period. The study team, consisting of trauma physicians, simulation specialists, simulation educators and HF experts, were present in the trauma bay prior to the scenario commencing and conducted each simulation. Two HF experts observed and documented LSTs as well as team and system performance.
To begin the scenario, the trauma team was notified in the usual manner through the hospital paging network; the manikin or actor arrival occurred 2 min following the page. The team was aware of the study in advance and the possibility that they may be asked to participate in an unannounced ISS. Only the TTL was aware of the specific time and date which helped reduce potential delays or overlap with patient care. All trauma team members were consented in advance to account for last minute changes in the call schedule and team composition. Our study coordinator crosschecked each potential participant on their arrival in the trauma bay to ensure they had previously consented. Any and all participants could opt out on arrival in the trauma bay without any reason required. On arrival the trauma team provided care in the usual manner, using actual equipment, personnel, resources and protocols available to them during actual trauma resuscitations. Scenario duration was 15 min followed by 15 min semistructured debriefing with a focus on LST identification and issues that impacted patient care.19
Data analysis
Modified framework analysis
We used a framework analysis to thematically code the video data.17 20 Three HF experts identified and categorised the LST events (eg, equipment malfunctions or behaviour inconsistent with previous team discussion) into several themes and subthemes, which were established deductively (ie, using previously published themes)5 21 22 and inductively (ie, newly created themes and subthemes). We defined ‘theme’ as a broad topic, idea or concept that incapsulates a variety of similar LSTs (eg, physical workspace). Subthemes exist under each theme and describe more precisely elements of the LST event.
Our team’s HF experts observed all sessions in person to provide context and greater understanding for the framework analysis which they conducted using video review. To maximise accuracy of LST events, a study team member and trauma physician (AP) reviewed the transcriptions and each video to ensure clinically relevant events were included and resolved any misinterpretations. Discrepancies in the assignment of LST events to themes/subthemes were resolved by consensus for sessions 1–4. Inter-rater reliability in LST assignment was established by session 4 (Cohen’s kappa >0.7), allowing a single HF expert to code the remaining sessions (5–12).
We charted the LST events into a framework matrix (figure 1). For each simulation (eg, surgical airway) we transcribed an LST event in each row, time stamped and assigned relevant themes/subthemes. Data contained in the intersection of a row and LST theme column show the LST subtheme applied to a specific event in a simulation session. This output matrix concisely summarises all LSTs within and across simulations, and can be filtered by scenario type and LST theme. The output also facilitated a count of the frequency of events categorised within each LST theme/subtheme.
(A, B) Examples from the framework analysis matrix output. The surgical airway latent safety threat (LST) event highlights when two separate LST subthemes are identified for a single LST event. For the purpose of this example, only two themes are displayed (unique columns in A and B, respectively). *The event timestamp represents the time during the simulation that the LST event occurred. Subthemes are shaded. Anaesthesia refers to anaesthesia team member. IV, intravenous access; RN, registered nurse; TPA, tissue plasminogen activator; TTL, trauma team leader.
Hazard scoring and risk prioritisation
To identify ‘critical’ LST subthemes, two trauma physicians independently scored each LST subtheme based on their perceived frequency (estimated based on clinical experience rather than observations during simulations) and severity. Both the frequency and the severity of each LST were assigned a score between 1 and 4, where 1 is infrequent or minor in severity and 4 indicates highly frequent or catastrophic. Any discrepancies between scores were discussed until consensus was established. From the hazard matrix, a ‘hazard score’23 was calculated by multiplying the frequency and severity scores. All LST subthemes with critical hazard scores (≥8) were subject to review by our trauma and ED clinical teams to develop threat mitigation strategies.23 If an LST was encountered that was deemed to pose an imminent risk to patients, our team presented these findings to hospital leadership.
Outcome measures
The primary outcome was a framework matrix output (figure 1), which lists LST events for each scenario, charted by timestamp and categorised by LST theme. Each LST subtheme was assessed using a hazard matrix23 to determine which would be considered ‘critical’ LST subthemes (hazard score ≥8). The frequency of events in each LST subtheme was also calculated. Each event could be assigned to more than one theme/subtheme. Additionally, repeated events of the same theme/subtheme were counted separately (eg, participants could not view the vital signs monitor on multiple occasions throughout the scenario was counted each time this event occurred). Secondary outcomes included the feasibility of regular trauma ISS based on department status (number of critical patients who arrived within 2 hours of ISS start time, number of patients waiting to be seen), cancellation rates and the duration of both the scenario and debriefing.
Results
We conducted 12 ISS sessions over 12 months. Each session consisted of one of four simulation scenarios. We conducted each scenario between two and four times indicated with parentheses: surgical airway (3), blunt trauma with MTP (4), trauma precipitated by medical event (3) and penetrating injury (2). Inter-rater reliability was assessed at the theme level for simulations 2, 3 and 4, with progressively increasing Cohen’s kappa values of 0.428, 0.557 and 0.845, respectively.
Using the framework analysis, we identified 843 LST events during 12 simulation sessions. These events were categorised into at least one of seven themes (an event may belong to multiple themes) and further classified among 38 LST subthemes, of which 23 were considered critical (hazard score ≥8) LST subthemes. All events categorised within a critical subtheme were considered ‘critical’ events by extension.
Table 2 summarises the number of total and critical events by scenario and the mean critical events per scenario. The percentage of LST events considered critical was similar across scenarios ranging from 47% to 67% of total LST events (online supplemental appendix A). Differences in the distribution of events across both themes and scenarios are evident. For example, the theme with the most LST events was ‘Communication: unclear responsibility and/or accountability’ (n=209), but the ‘Physical workspace’ theme accounted for the largest number of critical LST events (n=152). It is noteworthy that all (ie, 100%) ‘Communication: demands exceed individuals’ capacity’ LST events and most (ie, 97%) of ‘Team co-orientation and mental model formation’ LST events were critical. In terms of scenario differences, scenario 2 accounted for the largest number (n=216) and greatest proportion of critical LSTs (67%). In contrast, we observed in the less complex scenario 4, only 41 critical LSTs representing 47% of all LSTs.
Supplemental material
Count of critical latent safety threat (LST)* events by theme and subtheme
Twenty-three LST subthemes and 61% (516/843) of all LST events were categorised as critical (hazard score ≥8) (table 2). The four most frequently observed critical LST subthemes were (1) location of vital signs monitor not ergonomically optimised (ie, not easily viewed by team members) (92/843, 10.9%), (2) patient care activities delayed or not completed due to task overload (65/843, 7.7%), (3) instances lacking closed loop communication (61/843, 7.2%), and (4) team members absent for paramedic handover resulting in missed critical patient information (49/843, 5.8%).
Simulation feasibility
Before the study began, we conducted three presentations to multiple stakeholder groups at our institution to introduce the concept of unannounced ISS. We consented 455 potential participants who may be part of the trauma team response. A total of 124 individuals participated in 12 ISS sessions (mean=10.3/session). The mean simulation and debriefing time was 17.4 and 20.2 min, respectively. Three sessions were delayed (mean delay of 39 min) due to arrival of a trauma patient, but no sessions were cancelled. At the start of each simulation, a mean of 38 patients had registered and were waiting or receiving care in the ED. For context, approximately 270 patients visit our ED daily. Participant information is listed in table 3. Throughout the entire study, we had two participants decline participation on arrival. In each case, there were sufficient participants for the simulation to continue. In eight sessions, participants (mean=1.25 participants/session) left early from the debriefing for patient care-related activities.
Total participant counts for all 12 simulations
LSTs necessitating immediate intervention
Using video-based ISS we identified two LSTs that were deemed critical (hazard score ≥8) and were of imminent risk to patient safety.
Delays in blood administration for bleeding trauma patients
During the first session of scenario 2, we observed a significant delay (23 min) in blood administration following the activation of our MTP (ie, scenario 2 in table 1). We identified two primary reasons for the delay: (1) porters responsible for transporting blood from the blood bank to the trauma bay used circuitous routes, and (2) nurses made only one of the two necessary calls to initiate the MTP. In response, we introduced an optimal route to the porters for transporting blood, and trialled a single call process to begin the mass transfusion protocol (ie, nurse calls locating then is automatically forwarded to blood bank instead of making a second call).
Delays in set-up for cricothyroidotomy performance
During scenario 1, clinicians tasked with performing a cricothyroidotomy, a time-critical intervention, gathered four different pieces of equipment from separate locations around the trauma bay resulting in procedural performance delays. We subsequently bundled all four pieces of necessary equipment in a readily accessible kit found in a single location, which reduced the distance by fourfold to retrieve the equipment.24
Discussion
Overview of findings
We identified 843 LSTs during trauma ISS, categorised into seven LST themes and 38 subthemes, using the novel application of a video-based framework analysis. These findings provide a high degree of nuance and are traceable to time stamped video. For example, instead of concentrating broadly on the LST theme ‘team co-orientation’, a team tasked with safety threat mitigation may elect to address specific subthemes such as ‘team members absent for EMS handover’ or ‘shared mental models not established’, and review video examples pertinent to each subtheme. The ability to precisely review multiple LST events of the same subtheme across scenarios is an important strength of the framework analysis, as it provides clear evidence of how the same issue affects a diverse set of clinical scenarios.
Contextualising the identification of critical LSTs
Most of the LST themes and critical subthemes occurred during all four of the simulation scenarios. This suggests that at our institution, these findings are important across a variety of trauma care situations. These common themes may also be relevant to other institutions; however, a larger study is required for confirmation.
The theme ‘Physical Workspace’ accounted for the most critical LST events which included ‘location of vital signs monitors are not ergonomically optimized’ as the most frequently critical LST subtheme. This finding suggests the need to focus on a physical workspace redesign. In fact, as a result of these data, our institution invested in two additional vital signs monitors (ceiling mounted at the foot of the bed and wall mounted on patient right), offering 270° of monitor visibility in our newly designed trauma bay.
The critical LST related to delays to perform a cricothyroidotomy was categorised across two themes: ‘Physical Workspace’ (ie, location of the kit unknown) and ‘Communication: unclear responsibilities/accountability’ (ie, two team members searched for this kit simultaneously without coordination of their efforts). The identification of these two causal factors can direct quality improvement (QI) efforts towards aligning solutions to the causal factors. We might envision a suite of solutions that improves the design of the clinical space coupled with team training sessions emphasising team member coordination.17
Value of HF informed framework analysis of ISS video recordings
In contrast to other studies where mean LSTs/simulation range from 0.8 to 20.9, we identified 43 critical LSTs/simulation.5 8 13 22 25 We identified a higher number of LSTs than other studies likely due to our approach to the framework analysis, where we (1) used a detailed video review of our simulation scenarios, which allowed for multiple viewings and minimised recall bias, (2) allowed for a mix of deductive and inductive coding, and (3) engaged HF experts who emphasised a system-based view of LST events.
Direct observation and video review by HF experts resulted in the identification of LSTs that were not described by the participants during the debriefing. For example, the most frequent critical LST ‘location of vital signs monitor not ergonomically optimized’ in our framework analysis was never identified by participants during the simulation debriefing. Simulation participants may not identify—due to distractions, impaired situational awareness or simply acclimation to the trauma bay—their poor view of the vital sign monitors. This highlights the value of video-based expert review of ISS sessions to identify threats to and improve patient safety.
Reflections on study feasibility
Prior to undertaking this study, there were no regular ISS sessions within our trauma programme. The socialisation process to conduct unannounced ISS required substantial work to ensure these sessions could occur while respecting the clinical responsibilities of clinicians and needs of patients. We conducted presentations with multiple stakeholders and circulated email notifications of the process. The process was greatly aided by support from our hospital leadership. A substantial amount of work was required to consent all potential participants in advance so that on arrival there would be no delay in their participation in the simulation.
While we delayed the start to three sessions (due to the pending arrival of a real trauma patient), none were cancelled which differs from the experience of others.5 We attribute this zero cancellation rate, in part, to scheduling sessions in the morning before peak patient volumes. We also worked closely with our charge nurses and presented this study as adding ‘one extra patient’ to our ED volume per month, a helpful cognitive reframing strategy.
Limitations
This study has several important limitations. First, it is a single site study and the LSTs identified may not be representative of issues at other institutions. Anecdotally, clinicians from other institutions have remarked that these LST themes are similar to what they experience; further research is required to confirm this. Our goal is not to provide a generalisable taxonomy of LSTs, but rather trial a novel approach to LST identification during ISS. The application of video review coupled with a framework analysis is practical for those interested in an in-depth analysis of their processes during acute care. This approach is time consuming, but we have demonstrated it provides a more complete picture of system-based LSTs. Second, this study was conducted only during simulated trauma resuscitations and may not be applicable to other forms of resuscitation. However, the LST themes identified are plausible across a spectrum of resuscitation scenarios. Third, it is possible that our method did not identify every LST that occurred. However, its advantages are complementary to the most common strategies currently used (ie, participant recall during debriefing). Fourth, due to the severity of LSTs observed, we implemented an intervention in scenario 2 during the course of the study, possibly affecting our results. However, we note that scenario 2 remained the scenario with the highest percentage of critical LST events and the highest mean of critical LST events per scenario, suggesting that the framework analysis continues to draw meaningful observations in parallel with improvement efforts.
Future considerations
This study offers a preliminary investigation of ISS combined with a robust data analysis methodology to identify LSTs, but future work is needed to understand how these findings can be applied to clinical practice. We describe one example in our study where we discovered delays in time to blood delivery during our MTP activations. Based on our findings, we streamlined and optimised the MTP process which resulted in a 21% reduction in the time to blood delivery among real trauma patients, demonstrating rapid knowledge translation to applied settings.26 Therefore, we encourage future efforts to implement ISS in QI in close connection to patient-oriented outcomes. Future studies are needed to confirm and better define this promising application. Finally, a multicentre study using the same ISS scenarios may assess the generalisability of our findings.
Conclusions
This study describes the novel application of a video-based framework analysis, informed by the systems analysis common in HF methods, for LST identification during multidisciplinary, high-fidelity ISS. We observed common critical LSTs across a broad set of trauma situations, suggesting safety threats in the underlying processes of trauma care can be uncovered by this approach. The application of the framework analysis allowed for the systematic identification of LST patterns within and between simulations. Hazard matrix scoring aids in efficiently prioritising LSTs most in need of resolution to improve patient safety. Using this process, we gained a more complete understanding of the events linked to each LST, an important advancement to developing effective mitigation strategies and interventions. Further studies are required to establish the applicability of this technique.
Data availability statement
Data are available upon reasonable request. All deidentified participant data are available upon reasonable request from the study’s lead author (AP) who can be contacted at petro82@gmail.com.
Ethics statements
Ethics approval
The hospital’s research ethics board approved the study (REB No 15-046).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @petrosoniak, @KariLynWhite
Contributors AP and CMH conceived the study concept. All authors designed and implemented the study. MM and MF completed data collection. AP, MF, CMH and PT conducted the data analysis and data interpretation. The manuscript was drafted by AP and MF. All authors contributed to the edits and revisions during the manuscript drafting process. All authors approved the final version of the manuscript.
Funding This study was funded by St Michael's Hospital AFP Innovation Fund (SMH-15-007), Royal College of Physicians and Surgeons of Canada (15/MERG-06) and Simulation Canada (formerly Sim-One) (4-3-2015).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.