Article Text
Abstract
Objectives After introducing a team simulation training programme at our hospital, we saw a reduction in door-to-needle times (DNT) for stroke thrombolysis but persisting variability prompting further investigation. Our objective is to examine this gap through assessing: (1) whether there is an association between DNT and the clinical experience of neurology registrars and (2) whether experience influences the benefits from attending simulation.
Design Prospective cohort study.
Setting and participants Patients treated with intravenous thrombolysis between January 2016 and 2020 at a Norwegian stroke centre.
Primary and secondary outcome measures Using DNT and prior intravenous thrombolysis administrations (case-based definition of clinical experience) as continuous variables, a mixed effects linear regression model was performed to examine the association between clinical experience, DNT and simulation attendance. For dichotomised analyses, neurology registrars with 15 or more prior treatments were defined as experienced.
Results A total of 532 patients treated by 36 neurology registrars from January 2016 to 2020 were included. There was a linear association between clinical experience and DNT (test for non-linearity p=0.479). Each prior intravenous thrombolysis administration was associated with a significant 1.1% decrease in DNT in the adjusted analysis (ΔDNT −1.1%; 95% CI, −2.2% to −0.0%; p=0.048). The interaction between effects of clinical experience and simulation on DNT was not statistically significant (p=0.150). In the dichotomised analysis, experienced registrars had similar gains from attending simulation sessions (mean DNT from 18.5 min to 13.5 min) compared with less experienced registrars (mean DNT from 22.4 min to 17.4 min).
Conclusions Less experienced registrars had longer DNT in stroke thrombolysis. Attending team simulation training was associated with similar improvements for experienced and inexperienced neurology registrars. We suggest a focus on high-quality onboarding programmes to close the experience-related quality gap. Our findings suggest that both inexperienced and experienced neurology registrars might benefit from team simulation training for stroke thrombolysis.
- quality improvement
- stroke
- medical education & training
- stroke medicine
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study included a large number of patients (532) treated with intravenous thrombolysis from a comprehensive prospectively registered data set.
The wide range of variables included, and robust statistical models allowed for adjustment of potential confounding factors known to affect variability in door-to-needle times.
By assessing the impact of simulation training on door-to-needle times across different levels of clinical experience we were able to provide nuanced insights into the role of simulation training in medical education.
This is a single-centre study and factors specific to our context, such as onboarding procedures and simulation design, might limit generalisability of the findings.
The study design is susceptible to bias from unknown and uncontrolled temporal trends.
Introduction
The global burden of stroke is substantial and increasing. Globally one in four people will have a stroke in their lifetime, this number has increased over 50% during the last 17 years.1 Although efforts have been made to improve stroke care, unwanted variation in the delivery of care (ie, quality gaps) persists both within and between different hospitals.2 3 To alleviate some of the stroke burden, it is important to address unwanted variation across all dimensions of stroke care.4
One critical aspect of acute stroke management is the door-to-needle time (DNT) for intravenous thrombolysis (IVT) as it directly impacts patient outcomes.5 A recent meta-analysis showed that simulation training improves DNTs in stroke thrombolysis.6 In line with these findings, the first year after introducing a team simulation training programme at our hospital, the median DNT for IVT in patients with acute ischaemic stroke was reduced from 27 min to 13 min.7 Although we previously saw a reduction in DNT, there was persisting unexplained variability, prompting further investigation. The variations in clinical DNT are typically influenced by a range of factors including non-modifiable patient factors.8 In addition, based on our experience with simulated cases, we suspect that unwanted variation in DNT can be partly attributed to the clinical experience of neurology registrars. We also suspect that clinical experience influences the effects of simulation on DNT. Specifically, we hypothesised that: (1) Inexperienced neurology registrars have longer clinical DNT in stroke thrombolysis than their more experienced peers, and (2) inexperienced neurology registrars benefit more from attending simulation sessions, in terms of clinical DNT improvement, compared with their more experienced peers.
These hypotheses might be of particular interest due to the global and increasing concern of physician turnover.9 10 In contexts where care primarily relies on inexperienced physicians, treatment delays caused by a lack of clinical experience might significantly affect the quality of stroke care. Our first objective is therefore to evaluate whether there is an association between DNT and the clinical experience of neurology registrars. By understanding this relationship, we can identify whether clinical experience is a source of unwanted variation and a potential area for improvement of DNT. Our second objective is to determine whether the clinical experience of neurology registrars influences the effectiveness of simulation on clinical DNT. If the effectiveness of simulation is more pronounced among inexperienced registrars, this could imply either that they should be prioritised for participation or that changes to the design are needed to make the programme more applicable as a tool for continuous education.11
Methods
Study setting and data sources
Catchment area
Stavanger University Hospital is one of the largest stroke centres in Norway with a catchment area of 365 000 inhabitants. Annually around 5–600 patients are diagnosed with an acute stroke, approximately 20–30% of these receive IVT.
Current hyperacute treatment protocol
The emergency medical services, in consultation with the neurology registrar, are responsible for activating the criteria-based ‘stroke alarm’. This leads to pre-notification of the in-hospital stroke team (two acute care nurses, one neurology registrar, two radiographers and one radiology registrar). The neurology registrar is defined as the team leader. The ambulance crew prioritises immediate transportation, focused history taking, peripheral venous cannulation, measuring blood pressure and removing all jewellery during transport. Parts of the in-hospital stroke team are briefed (ie, sharing information and assigning tasks) by the neurology registrar prior to the patients’ arrival. On arrival, the patient is quickly examined and transported directly to imaging by CT, or MRI as indicated. In most cases, a non-contrast CT, CT-angiography and perfusion is performed and IVT is given if indicated. Whenever a large vessel occlusion potentially eligible for thrombectomy is detected, a separate ‘thrombectomy alarm’ is triggered (leading to pre-notification of additional team members). After initial treatment in the emergency room, the patients are moved to a neuro-intensive care unit for follow-up.
Onboarding of neurology registrars
Newly hired registrars usually go through a 1-day introductory course covering the most important aspects of acute stroke care followed by a 2-week period in training. There are two neurology registrars on call during daytime and one during night shifts. A neurology consultant and vascular neurologist are available by phone 24/7. There is no mandatory involvement of a neurology consultant, but most registrars will involve a consultant in decision-making for IVT.
Thrombolysis registry
All consecutive patients with a suspected acute ischaemic stroke receiving IVT at our centre are prospectively included in a local registry. The registry contains multiple variables including patient demographics, risk factors, stroke severity (eg, National Institutes of Health Stroke Score) and all relevant time points. Since January 2016, de-identified data about the treating neurology registrar for each patient was registered. The dates of each neurology registrar’s participation in simulation sessions was registered since the first team simulation session in February 2017. For each patient, the treating neurology registrar’s prior IVT administrations and attendance at simulation sessions were calculated. For neurology registrars that were employed prior to the first registration in 2016 we used data about time of employment and estimated 10 annual IVT treatments based on the mean number of annual treatments for other registrars. Data was collected using a form that is filled in by stroke nurses and neurology registrars for all patients with stroke, in addition to the patients’ electronic medical record and a registry of participants in simulation sessions. Data registration is overseen by a dedicated stroke research nurse.
Quality improvement project and team simulation training
A quality improvement (QI) project aiming to reduce DNT and improve patient outcomes at our centre was implemented in February 2017. Interventions included changes to streamline the previous hyperacute treatment protocol (eg, direct to CT-laboratory, parallel processing) and weekly team simulation training sessions focused on improving procedural adherence and team leadership skills for neurology registrars (eg, improving team situational awareness through briefs and summaries, using closed-loop communication). Both the results of our previous work, details regarding context, the QI project, all aspects of the team simulation and data sources can be found in a prior publication.7
Participants, variables and statistical analysis
Participants and variables
All consecutive patients treated with IVT between 01 January 2016 and 01 January 2020 at our centre were prospectively included. For the analysis of an association between clinical experience and DNT we only included patients treated prior to February 2017 (before QI). For the analysis of clinical experience and its potential influence on the effects of simulation, all patients were included. Thus, the group of patients treated by neurology registrars who had not attended the simulation consisted of patients treated both before and after QI.
The key variables in this study were DNT, clinical experience and whether the neurology registrar had attended simulation training prior to IVT treatment. DNT is defined as the time (minutes) from entering the door in the emergency room until a bolus dose of alteplase has been given. We used a case-based definition of clinical experience using each neurology registrar’s prior IVT treatments as a measure of clinical experience. For dichotomisations an experienced neurology registrar was defined as having given 15 or more prior IVT treatments. This number is based on the median number of cases per neurology registrar in our cohort. For simulation attendance, if the date of a neurology registrars’ attendance in a simulation session was prior to the admission date of the patient, the neurology registrar was defined as having attended the simulation prior to treatment. This was regardless of the number of sessions or time between simulation attendance and IVT treatments.
To reduce confounding from other sources than the clinical experience of the neurology registrar, patients undergoing initial MRI examination (typically patients with an unknown time of onset) were excluded as these patients usually have longer DNT due to systemic factors. Patients with an in-hospital onset of symptoms were also excluded as these patients’ door times will not be comparable to patients admitted through the emergency room. Limiting inclusion to only patients treated prior to February 2017 (before QI) in the analysis of an association between clinical experience and DNT was done to reduce confounding from several factors within the QI project potentially affecting DNT. Based on prior studies the most important factors explaining variability in DNT are age, National Institutes of Health Stroke Scale at admission, atrial fibrillation, hypertension, statin use, diabetes and prior stroke.12 These were therefore, together with gender, considered as potential confounders in the analyses.
Statistical analysis
Using DNT and clinical experience as continuous variables, a mixed effects linear regression analysis was performed on patients treated prior to QI to assess for an association between clinical experience and treatment times. A mixed model with random intercepts for each treating neurology registrar was chosen because of the correlated nature of DNT measurements within the same neurology registrar. Because of the non-normally distributed nature of DNT, the calculations were performed using the natural logarithmic transformation of DNT (logDNT). The effect estimates were expressed as expected percentage change in DNT given a one unit increase in clinical experience.13 Linearity of the effect of prior clinical experience on logDNT was tested using restricted cubic splines with three knots. A second similar model was produced to assess whether there was a significant interaction between prior clinical experience and effects on DNT from simulation attendance. For both models, unadjusted and adjusted effect estimates are presented with 95% CIs and p values from Wald tests. Adjustment variables were selected based on a data-driven approach, that is, by stepwise removal of the variables with the highest, non-significant p values and which also did not lead to a substantial change in the effect as compared to the fully adjusted effect estimate (<10% change in total).
Furthermore, for illustrative purposes, unadjusted mixed model analyses were performed with dichotomised clinical experience and simulation attendance, from which we provide predicted geometric mean DNTs (ie, exponentiated predicted mean logDNT) with 95% CI for different groups. Patients with missing data for any of the key variables (DNT, clinical experience or simulation attendance) were excluded. All statistical analyses were performed using Stata V.17.0 and IBM SPSS Statistics V.29.0.0.0 (Build 241). P values<0.05 were considered statistically significant.
Patient and public involvement
The local patient organisation for stroke victims (LHL) was involved in the design and conduct of the team simulation training intervention reported in this study. LHL was not involved in the design, conduct or reporting of this specific study.
Results
Patient inclusion
A total of 668 patients treated with IVT by 38 neurology registrars at Stavanger University hospital from January 2016 to January 2020 were included. Of these, 115 patients were excluded due to inhouse-stroke or MRI imaging prior to selection and 21 patients were excluded due to missing data, leaving 532 patients for further analysis. In the assessment of a potential association between experience and DNTs, the 148 patients treated before QI were included. A flow chart of included patients and different groups of patients used for the comparison of mean DNT can be found in figure 1. Baseline characteristics for all included patients are listed in table 1.
Clinical experience and DNT
The association between clinical experience and logDNT was deemed linear (test for non-linearity, p=0.479; figure 2). We thus proceeded with a mixed effects linear regression model. Unadjusted, we found that a one-unit increase in experience (ie, one additional prior IVT treatment) was on average associated with a statistically significant 1.3% decrease in DNT (ΔDNT −1.3%; 95% CI, −2.4% to −0.2%; p=0.023). After adjustment for patient age, hypertension, statin use and diabetes, the effect estimate was slightly lower (ΔDNT −1.1%; 95% CI, −2.2% to −0.0%; p=0.048). These results are summarised in table 2.
For dichotomised clinical experience, the unadjusted geometric mean DNT for patients treated by inexperienced registrars (ie, <15 prior IVT treatments) was 25.3 min (95% CI, 21.9 to 29.2) compared with 21.9 min (95% CI, 18.9 to 25.3) for experienced registrars.
The influence of clinical experience on the effects of simulation attendance
In a second mixed effects linear regression model, we found that for a registrar with median clinical experience (ie, 17 prior IVT treatments), simulation attendance was on average associated with a statistically significant 21.8% decrease in DNT (adjusted, ΔDNT −21.8%; 95% CI, −30.9% to −11.6%; p<0.001). For a registrar with experience corresponding to the first and third quartiles (8 and 28 prior IVT treatments, respectively), corresponding adjusted estimated effects of simulation training was ΔDNT −26.3% (p<0.001) and −16.0% (p=0.025). Model-based effects of simulation over the whole range of observed experience levels depicting both higher and lower levels of experience are plotted in online supplemental figure 1. The interaction between effects of any level of clinical experience and simulation training on DNT was not significant (p=0.150). When adjusting for patients’ age, atrial fibrillation, hypertension and statin use, the effect size was slightly smaller but following the same pattern (test of interaction, p=0.134).
Supplemental material
Correspondingly, we found smaller effects of clinical experience among those who had attended simulation training than those who had not (adjusted, −0.5% vs −1.2% per prior IVT treatment), but again, the differences in effects were not statistically significant (tests of interactions above). These results are summarised in table 3.
For dichotomised clinical experience, the unadjusted geometric mean DNT for patients treated by inexperienced registrars who had never attended simulation was 22.4 min (95% CI, 19.9 to 25.2) compared with 17.4 min (15.2 to 19.9) for those who had. Experienced registrars who had never attended simulation had a mean DNT of 18.5 min (16.2 to 21.1) compared with 13.5 min (11.9 to 15.2) for those who had. A summary of the means in the different groups according to both simulation attendance and clinical experience is found in figure 3.
Discussion
In this single-centre prospective cohort study, we found that inexperienced registrars had longer clinical DNT in stroke thrombolysis than their more experienced peers. Each additional prior IVT administration was associated with a significant 1.1% reduction in DNT. Registrars who had performed less than 15 prior IVT treatments had a mean DNT of 25.3 min compared with 21.9 min for their more experienced peers. The DNT reduction after attending the simulation was similar for experienced (18.5 to 13.5 min) and less experienced registrars (22.4 to 17.4 min). We did not find a significant interaction between the effects of clinical experience and simulation attendance on DNT (adjusted p=0.134). Experienced registrars thus have shorter DNTs but similar DNT reduction after attending simulation sessions compared with their less experienced peers.
Although we found evidence for an inverse association between a clinical experience (based on prior IVT administrations) and DNT in this study, in the broader literature the evidence from prior reviews on associations between clinical experience and healthcare quality is conflicting and depends on several factors including the medical field, the definition of clinical experience, onboarding procedures and the domain of quality being assessed.14 15 To our knowledge, the only study that assessed the relationship between DNT in stroke thrombolysis and clinical experience supports our findings.16 Another more recent study assessing door-to-balloon time in myocardial infarction also found that more experienced registrars had shorter treatment times.17 There are several mechanisms that potentially support a relationship between clinical experience and treatment times. Delays related to clinical decision-making may be shorter in more experienced registrars due to a higher proportion of fast ‘reflexive decisions’ as is suggested in prior reviews both for the broader medical field and specifically for stroke.18 19 Another qualitative study discussed several factors including anxiety, confidence and leadership skills as potential challenges for junior doctors handling critically ill patients.20 These challenges might be alleviated by clinical experience leading to faster decision-making. There is thus both plausible mechanisms and supporting data that experienced registrars achieve faster DNT in stroke thrombolysis than their less experienced peers. Our data does not provide insights for a particular mechanism underlying the difference.
We did not find a significant interaction between clinical experience and simulation attendance with regards to the effects on DNT. Our data therefore do not support the hypothesis of a significantly larger benefit from attending team simulation training for inexperienced neurology registrars compared with their more experienced peers. There might be a couple of reasons for this. First, although there is no significant interaction overall, there is a trend toward a weaker association between DNT reduction and simulation attendance for very experienced neurology registrars. In the lowest quartile of clinical experience, the benefit from attending simulation is numerically larger (−26.3% reduction in DNT with 8 prior IVT administrations) than for the highest quartile (−16.0% reduction in DNT with 28 prior IVT administrations). This trend can also be seen across the spectrum of clinical experience (online supplemental figure 1). Thus, there still might be aspects of the team simulation training that are more effective in inexperienced registrars (eg, increased familiarity with the real work environment and other members of the stroke team), while some aspects might be effective regardless of clinical experience (eg, system-level changes, culture and team coordination).21 Second, whether attending a simulation session is more effective for novices or experts depends in part on the design. According to the European Stroke Organisations standards of methodology for simulation, the design should optimally match both the level of the learner and the educational goal.22 Our findings might thus imply that aspects of the team simulation training design matched the level of the attending neurology registrar. There are several examples of team training interventions with design features similar to ours in the context of acute stroke.23 24 In this type of simulation, it might intuitively make sense to exclude experienced registrars on the assumption that only inexperienced registrars benefit from attending but our data does not provide firm support for this assumption. In conclusion, we found that experienced registrars have lower DNT than their less experienced peers but still have significant further DNT reductions after attending team simulation training.
Our study has several limitations. First, IVT treatment in acute ischaemic stroke is a team effort. Clinical experience in our study is only defined for the neurology registrar and any effects of clinical experience or lack thereof in other team members might have biased our results in a non-differential manner. Second, a physician’s clinical experience is a complex term that is not fully captured simply by measuring prior IVT administrations. What is gained (or lost) through the accumulation of experience could be everything from patient types, self-confidence, self-awareness or even cognitive abilities.25 Within the individual physician there could also be traits or individual experiences (eg, experienced complications of IVT) affecting decision-making skills and thus treatment times. Furthermore, age or experience from other medical fields were not included in our analysis potentially biasing our results in a non-differential manner. Third, due to the study design, any temporal trends in uncontrolled factors might have biased our results. Fourth, due to the design of the QI project, the effects of simulation attendance cannot reliably be separated from other interventions as part of the QI project. A proportion of patients treated by neurology registrars who had not yet attended simulation (144/266) were treated prior to concurrent efforts to streamline our treatment protocol as part of the QI project. This might have increased the overall effect size of attending simulation because ‘simulation attendance’ in part reflects the effects of concurrent procedural changes. However, you could also argue that the other part of the group of patients treated by registrars who had not yet attended simulation (122/266) were treated after the start of QI. These patients might not have represented a ‘true’ baseline for simulation attendance as these patients DNT was reduced by concurrent changes to streamline the protocol and other team members participation in simulation. This might have reduced the effect size of attending simulation and is probably the reason why even registrars that did not attend simulation had a relatively low predicted mean DNT (figure 3). Finally, our results are affected by contextual factors limiting generalisability. For example, our findings with regards to experience are influenced by local onboarding procedures and design features that might be unique to our simulation-programme. This is however somewhat alleviated in that the design of our team simulation training does not differ substantially from suggestions for simulation-based team training design in acute stroke by the European Stroke Organisation.7 22
Despite these limitations our findings might have important implications. Turnover of staff, including physicians, has been highlighted as an increasing problem.9 With a high turnover of physicians, a larger proportion of patients will be treated by less experienced registrars which in turn can negatively affect DNT. We know from several other studies that even a difference of minutes impacts patient outcomes in acute stroke care.5 26 Finding ways to compensate for the experience gained from treating real patients is important to close this quality gap. Furthermore, considering the effect sizes we reported, changes on a systemic level combined with simulation attendance produced gains that are larger than what is gained from clinical experience alone. Most neurology registrars gained on average around 5 min in DNT after attending one or more team simulation training sessions regardless of prior clinical experience (figure 3). This can be contrasted to the gains from real-life cases through the example of an inexperienced registrar with a mean DNT of 27 min (based on our pre-QI data): Each IVT administration for this registrar would reduce the following patients DNT by 1.1% corresponding to a very modest gain of 17.8 s. The role of simulation as a tool not only for novices but also as a tool for continuous lifelong education is highlighted in Gaba’s seminal paper about simulation in healthcare.11 Our data support the role of in situ team simulation training for stroke thrombolysis as a tool for improvement of both experienced and inexperienced neurology registrars.
Conclusion
We found that patients treated by inexperienced registrars had longer DNT in stroke thrombolysis than patients treated by their more experienced peers. Attending at least one simulation session was followed by similar improvements for experienced and inexperienced registrars. We suggest a focus on high-quality onboarding programmes to close the quality gap related to clinical experience. Team simulation training for stroke thrombolysis can be a useful tool both for purposes of onboarding and continuous learning.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study was performed with the approval of the regional ethics committee (Regional Etisk Komite), application number 2018/1895. All participating neurology registrars provided written consent for registration of de-identified information in a local registry and the analyses required for this work.
Acknowledgments
The authors would like to acknowledge Linn Haraldseid for acquisition and registration of data. The authors would also like to acknowledge the cooperation of the local patient organisation for stroke victims (LHL), all members of the stroke team in the study hospital and members of the quality improvement team for their engagement.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @SoffienA
Contributors SCA is the corresponding author and guarantor. SCA, HLE and MK contributed to the conceptualisation, methodology, drafting and critical revision of the manuscript. ID contributed to the formal analysis, data curation and critical revision of the manuscript. TWL contributed to the conceptualisation and critical revision of the manuscript.
Funding SCA is a research fellow funded by a Safer Healthcare Grant (University Research Fund, Number: N/A). Otherwise, the authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests SCA is a research fellow funded by a Safer Healthcare Grant (University Research Fund). The remaining authors report no disclosures.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.