Article Text
PDF
Abstract
Objectives In an effort to improve patient safety attitudes and skills among third-year medical students, two patient safety training sessions were added to their curriculum, complementing a previously implemented second-year curriculum on quality improvement, patient safety and teamwork.
Methods Safety attitudes and skills were assessed before and after students completed the medicine clerkship training and were compared with historical controls. Students identified and reported on observed safety events, with their reports matched to event type and harm score with contemporaneous safety reports from University of Missouri's Patient Safety Network (PSN). Comparisons were assessed by five internal safety experts using criteria for report submission “worthiness”, blame tone, target of blame and presence/strength of proposed solutions.
Results Students completing the third-year safety booster conferences expressed statistically higher comfort levels with identifying the cause of an error than did the student control group (p<0.05). Medical students proposed safety interventions that were more robust than those suggested by event reporters regarding similar events within our health system (p<0.0001). The worthiness and blame tone of medical student reports were not statistically different than event reports in PSN.
Conclusions Completion of two 1-h patient safety booster conferences in the third year of medical school led to increased student comfort in safety event analysis. Students documented stronger resolution robustness scores, suggesting similar training should be offered to PSN reporters. Medical students represent an underutilised resource for identifying and proposing solutions for patient safety issues.
- Patient safety
- adverse event
- medical education
- health professions education
- human factors
Statistics from Altmetric.com
Introduction
In Crossing the Quality Chasm, the Institute of Medicine encouraged health professional educators to embrace new approaches to teach patient safety skills.1 Despite the central role of physicians in patient safety improvements,2 significant patient safety knowledge and skill deficiencies exist in medical students and residents.3–5 Improving the patient safety attitudes and skills of physicians requires a systematic approach that encompasses curricular innovation, student assessment and attention to organisational culture.6 7
Responding to these challenges, the University of Missouri-Columbia (MU) initiated a patient safety curriculum for second-year medical students in 2003. This training introduced students to 8 h of instruction on patient safety fundamentals: human fallibility, causes of error, search for root causes, importance of blame-free investigations and proposing safety interventions. Learner attitudes measured immediately before and after curricular completion showed significant improvement in several safety attitudes; however, retesting of medical students one year later revealed a failure to sustain attitudinal improvements as well as a decline in comfort in error analysis.8 Given these results, additional curricular experiences were planned for introduction into the third-year medical student curriculum.
We postulated that third-year medical students would successfully identify and analyse significant patient safety concerns in clinical care settings and would propose effective safety improvements, if provided with a structured process and safe environment for this work. Edgar Schien has noted that professionals in training, in addition to mastering underlying basic science, must learn to apply this knowledge in day-to-day situations, with the skills and attitudes appropriate for the context.9 By providing experiential learning opportunities around real life patient safety incidents, we thought students would continue building their patient safety expertise.
Methods
Approval for studying the impact of third-year patient safety conferences was obtained from the Institutional Review Board of the University of Missouri Health Sciences Center.
Setting
The University of Missouri-Columbia, Department of Internal Medicine, is located within an academic health centre on the campus of the University of Missouri-Columbia. University of Missouri Health Care (UMHC) is comprised of hospitals and clinics that are owned by the University of Missouri. Within UMHC, the Office of Clinical Effectiveness (OCE) is staffed by patient safety and improvement experts. During the medicine clerkship, third-year medical students rotate through inpatient medicine units at both the 274-bed University of Missouri Hospital and the 118-bed Harry S Truman Veterans' Administration Hospital. Both hospitals maintain full-time patient safety staffs, have mature event reporting systems, conduct root cause analyses and emphasise learning from safety events. Within the Internal Medicine Department, a masters-prepared nurse with extensive patient safety experience facilitated a partnership with the OCE to promote quality and safety education.
Description of intervention
Beginning in January 2005, two mandatory 1-h patient safety “booster” conferences were introduced for third-year students during the 8-week internal medicine clerkship. Student participation was verified by sign-in sheets.
In the first one-hour conference, faculty reviewed two skills-based patient safety tools. The first tool, the root cause analysis (RCA) process, articulated a structured process for systematic event investigation that focuses on system complexities. For the medical students, we emphasised the questions: “what happened”, “why did it happen”, and “what could be done to keep it from happening again,”10 using a simplified RCA process successfully employed with several residency programmes in the United States.11 The second safety tool, designed by Gosbee as an adaptation of Reason's “Swiss cheese” model (figure 1), reflects the relative strengths of safety interventions intended to prevent health care errors, based upon safety and human factors engineering principles, as well as empirical evidence in healthcare settings.12 Simply changing a policy might be metaphorically linked to the strength of mosquito netting; by comparison, redesign of equipment to “force” safe behaviour is represented by a steel barrier, indicating that it would be much more likely to prevent future harm.13 Changes in architecture and work area design deficiencies are increasingly recognised as lasting solutions,14 15 as studies show that familiarity and training cannot easily overcome device design deficiencies.16 Automation and redundancy may reduce adverse events, but fall into an intermediate category due to limitations in understanding how they interact with complex processes.17 Changes in policy that require behaviour and work pattern changes can decrease efficiency—although have some value if carefully applied.18 Similarly, adding a warning label or sign has limited effectiveness as a sole intervention.19
Gosbee's adaptation of Reason's Swiss cheese model with robustness scoring instructions and examples.
In the first conference, faculty led students through an RCA brainstorming exercise of an adverse event. At the end of session one, students were given-an assignment to identify and summarise an actual patient safety event or concern using our structured RCA tool (figure 2). During the second session, several students described actual patient safety events, speculated on root causes and proposed system modifications to improve patient safety. Internal medicine faculty and patient safety experts facilitated these discussions, emphasising safety principles, reflecting on current system complexity and commenting on real-world practicality of their proposed solutions. Students were encouraged to report significant safety concerns through the hospital safety reporting system. A few cases were forwarded to appropriate institutional leaders for follow-up. A file of student cases was maintained in the Internal Medicine Department.
Case submitted by third-year medical student after first of two safety classes.
Description of analysis
The effect of this curriculum on safety skills was evaluated using two methods. Since 2003, medical students have been surveyed on patient safety attitudes and self-reported safety skills immediately before and after the M2 curriculum and at the end of the M3 year.8 Students completing the additional M3 booster conferences (January 2005 to June 2006) were compared with students who were not offered the M3 booster (July 2003 to December 2004) by calculating the difference in scores between matched pairs. For both the intervention and the historical control groups, differences were calculated by comparing survey results from the end of third year to results from the end of second year. The Cochrane–Mantel–Haenszel analysis was used to compare responses on student surveys. Adjustment for multiple testing was accomplished using the false discovery rate method of Benjamini and Hochberg.20
To determine if the booster conferences effectively prepared students to contribute to safety investigations and improvement plans, student-submitted reports were compared with contemporaneous reports from the UMHC patient safety reporting system. UMHC uses the “Patient Safety Network” (PSN), an internally developed, secure, web-based voluntary reporting system.21 As with the student reports above, PSN reporting screens prompt the reporter to suggest possible interventions to improve system safety. Five physician and nurse safety experts analysed and compared student-generated reports against matched PSN entries. All reports were de-identified in terms of names, dates, reporter origin (medical student vs PSN) and locations. Student reports were matched with randomly selected PSN reports of similar event types and harm scores generated during the same period. Independent review was followed by team consensus meetings regarding “worthiness” of report entry (ie, did this safety concern merit reporting?), degree of harm, event type, narrative blame tone, the target of blame and the estimated robustness of proposed interventions. When multiple solutions were proposed, the highest scored solution was recorded. The consensus data were used to compare the two reporter groups (medical students and PSN reporters) on three dichotomous responses (worthy of PSN reporting, presence of blame tone, presence of proposed solution) and one ordinal response (robustness of the proposed solution). The robustness score was assigned using the scale proposed by Gosbee, ranging from a score of 0 for no significant system intervention suggested to a score of 5 for a recommended equipment modification.12 A two-sided Fisher exact test was used to test the differences in PSN worthy reports and in the proposed solution. A χ2 test was used to test the difference in blame tone. A Cochran–Armitage trend exact test was used to quantify differences in solution robustness. A nominal significance level was set as 0.05 for all tests and analyses were conducted using SAS V.9.1 software.
Results
Safety attitudes and skill survey
On the patient safety attitudes and self-reported safety skills survey, 146 student survey pairs were analysed; 81 student survey pairs were in the control group with 65 pairs in the intervention group. At baseline, no differences in any of the patient safety attitudes or self-reported safety skills between the two student groups were noted. Students who completed the booster expressed a significantly higher comfort level in identifying the cause for an error. Specifically, mean scores increased in the booster group from 3.66 to 3.72, whereas scores in the control group declined from 3.58 to 3.27 over 1 year (p<0.05).
Student and PSN reporter description of events
Students completing the safety booster conferences submitted descriptions of 57 events during the 18 months. Two were illegible, thus excluded from the analysis. The remaining 55 events were prepared for the consensus team. Student-generated cases were matched on PSN event type, but when matching on harm score, there were three student-generated reports within one event type (adverse medication reactions) with a harm score of 0 and only one PSN entry with a harm score of 0. Thus, we “allowed” random selection by matching two additional PSN entries for this event type with a harm score of 1 (table 1).
Count of 55 matched cases (student and reporters) by harm score and event type
Following the consensus team analysis, there was no significant difference in PSN worthy reports (p=0.36) or in blame tone (p=0.12) between the two reporter groups (table 2).
Reporter group comparison of PSN worthiness and blame tone
Among M3 medical students who blamed, 50% blamed physicians and 33% blamed nurses. Among PSN reporters who blamed, 32%, 40% and 11% blamed physicians, nurses and unit clerks, respectively.
Student and PSN reporter solutions
All of the M3 students proposed solutions while PSN reporters only proposed solutions 76% of the time (95% confidence interval 65% to 87%). Further, a significant difference in the robustness of proposed solutions was found with median robustness scores of three for medical students compared with zero for PSN reporters (p<0.0001). The proposal of more robust solutions by the medical students was confirmed by Somers' D test statistic (p<0.0001; figure 3).
Reporter group comparison of events containing proposed solutions and solution robustness.
Discussion
Introduction of patient safety conferences for third-year medical students increased the students' comfort level with analysing causes of medical errors. This intervention reversed a previously described trend among third-year students at our institution to report lower comfort levels with safety reporting one year after completing a second-year patient safety curriculum.8 Furthermore, third-year medical students proved to be an untapped resource for identifying important patient safety issues in clinical areas. Not only did students consistently identify reportable patient safety issues within our care system; they also proposed safety interventions that were more robust than those proposed by the reporting staff of our hospital system.
We believe that few of these reports of safety concerns would have been brought forward without providing a structured forum for discussion of these issues in a trusted and collegial environment. A query of the PSN for reports submitted by “medical students” during the 18-month interval represented in this study revealed that only 0.1% of reports were submitted by medical students.
Little has been published evaluating medical student knowledge, attitudinal, or behavioural changes following patient safety curricular modifications. A 4-h patient safety/medical error curriculum for third-year medical students at New York Medical College increased perceived skills for medical error disclosure and increased awareness of the incidence of medical errors.22 A 5-hour patient safety training experience in the final year of the medical curriculum at Aberdeen Royal Infirmary led to improved student perception of medical knowledge. Their perceived control of events pertaining to patient safety was also improved 1 year post-training.23 At Thomas Jefferson Medical College, a one day third-year medical student seminar highlighting medical errors, event disclosure and professional liability improved students' attitudes and beliefs about patient safety. However, students told the study authors that they were uncertain about what defined an error and due to their limited clinical exposure, they often could not differentiate errors from other activity.24 While not associated with a specific curricular change, one report described four cases in which harm was, or could have been, averted with medical student intervention. These authors concluded that medical students are an overlooked, thus underused resource for detecting and preventing errors.25
Others have noted the reluctance of medical trainees to report safety concerns through formal reporting systems. In one study, reasons identified for not reporting included lack of time, extra paperwork and concern about personal reputation.26 In another report, medical learners requested that institutions develop arenas in which open discussions of errors can occur without fear of retribution.27 Even while appreciating that the third-year patient safety conference may constitute a “safe” environment in which to discuss medical error, we continue to encourage the use of patient safety reporting systems.
Third-year medical students demonstrated equivalent proficiency as those within our health care system in identifying appropriate (PSN-worthy) safety events of all types and harm scores. Medical students proposed more and stronger solutions than those of typical PSN reporters. However, since the medical students were completing this exercise as a required assignment, this may have prompted more proposed solutions than were found in the PSN system, where proposing a solution is an optional field.
UMHC's health system patient safety leaders have recently increased training opportunities for front-line staff to improve the robustness of proposed resolutions while minimising blame. More systematic efforts to follow-up on significant safety concerns raised by students, while maintaining student confidentiality, are ongoing.
Several study limitations were noted. The intervention student group, compared with the control group, contained more students who were in the second half of their third year. Although this might impact the comfort level with reporting safety events, previous work suggests that our students have typically not been comfortable with this skill at the end of their third year of medical school.8 While our consensus panel used the Gosbee tool for robustness scores, verification of the validity of tools for quantifying resolution robustness is imperative. Our results might not be replicable in a setting without early curricular exposure to patient safety fundamentals. Finally, the presence of strong faculty role models and collaboration between the health system and the Internal Medicine Department were deemed critical success factors in this initiative.
Conclusions
Completion of two patient safety booster conferences during the third-year internal medicine clerkship resulted in students reporting higher comfort levels with patient safety event analysis than did previous third-year students who had not been offered this training. Student safety reports described events that were worthy of reporting in a patient safety system and their proposed solutions were more robust than those of typical reporters. Further investigation is needed to identify more effective strategies to encourage students to consistently report safety hazards within hospital reporting systems and to engage in ongoing redesign of health systems to provide for safer care.
References
Footnotes
Competing interests None.
Ethics approval Although this was not a formal trial, because students were tested for safety attitudes and skills, ethics approval was sought and obtained (deemed “exempt” due to educational intervention) from the University of Missouri Health Sciences Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.