Article Text
PDF
Statistics from Altmetric.com
- Learning reporting systems
- patient safety
- healthcare operators′
- opinions
- human error
- qualitative research
- safety culture
Incident-reporting systems (IRS) are tools that allow front-line healthcare workers to voluntarily report adverse events and near misses. An IRS achieves two primary purposes: (1) to provide individual workers with a systematic methodology to promote continuous experiential learning; and (2) to provide organisations with a record of incidents for clinical risk assessment and prevention.1 2 Even if reporting of adverse events is mandatory, it is critical that clinicians learn from the experience.
The WHO has clearly outlined the basic principles on how to design and implement successful IRS in healthcare organisations.3 A successful IRS is one in which the information it yields is used to improve patient safety.4
The following characteristics of a successful IRS are well established in published literature: wide consensus between professionals and managers on the need to collect data on existing problems, time spent to design and start-up the system, use of well-recognised experts to manage the system and continuous marketing activities to ‘sell’ the system to the users.1
Each one of these characteristics has the prerequisite of an active commitment of both head clinicians and top managers in patient safety. Also, leadership is crucial, in particular for those who are in charge of running the daily operations of the IRS because the IRS needs workers to provide honest reports. Finally, an understanding of the basics of the system's approach is an important requirement to spread the safety culture, where an incident report is considered a mode in which to learn from errors and for improving the organisation rather than a self-claim.12–15
After the incident is reported, organisations must put in place a quantitative and qualitative analysis of the report, then follow-up with an in-depth investigation into the incident and finally develop, apply and monitor an action plan to anticipate and control the assessed risks. For front-line workers, it is particularly useful to have bounce-back (automatic response produced by the software) and then a structured feedback after the report.5 In healthcare, considering the high level of expertise and specialisation of the workforce, involving those who reported in the investigation process can be a first step to fostering learning and improvement, with the condition that confidentiality and deidentification be granted in the final report of the investigation.3 5
After all, the effectiveness of an IRS is still questioned, mostly when an IRS has been developed and implemented with large investments, personal effort and political support.7–9
The most valuable uses for an IRS are the rapid identification of incidents at the front-line level and the communication of the event to the managerial level of the organisational, or to an external agency dedicated to the analysis of reports and eventually to the investigation of major events or clusters.
The use of an IRS in daily clinical practice in many countries around the world can now offer the possibility to reflect on some myths accompanying their development. The daily practice with an IRS and concurrent studies of the effectiveness of patient safety provide more evidence on their shortcomings rather than on their benefits.10 11 The main point is that each report is more like a short novel than a systematic summary of a clinical process with negative results. These results are more valuable than the details of the individual stories. According to these considerations, the difficulty in coming to a standard classification of patient safety incidents can be attributed to the nature of an adverse event that is unexpected and therefore difficult to categorise a priori. The IRS is based on voluntary reports as stated previously. As a result, the statistical elaboration of data cannot provide generalisations about results or the development of provisional models.1 8 10 Studies on the healthcare workers' opinions and perceptions on IRS are needed.
In this paper, the authors present results of a survey conducted in 2006 and 2007 that examined healthcare workers' opinions and perceptions of patient safety incidents and the use of IRS in Italian hospitals. In hospitals without IRS, healthcare workers' opinions on patient safety incidents and their expectations for an IRS were assessed. In settings where an IRS had been in use for at least 1 year, workers' opinions on patient safety incidents, expectations for IRS and the effects of an IRS on patient safety were sought. Analysis of the differences between these two settings can reveal ways to improve and promote the use of IRS to improve patient safety in Italian Hospitals.
Methods
Sample
This research is part of a national project on clinical risk management (http://www.agenas.it/monitor2007.html). Hospitals participating in the national project were invited to participate in this study. Of those invited, four hospitals with an IRS (defined below) and 14 hospitals without an IRS elected to participate (see table 1). These hospitals are located in seven regions of Italy. Within each hospital, a representative sample of healthcare workers were selected to participate in the survey. The surveys were conducted between April/May 2006 and January 2007.
Sample
To be defined as having an IRS, a hospital must have met the following criteria: (1) the IRS must have been in use for at least 1 year; (2) formal training of healthcare workers on the basics of patient safety and incident reporting must have preceded IRS implementation; (3) near misses and unsafe acts must be reported; (4) reporting must be anonymous or made anonymous by the system; and (5) data from incident reports must be used to analyse contributing factors and to plan and implement corrective actions in the short and long term.
Survey participants were selected through a sampling procedure proportional to the size of the organisational unit (the hospital) they belonged to and to the professions that characterised the organisation. The professions were divided into two main categories: physicians and other healthcare workers (nurses and technicians). The size of the two samples (hospitals with and without IRS) was calculated with a random sampling procedure, without repetition, applied to the two different settings (margin of error 0.025). The sample for each setting was then subdivided in two ways: (1) proportionally respect to the dimension of each hospital participating in the survey, and (2) in each hospital proportionally with respect to each profession. The sample size required for the study in hospitals without IRS and in hospitals with IRS was 503 and 439 healthcare workers, respectively.
A list of reserves composed of 30% of the defined sample size was elaborated in order to integrate the sample in case of a high rate of non-responders (higher than 20%).
The specific dimension of each hospital and professional category is described in table 1.
Survey design
Two questionnaires were designed to evaluate workers' opinions and perceptions of patient safety incidents and their expectations for incident reporting and learning systems, and to explore differences in opinion between workers in hospitals with IRS and hospitals without IRS. The surveys were modelled after items in AHRQ's Patient Safety Culture Survey16 and the Safety Attitude Questionnaire17 18 (previously known as IHI Patient Safety Climate toolkit). The two questionnaires differed from each other only by two items (contact the lead author for more information about the two questionnaires). Workers in hospitals with IRS received one version, and workers in hospitals without IRS received the other.
Survey collection
The written surveys were administered in person during scheduled sessions. The surveys were administered according to an ‘assisted self-assessment’ technique, in which respondents filled in the questionnaire independently in the presence of a researcher who could provide further explanation of the questions, if needed.
Statistical analysis
Descriptive statistics were provided for each of the two samples (with and without IRS). In each sample, respondents' gender, profession and clinical specialty were described as a percentage the group. Professions were categorised as either physicians or other healthcare workers, which included nurses and technicians. The clinical specialties category was composed of individuals from the following fields: surgery, medicine, obstetrics and genecology, intensive care, radiology, laboratory and rehabilitation.18 The results of the survey were organised into two parts according to the two different investigated issues: ‘personal experience on patient safety issues’ and ‘opinion of incident reporting and learning systems.’
Differences between sample groups in the response of individual items and the difference within the two parts of the questionnaire were tested through the use of the χ2 test and the Fisher exact test. The possible confounding effects of professional type and clinical specialties was evaluated using the Cochran–Mantel–Haenszel (CMH) test. All statistical analyses were conducted using the software SAS System for Windows (V8; SAS Institute, Cary, North Carolina).
Results
Surveys were completed by 820 (87.0%) of 942 eligible workers, with 466 respondents working in hospitals without IRS and 374 working in hospitals with IRS (table 2). The response rate of workers in hospitals without IRS was slightly higher than that of workers in hospitals with IRS (92.6% vs 85.2%). In hospitals with IRS, 30.5% of workers were physicians, while 69.1% were nurses and technicians, in hospitals without an IRS, 26% were physicians, and 73.8% were other healthcare workers. In both settings, the specialties with the most sampled individuals were surgery and medicine, which is in line with the larger presence of these specialties in most hospitals. The responders were predominantly women in both settings.
Characteristics of the 820 respondents*
Personal experience with patient safety issue
There were statistically significant differences between the two settings around workers' personal experiences in clinical risk management (CRM). Workers were asked to reveal the methods by which they gained exposure to CRM (table 3). Workers in hospitals with IRS were more likely to report reading material in their unit and learning practices from colleagues, and to participate in clinical audits and meetings for CRM than workers in hospitals without IRS.
Response rate and rate of affirmative responses for questions relating to the personal experience on clinical risk management (question: Please, answer the following statement on the basis of your personal experience related to the clinical risk management)
Healthcare workers involved in incident investigation (formal in-depth analysis of the event) were a minority in both settings, 35.2% for hospitals with IRS and 40% hospitals without IRS. No significant differences were found between hospitals adjusting for professional profiles and clinical specialty categories (CMH=1.46, p=0.23).
In both settings, almost 90% of the workers who participated in a formal incident investigation agreed on the effectiveness of this activity with respect to a positive change on patient safety.
In both settings, 10% of interviewees had never been informed of a patient safety incident that occurred to a colleague, and the other 90% declared that they received information about an incident which occurred to a colleague at least once in their careers.
The way in which healthcare workers received communication on these patient safety incidents is described in table 4. In general, communication happened mainly during daily informal discussions with colleagues. This is more evident in hospitals with an IRS (74.3 vs 68.7).
Percentage of response on the communication flow of patient safety incidents (question: For the most part, in what manner do you learn about adverse events?)
With respect to the personal experience of either one or more patient safety incidents, 30.8% and 36.7% of the healthcare workers in hospitals with and without an IRS declared having not experienced a patient safety incident during their individual careers, and more than 40% of the participants of the survey in both settings reported having experienced between one and three patient safety incidents during their career.
Among healthcare workers who declared having not reported an adverse event, there was no significant difference between settings in the reasons as to why they did not. Nevertheless, in hospitals without an IRS, healthcare workers more frequently did not report patient safety incidents through a formal report because they were afraid of mistrust from colleagues and because they were sceptical towards any possible improvement after the report would have been filed (table 5).
Percentage of response on barrier to reporting in case of never reporting of adverse events (question: If you did not report an adverse event, what was the reason?)
In both settings, we found that the most frequent response was ‘none of the above.’ We then tried to understand if this kind of answer was due to a bias in the design of the questionnaires or in the characteristics of the responders. Data analysis showed that more than 50% of the interviewees who either did not answer or answered ‘none of the above’ also declared not to have experienced any patient safety incident. On the other hand, those who identified any of the proposed barriers to reporting had previously experienced at least one patient safety incident.
Opinion on incident reporting and learning systems
Healthcare workers differed on who should be the recipient of the formal report of an adverse event, either between the two settings (table 6) and among different professional figures (figure 1). There was a significant difference in the way of communicating adverse events between the two settings (p=0.001). In particular, in hospitals with an IRS system, the horizontal communication prevailed even for formal reports of patient safety incidents (38.6 vs 32.7), while in settings without an IRS the vertical communication was preferred (38.8 vs 26.0).
Percentage of response on the first recipient of the formal report of an adverse event (question: If you report an adverse event, who would you like to talk to first?)
Percentage of response on who should be the first recipient of the formal report of an adverse event for professional figures.
According to the general opinion of the interviewed healthcare workers, fear of punishment as a result of reporting a patient safety incident did not seem to be a barrier for using an incident reporting system because more than 50% of the total sample said that was ‘never or hardly ever a problem’ (CMH=3.91, p=0.27; data adjusted for the different professions). Quite interestingly, this answer was the same in hospitals with or without an IRS, while we expected to find more fear for a blame culture in settings where clinical risk management had not yet been formally implemented in daily practice.
More than half of the total healthcare workers involved in the survey, 60.44% in hospitals without IRS system and 59.29% hospitals with an IRS system still believed that an IRS can be used to assess professional responsibility and to blame individuals because of eventual errors. This was true both for physicians and for the other healthcare professionals. Despite this perceived inappropriate use of an incident report, more than 60% of the interviewees believed this system could help to reduce patient safety incidents, and it was not a fruitless effort. No difference between settings was found after adjusting for professional profiles.
The main factors contributing to the patient safety incidents were workload, stress, fatigue and organisation of work. Communication was not yet perceived as a major contributing factor, despite the evidence shown in published literature.5 14
Discussion
The statistical analysis did not show any important differences between hospitals with or without an IRS. In particular, in hospitals with an IRS, participation in clinical audits and meetings was higher.
Thus, the opinions and expectations on an IRS were very similar for healthcare workers, regardless of whether or not they belonged to a hospital with or without an IRS, or whether or not they belonged to a professional profile or a clinical specialty. There were probably other factors we did not explore in the survey that might have affected the development of the overall safety culture within a hospital and a professional community. One of these factors might be a general increase in the interest among clinicians in clinical risk management in the last decade. Indeed, incident reporting and reflection on patient safety incidents were only a fraction of the activities for patient safety at the hospital level.
In any case, from this survey, we can make some general consideration about IRS:
Experience of patient safety incidents and expectation of IRS. Regardless of whether a hospital had an IRS, workers reported similar rates of involvement in patient safety incidents. Seventy per cent of the respondents, in both settings (with or without an IRS) affirmed that they were involved in an incident. Of those with an available IRS, only 40% formally reported an event. This confirmed the presence, in general, of different reporting barriers in the IRS systems.15 19–22
Professional community and an IRS. The communication of risk remained in the community to which the professional belonged. The data showed that information on patient safety incidents was not expanded to the entire organisation. The frequency with which people knew about an incident was very rare (never or almost never). Informal communication of patient safety incidents was more integrated in daily practice, and horizontal communication seems to prevail. The perception of risks stayed local and very rarely (only when disasters occurs) became part of the organisational memory.
Value of an IRS. In both settings, workers expressed agreement that an IRS is useful for patient safety. Additionally, we found a general agreement on the effectiveness of patient safety incident investigation after formally reporting an IRS, particularly when front-line clinicians were involved in the analysis and reflection during formal clinical audits or mortality and morbidity meetings.
Limits of the study
As with any retrospective survey, one must take a respondent's recall bias into consideration when evaluating the study's reliability. Respondents were asked to recall incidents throughout the life of their careers. Additionally, the experience and consequences negative experiences can modify the memory of the event itself and make it more difficult for the individual to give an accurate depiction of the number of incident experiences. This is especially true when incidents are related to self-esteem and social trust.
Differences between professions should also be better explored in the settings of the survey, because we did not find any significant differences, even though we know from other studies that physicians and nurses usually show different behaviours towards an IRS.
The longevity of an IRS has not been considered in this survey. In fact, hospitals with an IRS had one requirement: to have had the system in place for at least 1 year. Finally, an IRS is quite a novel idea in Italy. Therefore, in the future, we need to better explore the relationships between healthcare workers' opinions of IRS, the number of IRS and the outcomes in terms of quality and safety of the hospitals with and without an IRS.
Conclusions
These research findings are in agreement with existing research on healthcare workers' experiences of patient safety incidents, as well as their positive opinions and expectations of an IRS, even though there are still barriers to formal reporting using the IRS.23 24 26 We shall consider developing further studies and interventions to appraise safety culture, in order to have more effective reports and occasions for learning from errors.24 25 After all, at the time of this survey, an IRS in Italy was still a fresh innovation, and as a result, it was difficult to provide an in-depth analysis on its impact in the world of patient safety. We can conclude that this research, as part of a wider national project aimed at spreading methods and tools for patient safety, has contributed to developing IRS in Italy, considering the sociocultural context of Italian hospitals.
Acknowledgments
The authors would like to thank all the healthcare operators who accepted to participate in the research, the Italian Ministry of Health and Gutenberg Sicurezza in Sanità, which funded this project. The research in all its phases (planning, conducting and reporting the work) has been realized with the contributorship of Agenas and all the members of the group of the research project: A dell'Erba, University of Bari, Bari; L Sommella, S.Filippo Neri Hospital, Rome; G Luongo, Regional Agency for Healthcare of Campania; G Simon, Regional Agency for Healthcare of Friuli Venezia Giulia; R Russo, University of Turin, Turin; M Bosio, Lecco Hospital, Lecco; AM Longo, Regional Agency for Healthcare of Rovigo.
References
Footnotes
Funding The Italian Ministry of Health and Gutenberg Sicurezza in Sanità.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Introduction