Article Text
PDF
Abstract
Objective Numerous studies have shown that a substantial number of patients suffer from adverse events (AEs) as a result of hospital care. However, specific data on AEs in acute cardiac care are scarce. The current manuscript describes the development and validation of a specific instrument to evaluate patient safety of a predefined care track for patients with acute myocardial infarction (AMI).
Design Retrospective patient record review study.
Setting and participants A total of 879 hospital admissions treated in a tertiary care centre for an AMI (age 64±12 years, 71% male).
Main outcome measure In the first phase, the medical records of patients with AMI warranting coronary angiography or coronary intervention were analysed for process deviations. In the second phase, the medical records of these patients were checked for any harm that had occurred which was caused by the healthcare provider or the healthcare organisation (AE) and whether the harm that occurred was preventable.
Results Of all 879 patients included in the analysis, 40% (n=354) had 1 or more process deviation. Of these 354 patients, 116 (33%) had an AE. Patients with AE experienced more process deviations compared with patients without AE (2±1.7 vs 1.5±0.9 process deviations per patient, p=0.005). Inter-rater reliability in assessing a causal relation of healthcare with the origin of an AE showed a κ of 0.67 (95% CI 0.51 to 0.83).
Conclusions This study shows that it is possible to develop a reliable method, which can objectively assess process deviations and the occurrence of AEs in a specified population. This method could be a starting point for developing an electronic tracking system for continuous monitoring in strictly predefined care tracks.
- Patient Safety
- Medical Error
- Patient Harm
- Process Assessment
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This study describes the development of a new valid instrument to objectively assess and monitor the occurrence of process deviations and adverse events in patients with acute myocardial infarction treated in a predefined care track.
This study focuses on identifying care track-specific process deviations, assessing the type and preventability of adverse events and the correlation between process deviations and adverse events.
This specific instrument can be used as a template for developing a quality instrument to objectify patient safety in a predefined care track.
This specific instrument has a substantial inter-rater reliability and is tested using a large number of patient records.
Limitations include the limited availability of information sources (ie, medical and nursing records only) and the absence of information regarding adverse events after discharge.
Introduction
Patient safety, defined by the National Academy of Medicine, formerly called the American Institute of Medicine, as the prevention of harm to patients, is the minimum pre-requisite for a good quality of care.1 In 1999, they published a report called ‘To Err is Human’, which drew attention to the fact that a significant number of patients suffered from injuries or even had died as a result of care delivered in hospitals.2 Subsequently, various studies in different countries reported that 2.9–16.6% of in-hospital patients experienced one or more adverse events (AEs) and that in 5–13% of the AEs the patients died.3–13 In these studies, an AE was defined as an unintended injury that results in disability at the time of discharge, death or prolonged hospital stay and is caused by healthcare management rather than by the patient's underlying disease process.
In various studies, a large variation in the incidence of AEs among the different hospital departments was shown (0.5–29.9%).4 ,14–16 However, these studies had a general hospital-wide approach and provided hardly any insight into causal relations on a departmental level. To develop specific interventions at each department in order to improve patient safety appeared even more challenging, particularly cardiology, which is a department with a high intervention rate and a large number of patients with life-threatening illnesses. A subset of studies contain results on the occurrence of AE among general cardiac patients, based on small numbers of patients and showing a substantial variety in the incidence of AE (13.3–29.9%).15 ,16 Therefore, sufficiently powered studies are needed for specific patient groups to gain more insight into the incidence of (preventable) AEs and to define ‘how safe is our care’.
Inspired by high-risk industries and best-practice hospitals, the aim of this study is to provide a system to review our work and to define if our work process is sufficiently safe. High-risk industries, such as the aviation and chemical industry, are required to perform structured assessments of all processes that contribute to a particular activity, which allows them to make a reasoned claim regarding safety. In the healthcare sector, best-practice hospitals such as the Intermountain Healthcare Group provide a framework such as the Quality and Patient Safety Plan on which an integrated and comprehensive programme to monitor, assess and improve the quality and safety of patient care is delivered.17 This study attempts to analyse process deviations and potential correlation with AEs in hospitalised patients who are treated according to a protocolised care pathway, in this case patients who suffer from an acute myocardial infarction (AMI). The method developed to assess AEs in patients with AMI is based on the Harvard Medical Practice Study (HMPS), which is a structured patient record review that has also been used in other AE studies.4 ,16 ,18 ,19 This manuscript describes how this commonly used method is adapted for our specific patient population. In addition, it was examined whether the linkage between process deviations and AEs will increase the uniformity of the assessments of AEs and creates the possibility to develop better improvement strategies.
Methods
Patient population
Patients who were admitted in 2012 and 2013 to the Leiden University Medical Center (LUMC) with an AMI warranting coronary angiography (CAG) or percutaneous coronary intervention (PCI) and treated according to the MISSION!-protocol were included.20 The LUMC functions as a tertiary referral centre performing PCI procedures on a 24/7 basis and serves an area of ∼750 000 inhabitants. The MISSION!-protocol contains a prehospital, in-hospital and outpatient framework for clinical decision-making and treatment for the different diagnosis in AMIs (unstable angina (UA), ST segment elevation myocardial infarction (STEMI) and non-STEMI (NSTEMI)). This study focused on the in-hospital programme (early reperfusion, same diagnostic trajectory like two-dimensional-echocardiography, structured medical therapy and disease education). Generally, patients are planned for discharge 12 hours after a CAG, or 48 hours after a PCI. The MISSION!-protocol is based on the evolving guidelines of the European Society of Cardiology.20–22 The MISSION!-patient records were extracted from the electronic patient file system (EPD-Vision, LUMC, Leiden, The Netherlands) by selecting the diagnose coding of a diagnosis–treatment combination for UA (11.203), STEMI (11.204) and NSTEMI (11.205). Patients with any of these three diagnosis codes were linked with the clinical database to select the patients who received an intervention procedure (CAG or PCI) within 24 hours after admission. This was applied to all patients admitted to the LUMC between 1 January 2012 and 31 December 2013. Patients with an urgent coronary artery bypass grafting after CAG were excluded because they underwent a different treatment path.
Review process
The method used in this study was based on a protocol originally developed by the HMPS. A modified version of this protocol was used in studies in Australia, Canada, Denmark, France, New Zealand, the UK and the USA.3 ,5 ,10 ,11 ,13 ,14 A Dutch protocol, based on the Canadian Adverse Events study, was used in studies in 2004, 2008 and 2011/2012.15 ,16 ,19 To identify high-risk patient records, the HMPS developed 18 triggers (ie, unplanned return to the operating room or hospital-incurred patient injury). The presence of one or more of these 18 triggers was established in phase 1. In case a trigger was found, the patient record entered a second phase, which focused on identifying whether harm was done to the patient, whether the harm was due to the care that the patient received, and whether the harm was preventable. To increase the uniformity of the assessment of AE and to gain more insight into patterns of AEs, the triggers of the HMPS were specified for acute cardiac care, thereby creating the opportunity to identify specific process deviations. In the first phase of the review process, we focused on identifying these process deviations in the patient records, and in the second phase AEs were identified. In case an AE was identified, it was also scored on preventability.3
Phase 1: specification of process deviations for AEs
In the MISSION!-protocol, the patient can have different workflows according to their electrocardiographic diagnosis at admission (STEMI, NSTEMI or UA). In the search for AEs, all process deviations from the MISSION!-protocol were identified. During this review phase, a process deviation was defined as every operation or treatment that differed from the MISSION!-protocol, such as additional procedures (a pacemaker implantation or second PCI), prescription of extra medication other than described in the protocol (use of antiaritmica, anticoagulation, inotropics or diuretics) or omission of a procedure (no diagnostics performed). If a patient had a transient heart rhythm disorder without treatment consequences, the process deviation was noted as an observation of the heart rhythm. Figure 1 shows all the defined process deviations. The review process of all medical records (nursing and medical records) was performed by a physician with work experience at the clinical department and who is familiar with reviewing electronic files. In contrast to the original triggers of the HMPS and other process deviation frameworks like the Process Deviations Analysis Framework, no judgements were made during phase 1 on whether the process deviation was ‘unexpected’ or ‘unplanned’.23 After identifying all process deviations, the process deviations were categorised into main categories and translated back to one (or more) of the original 18 triggers of the HMPS. Owing to the defined inclusion criteria stated in the MISSION!-protocol and restrictions concerning the availability of data from the patient records in the peripheral hospitals up to 30 days after discharge, it was decided beforehand that events experienced prior to (ie, unplanned admission before index admission), or after the index admission (ie, readmission) were excluded from the review process.
Newly defined process deviations in relation to the original Harvard Medical Practice Study.
Phase 2: determination of an AE and preventability
During phase 2, the nursing and medical records of the admission were reviewed for AEs by a clinical physician. If applicable, the records of patients who were transferred to another hospital during their admission were also traced. In cases where the physician found more than one AE in a patient, they were separately registered.
Event classification
First, it was assessed whether the event resulted in harm to the patient. The definitions are mentioned in online supplementary file 1. If the patient experienced harm that resulted in any disadvantage for the patient, such as prolonged admission, temporary or permanent (physical and/or mental) impairment or death, it was rated whether the harm was caused by healthcare (ie, an AE) and, if so, whether it was preventable (ie, caused by an error). Both the causation and the preventability were scored on a six-point Likert scale. Preventability of the AE was assessed by indicating a score between 4 and 6 on the Likert scale. This is in accordance with other AE studies (see online supplementary files 2 and 3).
supplementary files
To support the reviewer in the rather implicit judgement of determining the causation in healthcare of the AE, the causation score was preceded by structuring questions to direct if the injury was indeed caused by medical care rather than the underlying acute coronary syndrome (see online supplementary files 2 and 3). For example: ‘Does the timing of the (adverse) event suggest that the injury is related to the treatment?’ and ‘Is the lack of treatment or delayed treatment a recognised cause of this injury?’ Analogously, preceding questions were used to judge if an AE was preventable. An AE was found to be preventable when the performance of the practitioner fell short of the expected level of competence based on the professional standard. Appropriate management of the myocardial infarction was outlined in the previously mentioned MISSION!-protocol. Also, local hospital guidelines on precautionary measures to prevent common events, such as measures to prevent a delirium, were taken into account by the reviewer and the expert panel. The questions preceding the preventability score were also used to evaluate the complexity of the medical history and comorbidity of the patient. In addition, it was of importance to consider the potential benefit of the procedure, the calculated risk and the degree of emergency in treating a patient with a myocardial infarction. Therefore, the original preceding questions were augmented by two extra questions on whether the management of the AMI was appropriate, and on the estimated risk of an AE associated with the management (see online supplementary file 3).
Expert panel
In case of doubt regarding the causality and/or the preventability of an event, an advisory opinion of an expert panel was requested. The expert panel consisted of two consultants, cardiologists with a wide range of experience in interventional cardiology or electrophysiology. Both cardiologists were either involved in managerial tasks or departmental incident analysis. In addition, two cardiologists in their final year of training and involved in the daily clinical practice formed part of the expert panel. In case there was a discrepancy about the causation or preventability between the reviewer and the expert panel, the structuring questions were used to guide the discussion and reach a final decision (see online supplementary files 2 and 3). The expert panel was also involved in reaching consensus on the causality and preventability of events, which occurred frequently such as a groin haematoma.
Privacy
Guarding privacy and anonymity was considered to be a high priority. To warrant the privacy and anonymity, all people involved in the study signed a confidentiality agreement to maintain the confidentiality of the information. Study results were stored in a Microsoft Access 2010 Database on a safety disc which can only be accessed by individuals who are involved in the study.
Ethical approval
The LUMC gave a declaration of ‘medical–ethical permittance not necessary’ for this retrospective records study (reference number P15.133). The peripheral hospitals had formally consented to obtain data from outpatient clinical records, in accordance with their local medical–ethical committee.
Analysis and statistics
Patient characteristics and process deviations
For all patients, baseline characteristics such as age, sex, medical history and admission characteristics like length of stay, comorbidities, cardiac diagnosis and procedural characteristics (stenting or not) were retrieved. Continuous variables are presented as the mean with SD or median with 25th and 75th centile, where appropriate. Dichotomous variables are presented as numbers and percentages. The number of process deviations within each main category were calculated by summing the number of process deviations of each subcategory, after which they were plotted in a pie chart. Baseline comparability between patients with or without process deviations were evaluated by descriptive statistics and independent t-test or χ2 test, when appropriate. A p value smaller than 0.05 was considered to be statistically significant. Furthermore, the process deviation:AE ratio was determined to assess the effectivity of the new method based on identifying process deviations, and a χ2 analysis was performed to assess differences between patients with and without process deviations.
Inter-rater reliability
To assess the reliability of the assessment of the presence and preventability of an AE of the first reviewer, 10% (n=87) of the patient records were independently screened by an experienced cardiologist from another centre. This second reviewer was blinded to the outcome of the first review. To maximise efficiency, stratified sampling was performed. AE positive patient records were oversampled in relation to AE negative patient records, with a ratio of 2:1 (AE:non-AE). These patients were randomly selected from the whole MI study population. Consecutively, all patient records that contained an AE according to both reviewers were also reviewed on preventability. The percentage of agreement on causality and preventability is determined on a patient level and expressed separately for positive and negative rating. In addition, Cohen's κ statistics was calculated.24 To avoid any potential bias in the κ's coefficient, caused by the stratified sampling, the κ statistics were also calculated separately for the patient records with and without an AE.
Data accuracy
Data accuracy (missing data, inconsistent data) was checked on a regular basis and analysed using Microsoft Access 2010 and IBM SPSS Statistics for Windows, V.23.0, Armonk, New York, USA: IBM Corp.
Results
Process deviations and AEs
In total, 879 patients (age 64±12 years, 71% male) were reviewed, including the follow-up records of 274 patients who were transferred to three affiliated hospitals after a cardiac procedure in the tertiary care centre (table 1). In 347 patients (39%), one or more process deviations during admission (587 deviations in total) were found. The process deviations (n=587) were categorised into four main categories: observation, diagnostic, therapy or transfer (figure 2). Most process deviations were found during observations (especially observation of mental and physical symptoms or rhythm disorders) and therapy (especially anticoagulation therapy or return to catheterisation room for a second CAG or PCI procedure; figure 1). Patients with one or more process deviations were significantly older than patients without a process deviation (67±12 vs 61±11, p<0.001). In addition, female patients (66% vs 75% male patients, p=0.006) and patients with a lower renal function had a significantly higher risk for process deviations (table 1).
Baseline characteristics
Five hundred and eighty-seven process deviations of all 879 patients.
Translation of triggers to process deviations
After categorising the process deviations, the categories were translated back to one (or more) of the 18 original triggers of the HMPS (figure 1). Owing to the inclusion criteria of the study population, original triggers like ‘unplanned admission before index admission’, ‘unplanned transfer to another acute care hospital’, ‘injury related to abortion or delivery or neonatal complications’ were not applicable to this population. Moreover, ‘inappropriate discharge to home’ and ‘unplanned readmission after discharge’ were not included because the medical records were only reviewed while the patient was admitted to the hospital. Triggers like ‘dissatisfaction with care documented in the medical record’ and ‘documentation or correspondence indicating litigation’ were not used if they did not result in an alteration of the workflow. Eleven original triggers remained. Triggers used for diagnostic or therapy procedures especially became more specified by using process deviations (unplanned return to the operating room; unplanned removal, injury or repair of organ during surgery’ and other patient complication (no natural consequence of disease)). Likewise, the trigger ‘unexpected death’ was more specified in process deviations as ‘resuscitation’ or ‘positioning of an intra-aortic balloon pump’.
In 116 patients, 33% of all patients with a process deviation, an AE was found. The majority of patients with an AE had more than 1 process deviation (64 of 116 patients (55%), average of 2.0±1.7 process deviations per patient). In the group of patients with no AE, 150 of 231 patients (65%) had more than 1 process deviation, average of 1.5±0.9 process deviations per patient (p=0.005). No significant differences were found in patients with and without an AE in the distribution of the type of process deviations (table 2). Likewise, no differences in mortality rate (5 of 116 patients with AE died (4.3%) vs 13 of 763 patients without AE (1.7%), p=0.065). All patients who died (n=18, 2.0%) during their hospital stay experienced a process deviation (n=23 in 18 patients) and 5 patients an AE. The process deviations that were found in deceased patients were mainly related with therapy (21) or transfer (2) and no differences were seen between deceased patients with and without an AE in the type of process deviation (4 of 5 therapy deviation in patients with AE vs 17 of 18 therapy deviation in patients without AE, p=0.395). In patients without a process deviations, 2 patients (2 of 532, 0.4%) experienced a non-preventable AE.
Comparison of process deviations
Inter-rater reliability
The inter-rater reliability assessment was carried out for 87 patients (10% of n=879) by a second independent cardiologist with experience in the assessment of the presence of preventable AEs by means of medical record reviewing. On a patient level, there was an agreement in 73 patient records (agreement level of 84%). The positive agreement on the presence of an AE in the patient records was 87%, and the negative agreement on the absence of an AE in the patient records was 80%. No differences were found when performing an independent analysis of the agreement in patient records with AE and without AE (independent analysis AE positive agreement 86%, independent analysis non-AE: negative agreement 98%). In 45 patients, preventability was assessed on which there was an agreement of 84%. The κ statistics of the preventability was substantial (κ 0.67 (95% CI 0.45 to 0.90); Table 3). Of note, a common type of AE was a groin complication (groin haematoma), on which consensus was established by the expert panel. In principle, a groin haematoma was judged to be preventable. The expert panel decided that despite adequate action, a groin haematoma is less preventable in high-risk frailty groups like the obese, restless, elderly and patients with known peripheral vascular disease when using dual antiplatelet therapy.
Assessment of the inter-rater reliability between two physicians
Discussion
This study describes the development of a new valid screening tool for identifying process deviations and AEs in a specified patient population: patients with an AMI, treated according to a strict defined protocol.
First of all, process deviations were identified. We reviewed 879 patient records in which 587 process deviations in 347 patients were found. In 116 patients, 33% of all patients with a process deviation, an AE was found, leading to an AE:process deviation ratio of 1:3. This ratio seems higher in comparison to other studies where one AE was found in every sixth or seventh patient record with a trigger.3 ,4 ,18 However, it is important to take into account that previous studies were focused on a wider range of specialties, which can explain a lower prevalence of AEs. Interestingly, the type of process deviation is not associated with experiencing an AE; no differences in the type of process deviations were found between patients with an AE compared with patients without an AE. This finding could be explained by the fact that, despite process deviations being present in a patient record, they may not be the root cause of the AE. Various preventive actions may have taken place after the occurrence of the process deviation which could have averted patient harm. In addition, AEs are more likely to be caused by a combination of factors.25 This makes it difficult to develop a targeted approach to improve patient safety. More research, for instance based on incident analysis methods using more sources of information than the patient record, may help to identify other variables (ie, patient characteristics) that predict an AE. The majority of process deviations were found during clinical observation and therapy, which is in line with prior studies using the related original HMPS triggers.17 ,18
On the basis of the inter-rater reliability that was found in this study (causality and preventability agreement are both 84%), this adapted method appears to be a reliable and suitable instrument to use in this well-defined patient population. In previous studies, the reliability of the occurrence of AEs in general hospitals was moderate to substantial (κ ranged from 0.42 to 0.83, agreement ranged from 76% to 92%) and, if measured, it was moderate for determining the preventability of AEs (κ ranged from 0.33 to 0.69, agreement ranged from 68% to 91%; see online supplementary file 4).3 ,4 ,8 ,11 ,13 ,18 ,26–29 Recently, a study focused on hip fractures in elderly patients showed an agreement of 85% in the presence of an AE, with a κ value of 0.52.30 The substantial reliability in this study suggests that an assessment procedure shows reduced inter-rater variation when being performed in a specified population with a strictly predefined protocol. Nevertheless, a 100% agreement score was not reached. This is probably due to the different perspectives of physicians and limitations in our medical knowledge in causal relations; therefore, the assessment of preventability remains under debate until new scientific evidence is discovered. Previous studies on AEs were performed in general hospital populations and encountered a large variation among specialties with regard to the risk of the procedures employed and the severity of illness of the patients. As a result, heterogeneous numbers and causes of (preventable) AEs were reported among the different specialties.4 ,8 ,18 ,26 Focusing on one specific illness leads to a decrease in the workload for the hospital staff, makes it easier to plan the collection of data, and limits the interruption of the work flow. In the end, this specific instrument is likely to provide more valuable insights into the specific cause of an AE in patients with myocardial infarction and, consequently, possibilities to improve patient safety. Similar, AEs can show recurrent wrong patterns in the management within a clinic department.
Some limitations of the proposed method have to be considered. First, the validation method of the new process deviations tool can be considered incomplete because there was no direct comparison with an alternative screening tool, such as the original HMPS trigger tool. Therefore, we cannot exclude the possibility that other numbers of process deviations or triggers were found if a comparison using another tool was performed. In addition, tools for identifying process deviations, such as the Process Deviations Analysis Framework or the (Lean) Six Sigma Model, were not used for the design of the current method because they are not (yet) suitable for assessing whether process deviations are associated with undesirable (healthcare) outcomes.23 ,31 Besides, there are still considerable challenges when it comes to implementing process deviations frameworks in a healthcare setting. Current process deviation frameworks are highly measurement and data driven while a healthcare setting is mostly dependent on human behaviour, which is difficult to quantify. Another limitation is that this study depended exclusively on documentation in medical and nursing records. However, the likelihood that this has affected the quality of our study is low since previous studies showed that a record review method is sensitive for identifying AEs.32 Also, direct comparison of these results with different hospitals is difficult, as record reviewing highly depends on the level of record completeness and the use of a (electronic) patient record. To lower the possibility of hindsight bias, a prospective design with a weekly review process of a researcher could be considered.8 ,33
Future implications
In the future, it could be of interest to explore whether application of this new method makes it possible to assign AEs to uniform groups after which action can be taken. Furthermore, the eventual goal is to monitor patient safety with real-time process deviations and AEs measurements or, even better, to predict which patients are at risk for AEs to prevent harm. For this next step, existing approaches for measuring process deviations used in other fields can be helpful. An integration of HMPS and Lean Six Sigma, for example, may be beneficial for healthcare. HMPS's strength is to structure implicit relations and to define harm; on the other hand, Lean Six Sigma is a data-driven approach, which may facilitate detection of process deviations as indicators of AEs. It should be explored whether it is possible to develop an electronic tracking system, as part of an electronic patient records system, which enables the continuous monitoring of care. This will shift the emphasis away from focusing solely on medical errors, and more on real-time performance and measures that relate to future risks and resilience of organisations. This could be the starting point for the development of a hospital benchmark quality instrument to objectify patient safety as part of quality of care in a specific patient population.34 Bottom–up, this could serve as an incentive to improve safety, and top–down it will give insight to redesign patient work flows which can improve efficacy and quality of care. Structural process deviations seem more useful for educational purposes compared with individual preventable incidents. Although this method is focused on one illness, this approach may also be applied to other patient populations, or to evaluate other care tracks like the MISSION!-protocol, for example, the total hip arthroplasty procedures.17
Conclusion
In conclusion, this paper describes the development of a reliable method to objectively assess the process deviations and the occurrence of AEs in a specified population.
Acknowledgments
The authors gratefully thank L Relik-van Wely, cardiologist for screening the records for the assessment of the reliability. Furthermore, they also would like to thank the member of the expert panel, consisting of M Bootsma, electrophysiologist; F van der Kley, interventional cardiologist; WTJ Aanhaanen, cardiologist in training at the Leiden University Medical Center; M van Vliet, cardiologist in training at the Leiden University Medical Center.
References
Footnotes
Contributors MCdB and DCE designed the study. DCE drafted the manuscript, as well as collected and analysed the data. MCdB and CJWB contributed to the conception and design of the study, and provided daily supervision. CJWB helped in drafting the article and revising the content of the article. MJS supervised the conduct of the study and provided comments on subsequent versions of the manuscript. MFD supported the data collection. All authors read and approved the final manuscript.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Ethics approval Medical Ethics Committee of the Leiden University Medical Center (reference number P15.133).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.