Building a framework for trust: critical event analysis of deaths in surgical care

Introduction

Events over recent years have undermined patients' and society's trust in clinicians and healthcare institutions. For clinicians to restore and retain public confidence, they need to show that effective mechanisms exist for assessing events such as death and to justify patients' faith in the delivery of care.1

In the high profile world of cardiac surgery, the audited results of key elective procedures may be sufficient to restore confidence.2 In wider surgical practice, simple death rates are unlikely to be sufficient; case-mix (particularly for emergency admissions), institutional, and national issues can jeopardise the interests of high risk patients.3

Any critical incident review by peers working in the same speciality or subspecialty of patients who die under surgical care should take into account the nature of the patient and the circumstances of admission. In Scotland, the Scottish Audit of Surgical Mortality (SASM) is a national system of peer review of deaths that occur under surgical care that has been in place since 1994.4 Full details of how SASM works, and the annual reports, are on its website (http://www.sasm.org.uk/).

Assessment process

SASM uses the consultant surgeon responsible for the care of the deceased patient as a proxy for the secondary care system and includes all deaths under surgical care in Scotland (including the private sector) except for cardiac surgery and obstetric deaths, which are currently audited separately. SASM assesses the patient's last journey of care, not simply the role of a single clinician or clinical team.

The method of case assessment and the review process (fig 1) are now well established In brief, deaths occurring within 30 days of an operation or during the patient's last admission are identified by using multiple sources of case ascertainment (including mortuaries and ward secretaries), which compare favourably with the central register of deaths. Data are collected via validated general, orthopaedic, neurosurgical, and anaesthetic proformas. The proformas are anonymously assessed by a surgical assessor (and by an anaesthetic assessor if an anaesthetic has been given). Assessors work in the same specialty but a different region; guidance notes for assessors, assessors' forms, and hypothetical feedback letters are on SASM's website.

Fig 1

The pathway of data gathering and assessment starts with a surgical death and proceeds through the critical event pathway. Where there is a cause for consideration or concern, an adverse event may be identified and a case-note review requested. For 10% of the deaths where no adverse event is identified, the case is reassessed. The assessments and review are fed back to the individual clinician, who has the right to reply or request a further review, leading to a final feedback to the clinicians involved in the care of the patient before death

• Download figure
• Open in new tab
• Download powerpoint

Each surgeon receives individual feedback on every death assessed (including a detailed case-note based assessment if adverse events have occurred in the management of the patient) within 79 days (mean time in 2003). In addition, collated patient histories with a message for the wider anaesthetic and surgical community are mailed at regular intervals to all surgeons, anaesthetists, and trainees in Scotland. As a quality assurance measure, 1 in 10 cases (selected at random) are reviewed by a second independent assessor; in 2004, there was 98% concordance with the first assessor. The peer reviewers also individually assess a spectrum of real cases from the previous year and meet annually for group review of this portfolio to confirm the reliability of the process.

The requirement for clinical ownership and leadership to facilitate change5 is satisfied in SASM by the use of a tiered structure, including senior members of the relevant surgical colleges, professional staff, and lay representation (box). This process is to ensure that the quality of data collected to monitor clinical care is of high standard6. Compared with other surgical audits, SASM's methods seem to be more robust than most.7 However, changes in computer database systems, financial sustainability, the development of the adverse events coding system, and the new UK Freedom of Information Act continue to present challenges. The Central Legal Office in Scotland has assessed the SASM's process as sound (see SASM's website).

High compliance

SASM was started by surgeons and anaesthetists, is supported by professional bodies, is clinically led, and remains a voluntary process. This bottom-up culture of reflection rather than blame has led to a belief by the profession that the process is fair and open, with peer pressure to participate. Surgeons and anaesthetists participating in SASM to the required level (85%) of compliance are listed in the published annual report. Those who fall below the compliance threshold receive a letter from the chairman of the board to encourage them to participate more actively with SASM, an obligation commensurate with good clinical practice.8

Although in 2003 some 4084 out of 4478 deaths (91%) were audited, only 14 surgeons out of over 1200 surgeons and anaesthetists failed to comply to the required level. Neither an analysis of adverse events versus the number of completed proformas, nor additional peer review of patients selected at random, has identified selective reporting of events. Indeed, failure to retrieve case notes seems to be a common reason for incomplete compliance. With a substantially higher completion rate than most large clinical audits,9 there remains little evidence that making SASM compulsory would increase compliance further,10 but such a move would alter the basis of the relationship between the audit and the clinicians.

Extracted data analysis

The audit has reviewed some 45 000 deaths under surgical care in the past 10 years at a current cost per death of £59 ($113; €86). A national annual report collates the data in summary form. The reports consistently show the predominance of emergency cases (about 90%)—and in only half of these cases does the patient have surgery—with general and vascular surgery, followed by orthopaedics, accounting for most deaths. In recent years, some 10% of cases have gone on to case-note review; and despite 60% of case-note reviews failing to identify adverse aspects of care, this review rate has recently increased.11

Fig 2

Percentage of operations at which consultants have an input in various ways, 1994 to 2003

• Download figure
• Open in new tab
• Download powerpoint

Analyses of individual areas of surgery have included mortality from colorectal cancer,12 deaths in patients with a hernia,13 deaths after endoscopy14 and deep venous thrombosis prophylaxis.15 These analyses provide insight into surgical practice in the NHS in Scotland.

Trends over a decade

Over the 10 years since SASM was started, the annual reports show an elderly population (median age at death has risen from 75 to 76), with a stable overall annual mortality of 4500 deaths. The percentage of deaths that occur after elective inpatient surgical admission has remained static (at 0.27%) in 2003; the proportion of deaths occurring after emergency surgical admission also remained static (at 2.2%).

Consultant input from both surgeons and anaesthetists in the decision making process and in the operating theatre (fig 2) has increased compared with historical series from the same12 and other9 16 populations. Alongside this increase in consultant input, there was a decrease in the percentage of deaths in which adverse events in management contributed to or caused the death (figs 3 and 4). Whether the two are causally linked is unknown. Delay in transfer to surgical care, diagnosis missed by surgeons, and failure to use deep venous thrombosis prophylaxis have all declined as adverse events in the care of patients over the decade. Although the assessors' criticism in the choice of operation has changed little, criticism of the seniority of the surgeon operating and of delay in recognising postoperative complications has also fallen.

Fig 3

Percentage of deaths in surgical care for which adverse events in management were identified as contributing to death, 1994 to 2003

• Download figure
• Open in new tab
• Download powerpoint

Fig 4

Percentage of deaths in surgical care for which adverse events in management were identified as causing death, 1994 to 2003

• Download figure
• Open in new tab
• Download powerpoint

Although adverse events contributing to or causing death have declined over the past 10 years, and despite the fact that for the vast majority (94%) of patients who died in 2003 adverse events did not contribute to death—a figure comparable with other healthcare systems17—the media choose to focus on the 1 in 20 deaths in which such events contributed to death. Many of the adverse events identified by SASM relate to the hospital systems and the process of care. This contrasts with a perception that technical or intraoperative errors are common.18 19 The increase in “failure to use high dependency unit/intensive therapy unit” as an adverse event from 1994 to 1998 was highlighted in the annual reports and subsequently by the media. A working party recommended additional resources for improving existing high dependency and intensive therapy units in the NHS in Scotland and for commissioning such units in hospitals that did not already have them. As a result, this category of adverse event has declined markedly to 1% of cases since that time (fig 5).

Fig 5

Percentage of deaths for which “failure to use a high dependency unit/intensive care unit” was cited as an adverse event, 1994 to 2003

• Download figure
• Open in new tab
• Download powerpoint

Failure to use deep venous thrombosis prophylaxis has also been highlighted by the SASM feedback. Together with the publication and regular updating of the Scottish Intercollegiate Guideline Network (SIGN) on deep venous thrombosis prophylaxis,20 omission of deep venous thrombosis prophylaxis is now rarely identified (12/3907 (0.3%) deaths in 2003) as an adverse event for patients who die under surgical care. More recently, persistent concerns raised by SASM over postoperative care have resulted in SIGN producing consensus guidelines.21

What else might have accounted for the changes?

When SASM has heightened awareness and publicised concerns about areas of patient care, changes in practice have ensued and criticisms of adverse events have subsequently declined. However, other factors, such as SIGN guidelines, have undoubtedly contributed. Less rigorous peer review than a decade ago seems unlikely given that the proportion of cases undergoing case-note review has increased and the number of adverse events recorded as a consequence has not.11

What next?

In the first half of 2005, surgeons in Scotland will receive a pilot “individual annual report” that contains a range of individual and comparative data. This document may be used for voluntary submission in the appraisal process11 as it contains institutional and national data alongside the clinician data. Although this individual report is not designed to detect malpractice, clinicians with complete, peer reviewed individual reports are unlikely to be able to conceal avoidable deaths. The report should also empower clinicians during the two way appraisal process, which will highlight institutional responsibilities. As a clinical team, rather than a single doctor, may be responsible for a patient's care, the individual annual report may in time become a team document.

For the first 10 years of SASM, the consultant surgeon in charge of the patient was the point of contact but a consultant anaesthetist not involved in the case was not necessarily identifiable. Pressure from the consultant anaesthetists led to guidance from the Royal College of Anaesthetists in Scotland and the Association of Anaesthetists that from 2004 a named consultant anaesthetist will be recorded for all operations, thus allowing an individual annual report to be produced for anaesthetists.

The nursing profession will soon pilot a similar scheme to SASM in Tayside, Scotland, assessing nursing issues around the death of any patient on a surgical ward. Other areas of development include identifying patients who die outwith hospital but within 30 days of surgical care, which is currently undergoing a pilot study of SMR01 linkage.

Although the audit of non-fatal critical events is currently beyond the resources of SASM, a web based system of self reporting critical incidents, analogous to that used by pilots, may soon be available.

International perspectives

Healthcare systems in other countries have already adopted an audit process based on SASM. The West of Australia Audit of Surgical Mortality's first annual report was for 2002 (see http://www.surgeons.org/about/waasm.html). The report compared that region's data with those from SASM, and for many adverse events it identified a similar starting point to SASM's a decade ago. The SASM approach looks set to be rolled out elsewhere in the antipodes and will complement root cause analysis already in use.22 23 SASM, designed for review and reflection, but subject to the caveats of all peer review systems, may in future be used as a preliminary review process, with root cause analysis for further review of selected cases in both Australia and the United Kingdom. For future meaningful comparison between different healthcare systems, issues of coding taxonomy and adverse event codes will need to be resolved, and the same core questions will need to be asked.

Balanced governance

The adverse events identified relate to the process of care, rather than individual clinicians making errors. Examining the process of care at hospital and health board level should have a greater impact in the medium to long term24 than simply feeding the data back to individual surgeons and anaesthetists. Institutions should also be accountable for clinical governance and outcome, and the SASM process looks set to become involved in the concept of “balanced governance”—in which both clinicians and institutions are responsible for the care of patients. Until recently, hospital trusts and health boards seem to have shown little interest in individual hospital reports, which compare a given institution with others in Scotland. As many adverse events are concerned with process, managers are beginning to acknowledge the benefits of using high quality SASM data in balanced clinical governance. Perhaps it is also time for a similar process to start in other specialties, such as general medicine.

Editorial by Baxter, and p 1144

Both authors also hold positions on the management committee of the Scottish Audit of Surgical Mortality. We thank all participants in and contributors to the Scottish Audit of Surgical Mortality process; Z Ashaf, G McPhillips, and C Sproul for statistical analysis; H Burton for managing the data collection and storage; and Wendy Ritchie for secretarial support.