Background: reason for existence of database, when it began

The Society for Cardiothoracic Surgery (SCTS), the professional society representing cardiac and thoracic surgery in Great Britain and Ireland, has a long history of collecting and benchmarking adult cardiac surgery mortality data.1 As long ago as 1977, under the guidance of Sir Terence English, the SCTS established a cardiac surgery register whereby all units were asked to submit data to a central source for analysis. The data required were the number of operations in various specified operative groups, along with associated 30 mortality. All returns were paper based and thought to be of variable quality. The SCTS produced outputs of pooled mortality rates for the specific operations, and returned to each unit their mortality against the pooled data. There was no ability to stratify for case mix within the register.

In 1994 Bruce Keogh (now Professor Sir Bruce Keogh, NHS medical director) established a cardiac surgery database project where initially a small number of centres providing cardiac surgery care were asked to collect a more comprehensive dataset, including preoperative patient characteristics, operative details and postoperative outcomes to allow trends in surgery to be studied along with mortality outcomes; particularly of interest was to understand those outcomes against differences in case mix. A similar initiative was set up in northwest England, including a consortium of the four NHS hospitals undertaking cardiac surgery in the region. Together these projects showed that it was possible to collect and pool data from multiple hospitals using encrypted patient identifiers via software systems set up by the Central Cardiac Audit Database (CCAD, now part of the NHS Information Centre), to collect data on all patients from all hospitals in a defined geographical region and to supplement data collection with centralised analysis, governance screening processes and cooperative quality improvement methodologies. More recently the CCAD project has been embraced by more hospitals, and all NHS hospitals in the UK now contribute data, together with a number of private providers.

The SCTS data collection initiative has been clinically led and, as the CCAD concept has developed, other national initiatives have driven progress. It was apparent to some within cardiac surgery that there would be a need for national data collection and governance processes for several decades but the events in paediatric cardiac surgery at Bristol Royal Infirmary and the subsequent public inquiry brought these issues to the forefront of public and political attention.2 Indeed, the recommendations in Professor Sir Ian Kennedy's report that the outcomes of care by a consultant team should be available to the public have been ignored by the majority within British medicine. However, they have been taken seriously and implemented by those within the SCTS because of leadership from a number of people within the profession and scrutiny from outside by politicians and the media.

Surgical results by identifiable hospitals were first brought into the public domain by the Dr Foster Organisation (a private company with expertise in benchmarking administrative data) in conjunction with the Times newspaper in 2001. This analysis was based on the administrative Hospital Episode Statistics data, which are thought by many clinicians to be inaccurate, and named three hospitals as having high risk-adjusted in-hospital mortality after adjusting for case mix. The SCTS responded to this by publishing hospital mortality rates from the cardiac surgery database and then went a step further in 2004 by publishing the names of all consultant surgeons together with a statement as to whether they had achieved a defined mortality standard (crude mortality within 99.99% CIs of the national mean for isolated coronary artery surgery).3

In January 2005 a Freedom of Information Act became law in England and the Guardian newspaper published named surgeon mortality rates provided to them by the trusts later that year.4 The surgeons of northwest England published their named surgeon mortality rates with risk-adjusted comparisons at the same time through a peer review journal,5 and the SCTS then collaborated with the healthcare commission (the UK's organisational healthcare regulator at the time) to produce a public portal to disseminate named surgeons' mortality to the public later that year. This portal is updated regularly and now contains risk-adjusted mortality data for all hospitals and about 70% of surgeons for the 3 years of data from April 2006 to March 2009.6 Decisions about whether named surgeons' data are made available remains a choice for individual units.

The data: who is in the database, what is measured, how are individual patients identified?

The database contains information on preoperative patient characteristics, operative details and postoperative information. The dataset is prescribed by the SCTS and the current definitions were agreed in 2003 with an understanding that these would remain unchanged for 5 years to allow data collection to become embedded and to prevent frequent and potentially costly software upgrades.3 The SCTS has recently undertaken a major revision of the dataset to update it for contemporary practice, and it is planned that the new dataset will go live in April 2010. The dataset allows adjustments to be made for case mix—the initial risk adjustment algorithm which found widespread use within the cardiac surgical community was the Parsonnet Score,7 but over time the predictive ability of this algorithm drifted and it was replaced by the EuroSCORE.

The EuroSCORE was originally recommended for use as an additive score,8 but again as quality of care improved the accuracy of the score drifted and it has been largely replaced with the logistic EuroSCORE, a logistic regression model based on the same characteristics as the additive model.9 Over time the predictive ability of the logistic EuroSCORE has also drifted, because there have been marked reductions in mortality, such that observed mortality in the UK is now about 50% of that predicted by the score.

Recent national analyses of the SCTS database have used a complex recalibration of the logistic EuroSCORE whereby updated logit coefficients have been derived for some of the EuroSCORE fields to render the algorithm ‘fit’ for contemporary purposes. The major focus of the SCTS database project over time has been risk-adjusted in-hospital mortality, but recently more emphasis has been placed on measuring postoperative complications, length of stay and longer-term mortality.10 The last of these can be measured because all patient records in the database have an ‘encrypted’ NHS number that allows linkage with the office of National Statistic to allow life status at any time to be established. This has, for example, allowed us to generate 5-year Kaplan–Meier survival curves on 88 000 patients undergoing coronary artery surgery.

Data quality: accuracy, validation, completeness

The data in the database is thought to be of good quality but this is not subject to rigorous external validation. All hospitals and a number of private providers undertaking adult cardiac surgery in the UK submit to the database. It is believed that case ascertainment is complete, certainly for the NHS hospitals. For several years the SCTS undertook a voluntary validation system whereby a site visit occurred to look at an institution's processes to see that they complied with perceived best practice.3 These included validating documented systems and responsibilities for collecting the audit data, appropriate and timely feedback of data to clinicians for ‘real time’ feedback, a process to cross reference surgical activity in the SCTS database against theatre logs and the administrative database and a mechanism to cross check mortality on the database against other sources of mortality within the hospital. The learning and outcomes of these visits were disseminated through the SCTS website.

In the early stage of the northwest England adult cardiac surgery audit a number of reciprocal round robin site visits were organised and detailed case record validation against the database returns was undertaken. This was a tedious and labour-intensive process. No evidence of systematic gaming was identified from these visits and the single most important contribution to the degree of variance between data sources was the type of paper-based system available in each hospital—those with multiple sets of base and visiting hospital records showed more ‘errors’ against the database information than those with a single paper-based integrated care pathway, suggesting only the greater capacity for intraobserver variability in the former system. Given the increasing importance of surgical audit data for a variety of purposes it is important that the SCTS should reconsider its validation processes, possibly with online screening of submitted data to look for hospitals of potential concern, followed up by targeted site visits.

The completeness rates of the submitted data are generally good—the incidence of missing data for age is 1.4% and for gender 0.07% between 2004 and 2008.10 Most important fields for risk stratification have an incidence of missing data of <5%. The missing data for postoperative complication rates is somewhat higher at around 15%. This is coming down over time. For the purposes of risk adjustment for governance analyses we have taken the approach of being ‘systematically harsh’ on missing data in that extra risk points are only allocated when the presence of a risk factor is recorded and missing data are assumed to indicate an absence of that factor—it is hoped that this will further drive complete data capture.

Brief summary or example of data

The data collected by units is uploaded to CCAD after encryption of all patient identifiers. On upload a report is produced about the number of records and potential major and minor flaws in the data to allow correction to be made. The CCAD software allows views of the data including activity, the incidence of various risk factors, in-hospital mortality, risk-adjusted mortality, postoperative complications rate and length of stay. This allows for ongoing data validation and supports quality improvement within the organisation.

Use of data: overview of important publications, who has access and how do they use the data, how researchers get access?

The highest profile outputs from the database have been the national reports, known within UK cardiac surgery as the ‘blue books’. The most recent of these were the 5th national report ‘Improving outcomes for patients’ published in 20043 and the 6th ‘Demonstrating quality’ published in 2009.10 These are comprehensive reports which exhaustively document trends in cardiac surgery outcomes and practice and benchmark cardiac surgical mortality rates, including longer-term outcomes. The reports also have political significance—for example, the 5th report contextualised the UK cardiac surgical data collection initiative against the recommendations of the public inquiry into the events at Bristol Royal Infirmary. The recent 6th report applied the principles of the Darzi review ‘High quality care for all’11 to the specialty of cardiac surgery including, in addition to the obvious clinical outcomes data, some information on aspects of patient safety and patient experience, and thoughts about how this data will feed the forthcoming professional recertification agenda. Another high-profile output from the database is the publication of named hospital and surgeon mortality data to the public through the Care Quality Commission website.12 This presents detailed information about cardiac surgical diseases and their treatments, and presents results in a clear way for patients and their carers. This website receives in excess of 26 000 ‘hits’ each month.

An important use for the data will be in demonstrating quality of care for the purposes of individual practitioner professional recertification.13 The White paper ‘Trust, assurance and safety’ is changing the way the medical profession is regulated, and demonstrating satisfactory ‘success rates of treatments’ will be a part of this. The SCTS is developing a strategy to analyse and benchmark data for this purpose (B Bridgewater, O Nicholas, G Cooper, et al, unpublished data, 2009). It is planned that this type of methodology will apply more widely across British medicine in future, thereby increasing the importance of, and clinical buy-in to, national registries.

There has been much debate over time as to the ‘wrong and rights’ of publishing named surgeon data, but what is without question is that there have been marked improvements in risk-adjusted mortality for cardiac surgery in the UK over the past 10 years.3 10 14 There is no evidence that the initiative to collect, benchmark and publish these data has been associated with significant ‘risk-adverse’ behaviour among surgeons in the UK.14 This should be reassuring to all stakeholders.

The SCTS is developing a strategy to increase the research outputs from the database and has activated a data-sharing agreement for that purposes. Interested researchers can contact sctsadmin{at}scts.org.

Strengths and weaknesses of data

The strength of the data in the cardiac surgery database is the complete coverage of all NHS hospitals, and the mortality data are adjusted for risk and presented widely to patients and the public. This must be considered the ‘gold standard’ for transparency in British medicine, and it is a disappointment to many within cardiac surgery that this lead has not been followed by others. There is also great power in the numbers—the recent database report included over 400 000 operations with information on over 114 000 coronary artery bypass operations, 30 000 aortic valve operations and 10 000 mitral valve operations, which allowed important findings to be reported—for example, major changes in the type of prosthesis used for aortic valve replacements and poor outcomes related to late referral of patients with severe mitral regurgitation.10

The major weakness of the data is that it is not subjected to rigorous external validation and there is a important incidence of missing data in some critical fields within the dataset—for example, missing data on ‘valve pathology’ prevents optimal analysis. The SCTS has also not been able to frequently modify the dataset to account for changes in contemporary practice, which prevents accurate tracking of activity and analysis for novel and emerging treatments, such as surgical atrial fibrillation ablation and trans- catheter aortic valve implantation.