Article Text
PDF
Abstract
Aims To create an inclusive and accurate registry of all percutaneous coronary intervention (PCI) procedures performed in the UK for audit to assess quality of care, drive improvements in this care and to provide data for research.
Interventions Feedback to PCI centres with ‘live’ online data analysis and structured monthly and quarterly reports of PCI activity, including process of care measures and assessment of risk-adjusted outcome. Annual national reports focused on the structure of the provision of PCI across the UK, the appropriateness and process of its delivery and outcomes.
Setting All hospitals performing PCI in the UK.
Years 1994 to present.
Population Consecutive patients treated by PCI. Approximately 80 000 new procedures each year in recent years.
Startpoints All attempts to perform a PCI procedure. This is defined as when any coronary device is used to approach, probe or cross one or more coronary lesions, with the intention of performing a coronary intervention.
Baseline data 113 variables defining patient demographic features, indications for PCI, procedural details and outcomes up to time of hospital discharge.
Data capture Data entry into local software systems by caregivers and data clerks, with subsequent encryption and internet transfer to central data servers.
Data quality Local validation, range checks and consistency assessments during upload. No external validation. Feedback of data completeness to all units.
Access to data Available for research by application to British Cardiovascular Intervention Society using a data sharing agreement which can be obtained at http://www.bcis.org.uk.
- Percutaneous coronary intervention
- registry
- audit
- database
- coronary artery disease
- coronary intervention (PCI)
- non-coronary intervention
Statistics from Altmetric.com
- Percutaneous coronary intervention
- registry
- audit
- database
- coronary artery disease
- coronary intervention (PCI)
- non-coronary intervention
Background
The collection of data to describe the nationwide practice of percutaneous coronary intervention (PCI) in the UK started with the inception of the British Cardiovascular Intervention Society (BCIS) in 1988. From 1988 to 1991 annual surveys of PCI activity were published in the British Heart Journal.1–3 Annual reports from 1992 onwards are available for download from the society's website (http://www.bcis.org.uk), and summaries of some of the key data have been published as reports for distribution to participating hospitals and networks.4–7 The initial data collection process was essentially a descriptive survey of activity. Over time the process has evolved becoming more sophisticated with electronic methods for data collection developed by the Central Cardiac Audit Database (CCAD) group. The methods were tested in 1996 as a pilot in selected hospitals, and full UK-wide participation in the electronic collection of PCI data was achieved by 2005.
Organisation and funding
The data collection and analysis process is led by the BCIS. The BCIS audit lead (an interventional cardiologist) has the main responsibility for this process. Clinical and professional oversight is provided by the BCIS Data Monitoring and Analysis Group. The audit lead also works closely with the CCAD group, which provides the computing hardware, software development and management tools that deal with the processes of data collection and storage. The audit lead and CCAD are also responsible for developing a number of analysis tools and audit reports that are described in more detail later.
The audit has been funded by central government via the Department of Health since the start. Currently, this funding is managed by the Healthcare Quality Improvement Partnership, a government body whose aim is to increase the impact of clinical audit on the quality of care in England and Wales.8 The costs of local data entry are borne by the participating hospitals.
From April 2011, six of the UK cardiac audits including BCIS have been brought together in the form of an organisation called the National Institute for Clinical Outcomes Research. This sits within the Cardiovascular Prevention and Outcomes Centre in the Institute for Cardiovascular Science based at University College London. It is planned that collaboration, with sharing of resources and cross-linking of datasets, will enhance the potential value and output from these data.
Staff at CCAD provide telephone support via a help desk for technical issues and, together with the BCIS audit lead, respond to queries about case scenarios and definitions. A secure drop box can be used to analyse potential technical problems related to data uploads, file structures and field mapping errors.
The data: who is in the database, what is measured, how are individual patients identified?
Methods for data recording and transfer
The BCIS–CCAD database is intended to record information about every PCI procedure performed on all patients in the UK. Data are collected using a variety of database programs. Two of these are available free to users; one is part of the Lotus Notes software described below and present at all PCI centres, and the other is a Microsoft Access database available for download at http://www.bcis.org.uk. The remaining options are provided by commercial software developers. Regardless of platform, the local database generates a comma-separated values file to a standard format available at http://www.bcis.org.uk/pages/page_box_contents.asp?pageid=693&navcatid=25. All PCI centres then use a local copy of Lotus Notes software to manage the encryption and subsequent internet transfer of this comma-separated values file to the central CCAD servers. The same software provides a portal for data analysis and is used to generate reports described later. The Lotus Notes program and licence is provided without charge to all PCI units in the UK.
Data recorded
For each PCI procedure a total of 113 variables are collected. The current dataset is version 5.5.6 and is available in the form of an excel spreadsheet at http://www.bcis.org.uk. The variables include patient demographic features, indications for PCI, details of the PCI operators, technical aspects of the PCI procedure and adverse outcomes including complications up to the time of hospital discharge. Definitions of these variables are provided in the spreadsheet. Changes to this dataset have been as infrequent as possible to allow it to become embedded in system software and to reduce the cost of upgrades. As of March 2010 there are approximately 460 000 records in the BCIS–CCAD database, with approximately 80 000 new records being added each year.
Patient identification and tracking
The National Health Service (NHS) number provides a unique identifier for any person registered with the NHS in England and Wales. Mortality tracking is undertaken by the NHS Central Register using this unique patient identifier. It is a legal requirement for all deaths in the UK to be registered with this body. It is not possible to effect any form of burial or cremation or similar process for the deceased without such registration. Thus tracking yields very robust results. The NHS number is pseudonymised within the database, and patient addresses are not recorded. In Scotland a tracking system operates on date of birth and name only and is therefore less robust. No formal tracking exists in Northern Ireland. Other identifiers recorded in the database include hospital numbers, dates of birth and postcodes. Researchers do not have access to these aspects of the dataset, and hospital identity is also usually protected. However, age at time of PCI is provided, and Eastings and Northings of the centroid of the output area of residence, shared by between 1 and 80 addresses can be made available for geographical mapping with necessary permissions.
Data quality: accuracy, validation, completeness
Completeness
The aim is for all PCI procedures performed in any hospital in the countries of the UK (England, Scotland, Wales and Northern Ireland) to be recorded in the BCIS–CCAD database. Three aspects of completeness are addressed: first, the number of PCI centres participating in the CCAD data collection process; second, the percentage of the PCIs performed by a unit that have been entered into the database; and finally, the completeness of the fields in each record.
In recent years very full participation has been achieved.4 Data collection started with NHS hospitals in England and Wales, and then spread to all UK countries and to some private hospitals. In 2009, of NHS centres, 74 of 77 (96%) English centres were participating, all Welsh centres, six of seven Scottish and two of three Northern Irish centres. Uptake by private healthcare institutions has been less good, but because only a small proportion of PCI is performed in such institutions the vast majority of PCI procedures UK wide are being captured. By 2009, 97% of all PCI procedures performed in NHS hospitals in England and Wales had been entered and 93.3% of all PCI procedures in all hospitals across the entire UK.
Case ascertainment is measured by comparing a unit's self reported total number of PCI procedures (requested from all units by the BCIS audit lead each year) with the number of procedures seen in the database.
When assessing the completeness of data fields in each record, particular emphasis is placed on those fields required for risk-adjusted outcome assessment. In 2009, of participating PCI units, 79 of 90 had more than 90% completeness of data in the 12 key fields used for risk adjustment.
Participation in the audit and the completeness of each unit's data are published in an annual report sent to all hospital chief executives, cardiac network leads and PCI unit audit leads to try to encourage and improve participation.4
Data quality
During the upload of data to the central servers, some range checks and checks for internal consistency are applied. If a major problem is found that might cause the upload to contaminate the reliability of the complete dataset (such as a primary PCI procedure classified as a routine elective case) the record is rejected. Records with less serious inconsistencies are accepted, and an error log documents fatal and serious errors allowing units to clean and correct their data. There is currently no independent validation of data.
Use of the data
The main purpose of the BCIS–CCAD data collection is to help PCI units to benchmark and improve the quality of the PCI they provide for their patients. This involves a focus on three different but closely related areas: clinical audit against standards, outcomes analysis and research.
To provide information to those involved in treating patients by PCI, the data are analysed and presented in a variety of ways:
‘Live’ data analysis
The Lotus Notes software present in each PCI centre is programmed to provide a variety of cuts of the data that allows all units to compare their own PCI activity with the national aggregated data. These views include overviews of patient demographics, clinical characteristics, procedural and technical aspects of PCI and outcomes including complications. So for example, when comparing a local unit with national data a user can drill down progressively from all PCI procedures, to those performed for stable syndromes, to those performed using rotablation and observe demographic features and outcomes (figure 1). This represents a live view of all the data in the dataset at the time it is being accessed.
Live data view in Lotus Notes. The upper panel shows the national aggregated data, and the lower panel the data from the local percutaneous coronary intervention centre. Clicking on the + symbols allows the user to ‘drill’ further down into details. In this particular cut of data it can be seen that there were 80 690 cases in the national dataset in 2010, of which 31 102 were performed for stable coronary syndromes, and 743 where a rotablator was used (2.4%). Of these, the mean age was 72.1 yrs, 74.7% were male, 82% were treated with drug-eluting stents. In the lower panel, the single unit's experience shows a total of 969 cases. This centre used rotablation in 6.3% of cases compared with 2.4% nationally, and 93% were treated with drug-eluting stents (compared with 82% nationally).
Programmed reports
The Lotus Notes software allows users to generate specific analyses of their own data between chosen date ranges. Such reports include aggregate totals with multiple data cuts. A ‘delays’ report analyses call and door to balloon times for primary PCI. Risk-adjusted assessment of outcomes can be generated in the form of either a variable life-adjusted display plot or a cumulative funnel plot9 (figure 2). The current risk adjustment model used for these plots is that developed by the North West Quality Improvement Programme,10 but work is underway to update this model.
Cumulative funnel plot. The x axis shows successive percutaneous coronary intervention procedures. The observed major adverse cardiovascular or cerebrovascular event (MACCE) rates are plotted as a cumulative running percentage mean in dark blue, settling at about 1.7% after about 3000 percutaneous coronary intervention procedures. The predicted major adverse cardiovascular or cerebrovascular for each procedure is calculated using the North West Quality Improvement Programme model,10 and again a running cumulative mean percentage is plotted in pale blue. The observed and predicted rates of adverse events can therefore be compared, and are enclosed by the funnel-shaped curves that show the limits of statistical variation (3 sigma in red and 2 sigma in yellow, approximating to 99.8% and 95% confidence limits respectively).
Automated reports
Generating the reports described above requires access to the one computer running Lotus Notes on each PCI site. This is sometimes inconvenient and thus a system of automated reports has been set up to run in parallel. Reports as described above, but for a prespecified date range, are sent by email to the audit lead and any other interested parties to each PCI centre. The ‘delays’ report is sent monthly, and a cumulative funnel plot sent quarterly (figure 2).
National annual reports
Each year the audit lead analyses the previous calendar year's national PCI data and presents the findings to the professional society at their annual autumn meeting. The data are analysed to assess the structure, appropriateness, process and outcomes of PCI. Linkage to the NHS Central Register is also used in the analyses. As examples the mortality 30 days after following primary PCI for ST elevation myocardial infarction (MI) has been presented as a funnel plot (figure 3), and Kaplan–Meier survival analysis has been used to examine outcomes from primary PCI in patients aged over 80 years (figure 4). All these reports are then published on the society's website as slide presentations.11 An abbreviated version of this presentation is produced as a printed report and sent to commissioners, trust chief executives and network clinical leads, and is also available for download from the NHS Information Centre's website.
Primary percutaneous coronary intervention (PPCI) 30-day mortality funnel. Funnel plot of percentage mortality at 30 days for each PCI centre after PPCI for all patients treated in 2009 including those presenting in cardiogenic shock or requiring periprocedural ventilation.
Survival after primary percutaneous coronary intervention (PPCI) in patients aged >80 years. Kaplan–Meier survival curves for patients being treated by PPCI, according to age over 80 years. For this analysis, patients treated in 2008 were examined to allow appropriate numbers at risk at 1 year follow-up.
Wider collaboration
BCIS also works with a variety of professional bodies, policy-makers and government bodies to help improve quality of care. Collaboration with the Myocardial Ischaemia National Audit Project (MINAP)12 is also taking place, particularly with regard to those patients presenting with acute coronary syndromes. A joint dataset has been agreed and implementation is aimed at reducing some aspects of duplicate entry into these two registries.
Research
The research potential for this increasingly large and complete dataset has yet to be fully realised. A BCIS Data Applications Review group was established in 2010 to oversee and facilitate access to the data for research groups, and the data sharing agreement which contains the data governance framework can be accessed on the BCIS website. As of March 2011, eight projects have been approved. There are plans to expand the research potential of this dataset by cross-links with the other cardiac audits also hosted by CCAD under the umbrella of the newly established National Institute for Clinical Outcomes Research organisation at UCL.
Strengths and weaknesses of data
Despite the lack of a statutory requirement to participate, there is immense professional support for the PCI data collection process, and as a result the dataset includes an almost complete collection of all PCI procedures performed UK-wide. It therefore includes treatment of all patients, low and high risk alike, and contains a very large number of procedures. Mortality tracking in England and Wales is particularly robust. The audit has provided a high-quality national benchmarking process against which clinicians can assess the quality of their treatment and provision of care. The ability to assess risk-adjusted outcomes has been particularly important as PCI units increasingly treat patients presenting with ST elevation MI by primary PCI. This has resulted in an increase in crude mortality following PCI, and without case-mix adjustment incorrect conclusions might be drawn from this finding. The audit has confirmed that the observed increase is only due to case-mix changes, with no increased in adverse outcomes from each of the clinical subgroups themselves.
The importance of good clinically derived data has been highlighted by the recent difficulties of a commercial organisation that uses administrative data to provide assessments of hospital performance. This administrative dataset (Hospital Episode Statistics) cannot distinguish between patients presenting with ST elevation MI (with a predicted mortality of about 5%), and those presenting with non-ST elevation MI (with a predicted mortality of about 1%). After several false alerts the organisation had to suspend this aspect of its attempts to assess hospital performance.
Despite its strengths, the BCIS data collection shares the weaknesses of many other national registry programmes. There is always a balance between the size of the dataset and the willingness and ability to collect it accurately. In addition, changes to the dataset risk losing its collection from some units whose ability to modify data collection software is limited. Encouraging some private PCI units to participate in data collection has been difficult as there is less professional ‘ownership’ of activity in these centres. Strictly speaking funding for the BCIS–CCAD audit is for England and Wales, though the strong professional interest in collecting these data has meant that almost all units in Scotland and Northern Ireland do also participate.
While the assessment of mortality is very strong, and particularly appropriate for interventions aimed at reducing mortality (such as PCI for acute coronary syndromes) some other aspects of patient outcome are not assessed. In measuring outcomes following PCI for stable syndromes, an assessment of quality of life and angina status would be valuable but is not currently possible. However, planned linkage with the other CCAD datasets will allow determination of future events, such as recurrent MI and need for additional revascularisation.
The data collection process still requires a local copy of Lotus Notes software, and online analyses can only be performed from that computer. There are plans to address this by moving to a web-based system of data entry and upload, and possibly moving away from a Lotus Notes platform.
Apart from range checks and assessment of internal consistency and completeness, no independent data validation is performed. While mortality data are robust, the cause of death is not currently available from the NHS number linkage, and all other outcomes and complications are self-reported.
Future directions
As the treatment of patients with acute coronary syndromes increasingly involves the use of PCI, the BCIS and MINAP data collection processes are being brought closer together. The joint dataset (developed but not yet implemented) does make definitions consistent, but attempts to reduce duplicate data entry are hindered by the multiplicity of software platforms used by interventional units to collect the data. Web-based interfaces may deal with some of these issues.
The risk adjustment models used for BCIS need to be updated and work on new models is about to start. A number of research projects based on the dataset have recently been approved and should provide valuable insight into other important clinical questions.
The unique NHS number supports mortality tracking and can also be used for longitudinal tracking and to cross-link the dataset with other datasets that describe other aspects of the patient's treatments and outcomes. These datasets include the MINAP, which records patients presenting with acute coronary syndromes, and that of the Society for Cardiothoracic Surgery in Great Britain and Ireland, which captures all cardiac surgical procedures. This opens up the possibility of tracking the outcomes of a patient's treatment over time across a number of different subspecialties.
Conclusions
In parallel with enormous advances in the technique of PCI, the BCIS data collection process has evolved from a survey to a sophisticated method to assess and thus promote high quality of care for patients treated by PCI. Further important developments in this data collection and analysis process are underway.
Footnotes
Competing interests None.
Provenance and peer review Commissioned; externally peer reviewed.