Objectives To provide an accurate adjustment for mortality in a benchmark, developing a risk prediction model from its own dataset is mandatory. We aimed to develop and validate a risk model predicting in-hospital mortality in a broad spectrum of Japanese patients after percutaneous coronary intervention (PCI).
Design A retrospective cohort study was conducted.
Setting The Japanese-PCI (J-PCI) registry includes a nationally representative retrospective sample of patients who underwent PCI and covers approximately 88% of all PCIs in Japan.
Participants Overall, 669 181 patients who underwent PCI between January 2014 and December 2016 in 1018 institutes.
Main outcome measures In-hospital death.
Results The study population (n=669 181; mean (SD) age, 70.1(11.0) years; women, 24.0%) was divided into two groups: 50% of the sample was used for model derivation (n=334 591), while the remaining 50% was used for model validation (n=334 590). Using the derivation cohort, both ‘full’ and ‘preprocedure’ risk models were developed using logistic regression analysis. Using the validation cohort, the developed risk models were internally validated. The in-hospital mortality rate was 0.7%. The preprocedure model included age, sex, clinical presentation, previous PCI, previous coronary artery bypass grafting, hypertension, dyslipidaemia, smoking, renal dysfunction, dialysis, peripheral vascular disease, previous heart failure and cardiogenic shock. Angiographic information, such as the number of diseased vessel and location of the target lesion, was also included in the full model. Both models performed well in the entire validation cohort (C-indexes: 0.929 and 0.926 for full and preprocedure models, respectively) and among prespecified subgroups with good calibration, although both models underestimated the risk of mortality in high-risk patients with the elective procedure.
Conclusions These simple models from a nationwide J-PCI registry, which is easily applicable in clinical practice and readily available directly at the patients’ presentation, are valid tools for preprocedural risk stratification of patients undergoing PCI in contemporary Japanese practice.
- risk model
- percutaneous coronary intervention
- In-hospital mortality
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
We developed and internally validated a risk model predicting in-hospital mortality using a nationally representative sample of >650 000 patients who underwent percutaneous coronary intervention (PCI) in Japan to provide risk-adjusted mortality in a benchmark reports.
For clinical use, the ‘preprocedure’ risk model that did not require procedure-related variables and a simplified integer risk score based on the preprocedure model were also developed and internally validated, aiding in stratifying patients at risk of death preprocedurally.
Several variables selected in other risk models were not included in the Japanese-PCI registry and could further refine risk prediction.
The models underestimated the risk of mortality and could not provide an accurate estimation in the elective procedure, mainly owing to the small number of in-hospital deaths.
In Japan, percutaneous coronary interventions (PCIs) are widely available for both elective and acute settings, and >200 000 procedures are performed annually in >1000 hospitals. On the advent of technology and technique in interventional cardiology, the current in-hospital mortality rate is about 1%.1 2 However, PCI is applied onto a wide spectrum of coronary artery disease, and its indication varies substantially across hospitals, especially between low-volume and high-volume centres, suggesting an opportunity for improvement.1 2 Adjusted risk mortality is a core component of performance measures and used even for public reporting3; therefore, adjustment for the variation in patient risk across hospitals is critically important to enable an accurate assessment of each hospital’s performance and opportunity to improve.
A number of risk prediction models for short-term mortality after PCI were developed within the last decade.4–9 However, almost all of them share certain limitations: (1) they were not validated, (2) they were derived from small studies and (3) the parameters used cannot be easily assessed directly in the catheterisation laboratory. Furthermore, given the favourable patient and laboratory profile (eg, lesser risk of bleeding) for percutaneous procedures among the East Asian patients, PCI is performed onto significantly complex and ‘sicker’ group of patients in Japan. Hence, it remains a challenge for clinicians to use these existing models for the risk adjustment due to the differences in biological responses to medication or procedure among ethnicities and practice pattern among counties.10–12
The risk prediction model from its own dataset is mandatory, especially when considering the adjustment for mortality in a benchmark, eventually leading to improve quality of care in PCIs. Using the nationwide Japanese-PCI registry (J-PCI), this study aimed (1) to develop two separate PCI risk prediction models for estimating in-hospital mortality that can be applied to both acute and elective clinical settings, one based on all available variables including both preprocedure and procedure-related variables, and the other based on preprocedural variables only; (2) to internally validate the developed risk models and (3) to develop and validate a clinician-friendly simplified integer risk score to enable pre-PCI risk stratification in clinical care.
We analysed patients who underwent PCI between January 2014 and December 2016 (n=680 947) and were registered in the J-PCI. The J-PCI was established in 2007 and is a prospective Japanese nationwide multicentre registry of the Japanese Association of Cardiovascular Intervention and Therapeutics (CVIT) designed to collect clinical variables and in-hospital outcome data on PCI patients.1 13–15 Variables regarding patient background, clinical presentation, angiographic and procedure-related information, and in-hospital outcomes were collected from each patient. The CVIT registry subcommittee designed the software for the web-based data collection system, and each data manager in the participating hospitals submits data through this system annually. As registration in the J-PCI database is mandatory for the application for board certification and renewal, although participation in the J-PCI is voluntary, there is a high degree of data completeness. According to the annual reports of the Japanese Registry of All Cardiac and Vascular Disease, 773 359 PCIs (209 920 PCIs for acute indications and 563 439 PCIs for non-acute indications) were performed during the current study period (http://www.j-circ.or.jp/jittai_chosa/, accessed on 14 February2018). Since we included a total of 680 947 PCIs, approximately 88% of all procedures in Japan were estimated to be registered in our registry. The accuracy of submitted data is maintained by data auditing (20 institutions annually), which is operated by members of the CVIT registry subcommittee.
Written informed consent was waived due to the retrospective nature of the study.
The primary outcome measure of the J-PCI analysis was in-hospital mortality, which was defined as all-cause mortality during hospitalisation. Cardiogenic shock was defined as a sustained episode of systolic blood pressure <80 mm Hg and/or a cardiac index of <1.8 L/min/m2 (regardless of the measurement methods) despite the maximum treatment determined to be secondary to cardiac dysfunction, and/or the need for parenteral inotropic or vasopressor agents or mechanical support, including an intra-aortic balloon pump to maintain blood pressure and cardiac index above the specified levels within 24 hours prior to the initiation of PCI. In this registry, renal dysfunction was defined as the presence of proteinuria including microalbuminuria, serum creatinine≥1.3 mg/dL or estimated glomerular filtration rate ≤60 mL/min/1.73 m2 according to the Japanese Society of Nephrology guidelines.16 The other definitions of J-PCI variables are available online (http://www.cvit.jp/registry/jpci_definition.pdf, accessed on 14 February 2018).
The study cohort was randomly divided in a 1:1 ratio into derivation (n=334 591) and validation (n=334 590) cohorts. The demographic data and clinical patient characteristics were summarised by enrolment year, and the data were presented as mean±SD or as proportion (%), depending on the variables. Trend tests were performed using the Cochran-Armitage test for trend.
All variables in table 1 except for procedure details were listed as initial candidate variables. Age was considered as a continuous variable, while the others were considered as dichotomous variables. With regard to diagnosis, unstable angina, ST-elevation myocardial infarction (STEMI) and non-STEMI (NSTEMI) were included in the list of candidate variables based on clinical judgement. After accounting for multicollinearity among candidate variables, final candidates for inclusion in the multivariable model were selected as follows: age, sex, clinical presentation (unstable angina, STEMI and NSTEMI), previous PCI, previous coronary artery bypass grafting, previous myocardial infarction (MI), diabetes, hypertension, dyslipidaemia, smoker, renal dysfunction (renal dysfunction without requiring dialysis and renal dysfunction on dialysis), chronic lung disease, peripheral vascular disease, previous heart failure, cardiogenic shock within 24 hours prior to the procedure, number of diseased vessel (two-vessel disease, three-vessel disease and left main trunk lesion) and involvement of the left anterior descending artery or left main trunk as a target lesion. Age was considered as a continuous variable, while the others were considered as dichotomous variables. Multivariable logistic regression with a backward selection method was then performed to identify independent predictors of in-hospital death. Package ‘step’ in R was used for backward selection, in which the elimination process was based on Akaike’s information criterion.
From the derivation cohort, we developed two separate PCI risk prediction models for estimating in-hospital mortality that can be applied to both acute and elective clinical settings: one based on all available variables including both preprocedure and procedure-related variables (full model) and the other based on preprocedural variables only (preprocedure model). In addition, an integer score was assigned to each variable selected in the preprocedure model in proportion to the estimated regression coefficient (0.23) defined from an incremental risk ratio by age (10 years). The regression coefficient for each level of every risk factor was subsequently divided by this reference value (0.23) to compute its weights for the risk score.17 18
Using the validation cohort, the performance of the developed models was examined by using C-statistics to assess discrimination and by demonstrating calibration plots to evaluate calibration. After assessing discrimination and calibration across the entire validation cohort, we examined the model performance for clinically distinct patient populations. These included acute coronary syndrome (ACS) versus non-ACS, female versus male and advanced age (>70) versus younger age (≤70).
All variables had less than 3% missingness. To account for missing data, single imputation was used for each variable; ‘STEMI’ for type of MI, ‘transradial’ for access site and ‘no’ for others. Data were analysed using R, V.3.3.3 (R Foundation for Statistical Computing, Vienna, Austria). All p values were two sided, and p values <0.05 were considered significant.
Patient and public involvement
Patients and public were not involved in this study.
After excluding patients with missing data on age and/or sex (n=3410), outside the age range of 20–100 years (n=281), with missing data on outcomes (n=647), and with missing data on diagnosis (n=43), 669 181 patients from 1018 institutes were included in the analysis (figure 1). The baseline demographic data, lesion characteristics and procedure details are summarised in table 1, while in-hospital outcomes are shown in table 2, stratified by enrolment year. About 40% of PCIs were performed in patients with ACS, while 3% of patients developed cardiogenic shock within 24 hours prior to PCI. During the index hospitalisation, 4788 (0.7%) patients died after PCI. The bivariate relationships between patient characteristics and in-hospital mortality are shown in the online supplementary tables (online supplementary eTable 1 for the derivation cohort and online supplementary eTable 2 for the validation cohort).
Between 2014 and 2016, demographics and risk factors of patients undergoing PCI did not change in terms of clinically meaningful difference. With regard to the diagnosis at the time of PCI, the proportion of patients with silent ischaemic heart disease slightly increased from 15.9% in 2014 to 17.4% in 2016, whereas the proportion of patients who underwent PCIs as treatment for STEMI decreased from 18.6% to 17.3% (p values for trend <0.001). Procedures significantly changed during the study period. The rate of transradial approach increased significantly from 60.6% to 66.9%, in parallel with the decreasing trend of transfemoral approach (p values for trend <0.001). Similarly, the use of a drug-eluting balloon and drug-eluting stent showed an increasing trend, whereas the use of bare metal stent decreased significantly (all p values for trend <0.001). In patients with STEMI, the use of thrombus aspiration remained stable during the study period, although its trend demonstrated a significant increase (p value for trend=0.011).
Development of full and preprocedure models
Table 3 shows the results of the multivariable logistic regression analyses. Preprocedure variables selected in the final models were age, sex (female), STEMI, NSTEMI, unstable angina, previous PCI, previous coronary artery bypass grafting, hypertension, dyslipidaemia, smoker, renal dysfunction (renal dysfunction without requiring dialysis and renal dysfunction on dialysis), peripheral vascular disease, previous heart failure and cardiogenic shock within 24 hours prior to the procedure. All angiographic information, such as three-vessel disease, left main trunk lesion and involvement of left anterior descending artery or left main trunk as a target lesion, was retained in the full model.
Discrimination and calibration performance of full and preprocedure models
In the entire validation cohort, discrimination performance of the full model was excellent, with C-index of 0.929 (95% CI 0.924 to 0.935). The exclusion of the angiographic information from the full model resulted in only a slight decrement in the overall model discriminatory performance, and the C-index of the preprocedure model was 0.926 (95% CI 0.920 to 0.931). Both full and preprocedure models performed well even in the prespecified subgroups, and C-indexes ranged from 0.829 to 0.933 for the full model and from 0.820 to 0.926 for the preprocedure model, respectively (online supplementary eTable 3). There was excellent calibration performance of both models in the entire validation cohort (online supplementary eFigure 1. The calibration performance was also acceptable in the prespecified subgroups, although both models were likely to underestimate the risk of mortality in high-risk patients who underwent the elective procedure (online supplementary eFigure 2).
Development and validation of integer simple risk score
The integer points for each variable derived from beta coefficients in the multivariable models are listed in figure 2. The highest number of points was 12 (cardiogenic shock), whereas the lowest was −3 (dyslipidaemia). The possible total points ranged from −6 to 43. The agreements between the observed and estimated risks of mortality with the developed risk-scoring methods were assessed across five groups in the entire validation cohort (figure 2). Overall, the agreements were acceptable, although the integer score underestimated mortality risk in the higher risk patients, where the observed and predicted mortalities were 1.2% and 0.9% in the second highest risk group, and 6.9% and 6.6% in the highest risk group, respectively.
Using the nationally representative cohort of Japanese patients who underwent PCI with >660 000 procedures from 1018 institutes, we developed and internally validated risk models predicting in-hospital mortality in patients after PCI. The final full model included age, sex, clinical presentation, previous PCI, previous coronary artery bypass grafting, hypertension, dyslipidaemia, smoker, renal dysfunction, peripheral vascular disease, previous heart failure, cardiogenic shock within 24 hours prior to the procedure and angiographic information. The model performed very well in an independent validation cohort, as well as in various subgroups stratified by age, sex and clinical presentation. The model performance was retained even after excluding angiographic information, and a simplified integer risk score based on the preprocedure model also performed well. These developed risk prediction models provided the risk-adjusted mortality in benchmark report, compared an institution’s performance with that of national experience and stratified patients who were at risk preprocedurally.
For the purpose of risk adjustment for benchmarking and comparing institutional performances, rigorous adjustment is warranted. Significant differences were observed in patient demographics, biological response to medication or procedure and practice pattern in PCI between Japan and the Western countries.10 19 For example, patients in Japan are likely to have more advanced age, have lower body mass weight and experience bleeding complications than those in Western countries. Furthermore, the proportion of elective PCIs performed in Japan is greater than those in other countries. With regard to practice patterns, thrombus aspiration is still widely performed as treatment for patients with STEMI despite the recent downgrade of the relevant recommendations in the USA and European guidelines.20 21 In spite of these significant differences, some risk models or clinical scores that are derived from the Western countries have been externally validated and were proven useful in Japanese patients.10 22 23 Kohsaka et al applied the in-hospital mortality risk model derived from the National Cardiovascular Data Registry CathPCI Registry, which is the nationwide PCI registry in the USA, to the regional PCI registry from 15 Japanese institutes near Tokyo area and demonstrated that the US model was clinically applicable in Japanese population with excellent discrimination and acceptable calibration performances.10 However, given its limited calibration performance in high-risk patients, the risk model derived from its own dataset can provide an accurate adjusted mortality.
A number of risk models were developed from various registries to predict risk of mortality in patients undergoing PCI. Individual components varied among risk models, but several components, such as age, clinical presentation or urgency, renal dysfunction and cardiogenic shock, were consistently included in the model.4–9 Our developed risk models also allied with the previous models and included these core variables. Although angiographic information also served as a component of our full model, its incremental predictive value may be limited, given the sufficient performance of the preprocedure model. This finding is consistent with those of previous studies, and Peterson et al showed that in-hospital mortality was driven primarily by pre-existing patient comorbidities and makers of clinical instability.4 It allows patients and physicians to obtain a reasonable estimate of mortality preprocedurally.
Few studies reported the risk model predicting mortality after PCI derived from Japanese population. Tanaka et al developed the Kyoto model using the CREDO-Kyoto Registry, which is one of the most established PCI registries in Japan.24 However, the Kyoto model intended to predict long-term mortality after revascularisation and was not applicable to patients undergoing PCI for MI, because the dataset excluded patients who experienced acute MI 1 week prior to the index procedure. Furthermore, the dataset included patients who underwent revascularisation between 2000 and 2002 and could not be directly applied to current practice. Our study is the first to report the risk model predicting in-hospital mortality in Japan using the nationwide PCI registry in the contemporary PCI practice. Thus, the Japanese cardiovascular society can use this model as an instrument for the pre-PCI risk stratification and to distribute the risk-adjusted mortality in benchmark reports, leading to improve quality of care and clinical outcomes in PCI.
Several variables, such as prior PCI, hypertension, dyslipidaemia and current smoker, were identified as independent factors associated with a reduction in mortality. Although some controversies remain and they may be attributed to the confounding factors, their associations with improved mortality were consistent with those of previous studies.4 5 25–27 Smoker’s paradox is a well-known phenomenon and has been reported in patients undergoing PCI.26 27 Recent data have demonstrated a specific pharmacodynamic enhancement of platelet inhibition by clopidogrel among smokers,28 which may hold important implications for patients treated with PCI, given that clopidogrel is still widely used for patients treated with PCI in Japan.19 The protective effect of dyslipidaemia may be related to statin use before PCI, which our database does not collect as a separate variable. The diagnosis for dyslipidaemia may have acted as a proxy for statin use, which has been shown to reduce the risk of postprocedural MI and mortality in previous meta-analysis studies. Meta-analysis demonstrated that statin administration before PCI significantly reduced the risk of postprocedural MI, resulting in the reduction in mortality.29 30 The impact of pre-existing hypertension on in-hospital mortality remains controversial; however, a recent study from the Acute Myocardial Infarction in Switzerland Plus Registry reported that pre-existing hypertension was associated with increased risk of in-hospital mortality in patients presenting with ACS.31 Although the exact mechanism is unknown, pretreatment of hypertension with beta-blockers and renin–angiotensin–aldosterone inhibitors could have led to a favourable outcome. Previous studies have consistently demonstrated the protective effect of prior PCI in terms of reduction in in-hospital mortality in patients undergoing PCI.4 5 This may be attributed to the fact that patients with prior PCI were more likely to undergo subsequent ‘low-risk’ PCI, such as staged PCI. In contrast, those with prior CABG were more likely to undergo subsequent ‘high-risk’ PCI, such as PCI for multivessel lesion, left main trunk lesion and severely calcified lesion, resulting in an increased risk of in-hospital mortality.
This study had several limitations. First, J-PCI is a voluntary registry; the contributing hospitals may have a larger procedure volume than the average institutes in Japan. However, J-PCI included >85% of all PCIs performed in Japan and assured a high participation rate by linking to the application for board certification and renewal. Second, improving the quality of the database is a continuing issue in J-PCI. With regard to data accuracy, not only is data auditing important but also the education of persons inputting data at each site as it enables adherence to correct data definitions. Third, the candidate variables for the risk models were limited to those available in J-PCI. Actually, several variables selected in other risk models were not included in our dataset and may further refine risk prediction.
Fourth, the models underestimated the risk of mortality in high-risk patients who underwent the elective procedure, although their discrimination and calibration performances were acceptable. Although the exact reason remains unknown, lack of granularity for variables representing the complexity of PCI, such as SYNTAX score, chronic total occlusion and severity of calcification, may have led to the suboptimal calibration performance in this particular population. Since the developed risk models cover patients with a broad range of mortality risk, providing an accurate estimation for all patients is challenging, especially in low-risk patients, mainly owing to the small number of in-hospital deaths. Further studies may be required to generate separate models for patients with specific clinical presentations to improve risk prediction.
Fifth, the models have been validated using only J-PCI data; it has not been validated on an external dataset. Finally, J-PCI captured in-hospital outcomes that occurred only during hospitalisations where PCIs were performed; therefore, data on the subsequent outcomes when the patients were transferred to other hospitals and facilities were lacking. This may have led to the underestimation of mortality in J-PCI. We have recently launched a project involving selected institutes participating in J-PCI, aiming to obtain long-term follow-up data, which may provide additional insights.
Using the nationwide Japanese-PCI registry, we have developed and internally validated the risk models for predicting in-hospital mortality after PCI. The models performed very well in a broad spectrum of patients undergoing PCI as well as in clinically important subgroups. These models are expected to help improve the rate of patient mortality after PCI by facilitating pre-PCI risk stratification, improving hospital quality assessment and providing an accurate adjusted risk in benchmark reports.
The authors appreciate the contributions of all the investigators and staff involved in the J-PCI registry.
Contributors TI and SK had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: TI and SK. Acquisition, analysis or interpretation of data: TI, SK, KY, HI, TA, SU, KK, HK, HM and MN. Drafting of the manuscript: TI and SK. Critical revision of the manuscript for important intellectual content: SK, KY, HI, TA, SU, KK, HK, HM and MN. Statistical analysis: TI, HK and HM. Obtained funding: MN. Administrative, technical or material support: SK, TA and MN. Study supervision: SK, HK, HM and MN.
Funding This study was funded by the Grant-in-Aid from Scientific Research from Japan Agency for Medical Research and Development (Grant No. 17ek0210097h000) and the Japan Society for the Promotion of Science (Grant No. 16KK0186 and 16H05215). The J-PCI registry is a registry led and supported by the Japanese Association of Cardiovascular Intervention and Therapeutics.
Competing interests TI has a research grant from Boston Scientific. SK reports investigator-initiated grant funding from Bayer and Daiichi Sankyo, and personal fees from Bayer and Bristol‐Myers Squibb. HI receives lecture fees from Astellas Pharma, AstraZeneca, Bayer, Daiichi Sankyo and MSD. TA receives lecture fees from Astellas Pharma, AstraZeneca, Bayer, Daiichi Sankyo and Bristol‐Myers Squibb. MN receives remuneration for lecture from Daiichi Sankyo, Sanofi, Bayer, Nippon Boehringer Ingelheim, Bristol‐Myers Squibb, Terumo, Japan Lifeline, Abbott, Boston Scientific, Medtronic and Nipro, and investigator‐initiated grant funding from Sanofi and Daiichi Sankyo.
Ethics approval The study protocol of the J-PCI registry was approved by the Institutional Review Board Committee at the Network for Promotion of Clinical Studies (a specified non-profit organisation affiliated with Osaka University Graduate School of Medicine, Osaka, Japan) and complied with the principles contained within the Declaration of Helsinki.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement In addition to data in this manuscript, all data in the J-PCI registry are available to investigators from >1000 sites participating in the J-PCI registry. The dataset is stored in servers of the National Clinical Database (NCD). Local investigators can submit proposal for data use for research purposes.
Collaborators Member of CVIT Scientific Committee: Kazushige Kadota (Kurashiki Central Hospital), Nobuo Shiode (Akane Foundation Tsuchiya General Hospital), Nobuhiro Tanaka (Tokyo Medical University), Tetsuya Amano (Aichi Medical University), Shiro Uemura (Kawasaki Medical School), Takashi Akasaka (Wakayama Medical University), Yoshihiro Morino (Iwate Medical University), Kenshi Fujii (Sakurabashi Watanabe Hospital), Hiroshi Hikichi (Saga University).
Patient consent for publication Not required.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.