Objective Evidence-based performance measures have been increasingly used to evaluate hospital quality of stroke care, but their impact on stroke outcomes has not been verified. We aimed to evaluate the correlations between hospital performance measures and outcomes among patients with acute ischaemic stroke in a Chinese population.
Methods Data were derived from a prospective cohort, which included 120 hospitals participating in the China National Stroke Registry between September 2007 and August 2008. Adherence to nine evidence-based performance measures was examined, and the composite score of hospital performance measures was calculated. The primary stroke outcomes were hospital-level, 30-day and 1-year risk-standardised mortality (RSM). Associations of individual performance measures and composite score with stroke outcomes were assessed using Spearman correlation coefficients.
Results One hundred and twenty hospitals that recruited 12 027 patients with ischaemic stroke were included in our analysis. Among 12 027 patients, 61.59% were men, and the median age was 67 years. The overall composite score of performance measures was 63.3%. The correlation coefficients between individual performance measures ranged widely from 0.01 to 0.66. No association was observed between the composite score and 30-day RSM. The composite score was modestly associated with 1-year RSM (Spearman correlation coefficient, 0.34; p<0.05). The composite score explained only 2.53% and 10.18% of hospital-level variation in 30-day and 1-year RSM for patients with acute stroke.
Conclusions Adherence to evidence-based performance measures for acute ischaemic stroke was suboptimal in China. There were various correlations among hospital individual performance measures. The hospital performance measures had no correlations with 30-day RSM rate and modest correlations with 1-year RSM rate.
- ischemic stroke
- quality and outcomes
- hospital performance
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
As the largest stroke database in China, hospitals participating in the China National Stroke Registry encompassed different regions and had good representativeness.
We used the newer risk-standardised mortality method based on hierarchical models to account for variation in case numbers across hospitals or for intrahospital clustering effects.
Our study was observational and non-randomised.
The association between hospital performance measures and stroke outcomes did not necessarily prove causality.
Stroke remains to be the leading cause of mortality in China and was responsible for around 1.9 million deaths in 2013.1 Although evidence-based guidelines for stroke and transient ischaemic attack (TIA) have been developed and updated over time,2–4 there were variations and deficiencies in how these guidelines were applied.5–7 To reduce the national stroke burden, several national organisations have developed a set of evidence-based hospital performance measures to quantify and promote quality of stroke care.7–9
Evidence-based hospital performance measures evaluated the structure, process and outcome of stroke care, and provided a metric that could be tracked, reported and improved.10 Evidence-based hospital performance measures have also been increasingly used for certification of stroke centres,11 public reporting12 and remuneration for performance.13 Although there has been substantial improvement in the quality of stroke care in China, gaps still exist in adherence to the evidence-based hospital performance measures. In addition, although measuring outcomes after stroke has important policy implications,14 there has been limited evidence demonstrating the association between adherence to performance measures and better patient outcomes.15
The China National Stroke Registry (CNSR) was a national, hospital-based, prospective stroke registry which encompassed different regions and had good representativeness.16 The CNSR thoroughly investigated all indicators related to the quality of stroke care, including evidence-based acute performance measures and performance measures at discharge. Thus, it could reflect the real situation of stroke care in China. Using data from the CNSR, we sought to determine whether evidence-based hospital performance measures were associated with short-term and long-term outcomes in patients with acute ischaemic stroke in China. Specifically, we (1) reported the hospital variations in adherence to evidence-based performance measures, (2) assessed the correlations between each individual performance measure, and (3) quantified the associations between the performance measures and hospital-level, risk-standardised, 30-day and 1-year mortalities among patients with acute ischaemic stroke in China.
The CNSR was the first nationwide, hospital-based, prospective stroke registry of quality assessment in China, conducted between September 2007 and August 2008. It was sponsored by the Ministry of Health of the People’s Republic of China to establish a national stroke database to evaluate the quality of stroke care and outcomes. The study design of CNSR has been described previously.16 Briefly, the criteria used for hospital selection included the following: (1) having at least one stroke neurologist, (2) at least two sites were included from each of the 31 provinces and municipalities in mainland China, (3) commitment to participate voluntarily and (4) ability to conduct research.16 17 The CNSR recruited consecutive patients older than 18 years and diagnosed with ischaemic stroke, TIA, intracerebral haemorrhage or subarachnoid haemorrhage within 14 days of the index event. We collected patients’ demographics, adherence to performance measures during hospitalisation, and status at discharge, 90-day, 6-month and 1-year follow-up after symptom onset. Written informed consent was acquired from the patient or the legally authorised representative.
The characteristics of the hospitals were surveyed. According to the annual report on health statistics of China,18 hospitals were divided into three regions: eastern, central and western. Academic status was defined as being affiliated with a specific university or not. Hospital bed size was the number of total inpatient beds per site. Annual stroke discharge was the number of discharged patients diagnosed with stroke per year of each participating hospital.
Trained research coordinators at each hospital reviewed medical records daily to identify, obtain informed consent and enrol consecutive patients in accordance with the procedures indicated in a standard data collection manual. The key variables in the CNSR were assessed, including (1) demographic characteristics and medical history; (2) prestroke modified Rankin Scale (mRS) and National Institutes of Health Stroke Scale (NIHSS) scores on admission, vascular risk factors, and clinical care during hospitalisation; and (3) 30-day and 1-year all-cause mortalities after stroke. Patients or their authorised caregivers were contacted via telephone by trained research personnel with standardised scripts for follow-up at 90 days, 6 months and 1 year after stroke symptom onset.16 Specific death events and dates were recorded in detail, and 30-day and 1-year all-cause mortalities after stroke were confirmed.
We evaluated nine hospital performance measures, including seven ‘core’ measures for acute ischaemic stroke as recommended by the American Heart Association/American Stroke Association (AHA/ASA) Get With The Guidelines-Stroke (GWTG-Stroke)7 and two additional evidence-based secondary prevention metrics.4 These seven core performance measures included (1) intravenous tissue-type plasminogen activator in patients who arrived within 2 hours after symptom onset and treated within 3 hours (intravenous recombinant tissue-type plasminogen activator (rt-PA) <2 hours); (2) antithrombotic medication within 48 hours of admission (early antithrombotics); (3) deep vein thrombosis (DVT) prophylaxis within 48 hours of admission for non-ambulatory patients (DVT prophylaxis); (4) antithrombotic medication; (5) anticoagulation for atrial fibrillation (AF); (6) medications for lowering low-density lipoprotein (LDL) ≥100 mg/dL; and (7) smoking cessation. The two additional performance measures were antihypertensive medication for patients with hypertension and hypoglycaemia medication for diabetes mellitus. Details of the performance measures are described in table 1.
Adherence to hospital-level individual performance measures was calculated as the proportion of patients who received the indicated care among all the patients who were eligible for the indicated care. The overall hospital performance measures were measured as the composite score, which was calculated by the total number of interventions actually performed among all eligible patients at a hospital, divided by the total number of recommended interventions in all eligible patients.19 Although the composite score was analysed as a continuous variable, hospitals were also divided, for descriptive purposes, into quartiles based on this continuous variable. To ensure the stability of the measure, hospitals with fewer than 20 patients in the denominator of any measures were excluded.
Risk-standardised mortality rates
Risk-standardised mortality (RSM) rates were calculated using a multivariate hierarchical regression model in accordance with the AHA/ASA recommendations for risk adjustment of ischaemic stroke outcomes to compare hospital performance.15 The first level of the hierarchical model included patient characteristics. These patient variables were divided into three categories: (1) demographics and clinical features: age, gender, health insurance type (urban basic medical insurance schemes for urban and governmental employees and urban residents, new rural cooperative medical schemes for rural residents, commercial insurance, and self-payment), transport to hospital by emergency medical service and NIHSS score on admission; (2) vascular risk factors: hypertension, diabetes mellitus, dyslipidaemia, AF, coronary artery disease, previous myocardial infarction, congestive heart failure, valvular heart disease, history of stroke/TIA, peripheral vascular disease, current smoking and excessive alcohol consumption; and (3) other pre-existing comorbid conditions: chronic obstructive pulmonary disease, hepatic cirrhosis, peptic ulcer, previous gastrointestinal bleeding, Alzheimer’s disease/dementia, cancer, DVT/pulmonary embolus, renal dialysis, prestroke dependence (mRS ≥3) and blood glucose on admission. The second level included hospital-specific random intercepts that allow for different baseline mortalities between hospitals. The hospital RSM rates were obtained as the ratio of ‘predicted’ to expected mortality, multiplied by the unadjusted mortalities for the total population.20 21 The expected outcome for each hospital was the number of deaths expected at the hospital if the hospital’s patients were treated at a ‘reference’ hospital. The predicted hospital-level outcome was the number of expected mortalities at the ‘specific’ hospital and not at a reference hospital.
Numbers (percentages) were used to describe categorical variables, and median values with IQRs were reported for continuous variables. The correlations of individual performance measures, as well as performance measures and stroke outcomes, were evaluated using Spearman correlation coefficients. Bonferroni correction was performed in multiple comparisons. The interclass correlation (ICC) of 30-day and 1-year RSM rates was calculated. The relevant correlation coefficients and the percentage of the hospital-specific variations in RSM rates were explained using the squares of the correlation coefficients as indicators of the strength of the associations.22
All tests were two-tailed, and statistical significance was determined at the α level of 0.05 in univariate and multivariate analyses. Statistical analysis was performed using SAS V.9.3.
Patient and public involvement
No patients were involved in the planning of this study. Patients were recruited via the doctors and investigators at local hospitals in China. Once the results of the study have been published in a peer-reviewed journal, a summary of the main study findings will be distributed to the patients in Chinese language via the local doctors and investigators.
Study hospitals and patient baseline characteristics
Among 14 526 eligible patients diagnosed with acute ischaemic stroke from 132 hospitals throughout China, complete 1-year follow-up information was available for 12 173 patients (83.8%). We excluded 12 hospitals with fewer than 20 patients in the denominator of any measures. Finally, 120 hospitals that recruited 12 027 patients with ischaemic stroke were included in our analysis. Hospital and patient baseline characteristics are described in table 2. Among 120 hospitals, 62 (51.67%) hospitals were teaching hospitals and 70 (58.33%) were from eastern regions. The median hospital bed size was 1200 (IQR, 700–1861) and the annual stroke volume was 430 (IQR, 310–601). Among 12 027 patients, 7407 (61.59%) were men, and the median age was 67 (IQR, 57–75) years. Common vascular risk factors included hypertension, previous stroke/TIA, smoking history and diabetes. The median NIHSS score was 4 (IQR, 2–9).
Variability in performance measures
The overall composite score was 63.3% among the 120 hospitals. However, the composite score for each hospital varied considerably among the CNSR (table 3). The hospitals in the highest quartile (quartile 4) had a median (IQR) composite score of 85.1% (82.2%–89.9%), compared with 50.9% (44.2%–58.1%)%) for those in the lowest quartile (quartile 1).
In table 3, we showed hospital rates (median and IQR) for each process measure, as well as 30-day and 1-year RSM rates. For acute interventions, the early antithrombotics showed the lowest degree of variance, although the difference between the first and third quartiles remained significant. In contrast, there were twofold to threefold differences in the use of intravenous rt-PA <2 hours, DVT prophylaxis, anticoagulant for AF and lipid-lowering medication for LDL ≥100 mg/dL.
Correlations of performance measures
The correlations between two individual hospital performance measures varied widely (table 4). After Bonferroni correction (p<0.001), we found moderate correlations (Spearman correlation coefficient ≥0.40; p<0.001 for all) for all pairwise comparisons between antithrombotic at discharge and lipid-lowering drugs for LDL ≥100 mg/dL, antihypertensive medication and antidiabetic medication, and between antidiabetic medication and antihypertensive medication and smoking cessation at discharge. Other hospital performance measures had modest correlations or no correlations with each other (Spearman correlation coefficient <0.40 for all).
Association of performance measures with hospital-level RSM rates
The median 30-day and 1-year RSM rates (IQR) were 5.41% (4.91%–6.17%)%) and 13.55% (12.77%–14.48%), respectively (table 3). The ICC of 30-day RSM rate was 0.065, and the ICC of 1-year RSM rate was 0.041.
All performance measures showed no correlations with 30-day RSM rate. Several performance measures showed statistically significant, but modest, correlations with 1-year RSM rate, including antithrombotics at discharge (Spearman correlation coefficient=0.32), antihypertensive medication (Spearman correlation coefficient=0.30) and antidiabetic medication (Spearman correlation coefficient=0.31) (p<0.05 for all) (table 4). These performance measures individually explained between 0.04% and 5.02% of hospital-level variation in 30-day RSM rate, and between 0.08% and 10.05% of hospital-level variation in 1-year RSM rate (table 5).
No correlation was observed between the hospital-level composite score and 30-day RSM rate (Spearman correlation coefficient=0.16). Modest correlation was observed between the hospital-level composite score and 1-year RSM rate (Spearman correlation coefficient=0.32). The composite score explained 2.53% and 10.18% of the hospital-level variation in 30-day and 1-year RSM rates, respectively (table 5).
The present study found that ischaemic stroke care was suboptimal in China and there were various correlations among hospital performance measures. The hospital-level composite score had no correlation with 30-day RSM rate and a modest correlation with 1-year RSM rate. This finding suggested hospital performance measures cannot reliably infer short-term and long-term mortalities after acute ischaemic stroke. Our results underscored that the current performance measures of stroke care provided complementary, but not redundant, information with the measures of 30-day and 1-year mortalities.
When comparing our data with the GWTG-Stroke in America, we found that ischaemic stroke care was suboptimal in China. The GWTG-Stroke programme was developed by the AHA/ASA as a national stroke registry and performance improvement programme with the primary goal of improving the quality of care and outcomes of stroke and TIA in America.7 14 The comparison showed that many hospital performance measures in China were far down from the GWTG-Stroke,14 23 including the composite score (63.3% in 2007 CNSR vs 94.0% in 2007 GWTG-Stroke), intravenous rt-PA <2 hours (14.1% in 2007 CNSR vs 72.8% in 2007 GWTG-Stroke), early antithrombotics (80.3% in 2007 CNSR vs 97.0% in 2007 GWTG-Stroke), DVT prophylaxis (59.6% in 2007 CNSR vs 89.5% in 2007 GWTG-Stroke), discharge antithrombotics (71.0% in 2012 CNSR vs 98.9% in 2007 GWTG-Stroke), anticoagulation for AF (19.7% in 2007 CNSR vs 98.4% in 2007 GWTG-Stroke), lipid-lowering drug for LDL ≥100 mg/dL (42.6% in 2012 CNSR vs 88.3% in 2007 GWTG-Stroke) and smoking cessation (63.3% in 2007 CNSR vs 93.6% in 2007 GWTG-Stroke). Financial burden, lack of stroke centre certification and stroke education might be the main reasons.
The relationships between performance measures and stroke outcomes remained uncertain. Some studies have indicated that the use of performance measures was associated with lower short-term mortality and better functional outcomes.24–26 However, the links between these stroke performance measures and outcomes were not clear in other studies.27–30 There were several probable reasons for the modest correlations between hospital-level performance measures and RSMs in the present study. First, although clinical trials have shown significant relationships between performance measures and outcomes,31 32 the evidence-based performance measures were not mainly designed for overall short-term and long-term hospital mortalities. Second, there was relative little variation across hospitals in some performance measures, such as early antithrombotics after admission, which limited the ability to discriminate between hospitals based on their performance on these measures. Additionally, some performance measures, such as intravenous rt-PA, might have a greater effect regarding stroke-related disability than mortality.33 Finally, hospital mortalities, even risk-standardised ones, were more likely affected by many factors independent of the performance measures of stroke care quality, such as severity of stroke, patients’ or their family’s preference, economic level and clinical strategies that might contribute to a hospital’s performance outcomes.34
The present study also demonstrated significant correlations between certain evidence-based performance measures for acute ischaemic stroke care, indicating that hospitals performing well in one performance measure were more likely to perform well in other measures. However, other performance measures were less strongly correlated with each other. This finding implied different performance measures reflected separate components of quality in acute ischaemic stroke care. Our work indicated that hospital performance rankings were likely to be substantially affected by the performance measures under selection. A broader range of performance measures may be needed to comprehensively reflect the quality of stroke care.
Our study had several strengths. First, to date, the CNSR is the largest stroke database in China. Hospitals participating in the CNSR encompassed different regions and had good representativeness. Second, we used the newer RSM method based on hierarchical models to account for variation in case numbers across hospitals or for intrahospital clustering effects.15 Third, the baseline characteristics of patients, such as age, gender, vascular risk factors, comorbid conditions and stroke severity (NIHSS score), were adjusted in calculating the RSMs in accordance with the AHA/ASA.15 34 Additionally, besides short-term RSM, our study also analysed long-term RSM.
However, our study also had some limitations. First, our study was observational and non-randomised. The association between stroke care performance and stroke outcomes did not necessarily prove causality and may be confounded by previously discussed factors. Second, our outcome was mortality; other outcomes such as stroke recurrence and functional outcomes were also vital to patients, which should be assessed in future studies.30 Third, the hospital performance measures, stroke outcomes and the relationships between them might vary over time (such as extended time window for thrombolysis). Future research needs to determine the stability of the process–outcome relationship as quality improvement efforts would drive broader care adoption. Fourth, our data were collected between 2007 and 2008; therefore, interpretation of our data might be limited. Future research is needed to confirm our results. Finally, there were several differences in stroke healthcare delivery systems between China and other countries. The results reported herein came from the Chinese healthcare system and therefore may not necessarily be generalisable to other countries.
Adherence to evidence-based performance measures for acute ischaemic stroke was suboptimal in China. There were various correlations among hospital individual performance measures. The hospital performance measures had no correlations with 30-day RSM rate and modest correlations with 1-year RSM rate. More measures are needed to assess and improve the quality of hospital-level stroke care in China.
We thank all participating hospitals, colleagues, nurses, imaging and laboratory technicians, and the CNSR steering committee members.
XZ and ZL contributed equally.
YW and YW contributed equally.
Contributors XMZ and ZL analysed the data and wrote the manuscript. XB, LL, CXW, HL, JPB, QY and DW analysed and interpreted the data. ZL, CJW, YJ, XY and XMZ performed the research. YJW, YLW, ZL, YX and XQZ conceived, designed and supervised the study.
Funding Supported by grants from the Ministry of Science and Technology of the People’s Republic of China (2006BA101A11, 2008ZX09312-008, 2011BAI08B01, 2011BAI08B02, 2012ZX09303, 2013BAI09B14, 2013BAI09B03, 2015BAI12B02, 2015BAI12B04, 2017YFC1310900 and 2017YFC1310901), grants from the Beijing Municipal Science and Technology Commission (D15110700200000, D151100002015001, D151100002015002, Z161100000516223 and Z141107002514125), and the Beijing Tiantan Hospital Seeding Initiative.
Competing interests None declared.
Patient consent Not required.
Ethics approval Approval was obtained from the Central Institutional Review Board at Beijing Tiantan Hospital.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.