You are here

  • Home
  • Archive
  • Volume 2, Issue 6
  • Trends in Canadian hospital standardised mortality ratios and palliative care coding 2004–2010: a retrospective database analysis

Article Text

Article menu
XML
Health services research
Research
Trends in Canadian hospital standardised mortality ratios and palliative care coding 2004–2010: a retrospective database analysis
  1. Christopher AKY Chong1,2,
  2. Geoffrey C Nguyen3,4,
  3. M Elizabeth Wilcox5
  1. 1Section of General Internal Medicine, Lakeridge Health Oshawa, Oshawa, Ontario, Canada
  2. 2Faculty of Health Sciences, Queen's University, Kingston, Ontario, Canada
  3. 3Division of Gastroenterology, University of Toronto, Mount Sinai Hospital, Toronto, Ontario, Canada
  4. 4Division of Hepatology and Gastroenterology, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
  5. 5Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto Western Hospital, Toronto, Ontario, Canada
  1. Correspondence to Dr Christopher AKY Chong; caky.chong{at}gmail.com

Abstract

Background The hospital standardised mortality ratio (HSMR), anchored at an average score of 100, is a controversial macromeasure of hospital quality. The measure may be dependent on differences in patient coding, particularly since cases labelled as palliative are typically excluded.

Objective To determine whether palliative coding in Canada has changed since the 2007 national introduction of publicly released HSMRs, and how such changes may have affected results.

Design Retrospective database analysis.

Setting Inpatients in Canadian hospitals from April 2004 to March 2010.

Patients 12 593 329 hospital discharges recorded in the Canadian Institute for Health Information (CIHI) Discharge Abstract Database from April 2004 to March 2010.

Measurements Crude mortality and palliative care coding rates. HSMRs calculated with the same methodology as CIHI. A derived hospital standardised palliative ratio (HSPR) adjusted to a baseline average of 100 in 2004–2005. Recalculated HSMRs that included palliative cases under varying scenarios.

Results Crude mortality and palliative care coding rates have been increasing over time (p<0.001), in keeping with the nation's advancing overall morbidity. HSMRs in 2008–2010 were significantly lower than in 2004–2006 by 8.55 points (p<0.001). The corresponding HSPR rises dramatically between these two time periods by 48.83 points (p<0.001). Under various HSMR scenarios that included palliative cases, the HSMR would have at most decreased by 6.35 points, and may have even increased slightly.

Limitations Inability to calculate a definitively comparable HSMR that include palliative cases and to account for closely timed changes in national palliative care coding guidelines.

Conclusions Palliative coding rates in Canadian hospitals have increased dramatically since the public release of HSMR results. This change may have partially contributed to the observed national decline in HSMR.

  • Health Services Administration & Management
  • hospital
  • mortality rate
  • Palliative Care

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

https://doi.org/10.1136/bmjopen-2012-001729

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Article summary

    Article focus

    • The hospital standardised mortality ratio (HSMR), a ratio of a hospital's actual to the expected number of deaths, is a popular but controversial method of gauging hospital quality.

    • Because patients labelled as palliative are typically excluded from the calculation, the HSMR is potentially sensitive to changes in coding practices.

    • This study assessed whether the release of public HSMR data in Canada was correlated with changes in palliative care coding, and how such changes might have affected HSMR results.

    Key messages

    • After public release of HSMR data in Canada, national HSMR declined while rates of palliative care coding increased dramatically.

    • If palliative cases had been included, rates of HSMR decline would have been at least modestly attenuated.

    • Changes in palliative care coding practices likely contributed to the observed decline in HSMR.

    Strengths and limitations of this study

    • This study is based on over 12.5 million cases across Canada over 6 years.

    • The usual weaknesses of a retrospective analysis on database information apply.

    Measuring the quality of a hospital is an important but exceedingly difficult task. Different methods of capturing quality have been devised, including composite scores of compliance with various quality indicators1 to the adoption of balanced scorecard techniques from the business world.2 No universally accepted gold standard exists.

    The hospital standardised mortality ratio (HSMR) is a controversial macrolevel tool for measuring the quality of hospital care. The HSMR is a conceptually simple ratio of the observed deaths to expected deaths, multiplied by 100. An institution's number of expected deaths is calculated with a regression model using national data;3 having an HSMR greater than 100 implies having a mortality rate greater than expected for the types of patients admitted. Proponents argue that the metric is easy to understand, useful for tracking the effect of quality improvement initiatives and encourages hospitals to explore processes that affect patient safety.4 ,5 Opponents dissent that the HSMR has too many methodological weaknesses and has not been well-validated as a useful measure.6 ,7 Thus, while some countries have endorsed and publicly released HSMR results, other jurisdictions have felt that it is not a suitable tool for monitoring quality improvement.8

    One major concern of the HSMR is the potential for ‘gaming’ in which the rules of the calculation are exploited to achieve the semblance of a good outcome without actually changing care processes. Because calculating the expected number of deaths is dependent on how patients have been coded, an opportunity exists for a hospital to alter how sick their patients appear to be.9 In the UK, there has been evidence that systematic coding differences may affect HSMR results.10 Specifically, recoding a patient as palliative can be a simple way to exclude deaths that would otherwise have been included in the calculation. Since publication of the HSMR in the UK, there has been a noted increase in palliative coding,11 with two trusts increasing the number of deaths labelled as palliative to about 50%.12 In particular, a public inquiry into one UK hospital trust highlighted how abrupt changes in palliative care coding can quickly alter HSMRs.13

    In 2007, the Canadian Institute for Health Information (CIHI) announced plans to release HSMRs for all large hospitals in Canada (excluding the province of Quebec) participating in its Discharge Abstract Database (DAD) programme.14 Results were ultimately released in November 2007, and the public relations ramifications for hospitals with high HSMRs were severe.15 ,16 Similar to what occurred in the UK,13 defensive claims were made that poor rates were due to misunderstandings in palliative coding,17 although CIHI had prepared and released palliative care coding guidelines in 2006.18

    Since 2007, HSMRs across Canada have been declining, which has been relayed to the public as a sign that releasing the data has had a salutary effect.19 ,20 The purpose of this study was to explore how palliative coding in Canadian hospitals has changed since announcing plans to release the HSMR, and how that adjustment may have affected subsequent results.

    Methods

    Ethics approval and funding

    This study was approved by both the Lakeridge Health and Mount Sinai Hospital Research Ethics Boards. Funding was provided by unrestricted continuing medical education funds from Lakeridge Health.

    Data

    A request was made to CIHI to access its record-level DAD from Fiscal Years 2004–2009 with anonymity for hospital and patient identifiers. This database includes 12 593 329 discharges across 606 hospitals in Canada (excluding the province of Quebec) from April 2004 to March 2010, which constitutes our entire study population. The DAD includes a ‘most responsible diagnosis’ which is the principal condition responsible for the patient's stay in hospital and up to 24 other discharge diagnoses typed as comorbidity, secondary, transfer or other diagnoses.21 Details on the DAD are presented elsewhere.22

    The authors independently recalculated monthly HSMRs using the methodology released by CIHI and the same inclusion and exclusion criteria.3 The CIHI HSMR includes inpatient deaths only. A palliative diagnosis was identified with International Classification of Diseases-10-CA code Z51.5; CIHI excludes cases when Z51.5 is the most responsible diagnosis code. Total number of diagnoses/case was calculated as the sum of coded discharge diagnoses/case. Total number of interventions/case was calculated as the sum of Canadian Classification of Health Interventions codes/case. Crude percentages were calculated as the variable divided by the total number of cases each month. Charlson comorbidity scores were calculated using the methods described by CIHI.3

    While expected HSMR is based on all hospitals, CIHI only releases the HSMRs of hospitals which have at least 2500 annual cases meeting inclusion criteria. As hospitals were de-identified, we labelled institutions as having publicly released data as those with at least 15 000 qualifying HSMR cases over the six study years (ie, an average of at least 2500/year), which was 84 hospitals.

    Statistical analysis

    Data were analysed using SPSS V.19.

    Variables visually exhibiting seasonality on plots were adjusted using the SPSS multiplicative algorithm function, which yields seasonal component factors proportional to the overall series level.

    To construct a hospital standardised palliative ratio (HSPR), we used the same approach applied to building the HSMR.12 Using data from 2004 to 2005, we constructed a binary logistic regression model to define variables and coefficients that would predict the expected number of cases with palliative code Z51.5 as the most responsible discharge diagnosis (the CIHI HSMR is also based on 2004–2005 data). Inclusion criteria were all elective or urgent admissions to an acute care institution; cases with a discharge disposition of stillborn or cadaver were excluded. Final included variables were any diagnosis of metastatic cancer, length of stay, age, number of interventions and being a medical versus surgical case. Details on this model are found in the appendix. The HSPR is interpreted as the observed number of cases with palliation as the most responsible diagnosis/expected such number×100.

    To calculate an HSMR which included palliative cases (HSMR+palliative), we needed to estimate a coefficient value for these records. In the standard CIHI HSMR, the most responsible discharge diagnosis is given a coefficient, but no similar value was derived in cases identifying palliation as the main diagnosis. For many cases with palliation as the most responsible diagnosis we found that no further diagnoses were provided; when further diagnoses were coded, it was unclear which to consider the next most responsible condition. Therefore, we used three approaches to estimate HSMRs that included palliative cases.  First, we ran a sensitivity analysis in which the palliative coefficient would range from the minimum to the maximum possible values of the normally included discharge diagnoses. The purpose of this approach was to capture the entire probable range of HSMRs if another diagnosis had been entered as the main diagnosis.  Second, we estimated a palliative coefficient based on the expected mortality of patients in Canadian palliative units from other studies,23 ,24 which is detailed in the appendix. In the absence of primary data, this approach provided an indirect estimate of what the palliative coefficient should be based on existing literature.  Third, we reconstructed the usual CIHI HSMR binary logistic regression with all variables in the same categorisation, but with the addition of palliative care as one of the main diagnosis groups. This approach provides a coefficient for palliative cases within the existing CIHI statistical model. The usual CIHI HSMR regression was also reconstructed with complete elimination of the main diagnosis group as a variable. This approach used the existing CIHI regression without the single variable that differentiated palliative and non-palliative cases. Details of these three approaches are provided in the appendix.

    We defined April 2004 to March 2006 as the time prior to changes in palliative coding and HSMR publication, April 2006 to March 2008 as the time during and April 2008 to March 2010 as the time post. The t-tests were used to compare the first and last time intervals. Corrections for multiple comparisons were not needed as comparisons among other different time periods were not examined. Ordinary least-squares linear regression was used to analyse trends in clinical and demographics variables over time.

    Results

    Clinical and demographic characteristics of study population and crude mortality and palliative rates

    Table 1 outlines the baseline characteristics of the study population and their trends over the study period. Inpatients in Canada have been gradually getting older with increasing comorbidities. Mean age, mean number of diagnoses/patient, mean number of interventions/patient and Charlson scores have all been increasing. As expected, the crude proportion of inpatient deaths and the crude proportion of inpatients with a palliative code as either a most responsible or any discharge diagnosis have correspondingly been increasing as well (table 1 and figure 1).

    View this table:
    Table 1

    Descriptive statistics of clinical–demographic characteristics and crude mortality and palliative percentages for Canadian inpatients, April 2004 to March 2010 (n=12 593 329).

    Figure 1
    Figure 1

    Crude monthly percentage of deaths and percentage of cases with palliative care as any diagnosis of Canadian inpatients, April 2004 to March 2010 (n=12 593 329).

    A comparison of the percentage of inpatients with any palliative discharge diagnosis between hospitals with and without publicly released HSMR data is shown in figure 2. The visual trend of eventually more marked palliative coding in hospitals with publicly released data is supported by descriptive statistics. In 2004–2006, the mean percentage of cases coded with a palliative diagnosis was 1.67% in hospitals with publicly released data and 1.78% in hospitals without, for a mean difference of 0.11% (95% CI of difference 0.05 to 0.17, p=0.001). In 2008–2010, this pattern is reversed. Hospitals with publicly released data now have a higher percentage of cases coded with palliation at 3.40% compared to 3.18% in hospitals without (95% CI of difference 0.10 to 0.35, p=0.001).

    Figure 2
    Figure 2

    Crude monthly percentage of cases with palliative care as any diagnosis, grouped by inpatients in hospitals that do (n patients=4 791 714) and do not (n patients=7 801 635) have publicly released Canadian HSMR results, April 2004 to March 2012.

    Trends in standardised mortality and standardised palliative coding rates

    Seasonally-adjusted nation-wide monthly HSMR rates are illustrated in figure 3 and the corresponding rates in the time periods before, during and after HSMR publication are shown in table 2. HSMR rates are significantly lower in 2008–2010 than in 2004–2006 by 8.55 points (p<0.001).

    View this table:
    Table 2

    Mean monthly HSMR and HSPR for Canadian inpatients, April 2004 to March 2010

    Figure 3
    Figure 3

    Seasonally adjusted HSMR (n=3 816 181) and HSPR (n=12 593 172) rates for Canadian inpatients, April 2004 to March 2010.

    To compare with the HSMR, we constructed and calculated an HSPR expressing the ratio of observed cases with a most responsible discharge diagnosis of palliation to the expected number based on 2004–2005 data. As shown in figure 2 and table 2, the HSPR has been increasing, and has jumped 48.83 points in 2008–2010 compared with 2006–2008(p<0.001).

    The HSMR strongly correlates negatively with the HSPR (Pearson r=−0.862, p<0.001).

    Estimating the effect of including palliative cases on HSMR

    In the time period 2008–2010, under three palliative care scenarios, the HSMR would have decreased by at most 6.35 points (p<0.001) and may have potentially even increased by 2.30 points (p=0.028) (table 2). This is in contrast to the observed HSMR decrease of 8.55. While we would caution that the differing calculations make the HSMR+palliative statistic not directly comparable to the HSMR, one could roughly use these differences to estimate that the increase in palliative coding contributed to at least one quarter of the observed HSMR decline.

    In the final fiscal year 2009, of 84 hospitals with publicly released data over the study period, 6 had HSMR+palliative scores (when simply adding palliative care as a main diagnosis group) that were 10 or more points worse than the usual HSMR. The HSPR scores for these six hospitals were significantly higher than the remaining hospitals (347.4 vs 140.1, p<0.001).

    Discussion

    Across Canada, we demonstrate that since publication of the HSMR, hospitals have dramatically increased the number of patients labelled as palliative, a trend that may be more marked at sites with publicly released HSMR results. This change strongly correlates with the witnessed decrease in HSMR. Our results are in keeping with the UK experience, in which palliative care coding doubled since palliative exclusions began.12 The findings are also consistent with prior Canadian data suggesting that inclusion or exclusion of palliative cases can significantly alter HSMRs.25 In our study, including palliative cases would have at least modestly attenuated the rate of HSMR decline. Similarly, in the UK, including palliative cases would have changed whether some hospitals were labelled as high versus low performing.12 We suggest our results demonstrate that publicly releasing a quality indicator induces hospitals to improve their scores on that metric at least partially by modifying coding practices.

    Accurate coding is an essential part of administrative medicine and public health management. Any form of coding involves subjectivity, but ethically coding a patient as palliative can be a particularly difficult task requiring clinical judgment and a standardised, consistent approach.9 In a public inquiry to one UK trust, much debate was held on the intentions behind palliative care coding changes, although no formal finding of fact was made.13 Regrettably, the close timing between CIHI's release of new palliative care coding guidelines in mid-2006 and then plans to release the HSMR make it difficult to distinguish between the relative contributions of these two effects. For example, in the UK, the 2007 broadening and subsequent 2010 renarrowing of the palliative definition was associated with a commensurate expansion and retraction in the percent of deaths coded under palliation.13 Changes in national coding guidelines would be expected to change institutional coding practices. This is an important limitation to any pre- and post-based study, when two closely spaced interventions cannot adequately be separated. Also, it is challenging to determine how much of the observed increase in palliative coding is due to improved awareness of the importance of proper labelling and thus is appropriate and ultimately beneficial. Indeed, over the past decade there may truly have been an increase in palliative care resource utilisation as has been observed elsewhere.26 If our observed increased palliative coding rate is due to such a trend, then one might hypothesise the rise should correspond with an equally proportionate uptake in palliative care services. Such a study is beyond this paper's scope, but we speculate it would be difficult for the phenomenal rise in HSPR to be entirely accounted for by these effects.

    While our findings suggest that a component of gaming may exist, it is very important to note that even with including palliative cases, HSMR rates are probably still declining since 2007, albeit at a slower rate. Debate continues over the value and consequences of publicly reporting any quality metric.27 Publicly releasing the HSMR may very well have fostered adoption of evidence-based practices that improve patient safety and the quality of care, as anecdotally reported.19 ,20 However, because the HSMR is dependent on the subjective coding process, we feel the tool does not accurately quantify changes in hospital quality. As such, we agree with others who claim that as is, the HSMR is not a particularly robust instrument.28

    Strengths of this study include the size of the study population that spans across a nation with a high number of pre- and postimplementation observation points. Weaknesses include those shared with all non-randomised, retrospective observational database studies29 and the aforementioned inherent weakness of having a competing intervention (changes in coding guidelines) in our analysis. Specific weaknesses for our study include being unable to definitively label which hospitals had publicly released data. As well, we could not provide a definitive coefficient for palliative cases when calculating HSMR+palliative scores that would fit into the original model. In particular, the literature-based coefficient is based on data that is heavily biased towards patients with cancer. Our study also focussed solely on palliative care coding; evaluating other methodological issues such as the effects of possible shifts in comorbidity coding,10 inclusion of readmitted patients,30 or shifts towards out-of-hospital or other-facility deaths31 ,32 on HSMR was beyond the scope of this study; each may have had an effect on the observed decline. Also, it is important to note that in contrast to other countries, Canada is unusual in allowing palliative care to be positioned as the primary diagnosis; such a difference may limit extrapolation of our findings to other nations.

    Society demands transparency and accountability from their public hospitals, but identifying methods that openly encourage and accurately monitor quality remains a major challenge. We contend our findings indicate that to at least some degree, when publicly pressured to show improvement, administrators will seek ways to alter metrics in order to account for perceived local idiosyncrasies that they feel bias against their institution's performance. However, ‘real’ quality improvement more in the spirit intended may still occur. Recently, the English Department of Health has adopted use of the Summary Hospital-level Mortality Indicator which includes all palliative care cases and deaths occurring within 30 days of discharge;33 and HSMRs with palliative case adjustments can be calculated.12 Further experience should reveal whether such evolutions will lead to a truly accurate big-dot measure immune to variations in coding practice.

    References

    1. ↵
      1. Jha AK,
      2. Li Z,
      3. Orav EJ,
      4. et al
      . Care in U.S. hospitals—the Hospital Quality Alliance program. N Engl J Med 2005;353:265–74.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Chan Y-CL
      . An analytic hierarchy framework for evaluating balanced scorecards of healthcare organizations. Canad J Admin Sci 2006;23:85–104.
      OpenUrl
    3. ↵
      Canadian Institute for Health Information. Hospital Standardized Mortality Ratio. Technical Notes, Canadian Institute for Health Information, 2011:1–17.
    4. ↵
      1. Leatt P
      . Healthcare Papers. New models for the new healthcare. Do hospital standardized mortality ratios measure patient safety? Notes from the Editor-in-Chief. Healthc Pap 2008;8:4–5.
      OpenUrlPubMed
    5. ↵
      1. Jarman B,
      2. Gault S,
      3. Alves B,
      4. et al
      . Explaining differences in English hospital death rates using routinely collected data. BMJ 1999;318:1515–20.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Scott IA,
      2. Brand CA,
      3. Phelps GE,
      4. et al
      . Using hospital standardised mortality ratios to assess quality of care—proceed with extreme caution. Med J Aust 2011;194:645–8.
      OpenUrlPubMedWeb of Science
    7. ↵
      1. Black N
      . Assessing the quality of hospitals. BMJ 2010;340:c2066.
      OpenUrlFREE Full Text
    8. ↵
      1. Shahian DM,
      2. Wolf RE,
      3. Iezzoni LI,
      4. et al
      . Variability in the measurement of hospital-wide mortality rates. N Engl J Med 2010;363:2530–9.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Downar J,
      2. Sibbald R,
      3. Lazar NM
      . Ethical considerations for classifying patients as ‘palliative’ when calculating Hospital Standardised Mortality Ratios. J Med Ethics 2010;36:387–90.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Mohammed MA,
      2. Deeks JJ,
      3. Girling A,
      4. et al
      . Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals. BMJ 2009;338:b780.
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Robinson P
      . Hospital standardised mortality ratios and their use as a measure of quality. CHKS Market Intelligence Report: CKHS, 2010:2–6.
    12. ↵
      1. Bottle A,
      2. Jarman B,
      3. Aylin P
      . Hospital standardized mortality ratios: sensitivity analyses on the impact of coding. Health Serv Res 2011;46(6 Pt 1):1741–61.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      Closing submissions of counsel to the inquiry. The Mid Staffordshire NHS Foundation Trust Public Inquiry. London, 2011:1559–635.
    14. ↵
      The HSMR public release. Information Sheet. Canadian Institute of Health Information, 2011.
    15. ↵
      1. Talaga T,
      2. Cribb R
      . Humber hospital has GTA's highest death rate. Toronto Star 2011 14 December 2011.
    16. ↵
      Burnaby hospital has worst death rate in B.C. Vancouver Sun 2011 30 November 2011.
    17. ↵
      Statement by Humber River Regional Hospital.Toronto, 2011.
    18. ↵
      Revised coding guideline for palliative care cases submitted to the DAD in fiscal year 2007–08. Canadian Institute for Health Information, 2007:1–4.
    19. ↵
      1. Ogilvie M,
      2. Smysnuik S
      . Hospitaldeath rates fall in new era of disclosure. Toronto Star 2009 11 December 2009.
    20. ↵
      1. Stauffer K
      . CIHI data helps promote a culture of safety. Ottawa, Canada: Canadian Institute for Health Information, 2011.
    21. ↵
      CIHI Data Quality Study of the 2009–2010 Discharge Abstract Database. Ottawa: Canadian Institute for Health Information, 2012:115.
    22. ↵
      DAD—Frequently Asked Questions. Canadian Institute for Health Information, 2011:10.
    23. ↵
      1. Jenkins CA,
      2. Taube AW,
      3. Turner K,
      4. et al
      . Initial demographic, symptom, and medication profiles in patients admitted to continuing palliative care units. J Pain Symptom Manage 1998;16:163–70.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Napolskikh J,
      2. Selby D,
      3. Bennett M,
      4. et al
      . Demographic profile and utilization statistics of a Canadian inpatient palliative care unit within a tertiary care setting. Curr Oncol 2009;16:49–54.
      OpenUrlPubMed
    25. ↵
      1. Penfold RB,
      2. Dean S,
      3. Flemons W,
      4. et al
      . Do hospital standardized mortality ratios measure patient safety? HSMRs in the Winnipeg Regional Health Authority. Healthc Pap 2008;8:8–24; discussion 69–75.
      OpenUrlPubMed
    26. ↵
      1. Unroe KT,
      2. Greiner MA,
      3. Hernandez AF,
      4. et al
      . Resource use in the last 6 months of life among medicare beneficiaries with heart failure, 2000–2007. Archiv Intern Med 2011;171:196–203.
      OpenUrlCrossRefPubMed
    27. ↵
      1. Conway PH,
      2. Clancy C
      . Transformation of health care at the front line. JAMA 2009;301:763–5.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Shahian DM,
      2. Iezzoni LI,
      3. Meyer GS,
      4. et al
      . Hospital-wide mortality as a quality metric: conceptual and methodological challenges. Am J Med Qual 2012;27:112–23.
      OpenUrlAbstract/FREE Full Text
    29. ↵
      1. Hulley S,
      2. Cummings S
      . Designing clinical research. An epidemiologic approach. Baltimore: Williams & Wilkins, 1988.
    30. ↵
      1. van den Bosch WF,
      2. Kelder JC,
      3. Wagner C
      . Predicting hospital mortality among frequently readmitted patients: HSMR biased by readmission. BMC Health Serv Res 2011;11:57.
      OpenUrlCrossRefPubMed
    31. ↵
      1. McCormick B,
      2. White WJ
      . HSMRs and all cause mortality. London: Centre for Health Service Economics & Organisation, 2011:19.
    32. ↵
      1. Drye EE,
      2. Normand SL,
      3. Wang Y,
      4. et al
      . Comparison of hospital risk-standardized mortality rates calculated by using in-hospital and 30-day models: an observational study with implications for hospital profiling. AnnIntern Med 2012;156:19–26.
      OpenUrl
    33. ↵
      1. Campbell MJ,
      2. Jacques RM,
      3. Fotheringham J,
      4. et al
      . An evaluation of the Summary Hospital Mortality Index. The University of Sheffield, School of Health and Related Research, 2011:81.

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors CAKYC originally conceived of and designed the study, completed statistical analysis, interpreted results, wrote the first draft and reviewed and approved the final draft. GCN provided confidential housing for CIHI data (thereby allowing data acquisition), interpreted results, reviewed the statistical analysis, and reviewed and approved the final draft. MEW completed statistical analysis, interpreted results and reviewed and approved the final draft. CAKYC acts as data guarantor.

    • Funding Lakeridge Health Oshawa.

    • Competing interests None.

    • Ethics approval Lakeridge Health and Mount Sinai Hospital.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Additional data are available from the Canadian Insititute for Health Information, email: hsmr@cihi.ca, www.cihi.ca.