Introduction and background

The Hospital Standardised Mortality Ratio (HSMR) is a measurement tool used to express an observed mortality rate compared with an average expected mortality rate for a given hospital. It was designed to measure the quality of care provided by a hospital by correcting for patient factors such as age and comorbid illness. An HSMR >100 would indicate a higher than expected mortality, while an HSMR <100 would indicate a lower than expected mortality. The Canadian Institute for Health Information (CIHI) publicly released the HSMRs for many hospitals and regions across Canada for the first time in November 2007. According to CIHI,1 this was part of an effort to improve the quality of patient care by “help[ing] hospitals and regions to understand their mortality rates”.

Canadians have good reason to question this decision. The HSMR has been well studied over the past 2 decades, but it has never been validated as a reliable and efficacious measure of quality of care.2 The HSMR has a number of important limitations, including:

• Most deaths are not preventable, and most quality-related problems do not cause death.3

• The most important determinants of mortality are non-modifiable (eg, age, gender, comorbidities) and the HSMR cannot correct for unknown or unmeasured risk factors.

• The process of standardisation can introduce bias via the constant risk fallacy,4 inconsistent measurement of risk factors,3 variable admission thresholds,5 and variable access to rehabilitation and long-term care facilities.6

Furthermore, the value of releasing hospital performance data to the public is unclear,7 and it may certainly lead to negative consequences. Although CIHI advised people not to use the HSMR to compare hospitals, national news agencies were quick to assemble a rank list of every hospital in the report, and poor performers were left scrambling to issue public explanations for their results.8 Hospitals with high HSMRs suffer a loss of public image, which could result in lower staff morale and difficulty recruiting staff or attracting private donations. Patient advocacy groups have argued for public investigation of any hospital with an HSMR >100,9 and in some countries the HSMR is used to calculate reimbursement from insurance providers.10 Thus, the stigma of a high HSMR may deprive hospitals that have the greatest need for support. It also creates a powerful incentive to divert resources to improve the HSMR at the expense of other worthy goals (eg, palliative care, reducing Emergency Room waiting times), or deliberately manipulate the HSMR (‘gaming’) through creative statistical interpretation. One report even found that the introduction of public report cards for coronary bypass surgery was associated with worsening racial discrimination by cardiac surgeons selecting their patients.11

The HSMR does have supporters. It is relatively easy and inexpensive to collect and track over time, and some hospitals have successfully used the HSMR to identify quality of care issues and lower their HSMR.12 13 However, it is not clear that lowering the HSMR actually improves patient care. Indeed, while some studies have shown an association between the HSMR and rates of preventable death or poor quality of care, this association is inconsistent.14

Palliative care and the HSMR

Traditionally, patients receiving palliative care are excluded from the HSMR calculation. This would appear logical, as the mortality rate is clearly not an indication of quality care in patients who are expected to die. But this leads to an important question—how do we identify patients who are receiving ‘palliative care’?

Initially, this group included only patients whose most responsible diagnostic code was ‘palliative care’—that is those who received palliative care for the largest portion of their length of stay.15 However, this purely chronological criterion is difficult to apply clinically. A patient admitted with end-stage Alzheimer's disease might be admitted and treated for dehydration or pneumonia for 1 week without improvement, prompting a transition to purely comfort measures resulting in death after 3 days. This patient would not be considered ‘palliative’ by chronological standards, but many clinicians would have considered this to be a ‘palliative’ admission. On the other hand, a young man admitted with a myocardial infarction could suffer a massive brainstem stroke during a coronary angiogram on his first day of admission, prompting a transition to comfort measures. Assuming he lived for more than 24 h, his admission could be considered ‘palliative’ (by chronological standards) and his death excluded from the HSMR calculation, even though few clinicians would ever consider this to be a ‘palliative’ admission.

The arbitrary nature of the CIHI criteria may explain why ‘palliative’ patients had such a large and variable effect on the HSMR. In 2006, one hospital in Winnipeg found that its HSMR was 118 when palliative patients were included, and 55 when they were excluded.16 In Ontario, the overall effect of excluding palliative patients was unpredictable, but one third of hospitals saw their HSMR change by 10% or more.17 Many hospital administrators were left wondering whether their hospital should serve as a model for others or the subject of a coroner's inquiry.

In 2007, CIHI decided to publish an additional measure of HSMR that excluded ALL patients who had received palliative care, even if it did not account for a majority of their stay in hospital. The following year, CIHI abandoned this statistic, noting that it ‘does not reflect mortality trends for a growing number of organisations, as there have been changes over time in the identification of patients receiving palliative care’.15 Essentially, CIHI abandoned the second HSMR statistic because it didn't show the same trend as the first, while acknowledging that the identification and definition of ‘palliative’ patients was not consistent.

How should we identify ‘palliative’ patients?

The World Health Organization has published a widely-accepted definition of palliative care,18 but this definition does not specify which patients should be considered ‘palliative’. Would it be appropriate to simply identify ‘palliative’ patients as anyone receiving ‘palliative care’? Probably not. In our current model of care, palliative care is often initiated at an early stage of illness, and may be concurrent with ‘curative’ therapy.19 In light of this model, it would be inappropriate to categorise patients as exclusively palliative or non-palliative. In addition, hospitals have varying degrees of access to specialised palliative care services, which would introduce yet another bias into the calculation.

Can we use prognostic criteria? In Ontario, inpatient palliative care units will typically only admit patients with an expected survival of less than 3 months. Although this is a useful administrative criterion, it has little clinical usefulness. Patient prognostication is notoriously difficult, with a high degree of variability between patients. It is also harder to prognosticate for some end-stage conditions (eg, heart failure, chronic obstructive lung disease) than others (eg, cancer). Furthermore, even if we were able to reliably prognosticate, we would not be able to establish an appropriate ‘cut-off’ point beneath which a patient would be considered palliative. Decisions about treatment plans (eg, palliative, curative, life-prolonging etc.) involve complex considerations of medical facts and patient values. They should never be reduced to a simple prognostic calculation.

If there are no reliable clinical criteria to identify ‘palliative’ patients, then we must use arbitrary administrative criteria. For the purpose of the HSMR calculation, we would want the palliative status to apply to patients for whom mortality is not a relevant end-point, but not to patients who may have become palliative as a result of an issue related to quality of care. This would include patients who are admitted solely for the purpose of palliative care, as in the current CIHI definition. This would also include patients in whom a palliative diagnosis and treatment plan became apparent after minimal investigation. Patients should not be excluded from the HSMR, however, if their change to ‘palliative’ status occurred after an intervention (even if the intervention does not appear causally related to the change to palliative status). These latter patients might serve to identify a patient care issue. Furthermore, the duration of palliative care should have no bearing on whether the admission is considered ‘palliative’.

All stakeholders should recognise the danger of overusing the palliative status flag, particularly in ‘borderline’ cases. If too many patients are eliminated from the HSMR calculation, we risk missing an important change in patient outcomes that might indicate a problem with quality of care. Instead, Healthcare organisations should strive to apply the flag consistently (rather than frequently), and use the same coders whenever possible (rather than individual physicians, who might have differing definitions of ‘palliative’) to further reduce random variation in the HSMR.

What are the potential consequences of identifying inpatients as ‘palliative’?

Categorising living inpatients as ‘palliative’ or ‘non-palliative’ is not value-neutral, and may carry important consequences. Palliative status flags appearing in the medical record of living patients may have a positive effect by reminding clinicians to cancel unnecessary investigations or consult relevant services (eg, palliative care, chaplaincy), but they may also have deleterious effects. Patients may learn that they have been labelled ‘palliative’ before their medical team has had a chance to explain what this means. While most patients prefer honesty and disclosure, a poorly-handled revelation may cause major psychological trauma. Hospital staff may treat the patient differently, and there may even be an effect on prognosis and survival.20 The palliative status flag may also lead to ambiguity or conflict, as individual staff would disagree about whether ‘palliative’ patients should receive certain therapies (eg, antibiotics, feeding gastrostomy tubes), and some patients might be denied beneficial therapies based on their palliative status.

Given New York's experience with racial profiling by cardiac surgeons,11 we must be vigilant about the potential for ‘palliative’ flags to compromise care for individual patients. Indeed, since the palliative flag is a purely administrative classification, it should probably be used only in retrospect to avoid any possible negative consequences for individual patients.

An organisational ethics framework

An organisational ethics approach to the HSMR issue requires individual hospitals to ensure that in reporting any hospital data, they are doing so in a manner consistent with their espoused values, but also that is in keeping with the duties owed to various stakeholders.21 22 When hospitals aim to reduce their HSMR, other values and obligations may be compromised as a result. In particular, the consequences of labelling patients as ‘palliative’ may ultimately undermine patient care.

Given that (i) individual hospitals are responsible for calculating and publicly reporting their own HSMR; (ii) there is considerable variation in how hospitals choose to do this; and (iii) there is significant opportunity for negative consequences to multiple stakeholders; we propose an organisational ethics framework (table 1) for all healthcare facilities involved in public reporting of the HSMR (or any other internally-calculated medical statistics). Based on this framework, we propose six specific suggestions for any healthcare organisation that plans to report their HSMR to the public (summarised in table 2). First and foremost, staff and administration should be sure to educate themselves and the public about the meaning of the HSMR and palliative care. Second, instead of co-opting a clinical term (eg, ‘palliative’), hospitals should adopt a more ‘administrative’ term (eg, END—expected natural death) to describe patients who are excluded from the HSMR calculation. In turn, this administrative term should NOT be used in the clinical setting, and staff should be aware of the potential harms of using a term such as ‘palliative’ to describe an inpatient.

Third, hospitals should no longer use an arbitrary and ambiguous chronologic criterion to decide retrospectively whether an admission was ‘palliative’. Instead, admissions should only be considered ‘palliative’ if the goals of care were unambiguously comfort-oriented from the outset. This leads into the fourth recommendation—that the palliative status label be used consistently rather than frequently, and applied by consistent personnel rather than individual physicians. All stakeholders should recognise the danger of missing an important change in patient outcomes by overusing the palliative status flag. The validity of the HSMR can only be improved by reducing random variation, not by increasing the rate of ‘palliativity’.

Our fifth recommendation is to regularly remind all stakeholders about the limitations of the HSMR. The HSMR is used by an individual hospital over a long time frame as a means to detect potential problems with patient care. No HSMR value or trend by itself indicates a good or poor quality of care, and HSMRs should not be used to compare hospitals because of the biases and unmeasured variables that make these comparisons impossible.

Our final recommendation is for all stakeholders to be aware of the many incentives that make hospitals want to have a low HSMR, and how these incentives might affect behaviour in a negative way. Stakeholders should be aware of policies and practices that may lower the HSMR without improving patient care, and should avoid the temptation to prioritise the HSMR when allocating resources. Hospitals should strongly discourage any attempt to ‘game’ the HSMR by deliberately applying the palliative care status flag in a way that spuriously lowers the HSMR.3 This does not improve patient care, and may actually worsen it by rewarding existing practices and potentially concealing the effects of poor patient care. Governments and hospital managers should not create any incentives for lowering the HSMR, or even offer congratulations to units and hospitals that have done so. Even positive feedback may become an incentive, particularly when an institution fails to receive it consistently.

Summary

The HSMR has always excluded patients classified as ‘palliative’. In Canada, this ill-defined and arbitrary status flag made the HSMR particularly difficult to interpret. In this paper, we reviewed the Canadian experience with the palliative status flag, and explained why it was so difficult to define and apply consistently. We also highlighted some potential concerns about clinicians labelling inpatients as ‘palliative’ during their admission. Finally, we proposed an organisational ethics framework, and six specific suggestions for hospitals to use when publishing statistics such as the HSMR.