What happens when pay for performance stops?

Financial incentives for quality in pay for performance programmes are an attractive improvement method for healthcare payers worldwide, and the UK Quality and Outcomes Framework (QOF) is still one of the largest such programmes. Evidence shows that the QOF improved incentivised quality of care and reduced variation between practices. However, the effect for most indicators was relatively small, was not always persistent, and was partly balanced by negative effects on non-incentivised care.1 2 3 The QOF and pay for performance more generally are therefore clearly not magic bullets for improvement,4 and many uncertainties remain. The paper by Kontopantelis and colleagues (doi: 10.1136/bmj.g330) examines one of these important uncertainties—what happens when financial incentives for quality are withdrawn.5

Removal of incentivised indicators has several rationales, including lack of initial effectiveness, lack of continued improvement, and to allow the targeting of other priority areas. Over the nine years of the QOF’s existence, large changes to structural and organisational process indicators have been made, but relatively few indicators relating to clinical care have been retired. Instead, funding to introduce new clinical indicators has come largely from reallocating resources from organisational indicators. More radical change was constrained by concerns that withdrawing clinical indicators risked reduced delivery of important care. This fear was supported by evidence from two US studies, in which removal of financial incentives was associated with significant declines in performance, which in one case fell below the pre-incentive baseline.6 7

However, radical change is coming in 2014-15, intended to reduce bureaucracy and allow more holistic care of older, frailer, multimorbid people.8 NHS England is retiring 40 (mostly clinical) indicators and reallocating almost one quarter of the funding from QOF pay for performance to capitation.9 Changes on a similar scale have been announced in Scotland and Wales, although these differ in the detail of the indicators retired.10 11

The linked paper is therefore timely. It examines changes in quality of care for two indicators for which incentives were withdrawn in 2006 (influenza immunisation in people with asthma and measurement of lithium concentrations) and six for which incentives were withdrawn in 2011 (regular measurement of blood pressure, cholesterol, and HbA_1c in people with vascular disease and diabetes).5 In essence, the study found some evidence of reduced quality of care as measured by these indicators, but changes were generally small and often trivial, being largest for cholesterol monitoring in coronary heart disease with rates 1.2% lower than predicted. The robustness of the findings is reinforced by the careful analysis and a range of sensitivity analyses showing the same findings.

These results provide some reassurance to policy makers and others making recommendations about withdrawal of indicators. An important caveat remains in that the care measured by seven of the eight withdrawn indicators remained incentivised by the QOF, because failure to measure lithium concentrations or the specified intermediate outcomes means that patients automatically fail an associated incentivised “control” indicator (lithium in the therapeutic range, blood pressure controlled, and so on), meaning that practices still have incentives to measure.

Indicators planned for withdrawal in 2014-15 are more commonly stand alone, meaning that all financial incentives are removed. For example, NHS England is removing incentives for the measurement of cholesterol, glucose/HbA_1c, and body mass index in people with severe mental illness, which have no linked intermediate outcome targets. Whether these process measures improve patients’ outcomes is unclear, but physical disease care is clearly critical in this population given the 15-20 year reduced life expectancy compared with the general population, largely due to cardiovascular disease.12 The study does not directly consider the effect of withdrawing indicators like these, and declines in performance as was seen in the United States remain possible.6 7 Unfortunately, no routine, nationwide monitoring of whether the quality of care changes is likely; QOF data extraction measures only care that is incentivised, so removing the incentive removes routine measurement of quality. Given the financial cost of the QOF, and the potential consequences for patients, it is disappointing that the NHS does not collect routine data that would allow rapid evaluation of the effect of withdrawal.

Many other uncertainties exist about how the QOF and other pay for performance systems work or should be designed. Like other pay for performance programmes,6 7 the QOF itself was always more than a set of financial incentives, with important educational, informatics (computerised reminders, better searching, and recall systems), feedback, and public reporting interventions embedded within, and was implemented in the context of other related improvement activity such as development and implementation of clinical guidelines.3 Improvement is probably best driven through blends of different interventions, although what the best blend is, or whether pay for performance is an important element of the blend, remains uncertain.3 6

Evaluating the effectiveness of incentive design has been constrained by the QOF’s introduction across the entire United Kingdom simultaneously, meaning that no control group for comparison exists. This is set to change, with different indicators planned for withdrawal in each UK country.9 10 11 Other elements of incentive design are also increasingly variable, including payment thresholds and the time period during which practices must deliver care to obtain payment. What effect the significant changes to the QOF in 2014-15 will have is uncertain, but change was probably overdue,13 and the QOF’s increasing diversity will make it easier to evaluate the effectiveness of financial incentives in UK primary care, which will inform design of future incentives and indicators.