Does the quality and outcomes framework reduce psychiatric admissions in people with serious mental illness? A regression analysis

Background The Quality and Outcomes Framework (QOF) incentivises general practices in England to provide proactive care for people with serious mental illness (SMI) including schizophrenia, bipolar disorder and other psychoses. Better proactive primary care may reduce the risk of psychiatric admissions to hospital, but this has never been tested empirically. Methods The QOF data set included 8234 general practices in England from 2006/2007 to 2010/2011. Rates of hospital admissions with primary diagnoses of SMI or bipolar disorder were estimated from national routine hospital data and aggregated to practice level. Poisson regression was used to analyse associations. Results Practices with higher achievement on the annual review for SMI patients (MH9), or that performed better on either of the two lithium indicators for bipolar patients (MH4 or MH5), had more psychiatric admissions. An additional 1% in achievement rates for MH9 was associated with an average increase in the annual practice admission rate of 0.19% (95% CI 0.10% to 0.28%) or 0.007 patients (95% CI 0.003 to 0.01). Conclusions The positive association was contrary to expectation, but there are several possible explanations: better quality primary care may identify unmet need for secondary care; higher QOF achievement may not prevent the need for secondary care; individuals may receive their QOF checks postdischarge rather than prior to admission; individuals with more severe SMI may be more likely to be registered with practices with better QOF performance; and QOF may be a poor measure of the quality of care for people with SMI.


INTRODUCTION
The quality of care of people with mental health problems is of international concern. 1 2 While primary care is central to the provision of mental healthcare in England, there is increasing focus on the interface between primary and secondary care, and the potential for better quality primary care to reduce avoidable hospital admissions and contain health expenditures. 3 The role of general practitioners' (GP) care in preventing admissions is a matter of continuing debate, 4 5 including the mechanisms through which this might operate, 6 but the tough economic climate implies this will be a subject of focused attention. 7 Serious mental illness (SMI) includes schizophrenia, bipolar disorder and psychoses with considerable disability, prevalence [8][9][10] and an estimated economic burden of £14 billion. 11 In the UK, around 30% of people with SMI are treated solely by primary care clinicians 12 Strengths and limitations of this study ▪ This is the first study to investigate the relationship between general practitioner (GP) practice quality, as measured by four mental health Quality and Outcomes Framework (QOF) indicators, and psychiatric admissions in the English NHS. ▪ The data covered all practices in England and the results were found to be representative. ▪ The study used a consistent set of primary care quality indicators over the entire study period and employed longitudinal panel data estimation, therefore improving the robustness of results compared to previous research. ▪ A comprehensive set of GP practice and patient population characteristics were included in the models. ▪ An array of sensitivity analyses was undertaken and results were found to be robust. ▪ Aggregate practice-level data were used to examine the association between QOF and admissions and so we cannot be sure whether admitted patients had received a QOF review or had been exception reported. ▪ These are observational data and our results may be affected by unobserved confounders.
and compared with those without mental health problems, people with SMI are in contact with primary care services for a longer cumulative time. 13 14 The pay-for-performance scheme in primary care-the Quality and Outcomes Framework (QOF)-includes targets to incentivise GPs to improve the quality of care for people with SMI. 15 Although not an explicit aim of the QOF, several studies have examined whether better quality primary care can reduce hospital admissions. Analyses of relationships between QOF performance for coronary heart disease, asthma and chronic obstructive pulmonary disorder and hospital admissions found no effects, [16][17][18] but better performance for diabetes 19 20 and stroke 21 had a small negative association with emergency admissions. We hypothesise that this may be plausible for SMI. No previous study has tested this association for people with SMI. We investigated whether higher achievement on the SMI QOF indicators was associated with fewer psychiatric admissions for people with SMI.

METHODS
We carried out a retrospective analysis of routine data at GP practice level, estimating the effect of four QOF indicators (table 1) on psychiatric admissions to hospital using random effects Poisson regression.

Sample
Our data set included all GP practices in the English NHS between April 2006 and March 2011. We chose this period because the QOF definition of SMI was constant and there was a stable set of mental health indicators. We excluded practices with fewer than 1000 registered patients within a year as unrepresentative of the way in which primary care is normally organised. Practices were also excluded if the registered number of patients with SMI was below 5 as their QOF achievement was prone to large variations over time. However, we did not apply this exclusion to the analysis of bipolar disorder because registered numbers of patients with bipolar disorder were low for most practices. We excluded practices reporting inconsistent numbers of patients with SMI or bipolar disorder across indicators within a year and practices where the registered number was fewer than the number admitted to hospital. Finally, we excluded all admissions for patients who changed practice within a year as it was unclear which practice affected the need for inpatient care.

Data sources
We linked administrative data sets including the General and Personal Medical Statistics (GMS) data, the Attribution Data Set (ADS), the QOF data set and the annual GP Patient Survey data (table 2) using unique practice-year identifiers. Census (2001) data from the Office for National Statistics, measured at small-area level (ie, Lower Super Output Areas), were linked to practices on the basis of their practice population distribution as reported annually in ADS. Annual admission rates were calculated from Hospital Episodes Statistics (HES). All data sources are reported in the online supplementary appendix table 1.
Hospital admissions HES records diagnoses using the International Classification of Diseases (ICD) 10 classification system, whereas the QOF inclusion criteria are based on Read codes used in primary care. To identify the relevant population, we used the Health and Social Care Information Centre cross-mapping from Read to ICD-10. Psychiatric admissions were defined as hospital inpatient episodes in patients aged 18 years or over with a main diagnosis of SMI (ICD-10 codes: F20-F31); for the subset of bipolar disorder admissions, defining main diagnoses were ICD-10 codes F30-F31.
Just under one-quarter (23.3%) of psychiatric admissions in our data were coded as elective. However, some providers class all mental health admissions as emergencies; hence, elective and emergency psychiatric admissions cannot be consistently distinguished in routine data due to variation in coding. 22 On the advice of our study steering group (including policy experts, clinicians and people with SMI) we therefore pooled all admissions irrespective of how they were coded by the provider, and conducted sensitivity analyses.
Measures of practice quality During our study period, the QOF mental health domain included five indicators to incentivise proactive disease management for a population where low adherence to medication, or drug levels outside of a therapeutic range, may lead to relapse and hospitalisation (table 1). 23 Two of these indicators (MH6 and MH9) apply to all registered patients with a diagnosis of SMI, whereas the two lithium indicators (MH4 and MH5) apply only to patients with bipolar disorder. Our analyses excluded indicator MH7, which encourages follow-up of patients with SMI failing to attend their annual review, because practices that review all eligible patients cannot score on MH7 and so would not contribute to the analysis. We selected indicators on the basis of consistency over time within our study period (see online supplementary appendix table 2).
Under the QOF, practices may 'exception report' patients, 24 that is, remove inappropriate patients from the denominator used to calculate achievement. While exception reporting may reflect good-quality care, for example, because patients are deemed unsuitable for clinical reasons, it could also reflect 'gaming' by GPs, who can increase the number of points they earn by reducing the eligible population inappropriately. 25 The set of patients for each QOF indicator (all patients with SMI for MH6 and MH9, or all patients with bipolar disorder for MH4 and MH5) were divided into three mutually exclusive categories: those for whom the indicator was achieved (A), those who were exception reported (E), and those for whom the indicator was not achieved (NA). For each indicator, we calculated two measures of practice performance. The QOF incentive regime rewards GPs on the basis of reported achievement: which is set to 0 if all patients are exception reported. The percentage of patients with schizophrenia, bipolar affective disorder and other psychoses with a review recorded in the preceding 15 months. In the review there should be evidence that the patient has been offered routine health promotion and prevention advice appropriate to their age, gender and health status Patients with serious mental health problems are at considerably higher risk of physical ill-health than the general population, but are less likely to be offered health promotion advice. The annual review should cover: 1. Accuracy of prescribed medication 2. Issues related to alcohol/drug use 3. Smoking and blood pressure 4. Cholesterol checks 5. BMI 6. Risk of diabetes from olanzapine and risperidone 7. An enquiry about cough, sputum, and wheeze 47

Lithium indicator 1 [MH4]
The percentage of patients on lithium therapy with a record of serum creatinine and TSH in the preceding 15 months Lithium monitoring is essential due to the narrow therapeutic range (0.6-1.0 mmol/L) of serum lithium and the potential toxicity from intercurrent illness, declining renal function or co-prescription of drugs, eg, thiazide diuretics or NSAIDs which may reduce lithium excretion. It is therefore necessary to check calcium and thyroid function on a regular basis as well as renal function. There is a much higher than normal incidence of hypercalcaemia and hypothyroidism in patients on lithium, and of abnormal renal function tests. Overt hypothyroidism has been found in between 8% and 15% of people on lithium

Lithium indicator 2 [MH5]
The percentage of patients on lithium therapy with a record of lithium levels in the therapeutic range within the previous 6 months Sources: QOF guidance. 23 47 48 BMI, body mass index; NSAID, non-steroidal anti-inflammatory drug; QOF, Quality and Outcomes Framework; SMI, serious mental illness; TSH, thyroid-stimulating hormone. Given that the appropriate level of exception reporting is uncertain and we cannot distinguish admissions for patients who were exception reported from those who were not, we followed Kontopantelis et al 26 in using: as our preferred practice performance measure in our main analysis. We used three sensitivity analyses to investigate the effect of including exceptions in calculating achievement. First, proportions of exception-reported patients included in the denominator were successively increased by 10 percentage points from 0% (equation 1) to 100% (equation 2) to identify potential switching points (ie, the levels at which the sign and statistical significance of the estimated coefficient changed). Second, we stratified practices into tercile groups by their level of exception reporting and interacted the respective achievement rate with the population exception rate: This sensitivity analysis was restricted to indicators MH6 and MH9 as there was insufficient variation in exception rates for the bipolar indicators to classify them into terciles. Finally, we also tested a model including the population exception rate as a separate covariate.

Covariates
Data on hospital admissions and practice quality were linked to GP practice characteristics, their patient population characteristics and population characteristics including deprivation and other potential confounders recorded at small-area level (table 2). We also controlled for measures of access to care and modelled Primary Care Trust fixed effects to account for differences in resourcing of crisis resolution and home treatment teams providing alternative home care in emergencies and playing a 'gatekeeping' role in hospital admissions. [27][28][29] Year indicator variables were used to account for temporal trends. In order to reduce potential bias from unobserved practice-specific confounders, we included presample baseline admission numbers per practice

Analysis
Random effects Poisson regression models were estimated to relate the number of psychiatric admissions per practice to its QOF achievement, conditioning on potential confounding factors and a normally distributed GP practice random effect with zero mean and constant variance. 31 32 The numbers of practice-registered patients with SMI or bipolar disorder were used as exposure terms. We obtained cluster-robust Huber-White SEs to account for potential overdispersion. 31 Coefficient estimates are presented as incidence rate ratios (IRR) with 95% CIs, so that a coefficient less than 1 indicates that the variable reduced admissions and vice versa. We also calculated the average effect of a 1% increase in QOF achievement on admissions. All analyses were conducted in Stata V.13. Separate models were estimated for the SMI and bipolar admissions. The model for all SMI admissions included MH6 and MH9 as explanatory variables and for bipolar admissions included MH4 and MH5. Achievement scores were introduced as sets because in practice they are likely to be achieved jointly (the Pearson's correlation between MH4 and MH5 was 0.369, and between MH6 and MH9 was 0.585).
We conducted several additional robustness checks. First, we tested the effect of including QOF indicators separately (rather than as sets). Second, we estimated the model only on within-practice variation, where any time-constant (un)observed practice effects were conditioned out of the likelihood. Third, we used a dependent variable based on numbers of individuals admitted at least once in any given year (rather than total numbers of admissions) to test whether individuals admitted frequently ('revolving door' cases) 33 distorted observed admission rates. Fourth, we tested the effect of using only admissions coded as emergencies. Fifth, we ran models using 1-year, 2-year or 3-year lags between QOF scores and admissions. Lastly, we estimated separate (cross-sectional) models for each study year to check for temporal effects not already accounted for in our regressions.

Descriptive statistics
Our sample included 8234 GP practices that treated people with SMI during the 5-year period (38 774 practice-year observations; mean follow-up 4.8 years). The number of practices (8052) and practice-year observations (37 573) were lower for the bipolar sample because not all practices treating people with SMI also treated people in the subsample of bipolar disorder. The median number of people with SMI per practice was 39 (interquartile range (IQR)=22-64) and the median number of people with bipolar disorder was 6 (IQR=3-10). The median number of annual admissions per practice was 3.5 (IQR=1-5) for SMI, and 1.1 (IQR=0-2) for bipolar disorder.
Over time, average practice QOF achievement improved across all four indicators, whereas the exception-reporting rate declined (table 3).
Reported achievement, where exception-reported patients were excluded, was not statistically significantly associated with admissions for any of the four indicators (see online supplementary appendix table 3). However, the association between achievement on MH9 and psychiatric admissions was significant when at least 30% of exception-reported patients were included in the denominator (figure 1). Given the median SMI practice register of 39 patients, the overall ( pooled) MH9 exception rate of 12.5% (table 3), and a switching point of 30%, we calculated that approximately 1.5 exceptionreported patients per practice year (=39×0.125×0.3) needed to be included in the denominator for the positive effect of MH9 on admissions to be statistically significant. The association between QOF achievement on MH4 [MH5] and admissions for bipolar disorder was statistically significant if at least 0.02 [0.08] exceptionreported patients were included in the denominator.
Covariates generally had anticipated plausible and significant effects (see online supplementary appendix table  4 for results of all coefficients for the main specifications). Results for reported achievement rates (equation 1) for the main analyses and sensitivity checks are provided in online supplementary appendix table 3.

Sensitivity checks
Findings were generally robust to sensitivity analyses, including patients admitted at least once, within-practice effects, lagged QOF achievement scores and for only admissions coded as emergencies (table 4) although results were not always statistically significant. The effect of population achievement rates on admissions was similar across practices with high/medium/low exception rates. When the model also included the separate population exception rate, only achievement on MH4 had a positive and statistically significant effect on admissions. For all indicators, population achievement was significantly associated with higher admissions if the indicators were included separately rather than together. Cross-sectional analyses broadly supported the results for MH9, but showed that the estimates' association diminished over time. In addition, the lithium indicators (MH4 and MH5) were not significantly associated with admissions in individual years. None of the sensitivity checks found that the QOF indicators were associated with significantly lower hospital admissions.

DISCUSSION
Our study is the first to investigate the relationship between GP practice quality, as measured by four mental health QOF indicators, and psychiatric admissions in the English NHS. Contrary to expectations, we found that better performance was associated with a higher psychiatric admission rate on three of these indicators, although the likely magnitude of any effect was small.
The potential for higher quality primary care to reduce emergency admissions is an important issue 3 7 and existing research has addressed this in several disease areas, with mixed results. [34][35][36][37] The evidence on the effectiveness of the QOF on admissions is similarly mixed. [16][17][18][19][20][21] However, we are not aware of any finding that better quality care is associated with a significantly higher rate of admissions. Our study used longitudinal panel data rather than cross-sectional data, which allows us to control for unobserved time-constant confounders, therefore improving the robustness of results compared with previous research.
We explored the robustness of results to variations in exception reporting in practices and showed the results were sensitive to this. A previous study examining exception reporting found wide variation across practices and across indicators. 38 We found a high level of exception reporting for MH9. These may be because a patient is deemed unsuitable for clinical reasons, or because a patient received at least three invitations for review during the preceding 12 months, but did not attend, or they refused to be treated. It is not possible to identify an appropriate level of exception reporting, but some people with SMI may be harder to reach due to the nature of their mental health problems; thus, practices may face difficulties in establishing and maintaining contact, and some degree of exception reporting is, therefore, expected.
There are a number of limitations to our study. As with all observational studies, we cannot ascertain causality. Our results may be affected by unobserved time-varying confounders that correlate with achievement rates. If these factors are positively associated with admission rates and with achievement rates then our estimate of the effect of achievement rate on admissions will be biased upwards. Also, since QOF data are reported at practice level, we cannot be sure whether admitted patients had been reviewed or exception reported. This makes it difficult to ascertain timing and causality. For example, it is possible that those admitted then received a QOF check arranged on discharge from hospital which could explain the positive association. Alternatively, patients with SMI whose problems are more severe may be preferentially registered with practices that are better equipped to provide their care. These practices may achieve higher QOF scores, but also uncover more unmet needs and have more admissions because of their case-mix. Both explanations would imply that the estimated positive association between QOF performance and admissions is not causal.
A second data limitation is that a few of our control variables are time invariant because they are based on census data. It may be possible that the underlying factors (eg, the ethnic composition of neighbourhoods) have changed over our study period, which may bias our results in unknown directions.
Two further limitations arise from our defined inclusion criteria and outcomes. First, we counted all admissions for people with bipolar disorder even though some may not have received lithium therapy, reducing the likelihood of finding effects for MH4 or MH5. Second, we implicitly assumed that psychiatric admission is a poor outcome. However, QOF checks may uncover mental health problems best addressed by admission and we cannot distinguish appropriate admissions from those representing avoidable failures in care.
Finally, it is possible that QOF indicators do not accurately measure the quality of primary care for SMI. The QOF, like any other pay-for-performance scheme, may result in tunnel vision 39 or a focus on areas of activities which are incentivised, sometimes at the expense of other non-incentivised activities. 40 Thus, high QOF attainment may not necessarily reflect high-quality care.
Further avenues are ripe for exploration including analysis at the patient level, rather than practice level, which would allow for detailed exploration of the entire patient pathway, including identification of the timing of QOF checks for admitted patients. Other priorities include consideration of non-QOF measures of primary care quality that might reduce admissions more effectively and could be incentivised through pay-for-performance; identification of types of secondary care for which utilisation may be affected more by primary care; and investigating whether some practices are more successful than others in getting patients admitted and whether this correlates with their QOF achievement, particularly relevant in the light of high occupancy levels due to closures of mental health beds. 41 Finally, greater understanding of unmet needs for people with SMI is essential. 42 Prevalence of an unmet need relates to mental healthcare provision and to socioeconomic circumstances-the less integrated and continuous care and the poorer the life situation, the higher is the unmet need. 43 More precise estimates of unmet needs can inform policy initiatives to ensure primary care is appropriately equipped and incentivised.
While current policy places an emphasis on 'upstream' prevention and 'early intervention' to reduce the need for more intensive and expensive specialist care, 44 45 our findings raise doubt about whether improvements in primary care quality, as measured by the QOF, are likely to achieve this. This chimes more broadly with concerns about the effectiveness of current approaches to reduce avoidable secondary care use. 46 While the QOF was not explicitly designed to reduce hospital admissions, there may be effective alternative primary care interventions that could be incentivised in the QOF.