Article Text
Abstract
Objective To test the feasibility of applying population impact measures utilising local population data on established interventions for heart failure and diabetes mellitus.
Design Modelling study.
Setting Registered general practitioner (GP) population in a primary care trust (PCT)
Data sources Local data sources included the quality and outcomes framework, chronic disease registers for coronary heart disease and diabetes, hospital episode statistics and a range of published risk data in heart failure and diabetes.
Main outcome measures Number of events prevented in the population (NEPP) by increasing the uptake of established interventions expressed as the number of deaths, hospitalisations and cardiovascular events prevented.
Results Data from 17 GP practices (representing 55% of the PCT GP registered population) were used to derive the NEPP. A 10% increase in the number of eligible patients receiving ACE inhibitors (n = 191) could result in at least 18 fewer deaths (95% CI 9.8 to 27.1) and 32 fewer hospitalisations (95% CI 24.9 to 40.7) for heart failure every year. Only 45% of persons with diabetes with an above target total cholesterol were receiving a statin; increasing this to 75% (additional 921) could lead to 44 (95% CI 15.6 to 73.1) fewer cardiovascular disease (CVD) events over 5 years. Similarly, more rigorous blood pressure control in an additional 662 diabetic patients could result in 26 (95% CI −2.7 to 55.6) fewer CVD events over 5 years. There were differences in the potential impact of these interventions according to subgroups within the PCT, as defined by age and geography (locality).
Conclusions Local data and published literature estimates can be successfully combined to produce the number of events prevented within a locally defined PCT population (NEPP). Commissioners have shown interest in the utility of such a measure in identifying and quantifying areas for improvement.
Statistics from Altmetric.com
The importance of the commissioning process in improving healthcare provision and quality is becoming increasingly recognised.1 However, relatively little has been published on ensuring the commissioning process is evidence-based and systematic, rather than ad hoc or based on historical practice. A recent Public Health Information Strategy consultation document2 suggested that information systems needed to integrate health outcome information with the effectiveness of interventions. Evidence-based clinical information needs to be translated into population-based measures that can assist primary care trusts (PCTs) and its commissioners with their prioritisation and decision making.
Population impact measures (PIMs) have been devised to assess the impact of an intervention at a population level.3 An example is the number of events prevented in your population (NEPP) defined as “the number of events prevented by the intervention in your population” (over a defined period) (Box 1). Previously published studies4–7 have shown the potential value of the NEPP for local and national-level decision-making in prioritising interventions aimed at reducing the burden of coronary heart disease (CHD), mental illness and tuberculosis, implementing guidelines in primary care and assessing the impact of specialist services for heart failure.
NEPP- Number of events prevented in your population
NEPP = n×pd×pe×ru×RRR
n = number of people in the local population
Pd = proportion of population with the disease
Pe = proportion of population eligible for intervention (derived from the difference between “best practice” goal and current practice, adjusted for compliance with the intervention)
ru = risk of event in untreated population
RRR = relative risk reduction of intervention
In this study, we have used the NEPP, within a PCT-based population, to estimate the impact on health outcomes for those in two disease groups, diabetes and heart failure, who are already or could be in future exposed to the intervention.
Previous work using the NEPP has been based on literature only estimates of prevalence, baseline risk, reduction in risk associated with the intervention, compliance levels and likely uptake of the intervention. In this study, the uptake of the intervention can be measured directly as can the prevalence within the general practitioner (GP) population. A feature of the study is the use of actual clinical and prescribing data linked for each individual to provide a “real-world” estimate of impact in a PCT population.
More specifically, the study tests the feasibility of using locally derived population data from primary care
To calculate the number of deaths or hospitalisations that could be prevented among heart failure patients with increased ACE inhibitor provision.
To calculate the number of cardiovascular disease (CVD) events that could be prevented with improved lipid and blood pressure control in persons with diabetes.
To illustrate the potential of PIMs to assist with the PCT commissioning process.
Methods
Local data sources were identified from both primary and secondary care to populate the NEPP equation. Patient-level data were obtained directly from GP practice systems, both the diabetes and CVD registers were interrogated. Retrieval of data from GP practices that used the Egton Medical Information System (EMIS) was relatively straightforward requiring a predefined EMIS-based “aspects” query to be imported at each participating practice. The resulting data were then exported in Microsoft (MS) Excel format. Clinical and prescribing data were linked, aggregated and anonymised centrally at the PCT. The quality and outcomes framework (QOF) information was accessed centrally at the PCT.
Confidence intervals (CIs) for the NEPP calculation were obtained using a simulation method through an online calculator http://www.phsim.man.ac.uk/nepp/. We attempted to utilise as much locally derived data as possible. However, this was not always feasible, in particular hospital data from the hospital episode statistics (HES) tended to grossly underreport the disease admission types of interest. As a consequence, we reverted to published literature to derive the risk of hospitalisation.8–11
The NEPP calculation (Box 1) requires specific information to address the following:
What is the size and composition of the local population?
What proportion of the local population has the disease?
What is the risk of the outcome event of interest in the whole population?
What proportion of the population is currently taking this intervention?
What proportion of the population could potentially take this intervention (best practice, likely adherence, ability to tolerate, eg, side effects), taking adherence with the intervention into account?
What is the risk reduction evidenced from a particular intervention?
The primary care trust: what is the size and composition of the local population?
The PCT covers a population of 275 000 and comprises four geographical areas (localities) defined by local authority boundaries. The PCT has 33 GP practices ranging from single-handed to multipartner practices and each is affiliated to a particular locality. As the majority of GP practices use the EMIS system and familiarity among PCT audit managers was greatest with this system, it was decided to restrict participation to those practices using EMIS. Practices were invited individually to participate but their data was aggregated to locality level to prevent identification of individuals within small practices.
Prevalence of the disease (pd): what proportion of the local population has the disease?
The number of persons with diabetes was obtained from the QOF database. There was no specific database of heart failure patients although they are included as a subset of the CHD register. While there is some impact from ACE inhibitor treatment in all categories of heart failure, the impact is greatest in those with left ventricular systolic dysfunction (LVSDF). For this reason, we focused on LVSDF and used published estimates of the prevalence of LVSDF in the community.12
Prescribing and clinical data: how many people are receiving the intervention?
Prescribing data were obtained from GP practice computerised prescribing records using a simple EMIS search of the relevant disease register (diabetes or CHD) and exported in MS Excel format. Similarly, clinical data were obtained using a separate search strategy. Both the clinical and prescribing databases were then linked using the unique practice patient identifier (number). PCT audit managers performed the searches at each of the practices.
Risk of an event in an untreated group (baseline risk, ru): what is the risk of the outcome event of interest in the whole population?
The risk of a cardiovascular event among persons with diabetes was calculated as a weighted average for men and women, across all age groups from a previous study.8 This study used the 1998 Health Survey for England and the Framingham equation to calculate the mean risk of a CVD event among subjects of a particular age and sex. Local HES were reviewed to obtain the number of in-patient admissions for heart failure as a primary diagnosis. It was hoped that by considering this figure as a proportion of all those with heart failure in the community, we could derive the “risk” of hospitalisation for patients with heart failure. However, the local hospitalisation rates were very low and likely to be a gross underestimate, probably as a result of not recording heart failure as the primary diagnosis. Comparison with literature values confirmed this,9 and consequently, in order to provide a more realistic estimate, we used literature values derived from a prospective study for hospitalisation (table 1).8–10
The risk of death among heart failure patients is well documented in the literature and is applied here using an estimate from a UK-based population.10
Risk reduction of an intervention: what is the risk reduction resulting from a particular intervention?
Systematic reviews or good quality randomised controlled trials were used to obtain the relative risk reduction (RRR) of specific interventions (table 2).11 13–15 In common with the published literature, a 5-year RRR was used for diabetes,14 15 but for LVSDF, a 1-year RRR was used for both hospitalisation and death.10
Data handling
Data extracted in MS Excel format was uploaded to SPSS V.13 and the number of preventable events was then calculated by incorporating all relevant information required for the NEPP equation. A random sample of 257 linked records was used to estimate the prescribed proportions of statins in diabetic patients and ACE inhibitors in heart failure.
Ethics approval
Research Ethics Committee approval was sought but not required for the study. However, on advice, the approval of the PCT Governance Committee was requested and granted. Permission to access GP practice systems was obtained from participating GP surgeries.
Results
Results of data collection and calculations
Seventeen of the 26 practices using the EMIS computer system agreed to participate in data extraction, representing 55% of the GP registered patient population across the PCT. There were at least three participating GP practices from each of the four PCT localities. The results (tables 3–7) illustrate how calculations can be based on age groups (heart failure, tables 3 and 4) or within PCT locality (diabetes, tables 5–7). These can, in turn, be used to target interventions to particular age groups or to particular GP practices (provided there was agreement to be identified).
Heart failure and left ventricular systolic dysfunction
At the time of the study GP CHD registers only recorded patients with known CVD disease and heart failure, while some patients had a definitive LVSDF indicated there were inconsistencies in recording this feature dependent upon the extent of clinical investigation. This did not permit reliable and accurate derivation of community prevalence of LVSDF. Furthermore having identified a total of 5288 patients on the GP CHD registers, only 317 with definite LVSDF were identified. (Others coded with heart failure were unlikely to fulfil definitions used in the QOF; QOF criteria was altered shortly after the data collection had taken place. Practices were required to validate their heart failure patients against a more stringent definition that included the presence of LVSDF.) Compared with general population estimates this was considerably lower than expected. Consequently a published estimate of community prevalence of LVSDF (by age and sex) was used.12 We estimated that 66% of heart failure patients were currently receiving an ACE inhibitor and considered that a modest 10% increase in prescribing of ACE inhibitors could be achieved. Tables 3 and 4 show that a 10% increase in prescribing of ACE inhibitors (an additional 191 patients) could result in 18 (95% CI 9.8 to 27.1) fewer deaths from heart failure and 32 (95% CI 24.9 to 40.7) fewer hospitalisations annually in the population studied.
Diabetes
The PCT did not maintain a central register of persons with diabetes. However, QOF information (populated by GP practice returns) was used as a proxy to determine the prevalence of diabetes in the local population. Individual patient records, held at GP practices were used to obtain the required clinical and intervention data. Of the 9207 (table 5) patients on the diabetes register (in 17 participating practices), we observed that one third (2996) had a cholesterol greater than 5 mmol/l and 2648 (28%) of patients had a last recorded BP>145/85 mm Hg.
Improving cholesterol control (use of statins)
From a 10% random sample of those with higher than target cholesterol, we estimated that 44% overall were currently prescribed a statin. An increase in provision to 75% (ie, an extra 921 patients) was considered achievable and on that basis the NEPP indicated that 44 (95% CI 15.6 to 73.1), CVD events would be prevented over 5 years (table 6).
Tighter blood pressure control
Patients with hypertension can be on multiple drug therapy at varying doses, for this reason, we did not review detailed prescribing data. Instead, the “intervention” was defined as a “tightening of BP control”. This would entail a review of medication in this group to bring BP to below 145/85 mm Hg. On the assumption that BP could be better controlled or tightened in a quarter of this group (662 patients), the NEPP indicated that 26 (95% CI −2.7 to 56.6), fewer CVD events could be prevented in persons with diabetes over 5 years (table 7). The 95% CIs include no benefit, this being largely because of the wide CIs of the estimate of RRR, which also include no benefit.
Discussion
This study demonstrates the utility of population impact measures in quantifying the potential benefit in terms of untoward events prevented through improved uptake of established, evidence-based interventions. We estimated that across 17 participating GP practices, 18 deaths and 32 hospital readmissions might be prevented annually in heart failure patients, while 70 fewer CVD events might occur in diabetic patients as a result of improved use of statins and “tighter” control of blood pressure. Applying current tariffs, it is possible to translate the prevented events into economic savings that could be accrued over relatively short timeframes.16 The ability to incorporate local population specific data reduces the number of modelling assumptions and makes the results more relevant to health organisations charged with commissioning healthcare provision.
PCT response
Feedback was provided directly to the PCT commissioners and indirectly to GPs via relevant PCT groups. At the time the PCT were exploring different ways of using indicators to enhance practice-based commissioning. Having an intuitive, easy-to-interpret measure of the impact of improved patient management on the local GP practice population was welcomed. In presenting the information to the PCT, we provided examples of crude cost comparisons between further, targeted, provision of established interventions and the status quo. The PCT prioritised heart failure given imminent changes in the QOF criteria, which highlighted the need to validate diagnoses in these patients. A further piece of work was commissioned by the PCT to investigate the specific impact of improving interventions for heart failure.
With respect to diabetes, the PCT prescribing advisers and audit team used information derived from the current study to work with GP practices particularly on use of statins in those with above target total cholesterol. Given that positive working relationships with GPs have been developed by these teams, it was felt that they were best placed to feedback and develop in practice work.
Practicalities
A number of challenges, both pragmatic and political, were presented by this study, and these mainly centred on access to and extraction of GP-held data. This, despite regular feeds of similar data from practices to populate the QOF information held in PCT databases. Although some of the information required was slightly more detailed than for the QOF, it was no more so than routine GP practice audits undertaken elsewhere.17 This has implications for population indicators, which require GP-held data to produce locally relevant, valid and meaningful results. Given further refinement and enhancement of QOF reporting, it should in future facilitate the direct calculation of PCT-based NEPPs for a number of interventions.
Technical considerations limited the study to only those practices with EMIS software. Although this covered 55% of the GP-registered PCT population, future work will need to include those practices that use alternative systems. In addition, we also intentionally ran multiple searches on GP systems and linked the data sets, all of which may have overcomplicated the process. We have already identified ways in which the same information can be obtained with a single search on prevalidated disease registers.
Limitations
The main limitations of this study relate to the accessibility of local patient data and the pragmatic assumptions necessary to calculate the NEPP.
Previous work with population impact measures relied almost exclusively on literature-based values applied to hypothetical populations. Here we use real patients from, as far as possible, complete and well-maintained chronic disease registers in order to better model what would happen in practice within a UK-based GP-registered population. The starting point for the NEPP calculation is the derivation of an “eligible” population who could benefit from the intervention of interest; this takes account of those at risk, current uptake of the intervention and likely compliance.18 The combination of local GP data and a broad search strategy permitted robust derivation of the “eligible” population within the PCT.
In theory, it should also have been possible to derive the prevalence of the outcomes of interest (eg, hospitalisation in heart failure) using local hospital data. However, there may have been an issue with coding heart failure as the primary or even secondary diagnosis on admission. A search of the local hospital system failed to detect any of the heart failure patient group. Consequently, we used published hospitalisation estimates from prospective studies. The implications of using these estimates are their generalisability and the level of error inherent in their estimation. We used a UK “community” population-based estimate for baseline risk in heart failure and a non-UK-based estimate for outcome hospitalisation. It is noteworthy that heart failure patients are increasingly managed in primary care or home settings and so published hospitalisation rates may underestimate the prevalence of complications related to this disease.19
For CVD risk in diabetes, we used a pooled risk for both sexes and across all age groups (19%).8 While keeping the calculation of the NEPP in this group simple, this may not accurately reflect the number of events that could be prevented in different age and sex categories, as risk will differ in each of these. Future work could use separate age, sex and ethnic specific risks. Having also used a random sample of records to obtain the proportions receiving the intervention of interest in each disease, we would have preferred to examine intervention uptake on all available patients. We have addressed the relevant data extraction format to make this possible in future.
The use of published risk estimates (ie, baseline risk of disease and the RRR of the intervention of interest) carry with them a margin of error or imprecision normally denoted by CIs. To account for this, we derived the 95% CIs for the overall NEPP, which take account of precision of the risk estimates used. The NEPP and associated CI therefore indicate the range over which a true value for the NEPP is likely to lie. So, for example, the 18 deaths prevented in heart failure patients with increased ACE inhibitor use could be as low as 10 deaths or as high as 27 deaths. The wide CIs associated with the NEPP estimate for CVD events prevented as a result of tighter blood pressure control (table 5) illustrate the need to clearly define the intervention; tighter blood pressure control is a vague definition used in the UKPDS study.15 A further complication of using literature-based estimates is in ensuring that there is a like-with-like definition of the disease of interest, which is not always straightforward. For example, we used a strict definition of LVSDF for our prevalence estimate to fit with that used in NICE guidelines on prescribing ACE inhibitor.20
Conclusion
This study demonstrates the pragmatic use of PIMs to quantify the benefits of more rigorous implementation of established evidence-based practice. Despite practicalities and some modelling limitations, the use of local data to calculate the eligible population and the NEPP in a specific PCT population makes the impact of “local commissioning decisions” more realistic and relevant. The further development of locally derived NEPPs across a range of diseases seems a logical next step and offers a means of assessing quality improvement in commissioning decisions based on outcomes avoided, which is transparent and evidence-based.
Acknowledgments
We are grateful to the PCT audit managers for their assistance with this work.
References
Footnotes
Competing interests None.