METHODS

Overview

The clinical effectiveness study took a sample of patients with minor injuries attending the A&E department in 1996 and a second sample of patients attending the MIU in 1997. Each of these samples were subjected to a “double examination” and the findings compared.

The cost study used the dataset on all patients attending the A&E department in the financial year 1996/1997 and in the MIU in 1997/98. These data were adjusted for casemix differences to derive a “cost per workload unit”.

The cost consequences of follow up care were calculated from a second sample of patients to provide information on the follow up rates in each setting. Using actual costs of follow up visits, the total costs of follow up for minor injury patients in each setting were calculated.

Clinical effectiveness study

During 1996 the Royal Hallamshire Hospital (RHH) provided an A&E service 12 hours per day 7 days per week. After May 1997 this service was replaced by a nurse led MIU staffed and operated by nurse practitioners who rotated between the main adult A&E unit at the Northern General Hospital (NGH) and the MIU.

During the period from 7 August 1996 to 19 November 1996, 1447 patients with minor injuries attending the RHH A&E department were subjected to two examinations. The first was by the A&E doctor and then by one of two investigators. The investigators used pre-designed forms that recorded the “ideal” history and examination for the patient and the registrar made a judgement whether investigations were needed along with the treatment and follow up care needed. These decisions were based on departmental guidelines and “decision rules”. The A&E doctor was not aware of the findings of the research assessment and the researcher did not interfere in any clinical decision that was made. Further details of this methodology have been previously published.²² During the period 1 September 1997 to 31 January 1998 the same process was undertaken in the nurse led MIU and 1315 patients with minor injury were doubly examined firstly by the nurse and then by the research investigators.

At the end of the study the research investigators compared the “research assessment” as recorded in the research proforma with the “clinical assessment” as recorded in the A&E notes. The radiologist report was used as the “gold standard” against which radiological interpretation was judged. Errors were judged as “not clinically significant”, for example where a patient had not been asked tetanus immunisation status but was up to date with tetanus immunisation and therefore no treatment would have been needed. A “clinically significant error” was judged to have occurred if the treatment would have been different had the error not been made, for example where tetanus status was not asked but the patient needed an immunisation and this was not given. A “very significant error” was judged as one where there was a high risk of adverse outcome for the patient, for example where a tendon injury was missed.

Each separate part of the care was judged in this manner (history of the injury, previous medical history, examination, need for radiography, interpretation of radiograph, treatment, and ongoing care plan). Waiting time to see the clinician and department time were also recorded.

Cost study

When the changes to A&E services were announced steps were taken to harmonise the data collection and coding systems of the hospitals. During 1996 the RHH A&E department came under the managerial control of the NGH and thus accounting systems were also harmonised. Steps were taken prospectively to improve the completeness and accuracy of this routine data collection.

Revenue cost information was obtained directly from the costs of running the A&E service for the period April 1996 to end March 1997. Annual cost data for the MIU were calculated from the costs for the 11 month period May 1997 to March 1998 (the MIU opened on 1 May 1997). Budget lines were checked to ensure conformity. The costs for the A&E unit were adjusted for inflation (pay 2.5% and non-pay 0.4%). Costs included staff costs, investigations such as radiography, drugs, and disposable items.

Numbers of patients attending both departments were obtained for a one year period. However, the costs of treating the more complex cases attending an A&E department are greater than those for patients with minor problems who are discharged. Various methods have been used to adjust for the A&E resource use of different groups of A&E patients.²³ The NHS Information Authority has produced Health care Resource Groups (HRG) at discharge disposal for this purpose. We have used these as at the time the study took place they seem to be the best validated measure. Very recently these have been further updated.²⁴ The HRG methodology assigns a “multiplier” to be applied to the numbers of new patients attending a department that should correct for the difference in casemix. The multipliers used and the method were “minor case sent home” 0.75, referred to clinic 0.91, admitted 1.74 and died 2.31, and other 0.56. This allowed a revenue “cost per workload unit” to be calculated. These costs included investigation and treatment costs as well as the cost of patients reviewed within each department.

The cost consequences of planned follow up care were assessed in the following way. The database provided details of the intended follow up arrangements of patients. In the case of the A&E department the database was filtered to provide a dataset of all those patients with minor injury. The data collection forms for routine coding are clearly divided into “injury” and “non-injury” sections. We selected those patients with “injury “codes from both units but excluded those brought by ambulance. The same filters were applied to the MIU dataset. The follow up rates for general practice, outpatient clinic, and for the MIU main A&E review clinic were obtained. Costs estimates for each clinic were obtained from hospital accountants and for general practice from previously published data.²⁵ This then permitted the calculation of an extra cost of follow up per new “filtered” minor injury patient.

Analysis

Error rates (proportions of cases with errors judged by the “gold standard” of the research form/radiologist report) were compared using the χ² test. There were some differences in the casemix of patients in the groups mainly the location and type of injury. To adjust for these differences we used logistic regression analysis taking into consideration most possible confounders (age, sex, type, and location of injuries). For quantitative data such as age, waiting time, department times we used the t test to compare the means with 95% confidence intervals. All tests used were two tailed unless stated otherwise.

RESULTS

Effectiveness study

A sample of 1500 patients with minor injury were included in the A&E department arm of the study and 1447 were analysed, the remaining 53 (3.5%) were not included in the analysis because of lost clinical notes or part of the notes. A total of 1315 patient were included in the MIU arm. In the A&E department phase 1062 patients were seen by senior house officers, 297 by registrars, senior SHOs or consultants, and 58 by general practitioners. The groups were comparable in the age, sex, method of presentation (walking wounded or with recent minor injury), and triage category 4 (table 1). All patients who presented as 999 call even with minor injuries were excluded. There was a trend for more MIU patients to be female but this failed to reach significance (p=0.06). There was also some differences in the types of diagnosis and the areas of the body examined mainly more chest (A&E department 2.4%, MIU 0.3%) and head injuries (A&E department 6.4%, MIU4.1%). After initial analysis of the results we have also performed further analysis using logistic regression to adjust for these differences and to eliminate any confounding factors (table 6 on web site appendix).

Significant errors in any of the measures in the process of care (apart from too many radiographs) occurred in 191 (13.2%) of patients treated in the A&E department and 126 (9.6%) in the MIU (p=0.003). In some patients more than one error was made. Table 2 shows the error rate for each of type of error. Significant findings are that the nurses were better at recording a past medical history. The A&E department staff seemed to be better in examining the patient. However, after adjusting for the differences between the groups this was not statistically significant. Both groups requested many more radiographs than the research investigators thought necessary. There was no difference in the accuracy of radiological interpretation or treatment. Follow up was thought to be inappropriate in 3.2% MIU patients and 3.3% A&E department patients. There was only one “very significant error”, a missed tendon injury in the A&E department group. Waiting times were very much better in the MIU group. The mean waiting time for patients presenting to the MIU was 19 minutes while for those presenting to A&E department the mean waiting time was 56.4 minutes (p<0.0001, 95% CI of the difference 34.48 to 39.32). The total time spent in the department was also much shorter for patients presenting to the MIU (mean 51.5 minutes compared with 95.4 minutes for the A&E department p<0.0001 95% CI 36.27 to 42.75) (tables 2 and 3).

Cost analysis

During the year 37 427 patients attended the RHH A&E department and 13 616 the MIU. After applying the “multipliers” the number of “workload units” was 34 751 for the RHH A&E department and 11 329 for MIU (table 3). The costs for each unit are given in table 4. The revenue cost per “A&E department workload unit” using HRG methodology is £40.01 for A&E department care and £41.10 for MIU care.

Application of the “minor injury filter” to the both the A&E department and MIU patients yielded 17 909 patients attending the A&E department with minor injuries and 11 889 for the MIU. Analysis does show differences in casemix with more head/neck/trunk injuries in the RHH A&E dataset and more hand injuries in the MIU dataset. The MIU saw more cuts and less sprains and strains than the RHH A&E department.

The follow up rates for these patients were calculated for all clinics and there was a significant increase in referrals to general practice, fracture clinic, and eye casualty for patients treated in the MIU. In addition, more patients were being referred to the A&E department clinics. Cost for the returns to the A&E department and MIU are included within the total revenue costs given above. The follow up rates are given in table 5 along with the costs of outpatient attendances. This allows a calculation to be made of the total cost of referrals from each unit and thus an average “cost per case” for referrals can be calculated.

This table shows the total annual initial follow up costs of an MIU visit as £144 730 or £12.17 per “filtered minor injury case” and for the RHH A&E £164 923 or £9.66 per “filtered minor injury case”.

DISCUSSION

The reorganisation of A&E services causes heated debate, often without any evidence behind very emotive arguments. We have studied the effects of the changes in Sheffield in great detail and have published results on the changes to patient flows, process quality data, and overall revenue costs,²⁶ details of the effects on the care of patients needing thrombolysis,²⁷ and are preparing work on the impact of the changes on the care of patients with serious injury. The use of nurse practitioners has been widely advocated as a solution to many problems but it is only recently that evidence on the effectiveness of emergency nurse practitioners in an A&E department setting has been published.²² This trial compared the care of minor injury provided within the A&E department by A&E senior house officers or nurse practitioners. The study found that the nurse practitioner care was as good as that provided by SHOs. Measures such as patient reported outcome at 28 days showed no differences between the groups but patients in the nurse practitioner group had fewer unplanned follow up visits. However, the setting for this trial was an A&E department and therefore we thought it important to assess the care provided in the MIU setting.

The error rates for the “package of care” show significantly less process errors in the MIU patients (p=0.003). The error rate seems to be high but this is a comparison of real clinical situations against a very rigorous “gold standard”. Obviously where a significant error was made there was the potential for the patient to suffer an adverse effect. Very significant errors with a high probability of adverse outcome were very rare (one case).

There were differences in individual elements of the package of care. Past history was more frequently recorded in by the nurse practitioners whereas the A&E doctors seemed to have made less errors in examination. However, after adjusting for the differences in the groups this did not reach the statistically significant level (p=0.075). There was no difference in radiography requests or interpretation. Errors in treatment were similar in both groups. These trends were all noted in the previous study. The nurses did arrange more follow up but this was judged to be appropriate most of the time. The lack of available senior opinion in the MIU meant that the nursing staff were acting in safe manner by requesting further review.

Our conclusion is that the care provided by emergency nurse practitioners in the MIU is safe and the overall process of care is equal to and in some respects superior to that provided for minor injuries in the A&E department. The waiting times were much less. The nurse practitioners arrange more follow up but this was judged as appropriate given the setting and lack of senior advice.

The cost study was important as many perceive that MIU care is cheaper than A&E department care. It was difficult to make a direct cost comparison between the two groups given the different casemix. The HRG groups are the most widely validated measure of resource use in different types of A&E patients. Using this to adjust the workloads, the costs were almost equal with the A&E being slightly cheaper. The costs in the A&E department include many-non service costs such as educational time for the junior medical staff, teaching time of senior medical staff, research, and management time of senior staff. There was some management time built into the MIU costs but few costs of ongoing education or training. If these factors were to be taken into account the cost differential for the pure patient care parts of the service would have been wider. On the other hand the waiting time figures for the MIU were much better. Therefore some of this extra cost is probably “buying” better quality. Similarly the comparatively less pressured MIU work conditions may allow more time to be spent with patients, better recording of history, and treatment plans. These findings are in line with the use of nurse practitioners in primary care where they tend to spend longer with patients and make less process errors.

The A&E department costs include middle grade and consultant staff costs and therefore easy access to senior medical advice in the A&E department cohort. This may account for the lower follow up rate in A&E department patients. The study does not examine some of the quality benefits that may flow from a nurse practitioner service such as increased patient education.

The planned follow up rate for MIU patients was higher than that of the A&E department. Some of this follow up was to the MIU itself and thus the costs would be contained in the MIU costs. However, there were also increased follow up costs is general practice, the main A&E department, and in outpatient clinics. Our previous work did show that junior A&E staff have higher rates of unplanned follow up than nurse practitioners, perhaps indicating a reluctance to arrange follow up where it is needed. We were unable to assess that if unplanned follow up was greater in the A&E cohort in this study. It may be argued that the increased planned follow up represents better care, but the cost to the health service and to the patient in time and transport costs is significant.

Weaknesses of the clinical effectiveness study

A major problem in assessing the quality of care in patients with minor injuries is that there is a low incidence of adverse outcomes in this patient population and while the ideal study would have reviewed every patient at 28 days to assess outcome this was not feasible for large numbers of patients, many with comparatively minor conditions. We therefore chose to examine the process of assessment, treatment, and referral and compare these with a “gold standard” of treatment.

A significant source of bias of the effectiveness study was that the researchers were not blinded to the type of unit. This flaw was one reason we undertook the randomised trial with blinded assessments.²² It is reassuring that the findings of this trial are similar to those of the blinded study. However, there are some indications that more patients were referred for follow up in the MIU group, this did not account for any increase in the clinically significant errors as patients were not subjected to any increased risk, but has accounted for more use of resources. The knowledge that the A&E staff would have the ability to seek a senior medical opinion on site while the nursing staff did not have this facility may be the reason for these judgements. The lack of blinding could not have affected other process measures such as the accuracy of radiological interpretation. Another weakness in this study is that it was not possible to carry out outcome measurement at 28 days as was done in the randomised study.

Weaknesses of the cost study

Weaknesses of the cost study are that the “cost per minor case” was a derived figure and was not obtained from “bottom up” costing. We did not have the manpower to do such a detailed observational study. However, the findings agree with our previous cost study that did use direct observation of time spend by A&E staff and nurses in assessing these patients and did find that the nurse practitioners took longer to assess patients and salary costs were greater than those of A&E senior house officers. The NCO HRG methodology is the most extensively validated tool available for such cost comparisons. It has been refined very recently but this method was not available at the outset of the study. It has not been validated in an MIU setting and therefore costs such as educational time, teaching, research, and management time that are factored into the A&E budget are not large expenditures in the MIU budget.

This part of the study has not been corrected for the small differences in the patient population that were subject of correction by logistic regression in the effectiveness study. The differences were small and it is the most prominent being an excess of minor head and chest injuries in the A&E department group. It is unlikely that the costs of these injuries would be less than those for other minor injuries or have a major impact on the cost analysis.

SUMMARY

Nurse practitioners working in a MIU provide a clinically safe service. The revenue costs were greater but quality measures such as waiting time were better. There were additional costs attributable to increased follow up rates although again this may indicate better quality of care and prevent the need for unplanned follow up that was found in patients treated by A&E junior staff in our previous study.