Article Text
PDF
Abstract
Objectives To assess the costs of hospital accreditation in Australia.
Design Mixed methods design incorporating: stakeholder analysis; survey design and implementation; activity-based costs analysis; and expert panel review.
Setting Acute care hospitals accredited by the Australian Council for Health Care Standards.
Participants Six acute public hospitals across four States.
Results Accreditation costs varied from 0.03% to 0.60% of total hospital operating costs per year, averaged across the 4-year accreditation cycle. Relatively higher costs were associated with the surveys years and with smaller facilities. At a national level these costs translate to $A36.83 million, equivalent to 0.1% of acute public hospital recurrent expenditure in the 2012 fiscal year.
Conclusions This is the first time accreditation costs have been independently evaluated across a wide range of hospitals and highlights the additional cost burden for smaller facilities. A better understanding of the costs allows policymakers to assess alternative accreditation and other quality improvement strategies, and understand their impact across a range of facilities. This methodology can be adapted to assess international accreditation programmes.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This study identifies the cost of hospital accreditation across a range of acute care facilities.
We provide a purpose designed survey tool that can be adapted to cost accreditation internationally and across the acute, primary and aged care domains.
Our study assesses the direct financial incremental costs associated with the accreditation survey process.
The main limitations relate to the small sample size (n=6) as the wide variation in costs across hospitals of different sizes makes it difficult to accurately scale the results on a national level.
We reduced survey recall bias by verifying the data wherever possible by asking for documented confirmation of timetables and attendance logs for meetings and training sessions.
Introduction
Modern systems of quality management are complex and require monitoring to ensure compliance with rules and regulations. Health service accreditation programmes provide an important mechanism by which governments can direct quality improvement programmes, aiming to ensure that healthcare organisations maintain compliance with governmental legislation, regulations and guidelines, as part of normal business practice. While there are differences in detail, accreditation processes and designs are similar across the USA, Canada, Australia and parts of Europe.1–4 Common across these countries is the establishment of programmes to assess performance against clinical and organisational standards. A blend of methods is then used to determine performance against these standards including; external surveys, self-assessment surveys and indicator measurement.5 Health service accreditation programmes, particularly in acute care, are well established and have seen rapid spread.6–8 However, the cost of participation has remained largely obscure.9 While there are important benefits to be realised from accurately understanding the financial costs of participation in accreditation, the question deals with sensitive financial information, and is challenged by a lack of available tools and methodologies. As a result, our knowledge in this area is limited.10
One review showed incremental costs for undergoing healthcare accreditation of between 0.02% and 1.7% of operating costs when averaged over the accreditation cycle (studies=6, facilities=1–102).9 However, the results were mainly based on self-reported, single studies, from hospitals in the USA. The timing of the reports, one based on 1996 data, suggests the findings may be out of date with respect to current accreditation and organisational practices.11–13 Some studies did not include overall costs for all facilities so comparative estimates, as a percentage of overall costs, were not possible,14 ,15 but highlighted the difficulty of persuading hospitals to participate in such investigations. A larger study of 102 methadone clinics in the USA estimated accreditation costs at 1.7% of operating costs, averaged over the accreditation cycle and noted the proportionally higher costs borne by smaller units.16 The peer-reviewed literature on the costs of acute care accreditation appears limited and there is little evidence of costs being assessed over a range of acute care facilities. Our study aim was to independently estimate the incremental costs of accreditation across a range of acute care settings as part of a larger project to evaluate the costs and benefits of acute care accreditation in Australia.17 ,18
Methods
Study setting, design and participants
We reviewed the costs related to accreditation surveys conducted by the Australian Council on Healthcare Standards (ACHS) using versions four and five of their Evaluation and Quality Improvement Programme (EQuIP) during 2009–2012.19 We analysed activity-based costs relating to the accreditation survey process adjusted for each year of the 4-year accreditation cycle using our purpose-designed incremental costs audit tool.
Our mixed methods study design incorporated: stakeholder analysis; survey design and implementation; activity-based cost accounting techniques; and an expert panel to review and validate our results.20 Invited participants included public and private acute hospitals who were accredited by the ACHS EQuIP programme. We conducted interviews with participating hospitals during 2013 and 2014.
Costing methods
Previous studies indicated that accreditation activities were not specifically identified under current hospital accounting methods.21 We therefore determined to measure costs by surveying hospitals using a well-established activity-based costs approach that was developed in the 1980s to help identify costs more accurately in complex systems.22 We wanted to identify the direct costs that can be directly attributed to the accreditation surveys such a labour and material costs. We did not include indirect, or overhead, costs. This was partly due to inconsistent depreciation schedules causing us to use operating rather than total costs as the denominator and enable comparison across States, but also due to our focus on incremental costs and we determined that many of the overhead costs would not change over the accreditation cycle.
Focus group for stakeholder analysis
A focus group of health services researchers was convened to identify the stakeholders involved in costing accreditation activities. The group comprised seven researchers with, on average, 6.6 years working in health services research and having contributed to an average of 47 peer reviewed research papers. The group identified the main stakeholders for assessing the incremental costs of accreditation as: accreditation agencies; agencies involved in developing accreditation programmes, including designing and structuring the survey; and hospitals undergoing accreditation surveys. ACHS both developed and maintained the standards used during the study period in addition to administering the programme.19 The group agreed ACHS fees could be used as a proxy for the agency and standards development costs. ACHS largely uses surveyors employed in the health sector but some of these are seconded by their healthcare employers and are not paid by ACHS. Consequently, the group recognised the need to estimate any additional costs borne by the health system for these surveyors.
Developing the incremental costs of accreditation audit tool
In order to understand the activities involved in accreditation, members of the study team attended three hospital accreditation surveys as observers. Drawing on this experience, we developed an activity map of the connections between hospital staff groups and accreditation agencies (figure 1) in order to identify potential cost areas.
Accreditation survey activity map. ACHS, Australian Council on Healthcare Standards; EQuIP Evaluation and Quality Improvement Programme.
The research team used the activity map to identify accreditation activities that met our incremental costs definition. We drafted a purpose designed incremental costs audit tool through an iterative process, with feedback from the study team and ACHS staff and surveyors. We tested the tool at a pilot site and reviewed the findings to ensure feasibility of the tool. The finalised incremental costs audit tool, encompassing activity groups and activities, is presented in table 1.
Incremental costs audit tool—activity groups and activities
Implementation and analysis of the incremental costs audit tool
To recruit hospitals to the study, ACHS forwarded an invitation from the research team to their member hospitals. Consent to take part in the study was obtained from each hospital that responded to the invitation, and we arranged to conduct phone or in-person interviews as appropriate. A total of 11 hospitals initially agreed to take part in the study, but several hospitals withdrew due to reorganisation, staffing and timing issues. These hospitals comprised one private hospital, one medium sized hospital and three principal referral hospitals. Our study sample comprised all hospitals that did not withdraw from the study.
We conducted semistructured interviews to collect documentary evidence for the costed activities (eg, accreditation-related meetings, accreditation training and survey-related meetings) over the 4-year accreditation cycle. We interviewed senior managers, quality control managers and accreditation survey coordinators in each facility to determine the costs over the 4-year accreditation cycle. We cross checked the results of the survey with documentary evidence in the form of internal meeting requests, meeting minutes and the information packs sent to surveyors. In addition we obtained data on non-staff costs such as training costs and accreditation fees.
We used relevant State health authority guidelines to determine staff salary levels and translated the annual rates into total costs per hour based on government guidelines for measuring regulatory costs, which incorporates overheads and non-payroll costs (table 2).23 The recommended on-cost rate of 17.58% includes a payroll tax allowance of 4.5%.23 Although public hospitals in Australia are exempt from paying this tax, we did not adjust the on-cost rate as the overall figure (17.58%) was within the range of data we reviewed from individual hospital accounts.24–26
Calculation of hourly labour rates
To account for the change in staff rates over the 4-year accreditation cycle we used either current pay scales adjusted back to the year of the survey, or concurrent pay scales where available.27 ,28 In keeping with Australian accounting practices, we used an end of June fiscal year (FY) to align the results of our audit tool with hospital financial reports. We estimated an average wage inflation rate from 2010 FY to 2013 FY of 4.14% per year, which we used to adjust staff costs to the year incurred.29 ,30 We then added non-staff costs (eg, accreditation agency fees, accreditation workshop fees or external consultant fees) to assess the incremental costs of accreditation per year incurred.
To compare costs across different hospitals we calculated the incremental costs of accreditation as a percentage of operating costs, obtained from annual financial reports. A reorganisation of geographic health districts in 2011 created problems of non-compatibility across reporting years for some hospitals. We used adjustment rates to assess costs where annual report data was not available, and to account for the different base years for each cycle. We calculated our operating cost adjustment rate of 10.88% from the average change in recurrent expenditure across the survey years.29 ,30 We used different rates to adjust the staffing and operating costs as the latter incorporated a staff volume component, in addition to non-salary items. We then averaged the incremental costs of accreditation as a proportion of operating costs over the survey cycle.
Participating hospitals were grouped according to the Australian Institute for Health and Welfare (AIHW) peer groups.30 Specialist teaching and principal referral hospitals were classified as group A. The other groups were determined by size and activity using the number of episodes of admitted care in each facility: large hospitals comprised group B; medium hospitals group C; and small regional and remote hospitals group D.
We conducted a sensitivity analysis of our findings using a range of different levels for each of our main assumptions,31 including the on-cost multiplier and our adjustments for wage and hospital inflation. Our base case assumes an on-cost multiplier of 17.8%. Estimates for on-costs in the public health system were 10.71% in 2012 FY national data, similar to a previous survey of 10–13% in 2000.29 ,32 However, on-costs estimated from the annual reports of the hospitals in the survey show a higher range of 19–26%.33 ,34 We therefore used a range of 17.8% plus and minus 10 percentage points (27.8% and 7.8%), which covers both the high and low data points within this range. Our sensitivity analysis also included the effect of changing our staff and operating cost adjustors by 10% (from 4.14% to 4.55% for staff costs, and from 10.8% to 11.9% for operating costs).
Expert panel
We convened an expert panel to review the methods and results.35 The nine member panel comprised four health services researchers, three senior executives from the acute (two) and primary (one) care accreditation agencies, a senior representative from a heath safety and quality body and one accreditation quality consultant with 12 years as a senior executive in the aged care accreditation sector. We included accreditation representatives from the non-acute sector to broaden the diversity of the panel and learn from their experience in costing primary care accreditation. The panel session discussion focused on three questions: (1) do you think the results from the audit tool are representative of the incremental costs associated with the current accreditation process?; (2) if these costs are not representative, what is your estimate of these costs?; and (3) are there other incremental costs that have not been included, and if so what would be your estimate of these costs?
Results
Expert panel
The panel approved the methods and results obtained from the incremental costs audit tool developed by the research team. The panel determined that using an incremental cost approach had merit in terms of being used in previous studies in primary care and the fact that the accreditation standards were closely allied with existing policies, rules and regulations. Some of the panel members expressed surprise that the incremental costs were below 1% as the costs have been an issue in other research. Initially we collected data on the cost of remedial work arising from the accreditation surveys. However, the panel suggested these items not be included in the analysis, as that they related to compliance with the standards and not to the survey process. The panel also endorsed the weighting methodology for determining the overall costs to the health system, but recommended increasing the sample size in future research due to the wide variation in costs between hospitals.
Analysis of data from the incremental costs audit tool
Six acute public hospitals, from four different states, completed the incremental costs audit tool. The participating organisations were as follows: one specialist teaching facility; one regional referral hospital; one large regional hospital; one medium hospital; and two small rural hospitals. All hospitals had undergone previous accreditation cycles with ACHS.
The incremental costs of accreditation (ICA), averaged over the accreditation cycle, varied from 0.03% of total costs for the specialist teaching hospital to 0.60% for the smallest rural hospital (see table 3). We show the incremental costs of accreditation by survey year for ease of comparison across the different activities in the cycle. The incremental costs of accreditation are shown by year incurred.
Yearly ICA by hospital
Absolute costs were highest in the regional referral hospital (hospital 2) and lowest in the smallest regional hospital (hospital 6). However, the absolute costs for the specialist teaching hospital were only 8.5% higher than at Hospital 6, despite their difference in size. We also note the largest expenditure for each hospital was in the external survey years at an average of 0.4% versus an average of 0.072% in the two self-assessment years (years 1 and 3). We show the incremental costs per hospital at different stages of the accreditation cycle to highlight the differences between the hospitals in the sample (see online supplementary file 1).
Sensitivity analysis of the incremental costs of hospital accreditation
We conducted a sensitivity analysis (see online supplementary file 2) using on-costs at 10 percentage points above and below the base case, and operating and staff costs with a 10% adjustment. The results are mixed due to the differing effects of financial and accreditation cycles. In addition, the results from some hospitals are not particularly sensitive to changes in staff and operating cost adjustors as we only adjusted where we did not have actual data for that year. We did not include the overhead cost multiplier in our calculation of hourly labour rates; applying the government recommended rates of 25% would increase our ICA estimates by 20.5% to $A44.37 million, equivalent to 0.106% of total recurrent expenditure, including depreciation, in FY2012.30
Scaling the results to estimate national costs of accreditation
The variation in results between hospitals in the different AIHW groupings meant that using an overall average to determine national costs would not be representative. The variation within each group was also a concern due to the small sample size. The national costs, which we provide to illustrate the methods used, were calculated by averaging the incremental costs for each AIHW peer group. This was weighted by the admission activity in each group in order to account for the contribution of each group to total national hospital activity. In 2011 FY the 91 group A hospitals comprised 25% of total acute public hospitals and accounted for 73.3% of acute admissions.30 In contrast, the 155 small acute hospitals, AIHW group D, comprised 42% of the total number of public hospitals but accounted only for 2.7% of total admissions. Using admission activity weights, we estimate the weighted incremental costs of accreditation at $A35.94 million, based on 2012 FY data (table 4).
Estimating national ICA costs for acute public hospitals
Only a proportion of the surveyors are paid by ACHS and we needed to impute a cost for the remaining surveyors. Surveyor teams usually comprise an administrator, a clinician and a senior nurse, with the number of surveyors and days spent on the survey, varying with hospital size. The ACHS annual report indicates 470 surveyors were active in 2013 FY, with the majority employed within the health system.36 Using data provided by ACHS, we estimate there were a total 4169 surveyor days during 2011 and 2012. We used an average of the medical specialist, administrator and Director of Nursing hourly rates, with a conservative half-day travel time at the beginning and end of each survey to compute a total cost of $A2.96 million per year. Based on discussions with accreditation surveyors, we assume 30% are effectively being subsidised to attend the accreditation survey by their employing hospital, this equates to an additional cost of $A0.89 million, rising to $A2.1 million if 70% of surveyor costs are included (see table 4). Combining surveyor costs with the weighted ICA total we estimate national ICA costs of $A36.83 million, 0.1% of the recurring acute care costs of $A39.76 billion in 2012FY.37
Impact of the new National Standards
Our study estimates incremental costs relating to the EQuIP 4 and 5 programmes during our study period (2009–2012). In 2013, Australia moved to a process of mandatory accreditation for public hospitals with national standards developed by The Australian Commission for Safety and Quality in Health Care (ACSQHC). Our study included these as part of the accreditation fees but future studies would need to add these costs due to the role of ACSQHC and the National Safety and Quality Health Service (NSQHS) Standards in developing and maintaining the new mandatory standards.3 In a Decision Regulatory Impact statement ACSQHC estimated the impact of transitioning to the new standards for the accrediting agencies would be a one-off cost of $A1.058 million with a recurring cost of $A60 000 per annum incurred by each agency.38 ACSQHC did not publish an estimated cost of developing the NSQHS Standards but indicated the activities that had been core to the process (see online supplementary file 3).38
Using an average hourly cost rate of $A64.55 in 2010 FY,39 and allowing for attendance, preparation and travel time, we conservatively estimate an additional cost of $A1.02 million for the activities described above at the time of the survey development. This figure is much lower than the $A6.54 million in standard development expenses stated in the ACHS 2013 FY annual report, but the ACHS figures also include development of standards in the non-acute sector.36 Under this new accreditation framework, hospitals accredited by ACHS can elect to be assessed by an additional five standards developed from the previous EQuIP5 programme which relate to organisational processes. We understand there might be some adjustment to the accreditation agency fees to accommodate this change but it is not clear whether the overall development costs would be higher or lower.
Discussion
This is the first study to assess the costs of accreditation over a range of acute health services. The incremental costs of accreditation ranged from 0.03% of total operating costs for a specialist teaching hospital, to 0.6% of operating costs for a small rural hospital. At a national level we estimate costs at 0.1% of recurring operating costs for the acute public hospital sector, but put a caveat on these findings by acknowledging the small sample size. The national figures include an estimate of additional surveyor costs, but standards development costs and surveyor training programmes are included in the ACHS fees.
Impact of higher costs for smaller hospitals
Since other studies have not included a range of hospital types, this research is important in highlighting the impact on smaller hospitals with their relatively higher costs. Disaggregation of the results to focus on the smaller hospitals is complicated by the small sample size but the issues for smaller hospitals have been recognised. ACSQHC have published separate accreditation preparation guidelines to advise facilities with less than 50 beds, while recognising that these facilities still need to meet the required standards for patient safety and quality of healthcare.40 The higher costs for smaller hospitals could be related to a level of fixed costs for the survey process that would be unrelated to hospital size and activity. The average costs per admission for smaller hospitals were 43.5% higher than for principal referral hospitals in 2011FY.29 A better understanding of these apparent ‘fixed’ accreditation costs would help design activities to reduce the relatively higher burden for smaller hospitals, and help reduce overall costs. The senior quality manager at hospital 2 suggested that larger hospitals benefited from having a pool of dedicated and experienced accreditation personnel, and this resource was not available to smaller hospitals except where the smaller hospitals were accredited as part of a larger group. The manager also suggested that short notice or unannounced surveys would reduce the indirect costs of stress associated with the surveys. Hospital 1 had low absolute and relative costs, with average absolute costs only 8.5% higher than for the smallest hospital. The quality team in hospital 1 attributed this to using information technology to reduce the paperwork required for the survey and maintaining a policy of always being ‘survey ready’.
Implications for policymakers
Recommendations for policymakers to reduce higher costs for smaller hospitals could be focused on the way the surveys are conducted. Suggestions include: short notice surveys to maintain a ‘survey ready’ approach rather than reacting to accreditation timetables;41 improved provision for data collection to reduce the reporting burden; increased online reporting; providing technology platforms for accreditation tools; developing indicators to reduce survey requirements;42 and reducing duplication with State and Federal reporting requirements. This research provides a baseline of accreditation costs to compare the impact of the new accreditation programme. It is important from a policy perspective that these costs be considered in terms of the potential benefits of accreditation. However, these benefits have not been clearly identified or assessed for comparison to the costs.9 ,43
Strengths and limitations of the study
A key limitation of the study was the small sample size, and our expert panel suggested that increasing the sample size in future studies, especially for the group B and C hospitals, would improve external validity. Other limitations include: recall bias; the difficulty of assessing hospital costs over time; self-selection by the participating hospitals; and a focus on direct financial costs. To reduce recall bias we verified the data wherever possible by asking for documented confirmation of timetables and attendances. Using concurrent diary methods to detail accreditation activities would be more accurate over the 4-year accreditation cycle. Assessing costs based on geographically-based local health districts would: improve the accuracy of the financial reporting figures; would be in keeping with the way hospitals are increasingly assessed in Australia for accreditation purposes; and would provide us with greater certainty in estimating operating costs. The self-selection issue is difficult to address since hospitals need to opt into the research process, but discussions with staff intimated that they had little expectation of what the accreditation costs would be.
This study focused on direct financial costs. An Australian Productivity Commission report, investigating primary care accreditation costs, highlighted the intangible costs such as the stress and frustration for general practitioners with administration activities and dealing with government welfare departments.44 We did not estimate the intangible costs in our study but recognise that unpaid overtime and stress prior to, and during, the survey were common themes voiced by hospital staff during our cost discussion and in other studies.45 Our study sample comprised publicly funded hospitals and we did not calculate the economic costs of staff involvement. Imputing these costs for a private facility would need to include opportunity costs for staff involved in a for-profit enterprise. In addition, our estimate of the national costs of accreditation assumed that the study years were representative of accreditation activity and health funding. We mitigate this risk by having hospitals at different stages of the accreditation cycle so that the higher costs in survey years are spread over the 4-year study period. We also use the average change in operating costs over the study period but not all hospitals receive the same funding increases at the same time, and this could be a significant issue across the different States in Australia. Our total figure includes an estimate of surveyor costs that are not covered by the accrediting agency fees. Although this is a cost to the system, the benefits would include surveyors having access to safety and quality best practices at other hospitals and the ability to benchmark against similar facilities.
Conclusion
The higher costs for smaller hospitals represent a higher administrative burden that may contribute to their higher costs per admission.29 Our purpose designed activity based costing tool could be adapted to assess accreditation costs across international accreditation systems, and within the different health domains of acute, primary and aged care. The results from this study provide health providers and policymakers with a basis to assess whether this is a reasonable expenditure for an external assessment of an organisation's quality and safety efforts.
Acknowledgments
The authors thank the staff of ACHS and participating hospitals for their help in implementing the study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
- Data supplement 2 - Online supplement
- Data supplement 3 - Online supplement
Footnotes
Contributors JB and JW initiated the concept for the cost benefit analysis of accreditation. VM, DG and AH designed the incremental costs audit tool. VM collected and analysed the data, and wrote the first draft of the paper. JB and KF reviewed the assumptions and analysis. All authors made significant contribution to the final draft of the paper.
Funding This research is supported by the National Health and Medical Research Council Programme Grant (grant number 568612), and the Australian Research Council’s Linkage Project Grant (project LP100200586).
Competing interests None declared.
Ethics approval The University of New South Wales Human Research Ethics Committee has approved this project (approval number HREC 10274).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.