Article Text
Abstract
Objective Quantitative studies have shown the various benefits for having accreditation in hospitals. However, neither of these explored the general conditions before applying for an accreditation. To close this gap, this study aimed to investigate the possible association between joining an accreditation programme with various hospital characteristics.
Design A cross-sectional study was implemented using the databases of the 2013 Hungarian hospital survey and of the Hungarian State Treasury.
Setting Public general hospitals in Hungary.
Participants The analysis involved 44 public general hospitals, 14 of which joined the preparatory project for a newly developed accreditation programme.
Main outcome measures The outcomes included the percentage of compliance in quality management, patient information and identification, internal professional regulation, safe surgery, pressure sore prevention, infection control, the opinions of the heads of quality management regarding the usefulness of quality management and clinical audits, and finally, the total debt of the hospital per bed and per discharged patient.
Results According to our findings, the general hospitals joining the preparatory project of the accreditation programme performed better in four of the six investigated activities, the head of quality management had a better opinion on the usefulness of quality management, and both the debt per bed number and the debt per discharged patient were lower than those who did not join. However, no statistically significant differences between the two groups were found in any of the examined outcomes.
Conclusions The findings suggest that hospitals applying for an accreditation programme do not differ significantly in characteristics from those which did not apply. This means that if in the future the accredited hospitals become better than other hospitals, then the improvement could be solely contributed to the accreditation.
- quality in health care
- clinical governance
- organisation of health services
- accreditation
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Due to the high response rate, the findings of this study can be generalised to all public general hospitals in Hungary.
Since the hospital survey was conducted in a single country, different health policies or financial environments did not impact the findings.
The findings cannot be generalised to private hospitals.
The questionnaire used in the study was originally designed for a national survey, not as a research instrument.
Introduction
The quantitative studies that investigate the usefulness of accreditation in healthcare are based on the premise that the written regulations in the accreditation standards directly influence the process of care. From studies that address this topic, we know that the accreditation of hospitals is associated with more mature quality management, better clinical practice,1 safer patient care,1 2 increased compliance in documentation,3–5 shorter length of stay for psychiatric inpatients,6 better attitudes towards medication error reporting7 and lower proportion of errors in medical prescriptions8 but are not associated with better performance measures for acute stroke, heart failure and ulcers.9 Furthermore, accreditation was shown to stimulate the improvement in health service organisations and promote high-quality organisational processes10 while also predict organisational culture and leadership.11 Although accreditation does not affect outcomes directly, we assume that if a positive association exists between accreditation and the process of care, then this positive effect must also be reflected in various outcomes. Although none of the studies found any association between accreditation and patient satisfaction,12–15 Danish studies have revealed that patients in fully accredited hospitals had a lower 30-day mortality risk16 and a modestly shorter length of stay but not a lower chance of acute readmission compared with patients in partially accredited hospitals,17 and the process of care significantly improved after accreditation.18 Also, a recent study had also revealed a positive association between the compliance with accreditation standards and the level of evidence-based hospital care.19 In addition, a retrospective longitudinal study conducted in the USA showed that patients in accredited hospitals had significantly fewer major complication outcomes following laparoscopic bariatric surgery compared with patients in non-accredited hospitals.20
Because of the nature of the statistical methods of these studies, whether the statistically significant findings are directly associated with accreditation or if there is a common independent explanatory factor cannot be determined. Some studies assume that the dedication of the top management to a high quality of care could be the primary explanatory factor.1 21 Although there is no evidence for this assumption, if there are indeed any kind of independent explanatory factors, then this creates a situation in which it is very difficult to determine whether obtaining accreditation is the true cause of the high quality of care within the institution. One possible way to determine if this is a relevant question is to compare the activities of hospitals before applying for accreditation to those who have not applied for accreditation. If we identify significant advantages for the hospitals that did apply for accreditation, then the conclusions of previous research studies need to be reinterpreted, and future studies are needed to identify the independent factors that explain both the intent for accreditation and the high quality of care.
To understand the circumstances of the current study, the following paragraphs provide an abridged description of the creation of the Hungarian accreditation system for healthcare. To supplement the ISO 9001 and Hungarian Health Care Standards (HHCS) certifications already used in the Hungarian healthcare system and to follow the trend of European countries,22 23 the plan to introduce an accreditation programme was considered in the early 2000s. A decade later, in December 2012, the 2-year European Union (EU)-funded Social Renewal Operating Programme (TÁMOP) 6.2.5.A-12/1-2012-0001 project was initiated to create the foundations of a voluntary Hungarian accreditation programme for healthcare.24 With a high emphasis on patient safety and following the requirements defined by the International Society for Quality in Healthcare, the accreditation standards and the assessment method for inpatient and outpatient institutions and public pharmacies were created by the end of the project.24 25 In addition, the accreditation programme obtained its official name: the Accreditation of Healthcare Providers for Safe Patient Care (BELLA).
The next project, TÁMOP-6.2.5/B-13/1-2014-0001, included multiple interventions to improve the quality of Hungarian healthcare, and one of these interventions was to prepare medical institutions to join the BELLA accreditation programme. This preparatory project lasted from October 2014 to November 2015, and overall, 30 hospitals and 15 outpatient institutions participated in it.26 When voluntarily joining the project, the institutions guaranteed that after the project ended, they would apply for the BELLA accreditation; otherwise, they would have to pay back the financial support that was provided by the project. Because of this high level of commitment, we assume that these institutions take the process of being accredited very seriously.
Methods
Aims of the study
The aims of this study were twofold: first, to determine if a difference already existed between the general hospitals, before the decision of joining or not joining the nationwide preparatory project for BELLA accreditation was made. To this end, we compared the hospitals in areas such as the maturity of quality management, patient information and identification, internal professional regulation, surgical procedures, pressure sore prevention, infection control as well as the quality managers’ opinion of the usefulness of quality management and clinical audits. The second aim was to investigate how the financial status of the hospitals might have influenced the decision to apply for this preparatory project. The investigation of this aim could have gone both ways. Either the hospitals with better financial status were the ones that could afford to apply for the project, or because of the lack of it the hospitals joining were desperate to get the financial benefits that came with the project. As no similar studies have been conducted before, both of the initial assumptions were considered feasible.
Data collection and data processing
The questionnaire from the 2013 Hungarian national hospital survey was used in this study and was created by the first two and last two authors of this paper. The foundation of the questionnaire was the 2009 national survey of the Health Insurance Supervisory Authority which was also used to study the association between the ISO 9001:2008 and the HHCS certifications and various quality-related activities.27 In the revised 2013 questionnaire, 144 questions involved general information about the institution, quality management, patient information, patient identification, internal professional regulation, safe surgery, pressure sore prevention and infection control. The quality management dimension included various activities regarding quality planning, quality control, quality assurance and quality improvement. Patient information and patient identification were in the same dimension in which the former asked when the patient was informed and by whom, while the latter asked if there was a local protocol on how to identify patients and if so, which patient groups are included in it. The internal professional regulation dimension asked questions regarding reanimation and the usage of local protocols and clinical audit. The questions of the safe surgery dimension were mostly from the WHO surgical safety checklist,28 while the pressure sore prevention and infection control dimensions asked questions on how these activities are conducted and in what manner are these documented, respectively.
At the end of the dimensions that considered quality management and internal professional regulation, seven specific questions were asked of the heads of quality management on the usefulness of quality management, and three additional questions were asked on the usefulness of clinical audits. These were the only professional questions that had continuous answer options from 1 to 5, whereas the others had single or multiple-choice options. All questions covered the state of the hospital on the 31 December 2013.
For the national survey, permission was given by the National Institute of Quality and Organizational Development in Healthcare and Medicines, which supervises and controls the 105 state-owned hospitals. The data collection was conducted online with the EvaSys programme (EvaSys Education Survey Automation Suite, V.6, Electric Paper Evaluationssysteme, Lüneburg, Germany). The data collection was performed in two phases. In the first phase, which lasted from February to June 2014, all 113 hospitals, which were members of the Hungarian Hospital Alliance, were addressed. Because the response rate was too low, the Department of Health Policy of the Hungarian Ministry of Human Capacities was asked to support the survey. After an agreement was made, the second phase of the data collection was performed from October to December 2014. Because of the circular provided by the Department of Health Policy, the response rate nearly doubled, and in the end, 81 hospitals (71.7% response rate) answered the questionnaire.
Because of the heterogeneous characteristics of the hospitals, several exclusion criteria were applied. From the original 81 hospitals in the database, those that are not general hospitals were ignored. Due to the lack of a uniform definition for a general hospital, the decision was made that four types of basic inpatient care—inpatient medicine for adults, inpatient medicine for children, surgery and obstetrics—had to be provided in order for a hospital to be considered a general hospital. Of the remaining 46 hospitals, one had participated in the creation of the BELLA accreditation standards. Because it had a head start, we assumed that in this institution, both the quality management and patient care were already influenced by the accreditation standards. In addition, one university hospital provided answers at the departmental level. Although attempts were made to generalise the answers to an institutional level, the answers varied to such a degree that it made a reliable generalisation unfeasible. The remaining 44 hospitals had bed numbers higher than 130, which was one of the requirements to participate in the Deepening our Understanding of Quality Improvement in Europe research project.29 To check whether the response rate was sufficient for drawing general conclusions at a national level, the National Health Insurance’s Annual Report on Hospital Bed Size and Patient Flow for 2013, which contains a list of all the hospitals in Hungary on the 31 December 2013, was used as a reference point.30
The questions considering the quality management and patient care activities were dichotomised, and the answers that were positive were aggregated for each hospital by dimension. The answers concerning the opinion on the usefulness of quality management and clinical audits were treated separately, and, therefore, the first dimension had a possible score between 7 and 35, and the second dimension had a possible score between 3 and 15. Thereafter, the percentage was calculated from the possible maximum score for every dimension. In the survey, each question had the option of “I cannot/do not want to answer the question”, and the hospitals, in many instances, took the opportunity to select this answer. In addition, on rare occasions, some questions remained unanswered. For these reasons, the answering ratio for the questions for each dimension had to be 90% or above. In cases where a hospital answered less than 90% of the questions of a dimension, that dimension was ignored in the statistical calculation for the given hospital.
The financial status of the hospital was measured using the institutions’ overall debt in Hungarian currency (Hungarian forint). These data were obtained from the Hungarian State Treasury (MÁK) for each general hospital regarding the last month of 2013. To avoid the confounding factor of the hospital size, the hospitals’ debt was calculated per the number of beds and per the number of patients discharged in 2013 separately.
The list of hospitals that participated in the nationwide preparatory project for the BELLA accreditation was obtained from the National Healthcare Service Centre (ÁEEK). This list was the basis for arranging the hospitals that answered the previously mentioned 2013 survey into two groups, namely, participating and non-participating hospitals.
Statistical analysis
Cronbach’s alpha was used to test the internal reliability of the dimensions of the questionnaire, and the Kolmogorov-Smirnov test was applied to check the normality of the distributions of each dimension for both groups depending on their participation in the project. We rejected the normality if the P value was <0.01. A Spearman’s correlation was used to investigate the possible associations between the dimensions. To determine the statistical significance of the differences between the two groups, a Mann-Whitney U test and an independent t-test were used depending on the normality of the distributions. With the exception of the test of normality, the level of significance was set at <0.05 for the entire statistical analysis. The SPSS V.22 software programme (SPSS for Windows, V.22.0, IBM, Released 2013. Armonk, New York, USA) was used during the statistical analysis.
Results
Descriptive statistics
The 44 general hospitals included in this study represented 73.3% of all general hospitals in Hungary in 2013. Of the original 30 hospitals that joined the preparatory project for the BELLA accreditation, 11 were not general hospitals and of the remaining 19 only 14 took part in the survey. The answers of the 30 hospitals that did not take part in the preparatory project were used for comparison. The hospital-related characteristics of the 44 institutions are indicated in table 1. Of the four hospitals that completed the questionnaire and were owned by a private company or by a foundation, none were general hospitals. Thus, only public hospitals were analysed in this study.
Answering ratio of questions, normality and internal reliability
When the answering ratios of questions for each dimension were assessed, overall, 34 answers were excluded at a dimensional level (table 2). The test of normality varied according to dimension and participation. The internal reliability was strong in quality management (α=0.78) and pressure sore prevention (α=0.80), as well as in the opinion on quality management (α=0.89) and clinical audits (α=0.86). There was a somewhat below acceptance level of internal reliability in safe surgery (α=0.64), and the internal reliability was weak in patient information and identification (α=0.35), internal professional regulation (α=0.49) and infection control (α=0.57).
Correlation matrix
Spearman’s correlation matrix (table 3) reveals significant moderate-level associations between the dimensions of quality management and internal professional regulation (r=0.52, P=0.001), infection control and pressure sore prevention (r=0.48, P=0.001), internal professional regulation and opinion on clinical audits (r=0.39, P=0.019), and opinion on quality management and opinion on clinical audits (r=0.60, P<0.001). Since no strong correlation was identified, a common explanatory factor underlying the dimensions can be ruled out, and thus, these dimensions could be assessed independently from each other.
Comparative statistics
Comparisons of the participating general hospitals with the non-participating hospitals revealed no statistically significant differences in any of the measured activities (table 4). Additionally, the hospitals that participated in the preparatory project had lower means of debt per bed number and per the number of discharged patients than those that did not join, but the differences were not statistically significant (table 5). However, it is worth mentioning that the SDs of the debt ratios were very high. When attempts were made to remove the outlier data from the calculations, this has not decreased the SDs in a meaningful way.
Discussion
Implications for the BELLA accreditation
Because earlier studies had shown that neither ISO 9001 and the Hungarian Hospital Care Standards nor the HHCS had provided substantial benefits for hospitals regarding health-specific activities,27 31 it is imperative that we assess in what areas the BELLA accreditation will give added value to the hospitals in Hungary. To date, only one published study has addressed this question.32 Although this research only investigated the changes in patient safety culture among staff members before and after the development of the BELLA accreditation standards, these results might indicate the actual benefits after the hospitals obtain their accreditation. According to the findings, a statistically significant improvement was detected regarding organisational learning, continuous improvement, communication openness and teamwork.
Our study gives context for future studies investigating the impact of the BELLA accreditation. Since no significant advantages were detected among the hospitals applying for the preparatory project, future differences can be solely contributed to the accreditation. Furthermore, the notion that the hospitals only joined the EU-funded project for financial gains is questioned by the finding that the average debt rate of these hospitals was lower than those that did not join the project.
Implications for international studies
Surprisingly, no connection was found between quality management activities and the opinions on their usefulness (r=0.03, P=0.85). In addition, although most of the heads of quality management agreed with the usefulness of clinical audits, only 24.4% of the general hospitals reported conducting at least one in 2013.33 Furthermore, in many cases, nursing audits and process audits were reported as clinical audits; thus, the ratio of hospitals that use genuine clinical audits in Hungary is much smaller. All of these results might lead to the conclusion that the usefulness of asking the opinions of the heads of quality management on various activities is questionable. A possible explanation is that the opinions of the heads of quality management are a reflection of what should be, not of reality. For example, the heads of quality management may understand the importance of clinical audits, but without sufficient support from top management, they cannot be properly implemented. This assumption should be investigated in future studies.
The results of the comparative statistics suggest that the assumption that there might be an independent explanatory factor explaining both the intent for accreditation and high quality of care1 21 is unfounded. Because no significant differences were identified between the two groups of Hungarian general hospitals, we conclude that accreditation is not a method that selects hospitals which already perform better, but rather it is a tool that—if implemented correctly—can contribute to the further development of hospital activities. This also means that the benefits identified in quantitative studies regarding hospitals are indeed due to the accreditation.
Our study did not explore how the top management differs in hospitals that applied for an accreditation programme from those that did not. Since in countries where accreditation is not mandatory, the management has the authority to initiate the process to apply for one, it would be worthwhile to investigate the various attitudes and beliefs of management and to identify the key factors that make them more open to obtaining an accreditation for their hospitals.
Strengths and limitations
Because of the 73.3% response rate, the findings of this study can be generalised to all public general hospitals in Hungary. Alas, the sample size was not high enough to manage the high SDs of the debt ratios. Because the survey was conducted in a single country, including confounding factors, such as different health policies or financial environments, in the analysis was unnecessary. However, no private hospital participated in this study. This is due to the fact that there are not many private hospitals in Hungary, they normally provide specialised care and they rarely take part in government or academic surveys. Also, the 2013 questionnaire was originally designed for a national survey, not as a research instrument. The low Cronbach’s alpha score in three dimensions of patient care is attributed to this shortcoming. Nevertheless, the dimensions of quality management, pressure sore prevention and the opinions on quality management and clinical audits have strong internal reliability. Additionally, the answer option “I cannot/do not want to answer the question” acted as a double-edged sword. This option prevented hospitals from giving false information on specific activities because the question was ambiguous or it touched on a sensitive topic. However, this answer option led to a situation where not all questions were answered in each dimension per hospital, which weakened the overall level of evidence of the conclusions drawn from the statistical analysis. Because using valid information is more important than swelling the number of the sample, we believe that the benefits of introducing this option in the questionnaire outweighed the overall costs.
Conclusion
This is the first study that examines the hospital characteristics before applying for an accreditation programme through statistical methods, which resulted in two key findings. First, since no differences were identified in the maturity of quality management, patient care and the debt rate, both notions that either only hospitals with excellent performance or only hospitals lagging behind the rest are prompted to apply for accreditation are questionable. From this statement rises the second conclusion, that future differences in favour of accredited hospitals could be attributed to the effect of accreditation itself. Both of these conclusions give us a better understanding on the interpretation of results regarding research studies about accreditation in hospital care.
Finally, we suggest that future studies investigating the status of hospitals before applying for an accreditation should focus on the various knowledge, interests, attitudes and beliefs of the hospital management regarding accreditation and quality in general via interviews or questionnaires.
Acknowledgments
The authors would like to express their gratitude to the National Institute of Quality and Organizational Development in Healthcare and Medicines for authorising the 2013 survey in its hospitals, the Hungarian Hospital Alliance and the Hungarian National Committee of the European Organization for Quality for supporting the 2013 survey and the Department of Health Policy of the Hungarian Ministry of Human Capacities for first supporting and then facilitating the second phase of the 2013 data collection. The authors would also like to thank the hospital leaders for participating in the 2013 survey and all the hospital staff who were involved in completing the questionnaires.
References
Footnotes
Contributors VD and SG designed the study, wrote the first draft of the paper and coordinated the contributions from the other coauthors. VD, SG, BM and TG created the questionnaire and conducted the nationwide hospital survey of 2013. JS and OKBC designed and conducted the statistical analyses. CD planned and interpreted the analysis regarding hospital debt. All the authors made critical comments on the drafts of the paper. All the authors read and approved the final manuscript.
Funding The work is supported by the GINOP-2.3.2-15-2016-00005 project. The project is cofinanced by the European Union under the European Regional Development Fund.
Competing interests VD and SG were involved in and supported by the European Union-funded TÁMOP-6.2.5.A-12/1- 2012-0001 project, which sought to create the foundation of a Hungarian accreditation system called the Accreditation of Healthcare Providers for Safe Patient Care (BELLA).
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data available on request from the corresponding author.