Objective We aimed to develop a questionnaire for quantitative evaluation of the autonomy of public hospitals in China.
Method An extensive literature review was conducted to select possible items for inclusion in the questionnaire, which was then reviewed by 5 experts. After a two-round Delphi method, we distributed the questionnaire to 404 secondary and tertiary hospitals in Guangdong Province, China, and 379 completed questionnaires were collected. The final questionnaire was then developed on the basis of the results of exploratory and confirmatory factor analysis.
Results Analysis suggested that all internal consistency reliabilities exceeded the minimum reliability standard of 0.70 for the α coefficient. The overall scale coefficient was 0.87, and 6 subscale coefficients were 0.92 (strategic management), 0.81 (budget and expenditure), 0.85 (financing), 0.75 (financing, medical management), 0.86 (human resources) and 0.86 (accountability). Correlation coefficients between and among items and their hypothesised subscales were higher than those with other subscales. The value of average variance extracted (AVE) was higher than 0.5, the value of construct reliability (CR) was higher than 0.7, and the square roots of the AVE of each subscale were larger than the correlation of the specific subscale with the other subscales, supporting the convergent and discriminant validity of the Chinese version of the Hospital Autonomy Questionnaire (CVHAQ). The model fit indices were all acceptable: χ2/df=1.73, Goodness of Fit Index (GFI) = 0.93, Adjusted Goodness of Fit Index (AGFI) = 0.91, Non-Normed Fit Index (NNFI) = 0.96, Comparative Fit Index (CFI) = 0.97, Root Mean Square Error of Approximation (RMSEA) = 0.04, Standardised Root Mean Square Residual (SRMR) = 0.07.
Conclusions This study demonstrated the reliability and validity of a CVHAQ and provides a quantitative method for the assessment of hospital autonomy.
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
To the best of our knowledge, this is the first study to carry out a quantitative investigation of public hospital autonomy in China.
By surveying >400 public hospitals, we developed the first version of a Hospital Autonomy Questionnaire, designed to improve our understanding of the relationship between public hospitals and the government.
We verified the reliability and validity of our questionnaire, which provides a quantitative method for the assessment of hospital autonomy in China.
The survey was conducted only in Guangdong Province; thus, the generalisation of the results to other provinces should be made with caution.
This was a cross-sectional study that could not illustrate the underlying trend of hospital autonomy. In the future, panel data should be collected.
In developing countries, public hospitals are normally characterised by inefficient resource management, by low productivity due to their rigid hierarchical structures and by ineffective governmental administrative and financial controls.1 Turning public hospitals into autonomous entities is thought to be able to improve their performance.2 In China, since over 80% of hospitals are run by the state,3 increasing public hospital autonomy, which is also considered to be an effective way to improve the efficiency of public hospitals, is a key issue in health sector reform4 and is a particularly important issue in China. Studies on the evaluation of hospital autonomy in other countries have yielded mixed results. Govindaraj and Chawla,5 using the evaluation toolkit by Chawla et al, carried out five country case studies (Ghana, Kenya, Zimbabwe, India and Indonesia). However, the results varied by country, probably because of the short period of time that elapsed since the granting of autonomy in each country. Bossert et al6 measured hospital autonomy in Indonesia and found little evidence that its objectives were achieved. Sengooba et al7 performed another case study of eight hospitals in Uganda. Given the nature of the case study, the authors did not believe that their results should be used as the sole basis for national-level or international-level policy development. In addition, different authors carried out eight country or area case studies (UK, New Zealand, Australia, Hong Kong, Malaysia, Singapore, Indonesia and Tunisia) based on Harding and Preker's8 conceptual framework on hospital autonomy. Hawkins and Ham summarised these studies and found that autonomisation might improve efficiency in some areas but not in others.9 McPake et al10 reached a positive conclusion by studying five hospitals in Bogotá, Colombia, while Saleem M et al studied the autonomisation of teaching hospitals in Punjab and found that it has not yet yielded the hoped-for benefits.11 London investigated the impact of hospital autonomisation in Vietnam and also found mixed results, mainly due to the small sample size and data constraints.12 Some researchers attributed the mixed results to the small sample sizes and quality of data in these studies, which prevented researchers from conducting more elaborate statistical analyses and reaching robust conclusions.2
In China, public hospitals play a critical role in the healthcare system. According to the China Health Statistics Yearbook (2015),3 they provide medical services to 89.5% of the population, and the proportion of spending in public hospitals is 95.5% of the total national health expenditure in China. Currently, public hospitals are considered less efficient and a serious financial burden to the government.13 To address these problems, China began granting autonomy to some public hospitals in the early 1980s, including shifting partial decision-making control from the government to hospital managers, which has allowed public hospitals to make a profit from the medical services they provide. However, after >20 years, some research suggests that healthcare reform in China has not made significant progress.14 In 2012, China's State Council announced a new phase of healthcare reform, allowing for more private hospitals and granting greater public hospital autonomy to improve efficiency.15 Therefore, studying the impact of hospital autonomy on efficiency is important, and developing an effective tool for measuring such autonomy is the first challenge. With the Chinese government's support, and sufficient hospitals in Guangdong Province to provide a good sample, we carried out this study. The objective of this study was to develop a Chinese version of the Hospital Autonomy Questionnaire (CVHAQ), which can be used to evaluate the level of hospital autonomy and to help policymakers better understand the extent of policy implementation and formulate specific policies to redefine the relationship between health authorities and hospitals.
Theoretical model and questionnaire development
Although many researchers have proposed evaluation tools for hospital autonomy, there are three generic tools, namely, the Chawla et al,15 Over and Watanabe16 and Harding and Preker8 tools.2 The Chawla et al tool proposes three key areas to be analysed to determine the existing level of autonomy: administration, financing and inputs. The Over and Watanabe tool considers five elements of hospital structure: residual claimant status, decision right, degree of market exposure, availability of accountability mechanisms and extent of unfunded mandates. The Harding and Preker8 tool proposes five dimensions for analysis of the extent of hospital autonomy: decision right, market exposure, residual claimant, accountability and social functions. We modified the three existing tools to develop a CVHAQ and the flow chart of questionnaire development shown in figure 1.
A systematic approach was used to generate the final questionnaire.18 ,19 The development of items for inclusion in the CVHAQ was carried out in five phases. First, an extensive literature review was conducted to select possible subscales for the questionnaire. The theoretical basis for hospital autonomy was the conceptual framework from Harding and Preker,8 which contained five subscales: decision rights, residual claimant, market exposure, accountability and social functions (figure 2). At the same time, we also learnt from the other two related measuring tools, and nine subscales were created to evaluate the extent of Chinese hospital autonomy: strategic management, budget and expenditure, finance, human resources, day-to-day management, residual claimant, market exposure, accountability and social functions. Second, after in-depth interviews and focus-group sessions, 40 potential subitems were developed. Third, 10 carefully selected experts—including 2 lawyers, 4 hospital management experts, 2 government officers, and 2 professors versed in statistics—were invited to discuss relevant items for inclusion or exclusion. We used the Delphi method to evaluate the items. In two rounds, the experts gave their scores in terms of linguistic expression, importance, clarity and correlation for each item, and then we deleted the items whose importance score was <7.5. The remaining 32 items constituted the initial shape of the questionnaire. Fourth, a pilot test with 26 selected hospitals was conducted to judge the appropriateness of each included question. Fifth, in a formal survey, six undergraduate students were selected as questionnaire interviewers, and two researchers in our focus group were designated as quality controllers. After investigation, we used the method of exploratory factor analysis to screen the items, and finally we deleted 10 items and extracted 6 factors. (All loadings were larger than 0.5.) To enhance the interpretability of the factors with high loadings, we rotated the extracted factors to a simple structure using the Varimax procedure (orthogonal rotation) to see if the result was consistent with our theoretical model. Furthermore, we evaluated the reliability of the questionnaire and Cronbach's α coefficients in all subscales higher than 0.70. Meanwhile, we calculated the value of average variance extracted (AVE; >0.5) and square roots of AVE to test the validity of the questionnaire. After some necessary linguistic revisions, the questionnaire, including six subscale (22 items) indicators of Chinese hospital autonomy, was finalised and used to build a Chinese hospital autonomy model.
Responses were rated on a 5-point Likert scale for hospital autonomy (1=totally constrained, 2=very constrained, 3=partly constrained, 4=hardly constrained and 5=not constrained). Items were forward scored on a 1–5-point scale (1=1, 2=2, 3=3, 4=4, 5=5), and subscale scores were computed as the sum of each item score; that is, the higher scores indicated higher autonomy.
Quality control measures: item generation
The items in this questionnaire were generated through literature reviews, in-depth interviews and focus-group sessions. First, we reviewed PubMed and the China National Knowledge Infrastructure (CNKI) to collect pertinent literature. Second, we set up a focus group to analyse the pertinent literature and preliminarily determined the subscales of our questionnaire. Finally, the members of the focus group reviewed the potential respondents or experts in depth. When item generation continued until no new items emerged, we established an item pool with 40 items.
Then the Delphi method was used for two rounds.
The questionnaire was tested for how well respondents comprehended and correctly answered crucial questions, or to confirm that the questionnaire was user-friendly for the respondent and to ascertain how long it took to complete.
All of the questionnaire interviewers and quality controllers were systematically trained by the project investigator (PI) before the investigation. During the investigation, the questionnaire interviewers were responsible for connecting with respondents who were familiar with questions and reported regularly to the PI. The quality controllers were responsible for checking the integrity and accuracy of the finished questionnaire.
Double-blind input, consistency check and logical check were used to control the quality of data entry.
This survey was conducted in 2013, and the data collection process consisted of two parts. First, an electronic version of the questionnaire was sent to 404 secondary and tertiary hospitals in Guangdong Province with the support of the Health Bureau of Guangdong Province. Second, senior management staff of these hospitals were contacted and asked to organise persons familiar with these issues to complete the questionnaire. The completed questionnaires were then collected by the research team.
SPSS 19.0 for Windows was used for data analysis. First, Cronbach's α coefficients were calculated to evaluate the internal consistency reliability of the total questionnaire and each subscale. Cronbach's α coefficient ≥0.70 was considered as acceptable. Second, multitrait scaling analysis (using Pearson correlation analysis) was conducted to determine the item–subscale correlations. If the analysis showed a stronger correlation (correlation coefficient ≥0.70) between an item and its hypothesised subscale, and with a coefficient larger than those of other subscales, the questionnaire was considered well defined. To test the validity of the CVHAQ, an exploratory factor analysis was conducted, and the extracted factors were rotated to a simple structure by the Varimax procedure (orthogonal rotation) to see if the result would be consistent with the theoretical model. Varimax rotation can enhance the interpretability of the factors with high loadings. The values of AVE and construct reliability (CR) were calculated to test convergent validity and discriminant validity. A confirmatory factor analysis was then performed with AMOS V.19.0 to assess the fitness of our modified theory model. The fit indices of χ2/df (<2), Goodness of Fit Index (GFI; >0.90), Adjusted Goodness of Fit Index (AGFI; >0.90), Non-Normed Fit Index (NNFI; >0.90), Comparative Fit Index (CFI; >0.90), Root Mean Square Error of Approximation (RMSEA; <0.05) and Standardised Root Mean Square Residual (SRMR; <0.08) were used to assess the fitness of the model. Missing item values from individual surveys were imputed with the mean value.20
In total, 379 questionnaires were collected in this study. The characteristics of the participating hospitals are listed in table 1. The response rate was 93.81%. The percentage of missing values for each item of the subscale varied from 0.00% to 3.63%.
Exploratory factor analysis
To test the validity of the hypothesis, the exploratory factor analysis with Varimax factor rotation was conducted, and six components were extracted, which accounted for 73.05% of the total variance. Results are presented in table 2.
The mean scores, SDs and Cronbach's α coefficients are presented in table 3. All internal consistency reliabilities exceeded the minimum reliability standard of 0.70, among which the overall scale coefficient was 0.87, and six subscale coefficients were 0.92 (strategic management), 0.81 (budget and expenditure), 0.85 (financing), 0.75 (financing, medical management), 0.86 (human resources) and 0.86 (accountability).
Pearson correlation coefficients between subscales and items are presented in table 4. The results showed that items had higher correlation coefficients as compared with the theoretical subscale than with the other subscales.
Convergent validity and discriminant validity
If the value of AVE was higher than 0.5 and the value of CR was higher than 0.70, we considered the convergent validity acceptable. To test discriminant validity, we calculated the square root of every AVE value belonging to each latent subscale. The outcomes (table 5) show that the square roots of the AVE of each subscale were larger than the correlation of the specific subscale with any of the other subscales.
Confirmatory factor analysis
We used AMOS 19.0 to conduct confirmatory factor analysis, and seven indicators were calculated to test the fitness of our theory model. RMSEA and SRMR were <0.08, while GFI, AGFI, NNFI and CFI were higher than 0.9, leading to acceptable construct validity. The results are presented in table 6.
In this study, the hospital autonomy survey was modified to adapt to Chinese health policies, and the CVHAQ was developed on the basis of Harding and Preker's conceptual framework on hospital autonomy, as well as the tools developed by Chawla et al15 and Over and Watanabe.16 The reliability and factor structure of the CVHAQ were assessed. The results of this study showed that the CVHAQ was a reliable and valid questionnaire for the measurement of hospital autonomy. The confirmatory factor analysis confirmed the theoretical model, which reflected the extent of hospital autonomy. A higher score suggested the hospital had more autonomy. According to the Chinese Health Statistics Report (2015),3 government fiscal budgets provide about 7% of total revenue for public hospitals. Almost all secondary and tertiary hospitals must earn a profit from the medical services they provide, so there is little variance in residual claimant and market exposure between sample hospitals that could be captured by the questionnaire. Social function is a complementary reform to ensure that services that were previously cross-subsidised continue to be delivered, and does not belong to the scope of hospital autonomy. The six theoretical subscales included in the final CVHAQ well reflected the status of hospital autonomisation. The results of confirmatory factor analysis showed that all indicators of fitness were acceptable.21–23 This study showed that subscales 2 and 3 were correlated (r=0.59), which can be explained by the fact that the autonomy of the two subscales commonly depends on the economic policy for a hospital, which is set by the government; thus, finances, budget and expenditure are synergistically affected. Subscales 5 and 2 were also correlated (r=0.45). The underlying reason could be that the Chinese government spent little on human resources, and the cost was borne by the hospital.24 ,25
Characteristics of local hospitals in Guangdong made it very suitable for conducting this large-scale quantitative study. China has traditionally managed its public hospitals as administrative units. However, in the past 30 years, the Chinese government has been reforming its health system by reducing the health department's direct control over public hospitals and shifting the day-to-day decision-making from government authority to hospital managers, to reduce the government's financial burden and improve hospital efficiency.26 Guangdong, the frontier of China's economic reform, has >400 secondary and tertiary public hospitals. They are divided into national hospitals, provincial hospitals and municipal or county hospitals. Owing to different levels of government input and degrees of decentralisation, these hospitals vary greatly in autonomy, which created a unique opportunity for us to conduct this study in Guangdong. In addition, this study was supported by the Health Bureau of Guangdong Province, which resulted in a higher response rate from local hospitals. This helped overcome the issues of small sample size and data constraint existing in previous studies.
Although many studies on hospital autonomisation have been conducted, only a few were quantitative. Most previous studies were based on case study, and the results were subject to potential confounding factors, such as hospital size, management style, geographic location, etc.27 Empirical literature to date has done little to explore a Hospital Autonomy Questionnaire to quantify the extent of hospital autonomy. The CVHAQ developed in this study helped fill this gap in the published literature and provided a validated tool that can help overcome these shortcomings and enable reliable conclusions to be drawn.
In addition, the CVHAQ could lay a strong foundation for quantitative analysis of the relationship between hospital autonomy and its effect on healthcare reform. For example, a regression analysis can be done to explore the relationship between hospital autonomy and efficiency by using the hospital autonomy scores as independent factors.
Some limitations must be mentioned regarding our findings. The survey was conducted only in Guangdong Province; thus, generalisation of the results to other provinces should be done with caution. In our future study, we will investigate hospital autonomy in other provinces in China to further validate our questionnaire. In addition, we conducted a cross-sectional study that could not illustrate the underlying trend of hospital autonomy. In future studies, we plan to collect panel data to monitor hospital autonomy and explore the relationship between hospital autonomy and efficiency, to provide an evidence-based foundation for China's healthcare reform.
This study demonstrated the reliability and validity of a CVHAQ, and provided a quantitative method for the assessment of hospital autonomy in China.
The authors thank Qiumao Cai, Lina Yu and Chunxiao Wang at the Health Department of Guangdong Province for their assistance.
Contributors ZL, FL and YH obtained the funding. ZL, YH and LZ conceived the idea and designed the study. ZL and LY collected and processed the data. ZL, YH and LZ drafted the manuscript. All the authors critically revised the manuscript for important intellectual content; and read and approved its final version.
Funding This research was funded by the Guangdong Medical Fund of China (grant number C2013016).
Competing interests None declared.
Ethics approval Medical Ethics Committee, Guanghua School of Stomatology, Hospital of Stomatology, Sun Yat-sen University.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.