Article Text
PDF
Abstract
Objectives Public hospitals in Nepal account for a major share of the total health budget. Therefore, questions are often asked about the performance of these hospitals. Existing measures of performance are limited to historical ratio analyses without any benchmarks. The objective of this study is to explore the trends in inputs, outputs and productivity changes in Nepalese public hospitals from 2011–2012 to 2013–2014.
Setting and participants The study was conducted among 32 Nepalese public hospitals (23 district level and 9 higher level) for the three fiscal years from 2011–2012 to 2013–2014.
Outcome measures First, basic ratio analyses were conducted for the input and output measures over the study years. Then, Malmquist productivity change scores were obtained using data envelopment analysis. Aggregated as well as separate analyses were conducted for district level and higher level hospitals.
Results Real expenditures of the sampled hospitals declined over the 3-year period from an average of US$ 371 000 in year 1 to US$ 368 730 in year 2 and US$ 328680 in year 3. The average aggregated hospital outputs increased marginally from 8276 in 2011–2012 to 8613 in 2013–2014. The total factor productivity of the study hospitals declined by 6.9% annually from 2011–2012 to 2013–2014. Of the total 32 hospitals, productivity increased in only 12 (37.5%) hospitals and declined in the remaining 20 hospitals. The total factor productivity loss was influenced by a decline in technology change, despite an increase in efficiency.
Conclusions In general, productivity of the study hospitals declined over the study period. Availability and accessibility of accurate, detailed and consistent measures of hospital inputs and outputs is a major challenge for this type of analysis.
- Nepal
- Data Envelopment Analysis
- Hospital
- Productivity
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Hospital ‘inpatient discharges’, which is one of the most cost-intensive outputs, is adjusted for case mix.
Stability of the frontier and integrity of the model were tested before calculating the productivity growth scores.
Productivity scores for district level and higher level hospitals are analysed separately as well as together.
Human resource inputs could not be measured in terms of the physical units (full-time equivalents) because these data were not available.
Productivity growth is analysed only for last 3 fiscal years because the data were not available for earlier years.
Background
Public hospitals are under pressure to improve both effectiveness and efficiency. In developed countries, technological advances and the associated increase in costs are a concern, and in developing countries, there is a need to narrow the gap between the demand and supply of health services.1 2 The problem of inadequate financial resources is especially pertinent in low-income countries like Nepal. Improving the efficiency of Nepalese hospitals is therefore imperative as inefficiency results in lost opportunities to provide care to patients. At the same time, financing mechanisms that provide incentives to improve efficiency need careful assessment of performance.3 This study examines the productivity of public hospitals in Nepal. Measures of productivity provide policy-makers with information about accountability for public expenditure,4 and give hospital managers insight into how service efficiency might be improved.
Healthcare services in Nepal are financed through a mix of public and private funding. All services provided from primary healthcare centres are free for everyone. Maternity (delivery) services are also free to all, as are 40 essential drugs provided in hospitals with a capacity of ≥25 beds.5 However, other hospital outpatient, inpatient and emergency services are free only for targeted groups (ie, poor, destitute and disabled, senior citizens aged >60 years and female community health volunteers - FCHVs). Any people not in these groups must pay for these hospital services.
A study done by RTI International6 ,6 of the overall performance of the Nepalese health system shows that increasing per capita income and government investments are associated with increasing life expectancy and gradually declining infant mortality, child mortality and maternal mortality. This study also indicates that allocating resources according to needs can improve the efficiency as well as equity of health outputs in Nepal. However, questions remain about the performance of the public hospitals. Some empirical analyses have been undertaken,7 but existing performance measures are limited to historical ratio analyses without any benchmarks.8 Such indicators provide limited information for improving hospital performance.9
The configuration of public hospital infrastructure and its relationship with productivity was explored in a study conducted in four South Asian countries including Nepal.10 The study estimated productivity by production function analysis using unit costs as input variables and occupancy rates and length of hospital stay as output variables. However, the sample size of the hospitals from Nepal was relatively small, and thus limited analysis was conducted. Another study by Baral11 also focused only on efficiency. This study analysed technical and scale efficiency of 56 district level hospitals. Data envelopment analysis (DEA) was conducted for output variables—unadjusted volumes of the outpatient, emergency, delivery and inpatient days (no case mix adjustments) and input variables—number of doctors, nurses, beds and other technical staffs. Both of these studies analysed efficiency measures, which have significance for individual hospital managers, but have limited implications for improving decision-making at the policy level.12
The objective of this study was to explore variations in hospital inputs, outputs and the overall productivity in public hospitals in Nepal from 2011–2012 to 2013–2014 using DEA. It provides new insights into how productive the public hospitals are in Nepal and to the factors contributing to productivity change.
Theoretical framework
Productivity is basically a ratio of output per unit of input in a firm.13 It is a multidimensional concept that generally encompasses technical efficiency, scale efficiency, allocative efficiency and price efficiency. These components are inter-related to each other, at least partly. Therefore, every organisation that focuses on improving its performance should recognise, analyse and manage these key concepts regularly.9 In the case of firms producing more than one output by using multiple inputs, productivity is calculated by deriving output and input indexes.
According to Farrell’s definition of efficiency, a firm is considered to be technically efficient when it produces the maximum possible (feasible) outputs from a given combination of inputs or it is producing a given level of outputs from minimum inputs.14 The value of technical efficiency of a firm is considered as 1.0 (100%) when it is optimal. Similarly, allocative efficiency denotes combinations of inputs (an optimal mix) and outputs where marginal cost equals marginal benefit, in addition to being technically efficient.9 15
Prior to the 1990s, the concept of productivity considered only technical changes.16 More recently, productivity measures have included both technological and efficiency changes.16 17 The Malmquist Productivity Index (MPI) is one of the most common measures of productivity in service sectors.9 15 16 The MPI has been estimated largely by parametric methods such as Stochastic Frontier Analysis (SFA) and non-parametric methods like DEA. SFA calculates productivity by estimating a frontier for the given inputs and outputs based on a predetermined functional form of production. However, DEA does it through linear programming methods. DEA establishes a best practice frontier based on the observations of input and output mixes of different decision-making units (DMUs) and determines which DMUs are inefficient compared with a best practice frontier. It also quantifies the magnitude of their inefficiencies. This method does not require information on prices of inputs and outputs which are often not available in public sector organisations like public hospitals.18
There are different models of DEA, with Malmquist DEA being by far the most popular way of estimating MPI.16 19 Malmquist DEA estimates productivity change from period –t to period t+1 relative to the reference technology,20 which is not possible in other classical methods like Charnes, Cooper and Rhodes’s method,21 and decomposes the productivity growth in terms of technical change and efficiency (pure efficiency and scale efficiency) change.22 23
DEA estimates Malmquist productivity change in following way15:
Here, ‘d’ denotes distance function, ‘y’ denotes vectors of output for time ‘s’ and ‘t’ and ‘x’ denotes vectors of inputs for time ‘s’ and ‘t’. The first part of the right-hand side of the equation is a measure of the output-based technical efficiency in period –s and period –t, and the second part of the equation, within the square brackets, gives a measure of the output-based technological change for the two periods. For the output oriented productivity change measures as mentioned above, a change index of >1 indicates an increase in productivity and <1, a decrease over the study period. Similarly, if the value of the Efficiency Change (EFFCH) index that contributed to the productivity change is >1 (EFFCH >1), then efficiency has improved over the years, whereas if the value is <1 (EFFCH <1), then efficiency has deteriorated. Percentage change values in efficiency are given by (EFFCH–1) and (1–EFFCH), respectively. The same interpretation also applies for the technology change index.
The popularity of DEA in the analysis of efficiency and productivity of service sector organisations like hospitals is indisputable. However, it is not free from challenges. DEA estimates the relative efficiency of the firms under study. However, even the most efficient unit in the analysis may not necessarily be operating at maximum efficiency. Furthermore, as this model uses the actual volume of different inputs and outputs, the productivity indexes derived in DEA will be sensitive to the mix of inputs and outputs in the organisations under study.9 Therefore, it is important to consider case mix adjustments. DEA results are sensitive to measurement error as the model cannot accommodate statistical noise. Identification of possible outliers in the distribution of the sample is essential before performing DEA.3 DEA scores are sensitive to sample size (a small sample size tends to inflate the average efficiency scores and vice versa) and specifications of inputs and outputs. Therefore, appropriate decisions should be taken on the variables before estimating the scores.3 24
Methodology
Sampling
The DMUs in this study were 32 Nepalese hospitals (23 district level and 9 higher level hospitals) for the three fiscal years from 2011–2012 to 2013–2014. The sample size was determined by data availability because the Malmquist DEA method requires a complete set of input and output variables for all the DMUs under study.9
The sampling process commenced with a list of all the 99 public hospitals8 .Subsequently, four university hospitals were excluded because the operating mechanisms of these hospitals vary significantly from general hospitals without teaching status.25 26 The sample size was further reduced when completeness of data was checked for the hospital output indicators. These were total numbers of inpatient discharges, outpatients and emergency visits for all the three study years. The annual reports of the Department of Health Services (DoHS) for the respective years showed that complete data were available only for 37 of the 72 district hospitals and 23 higher level hospitals.8 In the case of the higher level hospitals, the basis for exclusion was data availability in the Health Management Information System (HMIS) data bank rather than the annual reports.
The samples of 37 district level and 23 higher level hospitals were screened further by assessing the availability of financial data and detailed case mix data for hospital inpatient discharges. This provided a sample of 24 district level hospitals and 9 referral level hospitals which had complete data for all the three study years. It was assumed that the underlying production function of these two types of hospital (both non-university public hospitals) is the same. One additional district level hospital was also removed following initial analysis as the distribution of its inputs and outputs revealed it to be an outlier (output to input ratio is >2.5 SD).3 So the ‘final’ sample was 23 district hospitals plus 9 referral hospitals. A rule of thumb provided by Banker and Morey for selection of sample size for DEA is adopted in this study. It requires that the number of study units should be more than or equal to three times the sum of numbers of inputs and outputs used in the analysis.27
Study variables
Annual recurrent expenditures of the study hospitals for salaried and non-salaried items were used as measures for labour inputs and other operational inputs, respectively. We could not find per unit prices of the different hospital inputs. Human resource inputs were measured in terms of expenditure amount (a proxy measurement) as well as physical quantities measured in terms of the ‘full-time equivalents (FTEs)’. The FTE data were available only for higher level hospitals. The expenditure amounts over the years were adjusted for inflation by the Wage Rate Index.3 13 15 The numbers of available beds were used as a proxy for capital inputs.13
Hospital outputs were measured in terms of the number of outpatient visits, emergency visits and inpatient discharges. Inpatient discharges, which are the most cost-intensive service in hospitals, were adjusted for case mix. Cost weights for individual inpatient cases by International Classification of Diseases (ICD) codes were not available from Nepal and we were unable to access cost weights from any other developing country with a similar economy and health system. Therefore, we estimated relative cost weights data for inpatient discharges by mapping the ICD-10 codes (diagnosis category) of the study hospitals against the cost weight databases of the Waitamata District Health Board, New Zealand. The concept here is that the relative cost weight gives an estimate of the resources used in delivering different inpatient services (in relative terms, not the absolute unit prices). The weight is calculated as: relative cost weight=costs of specific disease category/mean costs of all disease categories for specific hospitals. Then, weighted numbers of inpatient discharges for each disease category=number of inpatient discharges’ × cost weight.
Data sources
The Financial Administration Section of the DoHS provided financial information for district level hospitals. All other data were obtained from the central databank of the HMIS section, DoHS.
Data analysis
Data analysis was carried out in two moves. Ratio analysis was conducted for all the samples (33 hospitals) to provide trends in inputs and outputs over the years. Before running the second move—that is, a one stage DEA (using DEAP V.2.1 software)—the variables were checked for their distributions.3 The one district level hospital that was found to be an outlier was then removed following further checks for possible data entry error. Productivity change was then analysed using two models. Model A used all hospitals (n=32), all inputs variables (non-salaried expenditure, salaried expenditure and number of available beds) and all output variables (number of outpatient visits, case mix adjusted volume of inpatient discharges and number of emergency visits). Model B used only higher level hospitals (n=9), one input variable (number of FTEs) and two output variables (number of outpatient visits and case mix adjusted volume of inpatient discharges). We decided to perform this analysis focusing on labour productivity because FTEs give more accurate estimates of labour inputs.13 However, the sample size in this model (model B) is small because FTEs were available only for the nine higher level hospitals.
Multistage variable returns-to-scale (VRS) DEA efficiency scores were obtained for each of the models and productivity change scores calculated. Model A is the basic model in this analysis. As relative efficiency scores in DEA are estimated relative to the frontiers, it is important that the frontiers are stable and valid. Therefore, for model A, stability* of the frontier and integrity† of the model were tested.28 The productivity change estimates for the model B are basically a partial productivity (labour productivity), and so are not directly comparable with the scores in model A.
Results
The average annual recurrent expenditure over the 3-year period was US$ 1.06 million for the higher level hospitals (n=9) and only US$ 91 200 for the district level hospitals (n=23). The average number of available beds was 199 and 20, respectively, and the average FTEs (available only for the higher level hospitals) was 197. For the three output variables—that is, case mix adjusted inpatient discharges, emergency visits and outpatient visits—the average annual figures were 8790, 78 128 and 22 371, respectively, for the higher level hospitals and 1633, 18 536 and 4672 respectively for the district level hospitals. Table 1 provides a summary of observations for the study DMUs (n=32 for 3 years) by hospital type.
Distributions of the study variables by hospital type
Real expenditures of the sampled hospitals declined over the 3-year period from an average of US$ 371 000 in year 1 to US$ 368 730 in year 2, and US$ 328 680 in year 3. For higher level hospitals, expenditure declined from an average of US$ 1 109 000 in year 1 to US$ 970 860 in year 3. Similarly, the figures for district hospitals were US$ 94 250 and 87 870, respectively, in the first and last years. Average aggregated‡ hospital outputs increased marginally from 8276 in 2011–2012 to 8613 in 2013–2014. But when disaggregated by hospital type, outputs declined for higher level hospitals (from 19 786 to 19 207), but increased for the district level hospitals (from 3960 to 4640). For these hospitals, the number of outpatient visits increased over the whole period, but the volume of inpatient discharges and emergency visits declined in year 3. Total FTEs (only higher level hospitals) increased over the years. Figure 1A–C) show overall variations of the input and output indicators over the study years (indexed at 1000 for the year 2011–2012).
Input and output changes over the study years.
Tables 2 and 3 provide details of the annual productivity changes and annual means of the study hospitals. Based on model A, overall multifactor productivity declined by an average of 6.9% annually from 2011–2012 to 2013–2014. Of the total 32 hospitals, productivity increased in only 12 (37.5%) hospitals and declined in the remaining 20 hospitals. This multifactor productivity loss was influenced by a decline in technological change, despite an increase in technical efficiency (EFFCH). There was an annual 2.6% growth in technical efficiency with a 3.3% gain in annual pure technical efficiency against a 9.2% loss in technological change. Half of the hospitals showed positive changes in technical efficiency and scale efficiency with another 21.8% of hospitals showing no changes. However, technological change declined in three-quarters of the hospitals. The sensitivity of model A was tested by analysing ‘only district level hospitals’ and ‘only general (non-specialised) hospitals’ separately. Productivity of only the district level hospitals (n=23) decreased annually by 10.5% on average. Declining technological change (−13.1%) led the productivity loss. In the case of general hospitals (non-specialised, n=29), productivity declined by 8.1% which was led by a decline in the technological changes (−10.2%).
Malmquist index summary of annual means
Malmquist total factor productivity changes by hospitals (firm annual means)
Higher level hospitals lost an annual average of 3.8% labour productivity (labour being the only input in this model) over the 3-year period (model B). This change was due to a decline in technological change in all nine hospitals over the study period. There was considerable variation across the hospitals in all of the other indicators (change measures). We examined the sensitivity of model B for FTEs of technical human resources only (ie, excluding administrative positions).29 Overall productivity change did not vary but efficiency change scores increased.
Discussion
This study indicates that, overall, there was a marginal increase in outputs over the study period but inputs and overall productivity of the study hospitals declined. The analysis suggests that hospital outputs could have been increased by 25% for the given level of resources along with additional input savings and output slacks. Efficiency and productivity measurement is only one aspect of healthcare performance measurement. But in government sector organisations, productivity helps maintain accountability of an organisation for the use of public resources.4 9 In this study, the inpatient and outpatient counts increased over the years by small numbers but the unadjusted inpatients declined marginally. This matches the findings from a previous study that analysed efficiency of the district level hospitals in Nepal from 2009 to 2010.11 However, a Chinese study conducted among hospitals in Beijing for the period of 2006–2009 showed that there was a minimum of 7.83% growth in all of the input and output variables of the study hospitals.12
The earlier efficiency studies conducted in Nepal used unadjusted measures of the inpatient services (unadjusted inpatient days)10–12 and number of delivery services were treated as a separate output.11 However, in the present study, while the sample size was only 23, the volume of inpatient services was adjusted for case mix. We noted that, even though the district level hospitals look homogeneous in terms of their functional mechanisms, case mix varies from one to another. The VRS technical efficiency scores in this study compare well with the previous study but the scale efficiency is reported to be higher. This may be because of the smaller sample size in this study. Even though we followed the rule of thumb of Banker and Morey,27 that is, n≥3 (number of input variables + number of output variables) while selecting input and output variables in the model, DEA generally works better with larger sample sizes and tends to inflate the scores if there are small numbers of peers to be compared.3 15 24 The study of South Asian hospitals10 used a very different methodology from the present study. However, the general conclusions were similar—that is, Nepalese public hospitals are not working at an optimal scale.
This study suggests that public hospitals in Nepal could improve their efficiency through both reducing inputs and increasing outputs. Most of the district level hospitals have the potential to increase inputs so that they can produce more outputs with higher marginal benefits,30 whereas higher level hospitals are already operating at a point where returns-to-scale are decreasing. It is worth focusing on individual hospitals with efficiency scores in the upper scales as short-term interventions should yield productivity gains. Longer term strategies would be required to improve the performance of the hospitals with lower efficiency ratings. For the hospitals having an efficiency score in the upper range, it may be worth investing in technology (eg, technological innovations in processes as a means of improving productivity).
For the hospitals where output maximisation is not possible, input minimisation should be considered. Identification of specific remedial actions would require further analysis with more rigorous calculations than a general DEA.9 However, we were unable to do this due to data limitations. Availability of adequate input and output information for a larger number of hospitals and additional information for various environmental factors that affect efficiency and productivity would provide more opportunities for further analysis of determinants of efficiency so that hospital specific actions can be identified.
Measuring hospital inputs and outputs was challenging, given the scarcity of the relevant data, the difficulty in collecting existing data, the poor quality of some data, and the incompleteness of some of the datasets. Information on physical quantities of labour inputs was available only for the higher level hospitals. Labour inputs therefore had to be approximated by salaried expenditures in model A. However, this may not provide accurate measures of human resource inputs for each of the DMUs. Salaries and other allowances for hospital staff vary greatly among the hospitals in Nepal. Furthermore, where the human resource inputs are measured in monetary terms (expenditures), inefficiencies could be due to price rather than quantity. These problems of data availability also meant that the sample size was quite small and the period of data analysis (3 years) relatively short.
The use of New Zealand hospital cost weights for case mix adjustment may not represent the actual values, plus the volume of outpatient and emergency services could not be weighted at all. In addition, none of the hospital outputs could be adjusted for quality even though this may have influenced productivity over the 3-year period.4 15 29 The DEA frontier created in this study was stable and validity of the data analysis method (DEA) was confirmed by an integrity test.28 However, we cannot confirm that the efficiency and productivity estimations are free from data errors because the non-parametric method DEA does not account for the effects of statistical noise.
An important limitation in terms of data analysis is that we could not check for the factors affecting efficiency of the study hospitals despite the fact that the performance scores of individual hospitals in the sample vary significantly. This again was due to data unavailability. For example, we had to pick the hospitals purposively looking at availability of data for individual variables under study rather than selecting them randomly.
As data availability improves, future studies with larger sample sizes may be able to calculate a bootstrapped estimation using data analysis software other than DEAP V.2.1.31 Future studies covering longer periods may also be able to examine the effects of different policy and programme changes (eg, Free Health Care Policy 2006–2009, Public Private Partnership, etc) on hospital efficiency and productivity.
Conclusions
This study suggests that the productivity of Nepalese hospitals could be improved, although there were large variations in the performance of the individual hospitals. Availability and accessibility of data regarding financial aspects, actual human resource inputs as well as unit costs of different services (namely, inpatient, outpatient and emergency services) would open up avenues for further study in various aspects of hospital productivity. Given that there is no benchmark on hospital performance measurement practices in Nepal, this study provides some insights about what mix of inputs and outputs can be considered for optimal performance.
Acknowledgments
The authors are very thankful to Associate Professor Roger Marshall, postgraduate coordinator, School of Population Health, University of Auckland; Professor Paul Rouse, Department of Accounting and Finance, University of Auckland Business School; and Julie Harris, Senior Business Analyst of the Waitemata District Health Board who supported during different phases of data processing and data analysis. The authors acknowledge the support of Ms Franziska Fuerst, Deutsche Gesellschaft für Internationale Zusammenarbeit, Nepal, and Mr Ghanshyam Gautam, who provided guidance during the conceptualisation phase of the study. The authors also thank the anonymous reviewers for their helpful suggestions and contributions towards improving the final manuscript.
References
Footnotes
↵ * A stability test was conducted by independent removal of an input and an output variable consecutively from the model and analysing its impact on the efficient frontier membership as suggested by Avkiran.28
↵ † An integrity test was carried out by comparing the differences of various output/input ratios (raw, unweighted) for efficient and the inefficient decision-making units as suggested by Avkiran.28
↵ ‡ All the three outputs were aggregated using unit costs of the inpatient, outpatient and emergency services obtained from a sample survey data of district level hospitals that performed step down costing.30
Contributors PRS came up with the concept and shared with TA. PRS was involved in data collection, analysis and write-up which was continuously supervised by TA. PRS drafted the manuscript and TA analysed it critically. PRS and TA worked together in finalising the design of the work, decided on the final draft and agreed to submit. For the comments from reviewers and the editor, PRS worked first and TA reviewed it.
Funding This study received no specific funding. Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) – Support to the Health Sector Programme (S2HSP), Nepal, agreed to provide partial funding for data collection by enumerators. However, this could not be activated because the enumerator team could not go outside of Kathmandu valley due to the Earthquake that hit Nepal in April 2015 followed by a long strike in the Terai region.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement A copy of the thesis report from which this manuscript has been prepared is available at the University of Auckland, New Zealand library database.