Article Text
PDF
Abstract
Objectives To create a straightforward scoring procedure based on widely available, inexpensive financial data that provides an assessment of the financial health of a hospital.
Design Methodological study.
Setting Multicentre study.
Participants All hospitals and health systems reporting the required financial metrics in the USA in 2017 were included for a total of 1075 participants.
Interventions We examined a list of 232 hospital financial indicators and used existing models and financial literature to select 30 metrics that sufficiently describe hospital operations. In a set of hospital financial data from 2017, we used principal coordinate analysis to assess collinearity among variables and eliminated redundant variables. We isolated 10 unique variables, each assigned a weight equal to the share of its coefficient in a regression onto Moody’s Credit Rating, our predefined gold standard. The sum of weighted variables is a single composite score named the Yale Hospital Financial Score (YHFS).
Primary outcome measures Ability to reproduce both financial trends from a ‘gold-standard’ metric and known associations with non-fiscal data.
Results The validity of the YHFS was evaluated by: (1) cross-validating it with previously excluded data; (2) comparing it to existing models and (3) replicating known associations with non-fiscal data. Ten per cent of the initial dataset had been reserved for validation and was not used in creating the model; the YHFS predicts 96.7% of the variation in this reserved sample, demonstrating reproducibility. The YHFS predicts 90.5% and 88.8% of the variation in Moody’s and Standard and Poor’s bond ratings, respectively, supporting its validity. As expected, larger hospitals had higher YHFS scores whereas a greater share of Medicare discharges correlated with lower YHFS scores.
Conclusions We created a reliable and publicly available composite score of hospital financial stability.
- health economics
- statistics & research methods
- health policy
Data availability statement
Data are available in a public, open access repository. Hospital financial data for fiscal year 2017 was downloaded from the publicly available Centers for Medicare and Medicaid Services (CMS) Hospital Cost Reports (HCR) database. Hospital demographic data was downloaded from the CMS Hospital Compare database.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study addresses a growing need for a publicly available model for assessing the financial state of hospitals, which are otherwise assessed using covert, proprietary models.
We used a systematic and open scientific approach in selecting financial metrics and creating the Yale Hospital Financial Score (YHFS).
The financial metrics used in creating the YHFS were selected using statistical modelling whenever possible, though initial selection was done by the investigators based on proposed criteria, inherently introducing bias into the study.
The model was created using the most widely accepted ‘gold standards’ for assessing the risk of default though these scores have not themselves been systematically validated.
This score has been developed to aid health policy researchers and has not yet been validated in studies of longitudinal financial outcomes.
Introduction
The rapidly changing American healthcare environment exerts continuous financial pressure on hospitals. There is evidence that this pressure affects patient care: studies have shown both a higher rate of adverse patient events in hospitals under financial distress1 and improved quality of care when profitability increases.2 New policies, such as the introduction of quality-based reimbursement, may further propagate these adverse effects by placing additional selective pressure on already struggling institutions. Conversely, providing financial assistance to financially disadvantaged hospitals may prevent adverse events and improve care. Furthermore, many hospitals in the USA have closed since the start of the 21st century, often citing financial strain due to changes in policy and reimbursement as the ultimate cause. The financial standing of smaller hospitals is also an important consideration in antitrust investigations during the growth of competitor health systems. Empirical research on hospitals’ financial conditions, as well as their relationship to patient outcomes, is needed to inform any decision to change reimbursement policy and to evaluate any changes that are implemented. Such research faces an immediate problem in how to define and assess a hospital’s financial condition. Most modern studies rely on financial ratios, the Altman Z Score and credit ratings to assess hospital finances.1–3
Financial ratios are readily available and are therefore most frequently used by hospital managers,3 health policy researchers,2 4 5 and departments of public health6 to assess hospital financial condition and performance. There are, however, hundreds of candidate financial ratios, and there is no generally agreed on subset on which to focus. This problem is not unique to hospitals. Financial analysts typically wade through a sea of ratios, choosing the ones that make intuitive sense to them or that have been dictated by company policy.
There has been an effort to identity meaningful composites: single scores comprised of several financial ratios. The most extensively validated publicly available composite to indicate financial distress is the Altman Z Score.7 Altman focused on predicting the 2-year potential for bankruptcy and used a statistical approach on a sample of publicly-held manufacturing firms to develop the Altman Z Score. This score, a composite of five financial ratios, achieved 90% accuracy8 and several modern studies have shown its continued validity.9–11 Caution should be applied, however, in applying the Altman Z Score to assess the overall financial condition of hospitals. Revisiting the model in 2002, Edward Altman himself reminds us that the Z Score’s intended use is assessing distress in manufacturing companies.12 Studies extrapolating its use to healthcare5 13 or in predicting financial success14 have been limited and inconclusive.
Credit rating agencies, such as Standard & Poor’s and Moody’s, also create composite scores that attempt to capture the likelihood that a firm will pay back its creditors. These agencies use a combination of statistical studies, experience, and judgement to derive a rating from underlying financial statistics. Credit ratings are, to our knowledge, the best available indicator of financial health, but they are proprietary, often costly to obtain, and are simply unavailable for many hospitals. For hospitals with municipal credits, Merritt Research Services produces a ranking that is both specific to hospitals and aimed at financial condition more broadly than are the credit ratings.
There is no perfect model for assessing overall hospital financial standing, and current ‘gold-standard’ composite scores are unavailable to most health policy researchers. In this paper, we have undertaken a pragmatic approach to creating a simple, one-dimensional score that can be used to indicate the financial condition of a hospital. The YHFS involves a straightforward calculation that relies on widely available data that can be obtained at little or no cost. We assess the validity of the score by its ability to predict debt ratings, when they exist. We find that the YHFS explains 88.8%, 90.5% and 95.0% of the variation in three different credit ratings.
Methods
Hospital financial data for fiscal year 2017 was downloaded from the publicly available Centers for Medicare and Medicaid Services (CMS) Hospital Cost Reports (HCR) database.15 Hospital demographic data was downloaded from the CMS Hospital Compare database.16 We partnered with Merritt Research Services, an independent research and data provider, to investigate the creation of a new hospital financial model. Merritt chief executive officer, Richard Ciccarone, provided guidance in identifying key variables in hospital financial statements and in the use of statistical models to select the variables that are most important in predicting hospital financial performance.
Through our partnership with Merritt Research Services, we obtained a comprehensive list of 232 financial metrics that can be calculated using the data reported by hospitals in their annual financial reports (online supplemental appendix figure 1).1–3 6 7 17 18 We studied the financial analysis literature to assure that these metrics describe all aspects of hospital operations. To select metrics that provide a complete and diverse representation of hospital operations, we examined reports on financial management, consulted financial modelling literature, reviewed financial associations with patient outcomes2 and inspected the methodology used by credit rating agencies.17 18 We selected 30 ratios (figure 1) that, based on our research and the criteria below, appear to be essential in describing the overall financial standing of a hospital:
Supplemental material
Common use in bond ratings of financial institutions.
Ability to indicate the makeup of a hospital’s patient pool (ex. Medicare/Medicaid-predominant vs private insurance).
Predictability of asset management in the hospital.
Profitability, both from patient admissions and overall profit margins.
Predictors of economic performance. List of 30 metrics identified as strong predictors of economic performance of hospitals for use in a ranking score of hospital financial standing after initial evaluation of 232 potential candidates. Metrics in red were removed due to poor reporting rate (<10%) in the CMS cost reports database. Direction of metrics in green was changed to warrant positive correlation with utility. CMS, Centres for Medicare and Medicaid Services; NPR, Net Profit Ratio.
These 30 ratios were computed from numeric values found in the HCR dataset using financial ratio standard equations determined by the US Generally Accepted Accounting Principles.19 For example, profit margin is the quotient of net income/total revenue. Seven of the pre-selected 30 ratios could not be computed due to lack of data in the HCR database (defined as data missing for >10% of hospitals, shown in red, figure 1). The directionality of 5 ‘negative’ financial metrics was changed (shown in green, figure 1) to ensure that increasing utility was positive for all variables. Our sample consisted of 1075 US hospitals reporting financial data in 2017. This sample comprised of general acute care and teaching hospitals (535), health systems (301), hospital districts (102), critical access hospitals (93), children’s hospitals (32) and other hospital types (12). These hospitals had an average of 265 beds, and an average total annual discharges of 12 714 patients of whom an average of 37% were Medicare patients.
After the 23 ratios were computed, 90% of the 1075 hospitals in the sample were randomly selected as the analysis group and the remaining 10% were excluded from the analysis for later use in verification through cross-validation. We used principal coordinate analysis (PCoA) to examine trends in the dataset. This method was selected because it was felt to be best suited for analysis of this dataset and was one that the authors had most experience using. Financial data values were normalized and converted to natural numbers (≥0, µ=0, σ=1). To understand the relationships between these financial metrics, we used PRIMER analytic software (V.6. Lutton, UK: PRIMER-E, 2009) to graph their component measures (figure 2). The Euclidian distances between pairs of variables calculated by the software quantify how similarly two variables describe the dataset. We set a distance threshold of 50 and formed clusters for all variables with an inter-variable Euclidian distance of less than or equal to 50. Variables that clustered together (figure 2) were thought to provide similar information about the dataset and are therefore redundant. To eliminate redundant variables, we used principal components analysis (PCA). Performing PCA of the original dataset, we obtained nine principal components with a minimum eigenvalue of 1 that describe nearly 80% of the variation in the data. Returning to the variable clusters found through PCoA, we eliminated the variable(s) in each cluster that carried a lower weight in the PCA and were therefore felt to less comprehensively describe the dataset. Variables that had formed unique, single clusters in the PCoA were automatically selected for inclusion. In total, ten variables were selected to include in the hospital ranking score (figure 3).
Principal coordinate analysis (PCoA) of hospital financial metrics. (A) Cluster analysis of the Euclidean distances; data point labels are removed for visual clarity. (B) Magnified section of the mid-right sector of the cluster analysis with data point labels added.
The Yale Hospital Financial Score (YHFS). The 10 financial ratios comprising the YHFS were derived from hospital audit reports, with the corresponding weights used in calculating the composite model.
Moody’s Credit Rating is the most widely used credit rating for hospitals and was therefore selected as the presumed gold standard for our analysis. As such, we used Moody’s Credit Rating to determine the relative weight for each variable. We ran an ordinary least squares (OLS) regression model of the numeric Moody’s Credit Rating on the 10 variables. We used the coefficient for each variable as the relative weight of that variable in the model (online supplemental appendix figure 2). The weights were then standardised to a total of 100 as shown in figure 3. Values were ranked within a variable using the Excel percentrank function and the final financial score was calculated as the sum product of ranked values using the weights determined from the analysis.
Supplemental material
Evaluation of the model was conducted using OLS regression analysis. All statistical analyses in this study were performed using STATA statistical data analysis software (V.15, StataCorp).
Results
We constructed a compound score for evaluating hospital financial stability as outlined in the methods, henceforth referred to as the YHFS. We assessed the validity of this model through several tests. We had reserved 10% of the hospitals in the original dataset for cross-validation. After computation of the YHFS, an OLS regression model of the YHFS on the ten financial variables for the reserved hospitals predicts 96.7% (r2=0.967) of the variation in our model (online supplemental appendix figure 3A). We then compared our model to existing proprietary credit scores, used as gold standards. The CreditScope Rank is a proprietary rating constructed by Merritt Research Services, based on their evaluation of hospital finances. There was a significant correlation between our model and the CreditScope Rank (figure 4) and OLS regression showed that the YHFS explains 95.0% (r2=0.950) of the variation of the CreditScope Rank. The YHFS predicted 90.5% (r2=0.905) of the variation in Moody’s and 88.8% (r2=0.888) of the variation in S&P’s bond ratings (online supplemental appendix figure 3B,C).
Supplemental material
Validating the Yale Hospital Financial Score. Graphical comparison of the Yale Hospital Financial Score to the Credit Scope model for financial standing of hospitals.
To further test the validity of our model, we extend the analysis beyond fiscal data, using a multivariate OLS regression to examine the effect of non-monetary factors on the financial stability of a hospital. We found that larger hospitals are at lower risk of default, with a 1.7-point higher YHFS score for every additional 100 beds (0.017, 95% CI 0.01 to 0.02, p<0.001). Conversely, the share of Medicare patients seen at a hospital is associated with a 0.4-point lower YHFS for each additional percent of Medicare patients as a share of total annual discharges (−0.36, 95% CI −0.49 to −0.22, p<0.001). We found no association between hospital financial standing and teaching status of the hospital (6.61, 95% CI −4.19 to 17.4, p=0.23).
Discussion
The understanding of hospital financial standing is important for policy-makers, healthcare researchers and hospital operators. Financially stable hospitals have been shown to provide better patient care,1 2 and financial stability is an imperative consideration during the introduction of new policies, changes in reimbursement or continued growth of competitor healthcare mega-systems. There is currently no systematically validated model for evaluating the financial standing of hospitals.
We undertook a systematic approach in creating a publicly available and transparent financial score. Starting with a list of 232 financial metrics pertaining to all aspects of hospital finance, we selected the 30 most pertinent ratios and used statistical analysis to exclude redundant variables. This method identified 10 financial metrics that were used to construct a single score for a hospital’s risk of default compared with other hospitals. After these 10 metrics were isolated, we evaluated each of the ten ratios included in our model for their indispensability in predicting a hospital’s risk for default. Return on assets is a strong indicator of a hospital’s ability to generate revenue while limiting its costs, a reflection of payment models in the value-based care era. Analysis of hospital data has suggested that high profit margin can signify fewer competitors and a favourable payer mix.20 High return on equity allows a hospital to finance growth whereas low return on equity can make it susceptible to liquidation or acquisition,21 an increasingly common practice in the healthcare industry.22 Not surprisingly, measures of revenue and profit represented a total 51% of the YHFS score. Next to revenue and profit, debt is the second most important indicator for a hospital’s risk of default. Debt is a complex variable because more established and successful companies can take on higher debt at lower rates that can be leveraged to increase revenues in excess of expenses. Complicating the matter further, hospitals are at higher risk of bad debt than other operating entities because they are obligated to provide services before compensation can be guaranteed. To incorporate the intricate impact of debt on risk of default, we included four separate measures of debt: current debt service coverage, interest coverage, total debt and long-term debt to capitalisation, which collectively represent 40% of the YHFS score. Unlike other businesses, hospital may not collect payment until months or even years after a service is rendered due to complexities with insurance reimbursement. Therefore, sufficient cash on hand is critical for hospitals, who cannot rely on immediate or totally predictable payment, yet too much can signify poor management of funds.23 Finally, hospitals compete for a highly skilled workforce, and the salaries and benefits paid by a hospital reflect its ability to generate capital and attract talent, in turn drawing in patients and improving outcomes.
After constructing the model, we used three accepted means of evaluation to show its validity24: (1) by cross-validation with a portion of the initial dataset that is excluded from the model’s construction, (2) by showing a high correlation to a gold standard and (3) by replicating known associations with non-fiscal data. Since there is no agreed on gold standard in this field, we used accepted and widely used credit ratios. We first tested the association between the YHFS and raw financial data. We found that this dataset predicts 96.7% of the model’s variation in a small sample of hospitals excluded from its creation, suggesting that our methodology is generalisable. Although there is no gold standard for the scoring of hospital financial standing, credit scores predict financial instability and are sometimes used to model financial success. We found that our model is highly correlated with credit scores created by CreditScope (95.0%), Moody’s (90.5%) and S&P (88.8%). In the absence of a gold standard, these results support the value of our model.
We then studied Moody’s model more closely to assess where our model differed. To assess the impact of differences over time, we computed the YHFS for hospitals using 2014 data and compared it to Moody’s Credit Ratings for those same hospitals in 2014, then examined how the most varied hospitals have fared over this 5-year period. We found that the average difference between the score predicted by our model and the numeric Moody score in 2014 was 18/100 points (σ=15). This was uniformly distributed across each Moody’s score decile. A primary difference between the two models is that Moody’s focuses on absolute revenue whereas we used metrics that evaluate a hospital’s ability to generate and use funds. Second, we believe that the management and repayment of debt is paramount in the healthcare industry and used a more inclusive assessment of debt by including several measures of debt-management. Finally, we included measures of asset utilisation and efficiency that have been shown to be important in avoiding bankruptcy in the healthcare industry. While our score tended to closely resemble the score assigned by Moody’s, the score for 12 hospitals differed by more than 50/100 points between the two scales. In all 12 cases the Moody’s score was >50 points higher than the score predicted by our model. Since 2014, Moody’s has downgraded the credit ratings for 6 of these 12 hospitals and withdrew its rating for one hospital. In two of the six cases, the credit rating was downgraded by several tiers and in a third case a hospital was downgraded from a top tier credit rating to its second lowest rating. One hospital was sold, one merged with another health system and a third laid off 42 employees. In the hospitals whose credit scores differed by more than 25/100 points between the two models, only seven hospitals were scored higher by our model. For 4 of these seven hospitals, Moody’s has significantly increased their credit rating since 2014 and for the remaining three hospitals, Moody’s credit rating has not changed.
Our third test of validity was using our model to replicate known associations between hospital finances and non-fiscal data. Studies have shown that larger hospitals tend to be more efficient than smaller ones25 because of economies of scale, better recognition, or other factors. A 13-year longitudinal study of financial distress in public hospitals published in 2014 found an increased likelihood of financial distress in hospitals where Medicare payments comprised a greater share of total revenues.26 Finally, the finances of a hospital have not been found to be associated with the hospital’s teaching status.27 All three of these findings were replicated by our model score. Hospital scores in our model are higher for larger hospitals, lower for hospitals with a larger share of Medicare patients and are not associated with the teaching status of a hospital.
Despite our aim for using a rigorous scientific process in creating the YHFS, we recognise several limitations in our methodology. The initial attrition of financial indicators from 232 to 30 was performed by the investigators based on criteria proposed in our methods. While we strived to be methodical in our selection, this human factor inherently introduces bias into the study. Second, duplicate variables in each PCoA variable cluster were removed based on their weight by PCA, which suggests a stronger correlation with the financial dataset but does not guarantee a stronger power of prediction for the retained variable. Third, we created the final model using current ‘gold standards’ for assessing risk of default; though these models are widely accepted, they have not been systematically validated. Fourth, we used financial data from hospitals in the USA in creating the YHFS. Because hospitals in other countries face different financial stressors, this score will need to be further validated using country-specific financial data prior to use in another region. Finally, the construction of our model is limited to currently available financial data. Long-term assessment using longitudinal financial data will be critical in further appraising the model’s validity.
Implications
Numerous studies have shown an association between hospital financial distress and worse patient outcomes.1 2 A model of hospital finances would be invaluable in health policy research, though there is currently no perfect model. Existing methods are often proprietary and are either created with limited information or extrapolated from non-healthcare industries. An ideal model should draw data from diverse financial domains and have proven validity, rely on publicly available data, be easy to construct and provide a single composite score to facilitate comparison to other hospitals and to patient outcomes data. In this study, we have used publicly available data to create a financial score using transparent, systematic methodology that may be easily reproduced and used in health policy research. We have used currently available methods and associations to create a putative score for hospital financial standing. Such a model may screen for hospitals at risk for default, provide information regarding the fiscal effect of new policies, suggest correlations between financial performance and patient outcomes and indicate how different revenue streams affect overall performance.
Conclusion
We outlined a method for constructing a composite score for assessing hospital financial standing using publicly available data. We used three approaches to validate this composite, which we called the YHFS. First, we tested its reproducibility by showing a high degree of association between our model and financial data for hospitals excluded from the sample set used in its construction. Second, we tested its accuracy by showing a high degree of association between this model and proprietary credit ratings that are the current presumed gold standard. Finally, we showed that this model can predict variation in non-monetary data known to be associated with hospital finances. We believe that the YHFS may be useful for assessing the financial standing of healthcare institutions and we hope that this publicly available score may aid health services researchers in future evaluations of hospital finances.
Patient and public involvement
Patient and public involvement in the study was sought with the aim of enriching the creation of a financial score through open discussion in the medical and business communities. We met with investigators at the Yale School of Public Health and the Yale School of Management prior to initiating the study and discussed the current literature and methods for assessing hospital financial standing. The research question and outcome measures were inspired by a reported need from health policy researchers for an open, publicly available score for use in academic work. Open discussions regarding the design of the Yale Hospital Financial Score (YHFS) ultimately led to the recruitment of Richard Ciccarone and Rick Antle from the private and academic financial sectors.
In seeking to create a novel means of assessing hospital financial standing, the authors solicited open feedback regarding the study’s design and the choice of metrics. We used qualitative methods to gather data. This consisted of interviews with economists at the Yale School of Management and the aforementioned private and academic financial sector experts. Through these discussions, 30 metrics were selected from a list of 232. Subsequent variable selection through statistical analysis was inspired by work with biostatisticians at the Johns Hopkins Bloomberg School of Public Health Biostatistics Center with whom the study was discussed. All participants were informed of the time required to participate prior to their involvement in this work.
Data availability statement
Data are available in a public, open access repository. Hospital financial data for fiscal year 2017 was downloaded from the publicly available Centers for Medicare and Medicaid Services (CMS) Hospital Cost Reports (HCR) database. Hospital demographic data was downloaded from the CMS Hospital Compare database.
Ethics statements
Ethics approval
Ethical approval was not required for this study because it did not involve human participation or patient data.
Acknowledgments
The authors would like to express their gratitude to the faculty of the Yale School of Management for the helpful discussions in designing this study, and to Dr. Naveen Venayak and Dr. Maggie Stoeva for their invaluable guidance on statistical analysis.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @hmkyale, @@thehowie
Contributors Primary data analysis and preparation of the manuscript were executed by RZ. HF was the primary advisor for this work. HMK and RA were reviewing authors. RC assisted with data analysis and metric design. All authors met ICMJE criteria for authorship.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests RC is the president of Merritt Research Services, a data and research provider of credit information related to municipal bonds.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.