Objective To quantify which publicly reported hospital quality metrics have the greatest impact on a patient’s simulated hospital selection for hip or knee arthroplasty.
Design Discrete choice experiment.
Setting Two university-affiliated orthopaedic clinics in the greater Baltimore area, Maryland, USA.
Participants One hundred and twenty-eight patients who were candidates for total hip or knee arthroplasty.
Primary and secondary outcome measures The effect and magnitude of acceptable trade-offs between publicly reported hospital quality parameters on patients’ decision-making strategies using a Hierarchical Bayes model.
Results Publicly reported information on patient perceptions of attention to alleviation of postoperative pain had the most influence on simulated hospital choice (20.7%), followed by methicillin-resistant Staphylococcus aureus (MRSA) rates (18.8%). The understandability of the discharge instructions was deemed the least important attribute with a relative importance of 6.9%. Stratification of these results by insurance status and duration of pain prior to surgery revealed that patient demographics and clinical presentation affect the decision-making paradigm.
Conclusions Publicly available information regarding hospital performance is of interest to hip and knee arthroplasty patients. Patients are willing to accept suboptimal understanding of discharge instructions, lower hospital ratings and suboptimal cleanliness in exchange for better postoperative pain management, lower MRSA rates, and lower complication rates.
- patient preferences
- hospital quality metrics
- total hip and knee arthroplasty
- discrete choice experiment
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- patient preferences
- hospital quality metrics
- total hip and knee arthroplasty
- discrete choice experiment
Strengths and limitations of this study
Patients were engaged in the early stages of the study to identify which quality metrics should be included in the discrete choice experiment.
Discrete choice experiment is a rigorous method of eliciting and quantifying patient preferences for healthcare decision-making.
This study recruited patients from only two orthopaedic practices in one geographical region.
The collection and publication of healthcare quality data are key elements of the Affordable Care Act and other health reform legislation in the USA. Tools such as the Centers for Medicare and Medicaid Services (CMS) Hospital Compare website make hospital quality data available online to consumers. Patients are increasingly engaged in their care and their healthcare decision-making and seek out information from a variety of sources, including the internet.1 The CMS Hospital Compare website provides information regarding impressions of quality of care at over 4000 hospitals and includes over 120 measures.2 Despite the magnitude of information available online in today’s digital age, research suggests that patients are not optimising formal information sources on hospitals3 because they do not understand the information provided,4 are not interested in the nature of the information provided, or do not trust the data.3 Patient input and consultation have the potential to improve how hospital quality data are presented to healthcare consumers and to inform the collection, reporting and presentation of hospital quality data, yet there is limited research assessing consumer preferences regarding what published hospital quality data they deem most important.5
The evaluation of hospital quality data may be of particular interest to patients considering a planned, semielective surgery such as total hip or knee arthroplasty, as has been demonstrated in other countries using similar informed decision-making models,6 because patients have time to review data. In 2010, 310 800 total hip arthroplasty surgeries were performed among inpatients aged 45 and over in the United States. The number and rate of total hip arthroplasty surgeries among inpatients aged 45 and over increased from 138 700 procedures in the year 2000 to 310 800 procedures in 2010: a rate increase of 142.2–257.0 per 100 000 population.7 The demand for these procedures continues to grow as the American public to ages, making hip and knee arthroplasty patients among the top consumers of publicly available hospital quality data.
To our knowledge, orthopaedic patients have not had input into what hospital quality metrics are publicly available and there are limited data assessing patient preferences regarding what information they deem most important to inform their healthcare decisions. The aim of this study was, therefore, to determine which already available hospital quality metrics have the greatest impact on a patient’s simulated hospital selection for hip or knee arthroplasty.
The overall study design included two focus groups of patients eligible for primary or revision hip and knee arthroplasty to elicit important hospital quality metrics, followed by a discrete choice experiment (DCE) to quantify patient preferences for publicly available hospital quality attributes.
Patient and public involvement
Patients were engaged in this research early on. As is described in more detail below, patient stakeholders were consulted to assess whether the proposed research question was of interest to them and to define the attributes used in the data collection tool (a discrete choice questionnaire). Furthermore, patients are at the core of this work since the underlying goal was to identify patient preferences and to improve the information available to them for healthcare decision-making.
Discrete choice experiment
A DCE is a rigorous method of eliciting patient preferences for healthcare decision-making.8 This technique asks respondents to compare two or more hypothetical options, called choice sets, based on described attributes. Each respondent chooses a preferred option in a series of choice sets. The pooled responses from a sample of respondents allow the estimation of the relative importance each attribute (in this case hospital quality metrics), the acceptable trade-offs between these attributes, and the total satisfaction or utility that patients place on these metrics when making a decision, such as selecting a hospital for hip or knee arthroplasty.9 In this study, the DCE is a simulated hospital choice. To derive attributes for a DCE, qualitative methods are recommended, such as the use of focus groups as outlined below.8
Two focus groups were conducted with a total of 11 patients to determine the attributes for inclusion in the DCE survey. The patients were recruited from the University of Maryland Medical System surgery schedule for hip and knee arthroplasty. Consistent with the population who undergo arthroplasty, more participants were female (n=7, 63%) and the mean age was 61 years (Standard Deviation=10). In two, separate 90 min focus groups, participants were asked eight open-ended questions that focused on the following topics:
Primary sources of information for making healthcare decisions.
Considerations when choosing where to have surgery and what was most important.
Knowledge of and experience with CMS Hospital Compare website.
All discussions were audio recorded and transcribed verbatim. Thematic analysis was used to analyse the qualitative findings that emerged from the discussions. This is an established method to extract unique information from unstructured interviews such as those described in this study.10 11 The thematic analysis approach offers fewer preconceptions than other qualitative methods and is therefore subject to less bias.12 Two reviewers independently coded data collected from the discussions. Emergent themes and patterns were identified by initial inductive analysis. Through reviewer consensus, themes were condensed into overarching categories that represented and accurately explained the dataset. NVivo V.9 software was used to help facilitate management and analysis of the data. Finally, in the focus groups, patients were given a list of 37 measures relevant to their procedure that appear on the Hospital Compare website and were tasked with identifying and ranking those that they felt were most important. These measures informed the design of the survey for the DCE.
Focus group findings
None of the focus group participants had ever used the Hospital Compare website, but all expressed interest in using the tool to inform their healthcare decision-making in the future. Overarching themes that emerged from the open-ended focus group questions are presented in figure 1. In selecting a hospital for their surgery, patients sought guidance and information from various sources including friends and family, their primary care provider, websites on the internet, magazine or newspaper articles, and reflection on their own previous healthcare experiences. Before seeing the list of metrics available on the CMS Hospital Compare website at the end of the focus group sessions, focus group participants identified infection rates, complication rates, surgical volume for hip and knee procedures, and patient reviews as the things they cared about most when deciding where to have their surgery. They also placed value on how close the hospital was to their home, the cleanliness of the facility, the kindness of hospital staff, postoperative pain management practices, and appointment availability. Details regarding the measurement and reporting of each metric are comprehensively described at medicare.gov.2
Focus group participants ranked the following seven CMS Hospital Compare metrics as most important when selecting a hospital for surgery:
Methicillin-resistant Staphylococcus aureus (MRSA) blood infections (bacteraemia) (tied for #1).
Clostridioides difficile infections (tied for #1).
Rate of complications for hip/knee arthroplasty patients.
Patients who reported that their room and bathroom were ‘usually’ clean.
Patients who reported that their postoperative pain was ‘usually’ well controlled.
Patients who ‘agree’ they understood their care when they left the hospital.
Overall hospital rating.
The choice modelling platform in JMP V.13 (SAS Institute) was used to develop the choice sets in the survey. Each choice set consisted of two similuated hospital choices. Participants were asked to select hospital A or hospital B (see online supplementary figure S1 for details). Attribute levels were assigned as they appear on the Hospital Compare website along with national benchmarks listed for comparison (see online supplementary table S1 for details). Categorical attributes were coded using effects coding. Forty choice sets were developed using a D-optimal, fractional factorial, balanced design to maximise design efficiency. The 40 choice sets were randomised to 4 versions of the survey, each with 10 questions, to minimise respondent burden. Prior to completing the DCE, demographics and injury information was collected for each respondent. For each participant, one of the four survey versions was randomly administered.
Supplementary file 1
Patients who were candidates for primary or revision total hip or knee arthroplasty at two university-affiliated orthopaedic clinics in the greater Baltimore area were invited to participate in the DCE. Postoperative patients and those presenting for follow-up were excluded since they had already completed the hospital selection process.
There is no consensus on a sample size calculation for choice experiments, however, previous research recommends 50 participants per subgroup in the analysis.13 14 Based on this heuristic, 100 respondents would provide adequate power to model two subgroups.
Demographic and clinical characteristics of the respondents were described using means and standard deviations (SD) for continuous variables and frequencies and proportions for categorical variables. Parameter coefficients (or utility) of the attribute levels were estimated using a hierarchical Bayes model. This technique estimates the distribution of coefficients across the study sample and combines that with the individual respondent’s utility coefficients to derive posterior estimates of the respondent’s utility. Hierarchical Bayesian models use simulation techniques to estimate posterior and prior distributions of the respondents.15 This provides more specific estimates of the mean utilities and also enables individual-level utility estimates to be inferred. Model parameters, including interactions, were calculated iteratively using Gibbs sampling.15 The parameters for each attribute level represent the mean of these iterations and the utility of each attribute-level estimates the strength and direction of the respondents’ preference towards a given attribute. The relative importance of each attribute level was determined by constructing a ratio with the numerator equal to the difference between the maximum value and minimum value for the levels of a particular attribute. The denominator of the ratio is the sum of the values obtained in the numerator for all the attributes. This process normalises the scores to sum 100%. Acceptable trade-offs between attributes were determined using marginal rates of substitution.16
To perform the subgroup analysis, hierarchical cluster analysis was performed to determine the number of clusters in our sample that best predict preference heterogeneity among respondents. Based on cubic cluster criterion,17 two clusters were selected for our sample. All patient characteristics were compared by cluster to determine the variables associated with cluster membership using Student’s t-tests. In addition, individual-level utility estimates were compared by cluster using Student’s t-tests to determine the attributes associated with cluster membership. The p-value for statistical significance was set at p<0.05. All statistical analyses were performed using JMP Pro V.13 (SAS Institute).
Of the 194 patients meeting the screening inclusion criteria, 128 (66%) respondents completed the survey. Table 1 describes the 128 patients who agreed to complete the questionnaire. Of these respondents, 77 (60%) were female and most were white (n=102, 80%). The mean age of participants was 66 years (SD=10). Eighty-two (65%) were considering surgery because of knee pain or concerns, 28 (13%) because of hip pain and 17 (13%) reported having both hip and knee pain. The median duration of pain reported was 3.5 years (interquartile range: 1–10). Education level of participants varied with 35 (27%) having completed high school or less, 41 (32%) completed some college and 50 (39%) reported that they had obtained a college degree or higher. Thirty-one (24%) reported an annual income of <US$50 000, 40 (31%) reported earning US$50 000-US$100 000, and 40 (31%) reported an annual income of >US$100 000. Most participants had private insurance (n=68, 53%) while 23 (18%) reported that they relied on Medicare. When asked if they use information from the internet to make healthcare-related decisions, 47 participants (24%) said yes. The most common websites used by patients to get information were consumer reports (n=13, 10%), CMS Hospital Compare (n=8, 6%), other miscellaneous websites (n=8, 6%) and individual hospital websites (n=3, 2%) (table 1).
Of the attributes included in the survey, patients placed the strongest relative important on postoperative pain management (21%), closely followed by MRSA bacteraemia rates (19%) (figure 2). The understandability of the discharge instructions was deemed as the least importation attribute among the respondents with a relative importance of 7%.
Table 2 presents the parameter estimate of utilities (ß) for each attribute level. Attributes with a positive utility parameter and values further from zero indicate a stronger patient preference. A mean utility of 0.19 (95% Confiden Interval(CI): 0.16 to 0.22) was associated with a 1% increase in the proportion of patients who reported that their postoperative pain was ‘always’ well controlled, in our hypothetical hospitals presented in the choice sets. By comparison, 1% increases in patients who reported that their room and bathroom were ‘always’ clean and they always understood their discharge instructions were 0.09 (95% CI 0.07 to 0.12) and 0.08 (95% CI 0.05 to 0.12), respectively. A 1% increase in the proportion of patients that gave the hospital an overall score of 9 or higher was associated with a 0.12 (95% CI 0.09 to 0.15) gain in utility. The utility associated with complication rates, MRSA rates and C. difficile rates compared with national benchmarks follows a logical gradient. Rates that were no different than the national benchmarks were associated with positive utility. A negative utility (−0.43; 95% CI−0.88 to −0.04) was associated with hospitals that had insufficient samples to determine complication rates.
Trade-offs were assessed between attributes. Shifting from MRSA rates worse than the national benchmark to an MRSA rate that is no different than the national benchmark would provide equivalent utility as a 12% absolute increase in the proportion of patients that reported that their postoperative pain was always well controlled. By comparison, a shift from MRSA rates worse than the national benchmark to an MRSA rate that is better than the national benchmark would provide equivalent utility as a 19% absolute increase in the proportion of patients that reported that their postoperative pain was always well controlled. Similarly, moving from a complication rate that is no different than the national benchmark to a complication rate that is better than the national benchmark would provide equivalent utility as a 13% absolute increase in the proportion of patient who gave their hospital a rating of 9 or 10 on a 10 point scale.
In the hierarchical clustering analysis, two clusters emerged from our sample based on cubic cluster criterion (table 3). Sixty-one respondents (48%) were members of cluster 1 and 67 (52%) were members of cluster 2. Cluster membership was significantly associated with health insurance (p=0.02) and duration of pain (p=0.01). Cluster 1 membership had a disproportionately high proportion of patients with private health insurance (n=36, 64% vs n=29, 43%). Cluster 1 was also characterised by respondents with a reported median duration of pain of 2.5 years (IQR 1–5) vs 5 years (IQR 2–10) in cluster 2. There was significant heterogeneity in preferences between the two clusters with respect to complication rates, MRSA events and C. difficile events. Cluster 1 members had a very high utility associated with complication rates (ß=1.88, SE=0.06), MRSA events (ß=1.54, SE=0.02), and C. difficile events (ß=1.02, SE=0.03) that were ‘better’ than the national benchmark. Cluster 2 members had increased aversion to attributes that were ‘worse’ than the national benchmark for complication rates (ß=−1.64, SE=0.04), MRSA events (ß=−2.06, SE=0.04), and C. difficile events (ß=−1.43, SE=0.04). In addition, Cluster 2 viewed hospitals with an insufficient sample to provide complication rates more favourably than Cluster 1 (ß=−0.15 vs ß=−0.74, p<0.01). The divergent utilities associated with complication rates, and MRSA and C. difficile infection rates between the two clusters illustrate the opposing reference points for members of the two clusters. Cluster 1, a group with membership associated with private insurance and a shorter duration of pain, derives less marginal utility when performance measures improve from worse than national benchmarks to the national benchmark level and comparatively more marginal utility when performance measures improve from national benchmarks to better than national levels. Cluster 2, a group with membership inversely correlated with private insurance and associated with a longer duration of pain, gains comparatively more marginal utility when performance measures improve from worse than national benchmarks to the national standard and less marginal utility when performance measures improve from that national benchmarks to better than the national benchmark. In other words, improving hospital quality performance from worse than national levels to the national standard is more attractive to members of cluster 2 than it would be to members of cluster 1. Improving hospital quality performance from national benchmarks to better than national benchmarks would be more attractive to members of cluster 1 than it would be to members of cluster 2. There was no statistical difference (p=0.6) in the distribution of hip vs knee patients between the two clusters in table 3.
Postoperative pain management and hospital MRSA rates were the two most important attributes identified by prospective hip and knee arthroplasty patients when presented with a simulated hospital choice for their surgery. The understandability of discharge instructions was the least important of our included attributes. Patients would be willing to accept a 12% absolute decline in the proportion of patients that report their postoperative pain was always well controlled if the hospital reported MRSA rates that were better than the national benchmark. Hierarchical clustering analyses identified two clusters with cluster membership differing by health insurance coverage and duration of pain. The cluster differentiated by higher private insurance coverage and a shorter duration of pain significantly favoured hospitals with better than national benchmarks on complication rates, MRSA rates, and C. difficile rates. The cluster with less private insurance coverage and a longer duration of pain demonstrated a significantly stronger aversion to hospitals with worse than national benchmarks on complication rates, MRSA rates, and C. difficile rates.
To our knowledge, this is the first study to use the DCE methodology to quantify hip and knee arthroplasty patient preferences towards simulated hospital choice. By quantifying the relative value of each quality metric in selecting a hospital for an elective arthroplasty procedure, this study allowed us to describe acceptable trade-offs in publicly available quality metrics. The marginal rates of substitution enable us to quantify and compare acceptable trade-offs in this patient population. The cluster analysis provides insight into which attributes are most important to different segments of this orthopaedic population. Identifying patient preferences and hospital quality metrics that are most important to patients and healthcare consumers is an informative step forward to offer concrete suggestions for ways to improve tools such as Hospital Compare. A study by Rothberg et al, which included Hospital Compare statistics, revealed vast discrepancies between five online healthcare resources evaluating hospitals using the same outcome measures.18 Publicly available resources such as these must be accurate and should consider sample size and casemix adjustment.19 Hospital quality data must be appropriately standardised to ensure favourable outcomes, and to improve patient safety, education, and informed decision-making.20
The focus group findings are congruent with results from other studies that suggest patients often rely on information from personal contacts and experiences in addition to publicly available hospital data.21 Focus group participants ranked metrics describing hospital-associated infections, such MRSA and C. difficile rates, as most important to them. Infection control-related metrics also emerged as primary items of concern in the discrete choice questionnaire with better MRSA rates (compared with the national benchmark) having a high positive utility score of 1.69 (95% CI 1.26 to 2.12) and worse MRSA rates (compared with the national benchmark) having a high negative utility score of −1.88 (95% CI −2.27 to −1.55). Postoperative pain management was deemed as most important when selecting a hospital for hip or knee arthroplasty. Overall, patients were willing to accept suboptimal understanding of discharge instructions, lower hospital ratings and suboptimal cleanliness in exchange for better postoperative pain management, lower MRSA rates and lower complication rates.
The hierarchical cluster analysis revealed that patients with a private health insurance and shorter duration of pain had a stronger preference towards favourable outcomes for surgical complication rate for total hip or knee arthroplasty, MRSA, and C. difficile infection rates. Patients with a non-private health insurance and longer duration of pain, on the contrary, had a stronger aversion towards unfavourable outcomes of the studied CMS metrics. These measures could perhaps be surrogates for health literacy, suggesting that those with private health insurance are also more able to address their pain concern more promptly resulting in a shorter duration of pain. These findings suggest that this subpopulation prefers a stronger positive outcome rather than avoiding a negative outcome. This subpopulation also had a strong aversion to the sample too small level for surgical complication rate for total hip or knee arthroplasty, reflecting an understanding that a hospital facility may not perform enough procedures to accrue necessary data for this quality measure. This difference in health literacy may affect healthcare choices and decision-making strategy. These findings support the conclusions from a similar study conducted in the Netherlands that suggested that tailored presentation of data should be considered so that public information is more relevant and may motivate patients to use these resources in their hospital choice process.6
Postoperative pain management is a primary concern for hip and knee arthroplasty patients, as well as their clinicians. Studies suggest that severe postoperative pain is associated with an increased risk for complications, delayed return to normal functioning, progression to persistent pain states, longer length of hospital stay, and higher rates of readmission.22 A recent study using data from the 2015 National Survey on Drug Use and Health estimated that 38% of US adults used prescription opioids in the prior year and that among these 92 million persons, 13% reported prescription opioid misuse.23 Previous research suggests that the relative importance of attributes may be sensitive to a contemporary experience with that attribute, termed reference dependence. While the respondents’ temporal postoperative pain experience may have elevated the relative importance of that attribute compared with other attributes, the patient importance of postoperative pain management requires consideration.24 It is possible that patients place inordinate hope on being pain free or have the misconception that discomfort is incompatible with health. Observed patients preferences may be consistent with psychological and social influences on human illness behaviour. Coupled with the national opioid crisis, orthopaedic surgeons have the opportunity to play a critical role in reducing the severity of pain following surgery and in decreasing opioid use.
One possible limitation of this study is that it analyses stated preferences of the respondents to hypothetical scenarios and their actual responses may differ. Furthermore, participants in this study were recruited from only two orthopaedic practices, and therefore, patient preferences in other medical disciplines and other geographical locations may differ. This study included both primary and revision total hip and knee arthroplasty patients, but we did not explore the differences between these two groups. Finally, this study was not able to evaluate the importance of some key surgical outcomes such as joint range of motion and function. Despite these limitations, we believe that these findings have the potential to improve public reporting of hospital quality data by engaging patients in the development of tools such as CMS Hospital Compare and ensuring that data presented are focused on what patients care most about and that the data are presented in a way that makes sense to patients. A study by Fischer et al examined patients’ decision-making strategies in hospital selection from a psychology and cognitive science perspective. The authors theorised that during the decision-making process, patients wish to reduce information complexity and cognitive effort.25 Minimising the number of metrics presented and simplifying the data on public websites such as CMS Hospital Compare could potentially streamline this process for patients and increase use of such tools. Understanding patient preferences related to hospital selection for hip and knee arthroplasty is also likely of interest to orthopaedic surgeons and hospital administrators as they strive to improve patient satisfaction.
Publicly available information regarding hospital performance is of interest to hip and knee arthroplasty patients. Healthcare consumers are particularly concerned about postoperative pain management and hospital-associated infection rates when selecting a hospital for their hip or knee arthroplasty.
The authors thank all patient advisors who participated in the focus group sessions and who completed the survey. Without your valuable insights, this work would not be possible.
LMO’H and IC contributed equally.
Contributors LMO and ADH conceived the study. LMO, IC, NNO, RVO, DFD, ADH and TTM had input into the design of the study. LMO collected the focus group data and IC collected the discrete choice questionnaires. NNO and LMO analysed the data. LMO, IC and NNO interpreted the data. LMO and IC drafted the manuscript. NNO, RVO, DFD, ADH and TTM revised it critically for important intellectual content. All authors approved the final version to be published. All authors agree to be accountable for all aspects of the work. LMO and IC contributed equally to this paper.
Funding This work was supported by 5K24AI079040-05 (ADH) from the National Institutes of Health (NIH), and the Banting Postdoctoral Fellowship Program administered by the Government of Canada (LMO).
Competing interests None declared.
Ethics approval This study was reviewed by the University of Maryland School of Medicine Institutional Review Board (HP-00058264) and was deemed exempt from regulation based on the nature of the research.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Dataset available on written email request to authors.
Patient consent for publication Not required.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.