Objectives Publicly available hospital quality reports seek to inform consumers of important healthcare quality and affordability attributes, and may inform consumer decision-making. To understand how much consumers search for such information online on one Internet search engine, whether they mention such information in social media and how positively they view this information.
Setting and design A leading Internet search engine (Google) was the main focus of the study. Google Trends and Google Adwords keyword analyses were performed for national and Californian searches between 1 August 2012 and 31 July 2013 for keywords related to ‘top hospital’, best hospital’, and ‘hospital quality’, as well as for six specific hospital quality reports. Separately, a proprietary social media monitoring tool was used to investigate blog, forum, social media and traditional media mentions of, and sentiment towards, major public reports of hospital quality in California in 2012.
Primary outcome measures (1) Counts of searches for keywords performed on Google; (2) counts of and (3) sentiment of mentions of public reports on social media.
Results National Google search volume for 75 hospital quality-related terms averaged 610 700 searches per month with strong variation by keyword and by state. A commercial report (Healthgrades) was more commonly searched for nationally on Google than the federal government's Hospital Compare, which otherwise dominated quality-related search terms. Social media references in California to quality reports were generally few, and commercially produced hospital quality reports were more widely mentioned than state (Office of Statewide Healthcare Planning and Development (OSHPD)), or non-profit (CalHospitalCompare) reports.
Conclusions Consumers are somewhat aware of hospital quality based on Internet search activity and social media disclosures. Public stakeholders may be able to broaden their quality dissemination initiatives by advertising on Google or Twitter and using social media interactively with consumers looking for relevant information.
- Public Health
- Health Services Administration & Management
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
Study's novel focus on Internet searches and social media mentions to explore modern consumer approaches in gathering and sharing of information relating to hospital quality.
This study's results were limited by its reliance on one Internet search engine and on proprietary analytical tools.
Although freely available to researchers, we have no way of ensuring that Google's search volumes accurately represent Internet searches.
Similarly, our use of a well-accepted commercial social media monitoring technology relies on proprietary, unverifiable analytics.
Public reports of hospital quality are a national public health priority at the intersection of several important movements in healthcare: quality improvement,1 outcome transparency2 and autonomous, patient-centred decision-making by informed individuals.3 Activating awareness of, interest in, access to, and increasing the usefulness of existing public reports on hospital healthcare delivery performance can positively impact public health.4
To achieve these aims, public health stakeholders must take account of the mechanisms and networks by which information diffusion occurs through the new media platforms, particularly the omnipresence of Internet search engines (eg, Google), ‘Web 2.0’ (second generation Internet-based tools such as Facebook, a leading social network) and social media (web logs or blogs, microbloggers such as Twitter, forums, Internet chat boards and discussion groups).5 The Pew group notes that one in five Americans report that the Internet has helped them deal with at least one major life decision including health-related decisions.6 Among people who have a significant change in their health this proportion reaches one in four.6
The use of online social media and social networks also allows consumers to directly reach out to and mobilise friends, relatives and experts for help.6 More than 50% of online adults between the ages of 18 and 55 years use social networking sites.7 The use of social media is explicable as part of individuals’ well-documented need for autonomy, competence and relatedness, intrinsic and extrinsic rewards and well-being perceptions.8
Despite these new and increasingly popular mechanisms of information dissemination, little research has focused on how public health stakeholders can leverage such channels to complement existing strategies. In theory, such channels could allow fine targeting of subsets of consumers with well-defined interests and the presentation of relevant health-related information to them in cost-effective ways. Such outreach could, for example, allow timely matching of preferred providers to interested patients.
It is increasingly clear that merely placing hospital quality information on the Internet is insufficient.9 Consumers must become aware of the information, they ought to have confidence in the information which needs to become salient in their mind, allowing the information to become actionable. Media dissemination of medical information has been associated with changes in consumer behaviour. A significant drop in vaginal births after caesarean section (CS) followed reports of increased risk in the traditional media and on the Internet.10 Before the Internet age, there was a significant increase in faecal occult blood tests following the widespread media attention to President Reagan's colon cancer.11
However, the public (and referring physicians) appear to underuse online hospital quality reports. For example, the introduction of Hospital Compare, Medicare's public reporting initiative, was not associated with reductions in mortality beyond existing trends for acute myocardial infarction.12 Similarly, public reporting of provider and hospital CS rates via an interactive website in Virginia since 1996 has not controlled CS rate growth—indeed the rate of CS deliveries has risen faster in Virginia than in the USA as a whole.13
This shortfall between the promise of public reports in informing and empowering consumers to choose higher quality, safer or more preferred providers and their limited success in achieving such shifts in consumer choice in practice has been attributed to design and formatting,14 relevance and appropriateness.15 Others argue that methodological issues are important,16 that some online behavioural interventions have been effective,17 or that approaches for reporting quality information have been slowly evolving towards effective strategies that help consumers.18–21
However, an additional explanation may be that most consumers do not know that such information exists and/or that it could be helpful for them. Were this to be true, then one might expect few searches for such information on the Internet, and few mentions of such information or its sources on social media.
To our knowledge, no previous study has used Internet and social media search tools to understand whether consumers are actually aware of publicly available hospital quality reports, have searched for them, and have discussed these with others. Such tools could allow public health researchers to understand which entities produce and disseminate reports that have higher awareness or have been searched for more often. While such awareness is a necessary step in driving consumer activation higher it is clearly not sufficient, and our study did not investigate whether consumer behaviour changed as a result of searches or mentions of quality.
We used Google's in-house search and Adword analytics tools to understand the volume and types of hospital quality and quality report searches occurring through Google's market-leading Internet search engine. Google has maintained an approximate two-thirds of the market for Internet search, ahead of Yahoo and Bing.22 We also used a proprietary social media monitoring tool to investigate all instances in California in which any of six major publicly available hospital quality reports were mentioned on social and traditional media, and the attitude of users (their ‘sentiment’).
Although consumer awareness of such information is clearly necessary for it to inform consumer decision-making, however it is insufficient. Our study was not able to infer whether Internet search activity or social media mentions were associated with the processing of such information. We also did not analyse whether the information contained in the publicly available hospital quality reports was reliable, consistent, relevant and actionable. Moreover, our study was fundamentally limited by a focus on just one Internet search engine, Google. We ignore searches occurring on Yahoo and Microsoft's Bing, and also do not ascertain important other channels of accessing such information. For instance, we do not know how much direct Internet access occurs from consumers to a provider website, or how many individuals click through an advertisement on one site directly through to a particular hospital or public sector website on healthcare quality.
Despite these limitations, the findings in this study may still serve to inform public dissemination strategies. Beyond this, our presentation of advanced Internet search engine and social media analytical tools may allow other researchers to add such analyses to their own, unrelated studies.
Internet search volumes
To understand Internet search behaviour, we focused on the market-leading search engine, Google. We used two Google tools to understand Google search engine volumes. Google's business model depends on online users seeing or ‘clicking through’ display advertisements. These advertisements appear adjacent to search results and next to content on third party websites. Relevant advertisements are more likely to be viewed and clicked through, so Google's revenue depends on accurately matching advertisement keywords with those contained in searches or in published content elsewhere.
Google provides two research tools to investigate keywords and search terms. Both tools are free to researchers after the creation and registration of a user account. We first surfaced the most common search terms in the last year relating to three so-called seed terms: “hospital quality”, “best hospital” and “top hospital” using Google Adwords’ Keyword Planner.23 The degree of relationship is calculated by Google using proprietary algorithms incorporating similar user search and purchasing behaviour. For example, Google suggests that “patient safety” is a related keyword to “hospital quality”. However, AdWords did not surface “healthgrades” as a keyword related to our three seed terms of “hospital quality”, “top hospitals” and “best hospitals”. Google AdWords’ Keyword Planning Tool is designed primarily to suggest keywords for advertisers seeking to create matches between their own online display advertisements and others’ online content. Consequently, advertisers are barred from using a different entity's trademarks in their own keyword list.24
From the suggested keywords we then narrowed down searches based on an arbitrary threshold of 1000 searches per month and by our determination of their specificity to hospital quality (see table 1). We then used the Google Trends tool to quantify relative search volume by term, and by state, over the 12-month period from.25
Social media disclosures
To understand instances (‘mentions’) in which hospital quality terms and hospital quality reports were discussed in social and traditional media, we used a proprietary social media monitoring tool (MAP) from Marketwire/Sysomos (http://www.sysomos.com). Sysomos has mainly corporate customers which include Microsoft, Coca-Cola, Google, Marriott, Nike and other leading consumer good companies.
This tool was leased by us as a ‘software-as-a-service’, and allowed us to remotely access a database maintained by Sysomos to investigate hundreds of millions of current and historical data sources from all publicly disclosed blog, forum, social and traditional media sources (text, image and video) originating in California between 1 January 2012 and 20 December 2012 for the six public hospital quality reports listed below.
Social media tool queries were constructed using a Boolean structure to limit extraneous mentions (eg, reuters reporting on a hospital construction) and to capture abbreviations (eg, USNWR). In addition to mentions, several other commonly used marketing metrics were tracked by MAP.
‘Reach’, the degree of authoritativeness of the online content or user, was estimated by MAP using a proprietary algorithm. Reach is summarised for each mention as a score between 0 and 10 based on its authority and influence. This measure is based on a number of different factors as to assess the influence and reach of the media source. For example, if the mention is on a blog, the measure incorporates posting frequency, number of unique links into and out of the blog, and number of follow-up comments. ‘Impressions’ are an estimate of the number of total views of content seen and were similarly estimated by Sysomos’ proprietary analytics.
Finally, sentiment was analysed by MAP's proprietary natural language processing (NLP) software. This is again carried out automatically by MAP using proprietary algorithms, which summarise the sentiment of a set of such writings (eg, all social media writing regarding Healthgrades, say) by what proportion was positive, negative or neutral. Sysomos claims an 85% overall match rate between its MAP classification algorithms compared to gold standard manual (ie, human expert) categorisation.
Formally, sentiment analysis is the use of NLP tools to identify, extract, grade and categorise subjective information (eg, attitude of blogger to the content being discussed, mood of the person sharing some information on social media) contained within the writing. NLP tools effectively allow computers to understand human language.26 Similar tools have been used by health services researchers to assess the effectiveness of labelling adverse drug events,27 to mine patient medical record free text for evidence of diabetes care28 and to ascertain patient falls faster than incidence reports by applying NLP on imaging requests.29
Publicly available hospital quality reports
Publicly available hospital quality reports in California included the following six major sources. From the information and news industry we selected Thomson Reuters’ Best Hospitals ranking (http://www.100tophospitals.com), US News & World Report's ranking of top hospitals (http://health.usnews.com/best-hospitals) and Healthgrades, a large private supplier of national physician and hospital quality information (http://www.healthgrades.com/).
From government sources we used Medicare's Hospital Compare tool (http://www.medicare.gov/hospitalcompare/) and California's state Office of Statewide Healthcare Planning and Development (OSHPD) reports (http://www.oshpd.ca.gov). Lastly, we selected the non-profit CalHospitalCompare (http://www.calhospitalcompare.org). Hospital Compare, Thomson Reuters and US News hospital quality reports were not specifically related to maternity care.
Internet search volumes
We explored search volume for popular terms related to ‘top hospital’, ‘best hospital’ and ‘hospital quality’. We collected the intersection of all 2095 suggested keywords related to these three seed terms which had been surfaced by Google AdWords Keyword Planning tools.2,3 We then excluded search terms with less than 1000 monthly searches nationally (full list available on request).
From the resulting 75 search terms comprising 610 700 searches nationally each month on average, we then further excluded search terms relating to specific, named hospitals (eg, “dana farber cancer institute”), geographies (eg, “hospitals in los angeles”) as well as search terms not obviously related to quality (eg, “hospital gowns”, “hospital beds”) or not exclusively related to quality (eg, “hospital”, “medicare”).
This process yielded 21 search terms (table 1) comprising a total of 97 700 searches nationally per month on average. We focused further analysis on the most common search terms within this list. We used another tool, Google Trends, to track national and Californian online search activity on Google's search engine for terms related to hospital quality and quality reports between 10 August 2012 and 2 August 2013. Google Trends results are normalised by Google so that the peak search volume over the terms analysed is set to 100.25
In figure 1, we show national time trends for searches for the four most popular hospital-related terms as per table 1. These were “hospital compare” (presumably intended as a search for the Federal Government's Hospital Compare website), “hospital ratings”, “best hospitals in the us” and “us news best hospitals”.
Nationally, searches for the Hospital Compare term dominated those relating to the other search terms, except in mid-July, where the contemporaneous publishing of US News and World Report's rankings occurred. Interestingly, there was also a transient spike in search volume for “hospital ratings” at the same time, although not for the other two search terms.
In unreported analyses we examined search volumes for terms that focused specifically on physicians not hospitals, despite being algorithmically related to the hospital-centric seed terms according to Google AdWords. We found that “doctor reviews” and “best medical schools” dominated “doctor ratings”, and that national searches for all three terms were carried out throughout the year with greater variability than the hospital-related searches shown in figure 1. Within California, there were insufficient searches for “doctor ratings” to be reported by Google.
We also compared search volume in figure 2 between the leading not-for-profit publicly available hospital quality reports (Hospital Compare) and the leading for-profit one (Healthgrades). We found that “healthgrades” and its mis-spelled version “health grades” were each more commonly searched for than “hospital compare”, and that search volumes varied substantially over time. We also observed a very large, transient pair of spikes in search volume in mid-January 2013 for “healthgrades” and the mis-spelled variant which coincided with release of Healthgrades’ Top Hospitals for Outcomes list.
We disaggregated national searches into state-level searches, data not shown, and note in passing an unexpectedly varied distribution geographically for many terms, and these differences did not remain constant across the search terms.
Social media disclosures
Using the commercially available MAP software, we examined a complementary aspect of consumer awareness and interest. MAP tracked mentions of any of the public quality reports in consumer blogs, in comments on forums, and in social media more generally. In table 2, we summarise this measure of consumer activation for each of the six reports in 2012, focusing only on consumers self-identifying as originating in California.
Generally, there were few direct mentions of any of these reports on either social media or traditional media, with less than 1000 instances in each type of media. In unreported analyses, the typical posters of such content were healthcare providers, academics, public health agencies and commercial entities and stakeholders in the healthcare system. The estimated number of online users who were exposed to these mentions in aggregate was large for users of Twitter (>75 000 individuals) and for traditional media (>275 000) but small or not estimated by MAP for the other social media platforms.
Irrespective of social, online or traditional media type, the three commercial suppliers of hospital quality information appeared to dominate consumer awareness using this research tool. The federal government's Hospital Compare lookup tool was a distant fourth in mentions across all platforms, but in turn overwhelmingly dominated the state-level OSHPD and the non-profit CalHospitalCompare reports which were effectively absent from publicly disclosed social conversations.
To classify the sentiment expressed towards the reports, we used MAP's proprietary sentiment analyses to gauge the disposition of consumers to these reports. Twitter sentiment is typically ‘neutral’, given the short length. In blogs and traditional news media sources, reaction to content was generally highly positive for Hospital Compare, Healthgrades, US News and World Report's Top Hospitals and Thomson Reuters's Best Hospitals public quality reports. Given the absent mentions of the OSHPD and CalHospitalCompare hospital quality reports, MAP was unable to determine sentiment.
Our exploratory study used on market-leading Internet search engine, Google and Google Trend search tools to show substantial Internet search activity broadly relating to hospital quality, exceeding half a million monthly searches nationally with a more narrowly defined set of nearly one hundred thousand monthly searches. To put this into context, a high traffic health-related website such as WebMD is searched for on Google more than 1.8 million times per month (and additional inquiries are made directly by typing webmd.com into a browser, circumventing a search engine). Our main finding was that commercially produced hospital quality reports were searched for more commonly than government reports, although searches for HospitalCompare dominated other health-related quality keywords. Unsurprisingly, state-based variation broadly reflected number of consumers but variation differed by keyword searched for on Google.
We supplemented this analysis of awareness by examining social media activity using a different, social media monitoring research tool. This analysis revealed little social media and social network references to hospital quality reports generally, and the instances found tended to be from providers, experts, regulators, academics and not individual consumers. However, of the little activity documented, we again found that commercially produced hospital quality reports were more often mentioned than not-for-profit or government ones.
Our study has several strengths. Our focus was both nationally and on the racially and ethnically diverse, large state of California which suggests that the results may have external validity. Yet, our study also has several important limitations.
Most importantly, our study's ascertainment of consumer access to healthcare information is fundamentally limited to just one firm operating in one such channel. Within the space of Internet search engines, our focus on Google excluded Microsoft's Bing, Yahoo and other smaller but still important Internet search engines. Moreover, we have little understanding of how Google's users might differ from those of other search engines. Outside this space, our study was not designed to ascertain individual's use of, for example, advertisements in web articles (with direct clickthrough to a source of quality information) or the direct use of their Internet browser (to directly enter an already known website such as healthgrades.com). Thus, our study's results in the space of Internet search engines should be seen as a lower bound only.
A separate limitation is that our social media analyses were procured through the use of MAP, an analytical software-as-a-service from Marketwire/Sysomos a leading social media monitoring technology provider. This service uses proprietary, non-transparent NLP algorithms and search methods. It is possible that company-specific design decisions and negotiated upstream access to social media platforms may have affected the reported results.
Similarly, our Internet search analyses were performed using Google's proprietary research tools, and we have no ability to confirm that the data supplied by Google accurately or completely represent all searches. However, we have no reason to believe that the data were masked, distorted or otherwise manipulated by Google in ways that might have biased the results. To reiterate, we have little understanding of how representative Google users are of Internet search engine users in general, or even of Internet users using direct entry of website addresses into their browser or clicking through from ads directly to quality information sites. Despite Google's commanding share of the market, estimated at two-thirds,22 future work will address the extent of sample selection that could arise from focus on a single search engine. Finally, we have assumed without data that an individual searching for ‘healthgrades’ is interested in quality and searching for this reason. This assumption may be wrong, for example, as Healthgrades also informs users as to closest hospital to a particular zipcode or geographical area. Without data on intent, our assumption may therefore lead us to overstate interest in quality.
Many stakeholders have the objective to increase consumer awareness of hospital quality reports, and use of the Internet to disseminate such reports is common, increasing and promising. Yet, the largest and most recent study showed an apparent limited effectiveness of the flagship Hospital Compare reports in reducing mortality for key conditions requiring hospital admission.12 This shortfall between the promise and the actual has been attributed to many causes. In this study, our focus was on whether consumers know that such information exists and/or that it could be helpful for them.
Our use of Internet and social media search tools in this exploratory study suggests that consumers are somewhat aware of publicly available hospital quality reports, have searched for them and have discussed these with others.
We believe that these results suggest that there may be some lessons that could be learned by public stakeholders from the private sector. Such lessons could further their objectives of the enhancement of quality, greater transparency and more empowered consumers. To bridge the gap between the promise and the actuality of hospital quality reports, marketing approaches and consumer activation approaches such as widespread advertising might help ‘get the word out’. Despite substantial focus group testing of report contents and formats at CalHospitalCompare,30 for example, our study finds that their public reporting efforts appear not to have resonated with consumers after their initial launch in 2007.
Fine-tuning of existing public quality reports to enhance their visibility and usability may also help, as could a greater understanding of the crowded field of ratings-producing entities and the limited attention span of consumers. Understanding how consumers see, react to and use the information from all players in this space—private and not for profit—could allow organisations to benchmark their performance, tailor and distinguish their offering from others’ in the market.
Many of these efforts may require substantial investments. We estimate that Healthgrades spends in excess of US$50 million per year in producing, hosting and advertising its services. Replicating a similar approach would be infeasible for a county, but might be possible for a larger state health department. Such entities could advertise relatively cheaply on Google's search engine, paying for ‘clicks’ through to their own websites by individuals who search for hospital quality-related terms.
Many of the keywords in table 1 are priced at around US$1 per click at Google and on other search engines: an advertiser only pays this amount if the online user clicks the advertisement and navigates through to the advertiser's website. For example, a budget can be set in advance, limiting financial risk for a small agency. Similarly, cheap advertising can be purchased through sponsored tweets on Twitter. Certainly, at the federal government level such levels of expenditure are easily possible and might assist in elevating awareness and use of Hospital Compare.
Cheaper niche marketing efforts in building consumer awareness could also include engagement strategies using social media instead of other forms of traditional advertising or passive Internet website creation. Examples of campaigns on social media that have succeeded through identification of leads and direct interaction with online users span the education industry, entertainment, small business and many other ‘viral marketing’ settings.31 Stakeholders could exploit the advertising platforms on, for example, Facebook to serve advertisements to finely targeted (through Facebook's extensive private knowledge of their users interests and private disclosures) segments of individuals ‘in the market’ for a particular health-related message.
The use of social media is increasingly part of individuals’ well-documented need for autonomy, competence and relatedness, intrinsic and extrinsic rewards, and well-being perceptions.8 In addition, by virtue of its open source nature, social media enables much more efficient diffusion of information, network ‘cascades’ and permeation of many more social network.32 In contrast, the traditional highly passive view of online consumers individually receiving broadcast messages from a static website and acting on them has shown at best only modest evidence of success in practice.14 ,15
Finally, a focus on more narrow subsets of customers, conditions or geographies may also be the optimal use of scarce public health resources. Focusing on the most important set of ‘shoppable conditions’ rather than trying to cover every area of healthcare quality may be a more cost-effective strategy. Examples of such conditions with high variation in quality include maternity care.33–37 Targeted engagement at women of childbearing age, women who are pregnant and their family and friends would allow a focus on foreseeable delivery healthcare needs.
Both social media interaction (eg, through a tool like Sysomos’ MAP which allows real time identification of consumers posting relevant content on social media) and Internet search-linked advertising (eg, through Google's AdWords which allow fine delineation of what sort of individuals, where, etc, see advertisements) are feasible tools to reach such niche groups. Such tools could be fruitfully and potentially cheaply used to educate mothers regarding which hospitals in her neighbourhood have consistently lower primary CS rates, higher vaginal birth after cesarean (VBAC) rates, greater lactation consultation access or higher breastfeeding rates.
A strong alignment of interests in stimulating quality improvement, increasing accountability and improving consumer choices appears increasingly opportune for public health stakeholders. An increasing variety of cheap technology tools allow stakeholders to assess and drive consumer awareness of healthcare choices. Given the importance of hospital quality information to payors, providers, consumers and regulators, we are optimistic that the science of public reporting will continue to develop in the ways outlined here.
The authors are grateful to Marketwire for advice and technical assistance in obtaining the social media data using their Sysomos Market Access Platform.
Contributors MDH collected and analysed the data and wrote the first draft of the manuscript. MDH, EC-H and JND conceived and designed the study and revised the manuscript for critical intellectual content. All authors gave final approval of the version to be published. MDH serves as guarantor of the article.
Funding Funding was received by the Agency for Healthcare Research and Quality (R21 HS021868-01; PI: Huesch).
Competing interests MDH discloses support from grants to his institution by the Agency for Healthcare Research and Quality (AHRQ) and Lockheed Martin and receiving payments for consulting to the Institute of Medicine, the Parkland Center for Clinical Innovation and Precision Health Economics, a healthcare consultancy with clients in the pharmaceutical and medical device industries.
Ethics approval The study was qualified for expedited review under 45 CFR 46.110 (b)(2) and was approved by the Institutional Review Board of the University of Southern California.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement All data obtained during this study from Google Search and from Marketwire are freely available to academic researchers upon request to the corresponding author at firstname.lastname@example.org.