Article Text
PDF
Abstract
Objective To identify how parents judge the credibility of online health news stories with links to scientific research.
Design This qualitative study interviewed parents who read online stories about e-cigarettes and human papillomavirus (HPV) vaccination published by top-tier US news organisations. Researchers asked participants to describe elements of a story that influenced their judgement about content credibility. Researchers analysed transcripts using inductive and deductive techniques. Deductive analysis drew on cognitive heuristics previously identified as being used by the public to judge online health information. Inductive analysis allowed the emergence of new heuristics, especially relating to health.
Setting The US National Cancer Institute’s Audience Research Lab in Maryland, in August–November 2018.
Participants Sixty-four parents with at least one child between the ages of 9 and 17 residing in Maryland, Virginia, or the District of Columbia participated. Researchers randomly assigned 31 parents to the HPV vaccination story and 33 to the e-cigarette story.
Results Evidence of existing heuristics, including reputation, endorsement, consistency, self-confirmation, expectancy violation and persuasive intent emerged from the interviews, with participants deeming stories credible when mentioning physicians (reputation heuristic) and/or consistent with information provided by personal physicians (consistency heuristic). Participants also described making credibility judgements based on presence of statistics, links to scientific research and their general feelings about news media. In relation to presence of statistics and links, participants reported these elements increased the credibility of the news story, whereas their feelings about the news media decreased their credibility judgement.
Conclusions Parents used a constellation of heuristics to judge the credibility of online health news stories. Previously identified heuristics for online health information are also applicable in the context of health news stories. The findings have implications for initiatives in education, health communication and journalism directed towards increasing the public’s engagement with health news and their credibility judgements.
- medical journalism
- health informatics
- qualitative research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Over 60 parents participated in interviews.
Topics selected for topical relevance to parent participants.
Removal of the news publications’ names focused participants on story content but may have also taken away from the ‘real world’ experience of how the public reads online health news.
This study was conducted in a laboratory at the National Cancer Institute, which may have caused laboratory effects (ie, a participant’s reaction to the story influenced by the location of the interview).
Parent participants were from a defined geographical region and well educated.
Introduction
Every day, thousands of people—up to 68% of the US population—turn to the news media for health advice.1 The public’s reliance on the work of journalists for health information, much of which reports on biomedical research produced from scientific studies, influences knowledge and attitudes about health and ultimately behaviours.2 3 For example, researchers found that the news media, through articles directed at parents, physicians, policymakers and the general public, contributed to preventive human papillomavirus (HPV) vaccination rates below government targets.4 The news media, through such directed messages, act as a critical communication channel for transmitting biomedical research to the public.5–9
Like many communication channels in today’s information landscape, the news media present readers an overwhelming amount of health information.10 11 This influx of health information can lead to information overload,12 which, in turn, challenges readers in their ability to identify which health news stories have credibility (defined as ‘the believability of information’),13 and therefore worth their engagement. To mitigate this influx of information, researchers, in the broader realm of online information, have identified that readers use cognitive heuristics or mental shortcuts to make credibility judgements about information they encounter online.14 15
Through a series of studies, Metzger and colleagues identified six cognitive heuristics that individuals use to manage uncertainty and decrease the cognitive load necessary to assess the credibility of online information.13 16 These heuristics include reputation, endorsement, consistency, self-confirmation, expectancy violation and persuasive intent. For example, readers may judge information credible based on mental shortcuts such as if it appears on a website they deem reputable (reputation heuristic), if it is endorsed by a prestigious university (endorsement heuristic) or if the information is consistent with their already held beliefs (consistency heuristic). Klawitter and Hargittai identified that people who use websites with health information also employ these heuristics in their reading of health information online.17
However, there is a dearth of research about how the public attends to and uses heuristics to judge the credibility of online health news stories. Moreover, existing research does not address how parents, 43% of which use online information to make health decisions for their children,18 19 leverage these heuristics. This gap has implications for the optimal presentation of news, educational initiatives and ultimately public health. Thus, in this study, we explored the research question: Which, if any, cognitive heuristics or cues are used by news readers when considering the credibility of online health news stories relevant to adolescent health?
Methods
We conducted a qualitative interview study using thematic analysis guided by a constructivist epistemology (the perspective that knowledge is co-created by individuals). This study was a component of a larger mixed-methods initiative to understand how parents read online news articles citing health research potentially relevant to healthcare decision-making for their child(ren). The National Cancer Institute (NCI) (Protocol #18-NCI-00551) and Uniformed Services University of the Health Sciences Institutional Review Boards (IRB) (Protocol #: HU-MED-83-9908) reviewed this study and determined it to be exempt from further review. Per the regulations of these two bodies, access to the interview data is strictly controlled and limited to the core research team, making it impossible for us to publicly deposit these data or make them available on request.
Patient and public involvement
Patients and the public were not involved in this study.
Recruitment
Recruitment, data collection and analysis occurred from August 2018 to December 2018. Based on power calculations for the overall initiative, we recruited 90 participants; 64 of whom participated in the qualitative component of the study presented here. A professional recruitment company identified all study participants and conducted participant screening to ensure that participants met study inclusion criteria. Because news story stimuli focused on two topics pertinent to adolescent health (HPV vaccination and e-cigarette use in schools), recruitment focused on individuals for whom these stories were likely to be salient (ie, parents and caregivers of children in the HPV vaccine-eligible age range). Inclusion criteria required that participants be parents or guardians of at least one child, age 9–17, and a resident in Maryland, Virginia, or the District of Columbia. We required that participants had one or more children within this age range to ensure that the study stimuli (ie, health news stories on HPV vaccine and e-cigarettes) would have topical relevance for participants’ family health. Participants were compensated $75 on completing the study.
Data collection
We collected all data at the NCI Audience Research Lab. During the informed consent procedures, researchers explained to participants that the purpose of the study was to better understand how parents read online health news. Following informed consent, we randomly assigned each participant to read a brief online news story while an eye tracker documented their ocular patterns. We then interviewed each participant after they read the assigned news story. Per the Consolidated criteria for Reporting Qualitative research (COREQ) checklist for qualitative research, team members contributed to the study in the following manner: LAM, an Associate Professor of Medicine with a PhD in health professions education, and MK, a former postdoctoral fellow at NCI with a PhD in health communications, conducted interviews (10–45 min each). Using real-time video, LLM observed all interviews and all three researchers observed each participant’s news-article reading. LLM is an Assistant Professor of Journalism with a PhD in Learning Sciences Technology Design. Researchers had no previous knowledge of the participants.
The research team used a semistructured interview guide to conduct the interviews, which was based on a review of the literature and feedback from a pilot study conducted with nine parent participants in spring 2017 (see online supplementary appendix A for the interview guide). In interviews, participants described their level of trust in the assigned article and the characteristics of the news story that contributed to its credibility.
Supplemental material
Data collection focused on participants’ reactions to the online news story they were assigned. This study focused on two news stories published in 2016 by the Los Angeles Times and The New York Times. Both articles were included in both the pilot test conducted in 2017 and the full study in 2018 for consistency. These stories were identified based on a listing of 2016 news stories that featured links to cancer research that was compiled by the author team for an earlier study.20 From this listing, we focused on news stories reporting on e-cigarettes and the HPV vaccine as these two topics have been previously identified as relevant areas of cancer prevention among parents of adolescents.21 The two specific stories were selected for their inclusion because they contained links to journal articles, were published in two online news sources with national readership and were both less than 1000 words in length. Additionally, using the Flesch-Kincaid Readability Tests, we calculated that both news stories were scored at the college level.
In keeping with the original presentation of the news story, a photo accompanied each news story. All articles contained several clickable links, which were featured in the original online news story, including to a freely accessible full-text version of a scientific study and related websites. Links appeared in the original news stories and highlighted text as either a single word (eg, Pediatrics) or a short phrase (eg, Centres for Disease Control and Prevention) underlined in blue, indicating additional information accessible by clicking.
One article, published in The New York Times,22 explained the HPV vaccine’s role in reducing HPV in teenagers, as reported in a study in Paediatrics.23 This article, as presented to the participants, contained 949 words and 7 links. The other article, which ran in The Los Angeles Times,24 discussed teens’ e-cigarette use, contained 684 words and 4 links, including one pointing to a study in JAMA.25 The articles, as published by the news organisations, included multiple internal links that connected a reader to pages within the publication. For example, in the HPV vaccine stimuli, the first sentence of the article originally included the link text ‘cervical cancer’, which directed readers to The New York Times page on general wellness. Previous research recommended a need to focus on selective exposure to heuristics26; thus, to simplify the stimuli and focus our participants on the text of the news story, we removed most internal links as well as all advertisements, the journalist’s name and the masthead of the news publication. We informed participants that the article came from a national news publication and that they should engage with the news story as though they were at home. No further comments were made about the source.
Data analysis
We audio recorded and deidentified each interview; interviews were transcribed verbatim and reviewed for accuracy. Following this process, we began preliminary analysis of the transcripts.
To identify, analyse and report patterns found in our transcripts, we used iterative rounds of thematic analysis.27 Through close line-by-line reading, we identified and defined themes within the data that were of importance to answering the research question. In our analysis, we combined both inductive and deductive techniques. The deductive component drew on Metzger’s identified cognitive heuristics16; the inductive portion encouraged the emergence of new heuristics from the qualitative data, especially those related to health, which has not been a focus of Metzger’s work.
We began analysis following our first round of interviews (n=10); all transcripts were coded using Dedoose software.28 Throughout data collection, all researchers actively reviewed transcripts, considering and discussing the resonance and fit of the codes, ultimately raising the level of analysis from categorising to conceptualising.
For each stimulus, we worked to achieve information power29 (the state of having interviewed enough participants to answer the research question) and ensure that we identified all relevant themes from the data. As such, the amount of data captured enabled us to answer our research question. However, due to the nature of the larger study, we conducted interviews with all participants despite agreeing as a team that we had sufficient information power after interviewing the initial 30 parents.
Results
We interviewed 64 parents and guardians. Participant demographic characteristics are reported in table 1. Across all participants, 31 were randomly assigned to view the HPV stimuli and 33 to the e-cigarette stimuli.
Descriptive characteristics of the study sample (N=64)
In our analysis, we identified evidence of Metzger’s six cognitive heuristics (reputation, endorsement, consistency, self-confirmation, expectancy violation and persuasive intent). Additionally, we observed participants describing how the presence of numbers and statistics, the inclusion of linked scientific research and their general feelings about the news media influenced their judgrments about the credibility of the story they read. We did not observe major differences in how the parents applied the heuristics dependent on which news story they were assigned except that participants did not describe using the endorsement heuristic when reading about e-cigarettes. Major differences were not expected as these heuristics have been used across a variety of information topics featured on health websites with limited differences observed.12 14 15 Additionally, the observed similarities across the two stories allowed for broader generalisations about cancer prevention news stories to be drawn among the entire pool of participants. To provide evidence of our findings, we present illustrative quotes that include the stimuli (ie, H=HPV; E=e-cigarette news stories) and the participant’s number in the study.
Reputation
The reputation heuristic is evoked when individuals judge the credibility of information based on whether they recognise the source.16 In this study, we did not divulge the source (ie, the name of the news publication). However, we observed participants applying the reputation heuristic to information sources that were mentioned as contributing to or referenced in the article, such that many participants noted that the presence of mentions of universities, non-profit organisations (eg, Tobacco-Free Kids) or government agencies. Generally, the presence and recognition of these entities bolstered participants’ trust in the news story. In several cases, participants described their judgement as rooted in the entity’s known reputation and specifically named the source. For example, “There’s a link in the news story to credible places like Yale and Cornell and places that you feel like you could potentially trust the information.” (H26) Another participant noted, “There were some clues. Like here I read the American Association of Medicine; that makes me think it’s trustworthy. I think it’s a good thing that there are government agencies. I still think these are very respected from the U.S. population. So, if I tend to read something from NCI, CDC or WHO, I think I would trust it.” (E67) If participants perceived an entity as having longevity, their belief in it was enhanced. “Many of these research institutions are on point. They have been around for a lot of years, and you don’t last long in the game if you are not on point.” (E61)
Overall, participant mentions of institutions by reputation were primarily positive, but not always. One participant raised the following point: “It was a good article, but sometimes you know — [the] CDC, you know [they’re] with the government. Who sponsored the study? You know even with the government, what comes to mind is the Flint, Michigan water thing. Wasn’t the government involved in that? Yet, it still happened.” (E16)
Endorsement
The endorsement heuristic suggests that people make credibility judgements based on whether the information is recommended by those who they know or a group of unknown individuals presented in aggregate form.14 Due to the design of this study, participants were not familiar with the scientists or physicians featured in the news story, and thus we did not observe this heuristic on the individual level. However, participants spoke broadly about doctors and scientists as groups of professionals who positively influenced their decisions to trust the article. “I really trust the doctor [in the news story]. They just know more.” (H33) Another participant noted: “There are all these PhDs that were quoted. It made it feel more real.” (H89) Readers of the e-cigarette news story did not describe this heuristic.
Consistency
The consistency heuristic (ie, the ‘bandwagon heuristic’28) focuses on credibility judgements based on the belief that if other people find information credible, then an individual will also find it credible.14 In this study, participants’ use of this heuristic was most pronounced when the news story was consistent with their interactions with or information received from their personal physician. One participant indicated that she trusted the presented article by saying, “This was the same information that my primary care doctor had shared with us. I felt comfortable because I don’t think she would give me false information.” (H56) Another participant noted, “These are things that I have heard from my own doctor, so that kind of validated it.” (H30)
The consistency heuristic is also associated with individuals’ efforts to triangulate the alignment of information with that found in external sources. Several participants mentioned checking consistency as something that they would do to verify the news story’s information. For example, one participant explained, “I might just google vaping vs cancer cause and see what stats are out there. If I followed the links, I want to see where they took me. If it were places like NIH and WebMD, I’d be okay.” (E76) Another participant noted a desire to check specific data points to inform their credibility judgements, “Now if I see something stating here, 14 million Americans will contract the virus and clear it. So, I would initially google it to see if that one fact right there is valid. If that’s valid then I can trust and adhere to what they are saying.” (H44) However, while participants had immediate access to the internet while reading the news story, none took these described steps to seek additional information beyond that which was linked from the news story.
Self-confirmation
Metzger describes the self-confirmation heuristic as the tendency for people to judge credibility based on whether information aligns with their self-held beliefs and to reject that which does not.14 Multiple participants judged the assigned news story based on whether it confirmed their existing beliefs or if it aligned with personal experiences. If the news story confirmed such previously held beliefs it was deemed credible; if not, it was suspect. One participant noted, “Because it has some of the things that I sort of know, I would trust it. It talks about the same age range that my older one was told to get the shot. So, some of the things I already knew were validated.” (H26) Similarly, a father noted, “I thought this was a good article. I would give it a 7 out of 10. It was informational and fact-based. This also reinforced a lot of things that I already knew about the topic.” (E58)
Expectancy violation
The expectancy violation heuristic asserts that individuals will find information less credible if it violates their expectations, such that if an information source contains elements or features that are unexpected (eg, pop-up ads, request for personal information). Conversely, individuals will consider a source to be of higher credibility if it manages to not to violate their expectations.14 In this study, we found that participants remarked on the latter condition and felt that the news story they read presented them with what they would expect from a health news story and therefore found it credible. One mother noted, “I didn’t read anything that made me think this was slanted or biased in any way. I trusted it.” (H50)
Persuasive intent
Metzger described the heuristic of persuasive intent as an individual’s tendency to judge information as not credible because they find it biassed, often in regard to commercial purposes.11 Research on this heuristic has generally focused on the presence of advertising on websites.14 15 29 As previously discussed, all advertisements in the news articles were removed. Thus, participants did not comment on this aspect. However, participants did describe credibility judgements based on whether they perceived bias in the content of the news story. For example, a participant noted that she found the news story credible: “I thought it was presented in a very straightforward manner… It didn’t seem like anyone mentioned had an axe to grind. It mentioned some controversy around the vaccine but didn’t provoke the controversy.” (H26) Another participant commented, “It was well written, and they let you know that there is a lot more information that needs to be done. So, they didn’t blow it up; they left it just where it is, and I like that. I think that’s important to know I’m reading the truth.” (E61)
In relation to persuasive intent, several participants commented on the importance of balance in the news story for judging credibility. A participant explained, “It was persuasive in the way it presented the information. It was definitely trying to point out the benefits. Although it did point to some of the pitfalls with the research. I think it was appropriate because it did acknowledge why some people would want to vaccinate and some wouldn’t.” (H65)
Presence of numbers and statistics
Multiple participants described that the presence of numbers and statistics helped them judge the new story’s credibility and, in some cases, served as ‘cues’ (E74) for credibility. As is common in news stories reporting on research,30 both news stories presented basic descriptive statistics. For example, the HPV vaccine news story included the following: ‘Despite the vaccine’s proven effectiveness, immunisation rates remain low—about 40 percent of girls and 20 percent of boys between the ages of 13 and 17’.22
In several cases, participants described the presence of numbers and statistics as reassuring. “The statistics also helped. The data definitely drove the point home and made it more credible.” (E91) Another participant noted: “Numbers always help me trust more. Numbers and percentages because I don’t know. If you see a higher percentage then you tend to be more okay with something.” (H77) Related to reassurance there was a sense among participants that numbers presented truth. For example, a participant noted: “I prefer percentages, especially when you are with someone with cancer. You need the facts. I need the numbers. Don’t tell me maybe. I need the numbers because the numbers are usually based on facts. People don’t usually make up percentages. They are usually based on facts, so I look for those.” (H83)
In relation to numbers and statistics, we observed that participants described the general presence of these elements as symbols or markers of credibility, but rarely described how they interpreted the meaning of the numbers within the context of the news stories. Moreover, generally, participants felt positive about the presence of statistics; however, several participants across both stimuli noted that they were unable to decipher them or found them confusing. Lastly, when referencing numbers and statistics, participants spoke in generalities, rarely pointing to a specific data point in the article or referencing the meaning of the numbers and statistics within the context of the news story.
Links to scientific research
Participants noted that the presence of links to scientific articles factored into their credibility judgements. Participants remarked that the presence of the links, whether they clicked them or not, provided opportunities for confirmation of the story’s information and offered value through the easy access of the scientific study. “I liked that they did provide the link. They were trying to be balanced. I like it when they give you the tools to get to the information on your own. Otherwise you have to dig around on your own.” (H11) Another participant commented, “In this case, they did include the links to the independent studies, which gives you the chance to go to them and judge the validity of the study.” (E55)
Overall, participants described the presence of links in the news stories to be positive. However, only a minority of participants clicked on them. Participants cited multiple reasons for not clicking, including that they thought the news story provided enough information or they believed that the scientific article would be too difficult to understand. One participant said, “I love to gather information, but I don’t want to read an academic article. This was good [the news story]. If I want to learn more, I can, but I can walk away from this article feeling like I learned something.” (E52) Additionally, participants described clicking links in negative terms (eg, that a link lacked context and they did not know where it might take them, that a link would distract from the news story, or that it could infect their computer with a virus).
Media attitudes
Prior to reading the news story, we explained to participants that the story had been published by a national news publication (again, we did not reveal the publication’s name). However, multiple participants still described general feelings about the news media, particularly how the news media in general played into their credibility judgements. Participants discussed their attitudes about the news media in negative terms, particularly regarding their perceptions of journalists and the motives of news publications (eg, the need to generate attention or ‘drive clicks’). (E55) One participant discussed the credibility of the news story, explaining “Based on scientists, it’s okay, but the journalists—I don’t know—because the journalists can make up any story. I’m not saying that they make fake stories or anything, but I just don’t trust the stories because it’s not 100% accurate to make the company look better just to compete in the market or sometimes they have to add more and more information, which might be right or wrong.” (E70) Another participant said, “You know, I just don’t trust journalists usually. They can make up a story, but these things like the ages doesn’t seem made up. But what if it is? So, I just don’t trust it.” (H33)
Discussion
Participants in this qualitative study described using a constellation of cognitive heuristics to judge the credibility of online health news articles that include links to scientific research articles. Among the heuristics used, we identified the six heuristics as proposed by Metzger and colleagues.16 To our knowledge, this study represents the first time these heuristics have been observed in the context of online health news stories. This suggests that researchers can extend these heuristics to better understand how readers of health news stories make credibility judgements. Our findings also propose an extension of Metzger’s scholarship through the introduction of three new heuristic types: presence of statistics and numbers, links to scientific research and news media attitudes. We now focus on these three new heuristics in relation to the existing literature.
Multiple participants described the presence of numbers and statistics as contributing to their credibility decisions about the presented news articles. This finding supports news media research that the inclusion of numbers and statistics bolster readers’ trust in news articles.31–33 Researchers have proposed that the inclusion of statistics and numbers represents to readers a presentation of factual information that can be verified, which increases credibility.31 34 In our study, we observed that participants focused on the presence of these elements as symbols or markers of credibility, but rarely described how they interpreted their meaning within the context of the news stories. In some cases, participants remarked that while numbers and statistics provided credibility cues, they were unable to interpret or understand their meaning. This finding is consistent with previous research in public health communication noting a ‘rudimentary understanding of quantitative findings’ and difficulty with data interpretation among lay audiences.35 This symbolic use of numbers and statistics speaks to a tension that researchers have identified: how readers understand numbers in the news media versus how they are persuaded by them.31 This is especially concerning in the context of health news, which may be used by parents to make medical decisions about their own health and the health of their child.36 To this end, there is a movement in health communication and journalism education to improve the communication of health data presented as numbers and statistics in ways that are accessible to readers and that encourage readers to interpret their meaning in relation to their own health. For example, the Columbia University Journalism School and other universities offer a host of courses and a master’s degree in data journalism, which focus on the presentation of data, including statistics, in news stories in accurate and compelling ways.37
Online news stories, including those focused on health, frequently incorporate links to internal and external sources of further information. For example, a recent pilot study found that in 2016 over 67 000 cancer news stories linked to more than 11 000 scientific studies.10 Research has shown that journalist’s inclusion of links in news articles to source documents, including scientific studies, increases readers’ perceived transparency of the story and positively influences their perceptions of media credibility.38 In this study, participants confirmed this research by noting the link presence as a cue for credibility. However, while this is encouraging, participants’ behaviour, which included limited clicking of links to the included scientific papers, suggests a potentially missed opportunity for further learning and signals a need for future research to understand the hesitation to click and directly engage with primary research sources. To further quantify this lack of clicking, researchers might consider partnering with news media outlets or publishers to better understand the overall volume of clicks to scientific studies. With this information, researchers and journalists could begin to answer questions such as what link characteristics attract attention or in what types of health stories are readers more likely to click links to scientific research. This information could allow for targeted education to encourage the clicking of links to scientific research and influence the display of news stories and how scientific journals might better present information to the lay public. Historically, these journals have likely considered the public outside their purview or reach. However, the common inclusion links to scientific research in news articles suggests an opportunity for further health communication to the public.
In terms of readers’ reluctance to click on links, more research is warranted. For example, readers may not click because they suspect that they will not understand the information presented or, as found with physicians, their access to information will be deterred by paywalled scientific links, prompting for passwords or payment.39 Researchers have yet to determine if experiences with paywalled scientific literature have a similar effect on the public. However, the increasing nature of public access to the research literature40 suggests that this could be an important line of research.
We observed that participants judged the credibility of their assigned news story, often negatively, based on it being a product of the journalistic system and not necessarily on the merits of the article itself. It is possible that this finding is an outgrowth of the current news media climate, which based on recent nationwide surveys suggests that trust in the media is low.41 42 In the context of health information, this has set off alarm bells for physicians and public health professionals.43 A recent JAMA article on ‘fake news’ warns that this current threat to scientific communication is making it difficult for the public to discern science from science fiction while underscoring the potential negative impacts to patient health (eg, delayed engagement with screening, refusal of treatments).44 To mitigate the impact of the current situation, researchers have suggested a need to support healthcare journalists, enlist healthcare professionals to amplify truthful health information and to actively correct misinformation when it appears in the media.43
Limitations
This study has several limitations. As this study took place in a laboratory at the NCI, we acknowledge that there may be laboratory effects, such as the location of the interview influencing the participant’s credibility judgement of the news story. Although the interviewers were not identified as researchers per say, it is possible that some of the participants believed us to be and therefore modified their behaviours so as not to offend. Although we interviewed 64 participants, our population was restricted to a specific geographical area and focused on parents. It is possible that participants from another region or those without children may have reacted differently. However, based on the previous research of these heuristics, which has been conducted across multiple populations,12 14 15 and the alignment of our findings with this research, we feel that our findings may have broad applicability. We removed the name of the news publication from presented news stories, meaning participants were not able to rely on the newspaper names as a cue. We recognise that this may have taken away an element of the real-world context of the reading experience. It is possible that had we retained the names of the news publication (ie, Los Angeles Times and The New York Times) this additional information may have impacted readers’ judgement of the story. Future researchers might consider retaining information that identifies news publications as an additional element for analysis. In our study design, while we attempted to select news stories similar in length and reading level, the stories were nevertheless written by different authors publishing in two different news sources online news sources. However, based on our reading of the articles and that participants reacted similarly to the news stories, this design decision seems not to have greatly impacted this study. Consideration of differences between news sources is an area ripe for future research. Lastly, we asked parents to read news stories that pertained to their children’s health and not their personal health. It is possible that parents may use different heuristics for matters of their own health; however, we did not observe this in our study and would recommend future researchers more closely examine this possibility.
Conclusion
In this study, we identified that parents use a variety of cognitive heuristics when making credibility judgements about online health news articles containing links to scientific research. The identified heuristics aligned with those used by the public to discern the credibility of online information, broadly suggesting that these heuristics are applicable to health news. The findings have implications for initiatives in education, health communication and journalism directed towards increasing the public’s engagement with health news and their judgement of its credibility.
Acknowledgments
We would like to thank Dr Anita Ousley and Silvia Salazar of the NCI Audience Research Lab for their support throughout the study. Additionally, we acknowledge Dr Bradford Hesse who was instrumental in the early conceptualisation of the study and provided support throughout. We also appreciate the efforts of Emily Harvey, a research assistant who helped code transcripts.
References
Footnotes
Twitter @laurenmaggio
Contributors LAM made substantial contributions to the conception or design of the work, the acquisition, analysis or interpretation of data for the work; drafted the work, revised it critically for important intellectual content; approved the final version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. MK made substantial contributions to the conception or design of the work, the acquisition, analysis or interpretation of data for the work; drafted the work, revised it critically for important intellectual content; approved the final version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. LLM made substantial contributions to the conception or design of the work, the acquisition, analysis or interpretation of data for the work; drafted the work, revised it critically for important intellectual content; approved the final version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding This work was supported by internal funds from the National Cancer Institute.
Disclaimer This article was first-authored by an employee of the US government. The opinions and assertions expressed herein are those of the author(s) and do not necessarily reflect the official policy or position of the Uniformed Services University of the Health Sciences, the Department of Defense or the National Cancer Institute.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available. The National Cancer Institute (NCI) (Protocol #18-NCI-00551) and Uniformed Services University of the Health Sciences Institutional Review Boards (IRB) (Protocol #: HU-MED-83-9908) reviewed this study and determined it to be exempt from further review. Per the regulations of these two bodies, access to the interview data is strictly controlled and limited to the core research team making it impossible for us to publicly deposit this data or make it available upon request.