Article Text
PDF
Abstract
Objectives To investigate the quality of nutrition articles in popular national daily newspapers in the UK and to identify important predictors of article quality.
Setting Newspapers are a primary source of nutrition information for the public.
Design Newspaper articles were collected on 6 days of the week (excluding Sunday) for 6 weeks in summer 2014. Predictors included food type and health outcome, size of article, whether the journalist was named and day of the week.
Outcome measures A validated quality assessment tool was used to assess each article, with a minimum possible score of −12 and a maximum score of 17. Newspapers were checked in duplicate for relevant articles. The association of each predictor on article quality score was analysed adjusting for remaining predictors. A logistic regression model was implemented with quality score as the binary outcome, categorised as poor (score less than zero) or satisfactory (score of zero or more).
Results Over 6 weeks, 141 nutrition articles were included across the five newspapers. The median quality score was 2 (IQR −2–6), and 44 (31%) articles were poor quality. There was no substantial variation in quality of reporting between newspapers once other factors such as anonymous publishing, health outcome, aspect of diet covered and day of the week were taken into account. Particularly low-quality scores were obtained for anonymously published articles with no named journalist, articles that focused on obesity and articles that reported on high fat and processed foods.
Conclusions The general public are regularly exposed to poor quality information in newspapers about what to eat to promote health, particularly articles reporting on obesity. Journalists, researchers, university press officers and scientific journals need to work together more closely to ensure clear, consistent nutrition messages are communicated to the public in an engaging way.
- nutrition communication
- media
- newspaper
- public health
- obesity
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
A large number of nutrition articles from newspapers were analysed for article quality using a validated quality assessment tool.
Key factors were tested for prediction of article quality adjusting for other factors.
Newspaper articles were collected over 6 weeks, but longer time periods may be needed to explain some of the differences in article quality due to variation in quality each week.
Popular sources of news such as online newspaper articles and news on social media were not included in the analysis.
Introduction
Chronic conditions such as obesity, cardiovascular disease (CVD), type 2 diabetes and stroke are leading causes of death, accounting for 86% of total deaths in the UK.1 As a result of lifestyle factors such as poor diet, physical inactivity and excess weight playing key roles in the development of these chronic conditions,2 3 33 000 deaths each year could be avoided if the UK dietary recommendations were met.4 Raising knowledge and awareness of dietary guidelines in an effort to educate and encourage the public to make a conscious decision about their dietary intake could help to significantly improve the health of the population and reduce the incidence of these conditions.5
The media is comprised of the internet, radio, television, smartphones and printed newspapers and media communications, many of which have been shown to have an influential effect on the public’s knowledge and awareness of health issues and which therefore have the potential to promote positive behaviour change.6 7 Only a decade ago, tabloid and broadsheet newspapers were the primary source of health-based information8; however, news from social media sources such as Facebook and Twitter are now popular. Nevertheless, despite a dramatic increase in the use of online media,9 printed newspapers remain an efficient way of providing the public with essential information to enable them to make informed decisions.10 11
Previous research has shown that nutrition coverage has often been sensationalist, with the headlines not accurately reflecting the scientific research12 and based on reporting preliminary research as a ‘breakthrough’.13 The media have been criticised for their classification of ‘newsworthy’ stories,13 and one study reported that 72% of articles were based on low-quality scientific evidence.10 It is common to present contradictory messages or an unbalanced view about health and nutrition in many media articles.14–16 However, newspapers do not exist to provide a free public health service to the public but to provide newsworthy articles.17
A review of the quality of 160 health-based articles (although not necessarily nutrition-related articles) in eight UK newspapers over 4 weeks revealed significant differences in the quality of reporting between newspapers,18 with The Times publishing the highest quality articles and The Sun the lowest. Their findings highlighted aspects of an article related to editorial policy that affected the quality of reporting such as article length, journalist and credibility of source; however, they did not explore how these predictors of quality explained variation in quality by paper type or whether they interacted with each other. Therefore, the main aims of this study were to use the existing validated quality assessment tool by Robinson et al 18 to assess the quality of nutrition coverage in particular in five of the highest circulating printed newspapers and to determine the most important predictors of article quality to explain any differences in article quality between papers. We also made recommendations to improve the quality of future nutrition and health reporting in the media.
Methods
Data collection
Five of the highest six circulating tabloid and broadsheet national newspapers in the UK were examined in the summer of 2014. Four tabloid newspapers (The Sun, The Daily Mirror, The Daily Mail and The Daily Express) and one broadsheet (The Daily Telegraph) were included in this study. We omitted The Daily Standard from the included list, as it is not available outside London. Both tabloid and broadsheet newspapers were included to understand whether there were any differences in predictors of quality of the nutrition coverage in these forms of media. Audiences vary between the two types of newspaper with tabloids generally targeting audiences with a lower socioeconomic background.19
Printed editions of the five newspapers were collected on 6 days of the week (Monday–Saturday) for 6 weeks from 30 June 2014 to 9 August 2014. Sunday was excluded from the data collection as a pilot study revealed repetition of nutrition/health articles from previous days. Each printed newspaper was scanned by a researcher in its entirety. Articles covering an aspect of nutrition (as an exposure) and an aspect of human health (as a health outcome) were identified and extracted for inclusion in this study. Articles were excluded if (A) they covered nutrition but without a related health outcome (eg, the use of cucumber as a beauty therapy) or (B) they covered a health outcome such as heart disease without discussing diet. Articles from opinion columns were also excluded. This process was carried out in duplicate and independently by a second researcher, and the selected articles were reviewed by a third nutritionist. Articles that did not adequately meet inclusion criteria were excluded.
Where sufficient information was provided, original research was located using PubMed and other online databases. Articles with insufficient information to locate original research or not based on published research were not excluded. Each article was coded with a unique ID number. Descriptive data, such as the newspaper title, article size, date and day of publication and journalist’s name, were extracted for each article. Articles were categorised into aspect of diet and health outcome covered in the publication. Dietary components were broadly categorised according to The Eatwell guide20 but with high fat and high sugar foods separated into different food categories as these are usually covered separately in the media.
The size of the article in column inches was measured using a standard method (column inches high × number of columns). Articles were then categorised into either small (≤19.9 inches), medium (20–34 inches) or large (≥35 inches) based on space allocated to articles. The cut-off points for these categories were based on the average column inches for less than half page, half a page and more than half a page. Articles were categorised as being anonymous with no journalist name provided or as named if the author of the article was provided (known as a byline).
Quality assessment measure
Each article was reviewed and graded using a validated quality assessment tool.18 The tool assessed different aspects of reporting quality such as generalisability and significance of findings, editorial content, credibility of source and representativeness of research used. The tool consists of 21 items, and points were awarded or deducted based on whether the article met the criteria. Items 1–8 and 18–21 were considered essential criteria; for these questions, points were deducted if the criteria were not met. Items 9–17 were considered desirable, and points were awarded if the criteria were met and zero if the criteria was not met (see the complete list of questions published by Robinson et al 18). Articles could receive a maximum of 17 points or minimum of −12. Following grading, articles were categorised based on the quality score (as recommended by Robinson et al 18) with poor quality articles scoring less than zero, satisfactory articles scoring 0–10 and high-quality articles scoring more than 10.
Statistical analysis
Descriptive statistics were conducted to obtain frequencies, median values and IQR for quality score. In all the models, due to the lack of normality in the distribution of quality scores, the scores were categorised into two groups: poor (quality score of less than zero) or satisfactory (quality score of zero or above) based on the work by Robinson et al.18 Descriptive data were provided for the different categories of food and health covered by the articles, anonymous reporting, article size and days of the week including median and IQR of quality score for each category. Logistic regression models were generated with article quality score as poor or acceptable as the binary outcome variable. In the first model, differences in quality score by newspaper type were tested without adjusting for any predictor variables. The newspaper that published the most articles was used as the reference category. Pairwise comparisons between papers were reported with Bonferroni corrections (to reduce the risks involved with multiple testing). In the second model, predictors were included in the model, namely, day of publication, article size, whether there was a named author (byline), the health outcome reported and food type covered in the article. In both models, due to the number of weeks sampled being a smaller subset of weeks over the year, the results were clustered within weeks using a sandwich estimator.21 To determine whether newspaper type and each predictor were explaining significant amounts of variation in quality score, we took a nested model approach. A likelihood ratio test was used with each factor in turn, comparing the model without and with each factor, and P values of each test were reported. The reference category for each variable was the most common category that had the largest number of articles, and each of the remaining categories was compared with the reference in the tables. Pairwise comparisons with Bonferroni corrections were reported. Residuals of the models were checked for approximate normality. Analysis was conducted using StataIC V.14 with level of significance set at P value of <0.05. Key aspects of the articles identified by the quality assessment tool that were particularly unlikely to be met were discussed as well as any substantial differences between newspapers.
Results
Descriptive analysis
In total, 141 different articles were published over the 6-week period (see table 1) in the five newspapers. Five articles on heart disease were excluded, which were initially included, as they focused on statins rather than dietary intake. A mean of 24 articles were published each week, and a mean of four articles were published each day. The Daily Mail had the most publications relating to nutrition and health over the period studied (n=40). Their articles accounted for 28.4% of the total publications and therefore was used as the reference category in subsequent analysis. In contrast, The Sun published the fewest articles (n=20), accounting for 14.2% of the total publications. Papers varied in the proportion of small articles and anonymous articles, and none of the papers published high-quality articles as defined by the quality assessment tool (see table 1).
Descriptive information on quality scores, article size and whether named journalist listed by newspaper name
Quality assessment
The quality scores across the newspapers ranged from −9 to 10, with an overall median score of 2. In total, 44 (31.2%) articles were rated poor quality (score of less than zero) and 97 (68.8%) were rated satisfactory quality (score of 0–10). There were no high-quality articles (score of more than 10). The median quality scores varied between paper type; the lowest being 0.5 for The Sun and the highest being 3 for The Daily Telegraph. The percentage of articles that achieved a score of zero or above (and therefore defined as satisfactory quality) varied between papers and was lowest for The Sun at 55% and highest for The Daily Telegraph at 75% (see table 1). Median scores for weeks 1–6 varied and were −4, 3, 3, 0, 3.5 and 5 consecutively. Weeks were adjusted for in the analysis. Logistic regression results using The Daily Mail as the reference category indicated there was an overall significant effect of newspaper type on per cent of articles of satisfactory quality (P<0.01), but none of the individual papers had a significantly different per cent of satisfactory articles compared with The Daily Mail, and none of the pairwise comparisons were statistically significant.
We investigated the importance of five different predictor variables. Quality scores varied by day of the week. Median scores for Monday–Saturday were 1, 0, 0, 4, 4 and 2, respectively, with higher scores on Thursday and Friday and lower scores on Tuesday and Wednesday. More articles were published on Tuesday than any other day, and therefore this was used as the reference category in subsequent analysis.
There were 48 named journalists across the 141 articles. These journalists were responsible for publishing 98 (69.5%) of the articles reviewed. The remaining 43 (30.5%) articles were published anonymously (table 1). The Sun had the highest number of anonymous publications (n=8, 40.0%), and The Daily Express had the least (n=6, 20%). Articles with a named journalist had a median quality score of 3 compared with a median score of −2 for articles that were anonymous.
The majority of articles were categorised as small (n=87, 61.7%) (table 1). Small, medium and large articles had median quality scores of 1, 3.5 and 5, respectively. The Daily Express had the greatest number of large-sized articles (n=10, 33.3%), while the broadsheet, The Daily Telegraph, had the largest number of small articles (n=23, 82%) (see table 1).
The majority of articles discussed diet and nutrition in relation to their effect on health and well-being. Conditions covered most often were obesity (n=35, 24.8%), CVD (n=34, 24.1%) and neurological disorders (n=22, 15.6%). The main dietary components covered energy (n=27, 19.1%) and fruits and vegetables (n=25, 17.7%). Quality scores varied across different health outcomes and different food topics (see table 2). Articles focusing on obesity were of the lowest quality compared with all other health categories (table 2) with a median quality score of −1. Out of the different food topics covered, high fat and processed foods had the lowest quality score with a median of zero.
Number, per cent, median scores of article quality and IQR for each of the eight different categories of food type and eight different categories of health outcome
We investigated which of the different factors were important at predicting article quality when all the predictors were included in a logistic regression model and where each was adjusted for the remaining predictors. The full model explained 34% of the variation in article quality score. The odds ratios (the odds of an article being defined as satisfactory for each category compared with the odds for the reference category) are displayed in table 3.
Predictors of quality score for different factors including paper type, week, day, food category, health category, named journalist and article size
Likelihood ratio tests used to test the contribution of each variable to the model indicated that paper type was not a significant predictor of article quality once other factors were taken into account (see table 3). Article size was also not a significant predictor of article quality when other factors were taken into account. However, day of the week, food category, health category and whether the journalist was named were all significant factors (see table 3) predicting article quality.
For day of the week, compared with the reference category of Tuesday, Monday had significantly higher odds of having a satisfactory score with articles published on Monday having nearly four times the odds of receiving a satisfactory score compared with Tuesday when adjusted for other factors. Articles published on Saturday had particularly low scores with significantly lower odds of having a satisfactory score compared with Tuesday and also Thursday (the latter result from pairwise comparisons) when adjusted for other factors. These results are different from the unadjusted figures where articles on Tuesday received a lower score than Saturday indicating that other known or unknown factors that reduce quality score may be more common on Saturdays. Compared with articles reporting on obesity, articles reporting on cancer, CVD and diabetes had more than 10 times the odds of receiving a satisfactory quality score. No pairwise comparisons were statistically significant. Articles with no byline were far less likely to receive a satisfactory score. Although food categories made a significant contribution overall to article quality score, no pairwise comparisons were statistically significant.
We investigated which of the 21 questions making up the quality score for each newspaper scored particularly badly. Table 4 provides a breakdown of the scores for each of the 21 items for individual newspapers. The analysis revealed that 54% of articles ranked negatively for Q1 and 40% ranked negatively for Q2, which meant that more than half the articles were not based on published research or did not cite the journal of publication and nearly half did not provide an author name. It would be particularly difficult to locate and read the original research article without this information. The newspapers differed in what proportion of their articles met these two criteria. The majority of articles omitted essential information such as number of participants (Q4) and whether the findings differed from previous research (Q5) (61% and 73%, respectively), but these results did not vary substantially by newspaper. Furthermore, the majority (90%) of articles did not state whether the results of research were statistically significant (Q11). The Daily Express had the most negatively scored articles for Q19, meaning the article had the ‘potential to cause undue harm or optimism’. The Sun and The Daily Express were most likely to score negatively for Q21, stating a ‘breakthrough’ or ‘cure’ in articles. The majority of articles (70%) quoted a second opinion from a specialist (eg, health professional, nutritionist or academic). Different newspapers scored differently on different questions, although no newspaper scored poorly on all questions.
Percentage of articles meeting and not meeting the criteria for each of the 21 items in the validated quality assessment tool
Discussion
This is the first study that explores in detail a range of predictors of quality of nutrition-related articles. We found that there were differences between papers in the per cent of articles with an acceptable quality score when no predictor variables were included in the model. However, when predictors such as food and health type reported in the article and whether there was a named journalist were taken into account, there was little variation between different newspapers. Therefore, the main differences in article quality were explained by the article content and author of the article. Quality of articles also varied by day of the week. These differences in article quality could possibly be related to editorial policy and other factors that were not considered here; however, these factors explained a third of the variation in per cent of articles reaching an acceptable quality level. Articles with the lowest quality scores were those covering obesity and high fat and processed foods and written anonymously. The poor quality of articles on obesity was particularly worrying. Poor quality reporting can lead to readers being confused or uninterested in the poor information provided;22 a serious concern given that obesity affects a quarter of the UK adult population23 and many readers may rely on information from newspapers about how to lose weight.24 There are high levels of stigma around the subject of obesity and its possible causes and solutions that may lead to journalists (as well as health professionals) potentially including information in their communications that is based on their belief system as well as the scientific evidence.25
Journalists have the complex role of translating scientific information to the lay public, and it is important that the authors have sufficient understanding to ensure the correct balance between portraying scientific information accurately and making the information clear and readable. However, journalists must make the story ‘eye-catching’ and ‘appealing’ for the public, which can lead to nutrition articles containing sensationalist reporting, alarmist headlines or contradictory information, resulting in confusion or distrust of dietary recommendations.14 26 Journalists are in a position to shape social norms and attitudes through their choice of topics to publish and therefore may influence understanding of, and appetite for, particular stories but, ultimately, the role of journalists is to provide news that is interesting and sells newspapers and not to act as a public health service to the masses. Of the five newspapers reviewed, some papers published more nutrition articles than others, a finding that is consistent with previous research.18 However, it may be more beneficial to the public to have fewer higher quality articles rather than many articles of low quality. Articles may be published in newspapers if the editors believe it will be of interest to readers and therefore a large number of articles can be seen as a positive sign that readers (the public) are interested in nutrition and health. However, the public do not want poor quality reporting. One study reported that more than three quarters (81%) of those surveyed said they only wanted to hear about findings once ‘there is acceptance among nutrition and health professionals’.27 The current situation needs to take these views into account. We did not collect relevant information to determine why quality of articles varied by day, and the reasons for this need to be explored further.
University press officers, researchers and scientific journals also have a key part to play in improving the quality of research reported in the media. A content analysis28 revealed that academic press releases play an influential role in the quality of news articles but highlighted that many of the exaggerations of media articles stemmed from exaggerations in academic press releases. Nevertheless, the best quality newspaper articles are based on scientific research (usually based in a university) that is published in a scientific journal rather than unpublished research promoted by Public Relations agencies. Improving the quality of reporting in the news perhaps lies first with universities and scientific journals providing easier to understand information that can be understood by a non-specialist audience. Scientific journals have embargo policies that could contribute to the differences by day of the week. Some newspapers were more likely to report on studies that were not from scientific journals, and therefore one recommendation is to encourage all newspapers to increase the proportion of articles based on published studies and to cite the study in the newspaper article.
Previous research has highlighted that the mass media can be an effective tool that health professionals can use as a way to increase public knowledge of aspects of public health such as physical activity6 7 or drink-driving,29 and therefore it is beneficial for scientists to work with the media more closely to increase the proportion of high-quality articles. The best quality articles are more likely to have certain attributes. Higher quality articles are more likely to be written by a named journalist (with a byline), often with a declared interest in health; however, a third had no name provided. It has previously been suggested that the unnamed author may know less about health issues and have had little training in this area30; however, this is not necessarily true. Health journalists could be more likely to publish articles without a byline due to differences in editorial policy between newspapers. Articles that have come from press releases may be more likely not to have a byline, and therefore we support more transparency on the source of information and recommend that more nutrition articles are published by a trained health journalist. Although we did not conclude that article size was a key factor when other factors were taken into account, we believe that articles need to be large enough to cover many of the main points, a finding reported in previous research.10 18 It is unclear what the optimum size is for an article, but it needs to be large enough to successfully provide sufficient context for readers to understand the main points of the research, the conditions attached to the research and the quality of the study design.
Training for journalists is available in the UK such as that provided at the Science Media Centre in London, although little is offered on nutrition and the Centre receives corporate funding that may mean it is not neutral. We recommend more rigorous training of journalists in scientific study design and more dialogue between journalists and scientists to improve the choice of studies covered in the news. A recent review of media quality in Australia concluded that although quality of news media was low, it had recently improved with benefits and harms more accurately provided. This was mainly limited to online news articles31 but indicates that progress can be made. This will only be achieved if journalists, scientists and academic press offices work together as has previously been highlighted.32
There are a number of notable limitations to this research. Data were only collected for a limited period from a limited number of papers. It is likely that there are differences between newspapers, although we saw little difference between newspapers here. It is likely that some newspapers that we have not included are different in format and editorial policy and vary in the quality of their nutrition related articles. Therefore, it is possible that we have not captured a true picture of the quality of nutrition articles in all newspapers. It is also likely that fluctuations may occur when a nutrition topic of particular interest is covered in the news that may increase the proportion of larger articles written or the number of articles categorised under a particular health outcome. Importantly, most newspapers have reported declines in circulation figures as more people are turning to alternative sources, for example, online news websites and blogs,33 although the newspapers that we included in our survey (mostly tabloids) do also have an online presence. Additional articles will have been published on the online version, but we did not explore this. More research is required to assess online sources of news in order to capture a complete picture of the quality of nutrition-related articles. A validated tool to assess quality from a range of online news sources is needed in order to achieve this. Some of the methods used to measure article attributes do not have universally agreed standards, for example, methods for measuring article size. These methods are prone to measurement error and could be improved in future.
In conclusion, it was highlighted in the 1990s34 that health research was often misrepresented and preliminary research reported as a breakthrough. These findings are mirrored in our study, indicating that despite steps being taken to improve the situation, many of these issues still persist. It is therefore essential that further measures are made to improve the quality of nutrition coverage and minimise the damage to public health.35–37 First, we propose that journalists have adequate training in issues related to scientific methods and health. Second, newspaper editors should consider publishing a smaller number of higher quality articles based on studies published in scientific journals. Third, researchers, health professionals and university and journal press officers are key and could assist in providing clear information that follows a standard format to media sources as well as support with training. Finally, all parties need to work together to ensure that nutrition coverage and health messages published for the public are both clear and informative as well as interesting and exciting. Establishing common ground between stakeholders is central to improvement.
Acknowledgments
We are grateful to Dr Jayne Hutchinson for statistical advice.
References
Footnotes
Contributors CELE provided the original research idea, supervised the research, wrote the first draft of the discussion and critically reviewed the first and subsequent drafts of the manuscript. ARK checked and analysed the data, wrote the first draft of the manuscript and contributed to all subsequent drafts. NA and NJ contributed to the design of the survey, collected the data, contributed to the analysis of the data and reviewed the final draft of the manuscript.
Funding The research was carried out in the School of Food Science and Nutrition at the University of Leeds with no external funding.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available for analysis.