Objective To determine if observational studies examining the association of wholegrain foods with cardiovascular disease (CVD) with food industry sponsorship and/or authors with conflicts of interest (COI) with the food industry are more likely to have results and/or conclusions that are favourable to industry than those with no industry ties, and to determine whether studies with industry ties differ in their risk of bias compared with studies with no industry ties.
Design Systematic review and meta-analysis of observational studies.
Data sources We searched eight databases from 1997 to 2017 and hand searched the reference lists of included studies.
Eligibility criteria for selecting studies Cohort and case–control studies that quantitatively examined the association of wholegrains or wholegrain foods with CVD outcomes in healthy adults or children.
Results 21 of the 22 studies had a serious or critical risk of bias. Studies with industry ties more often had favourable results compared with those with no industry ties, but the Confidence Interval (CI) was wide, Risk Ratio (RR)=1.44 (95% CI 0.88 to 2.35). The same association was found for study conclusions. We did not find a difference in effect size (magnitude of RRs) between studies with industry ties, RR=0.77 (95% CI 0.58 to 1.01) and studies with no industry ties, RR=0.85 (95% CI 0.73 to 1.00) (p=0.50) I2 0%. These results were comparable for studies that measured the magnitude using Hazard Ratios (HR); industry ties HR=0.82 (95% CI 0.76 to 0.88) versus no industry ties HR=0.86 (95% CI 0.81 to 0.91) (p=0.34) I2 0%.
Conclusions We did not establish that the presence of food industry sponsorship or authors with a COI with the food industry was associated with results or conclusions that favour industry sponsors. The association of food industry sponsorship or authors with a COI with the food industry and favourable results or conclusions is uncertain. However, our analysis was hindered by the low level of COI disclosure in the included studies. Our findings support international reforms to improve the disclosure and management of COI in nutrition research. Without such disclosures, it will not be possible to determine if the results of nutrition research are free of food industry influences and potential biases.
PROSPERO registration number CRD42017055841.
- industry sponsorship
- conflict of interest
- food industry
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
This is the first systematic review and meta-analysis to evaluate the association of industry sponsorship and author conflicts of interest (COI) with the results, conclusions and risk of bias of primary nutrition studies examining the effect of wholegrain foods on cardiovascular disease outcomes.
We conducted a comprehensive search and followed explicit and well-defined inclusion and exclusion criteria for the included studies.
Although our sample was small, we searched several databases and reference lists of included studies.
We did not attempt to contact the authors of studies lacking a COI disclosure statement; thus, we may be underestimating the number of articles that had authors with COI.
Our assessment of risk of bias in the included studies was based on a tool that is under development, but changes to the tool are unlikely to affect the risk of bias ratings.
Dietary guidelines are designed to promote well-being and reduce the risk of non-communicable diseases. Recent evaluations of the development of dietary guidelines have identified concerns with the methods of the systematic reviews and how evidence from these reviews is synthesised into final recommendations.1–3 Several countries including the UK, USA and Australia have dietary guidelines offering recommendations around the consumption of wholegrain foods.4–6 The guidelines conclude that there is a probable association between wholegrain consumption and a reduced risk of cardiovascular disease (CVD).4–6 These recommendations are supported by recent systematic reviews and meta-analyses of prospective cohort studies, which have found a consistent, inverse relationship between wholegrain intake and CVD risk and mortality.7–9 However, the beneficial effects of wholegrains on CVD when assessed in randomised controlled trials (RCTs) are uncertain.10
Wholegrain products can be defined in various ways, including by the species (eg, wheat, oats), components (eg, endosperm, bran, germ) and percentages (eg, 25%–100%). While some food regulators use a definition of 100% retention of wholegrain content, the epidemiological literature typically uses 25% or more retained content. In the development of the Australian Dietary Guidelines, the most common definition for wholegrain foods was those containing 25% or more of wholegrains.11
Dietary guidelines use a variety of methods to assess bias in primary research studies, but these do not assess one potential source of bias—financial conflicts of interest (COI).12 Across a variety of research areas, industry sponsorship and author COI have been found to be associated with outcomes that favour the study sponsor.13–15 Even when controlling for methodological biases, industry-sponsored studies are more likely to have results that favour the sponsor’s product than those studies with no or other sources of sponsorship.13 Industry sponsors may bias research via the questions they ask (research agenda), how they design and conduct a study, the selection of results they report and through ‘spin’ on conclusions.16–19
A systematic review of methodological studies that compared food industry-sponsored studies with those that had no or other sources of sponsorship found that food industry-sponsored studies were more likely to have favourable conclusions than non-industry-sponsored studies.20 However, there were insufficient data to quantitatively assess the association of sponsorship with study results. Only one methodological study examined the association of author COI and conclusions, and found a statistically significant association between them.21
Funding sources and author COI may be a risk of bias in studies of wholegrain consumption as these studies could test formulated or processed wholegrain products, such as breakfast cereals. Industry sponsors may gain financially from finding that these types of products have health benefits that can be used to market their products. There has been no assessment of the association of food industry sponsorship and author COI with the food industry and the statistical significance of results, effect sizes, conclusions and risk of bias of observational studies examining the cardiovascular health benefits of wholegrain consumption. The primary objective of this review is to determine whether:
Primary studies examining the association of wholegrain foods with CVD with food industry sponsorship and/or authors with COI with the food industry are more likely to have results and/or conclusions that are favourable to industry than those with no industry ties.
This review also examines whether any differences between industry and non-industry-sponsored studies could be related to their methods or interpretation of results.
The secondary objectives of this review are to determine whether:
Studies with food industry sponsorship and/or authors with COI with the food industry differ in their risk of bias compared with studies with no industry ties.
Studies with food industry sponsorship and/or authors with COI with the food industry have a higher level of discordance between study results and conclusions, with the conclusions more likely to be favourable compared with the results.
We conducted a systematic review of observational studies examining the association of wholegrain consumption with CVD.
Literature search strategy
The search was based on the Process Manual used in the development of the 2013 Australian Dietary Guidelines22 and the advice of an information specialist. We searched the following databases from January 1997 to October 2017: MEDLINE; CINAHL; PubMed; PreMEDLINE; Cochrane Library; PsycINFO; Science Direct; and ERIC. The search strategy we used for Ovid MEDLINE is shown in online supplementary file 1. We adapted this strategy for the other databases. We also hand searched the references lists of identified studies and reviews. The search also included terms for RCTs to identify relevant trials for a future systematic review.
The RCTs identified in our search were included in another review currently under development. We selected observational studies for this review. This review included primary nutrition studies of cohort or case–control designs that quantitatively examined the benefits or harms of wholegrain consumption related to CVD outcomes in healthy children and/or adults.
We included studies that defined wholegrains in any way, as defined by the author of the included study. If total wholegrain consumption had been assessed in the study, we included this as our only exposure. If total wholegrain consumption as an exposure was not available, we included any type of wholegrain consumption (ie, wholegrain cereal, breakfast cereal, bread, rice, etc) as our exposure. We included studies that compared wholegrain food to other foods or compared various levels of wholegrain consumption. We included the result representing the effect of the highest level of wholegrain consumption compared with the lowest level of wholegrain consumption (eg, ‘yes’ to wholegrain consumption vs ‘no’ to wholegrain consumption, tertile 3 vs tertile 1, quartile 4 vs quartile 1, quintile 5 vs quintile 1). If our prespecified rules for selection did not uniquely identify one exposure for inclusion in the meta-analysis, we randomly selected one result.
We included studies that had a clinical outcome measure related to CVD, defined as mortality related to specific cardiovascular events, and/or cardiovascular events (eg, first myocardial infarction, total stroke, etc). If ‘cardiovascular disease mortality/death/s’ (verbatim) had been assessed, we included this as our only outcome. If not, we included any type of CVD mortality (eg, coronary heart disease mortality, stroke mortality, etc) as our outcome. If there were no mortality outcomes assessed in the study, we included any CVD event as our outcome. If a study assessed subgroups of CVD deaths and events (eg, intracerebral haemorrhages, ischaemic stroke) and also assessed them collectively (eg, cerebrovascular diseases), we took the result that had assessed them collectively. If our prespecified rules for selection did not uniquely identify one outcome for inclusion in the meta-analysis, we randomly selected one result.
We excluded conferences presentations, opinion pieces and letters to the editor. We had no language restrictions.
Types of outcome measures
We hypothesised that studies with food industry sponsorship and/or authors with a COI with the food industry would be more likely to have favourable findings than those with no industry ties. We assessed three primary outcomes.
Statistical significance of results favourable to the sponsor
Favourable results were defined as results that were favourable to the sponsor’s product(s), either indicating greater health benefits or less harm than the comparator. Specifically, for studies of health benefits of wholegrains, favourable results were defined as those that were statistically significant at the 0.05 level (two tailed). For studies of harms of wholegrains, favourable results were defined as those where harms were not statistically significant at the 0.05 level or there were a statistically significant higher number of harms in the comparator group. Otherwise, results were classified as unfavourable.
Effect size of results
Effect size was defined as the risk ratio (RR), hazard ratio (HR) or odds ratio (OR) of the association between wholegrains and a clinical outcome of CVD. We compared the magnitude of the pooled effect estimates in studies with food industry sponsorship and/or authors with a COI compared with studies with no industry ties.
Conclusions that suggested that the wholegrain intervention being studied was beneficial to health and/or safe were considered favourable to the study sponsor. Otherwise, the conclusions were considered unfavourable.
We assessed two secondary outcomes.
The risk of bias of the included studies
We hypothesised that studies with industry sponsorship and/or authors with a COI with the food industry would have the same overall risk of bias as those with no industry ties.
Concordance between study results and conclusions
We hypothesised that studies with industry sponsorship and/or authors with a COI would be more likely to have discordant results and conclusions, with results not favouring the sponsor and conclusions favouring the sponsor, than those with no industry ties.
Selection of studies
Three investigators (NC, SM and JT, working in pairs) independently screened the titles and abstracts of all retrieved records for obvious exclusions. Full text of potentially eligible studies was then retrieved, and three investigators (NC, SM and JT) assessed these against our inclusion criteria. Agreement was reached by consensus.
Data collection and analysis
Three assessors (NC, SM and JT) independently extracted the following data from each included study. Discrepancies in data extraction were resolved by consensus. If agreement could not be reached, a fourth assessor (LB) adjudicated the outcome.
From each study, we extracted:
Year of publication.
Study design (cohort or case–control).
Sample size of study.
Age of participants.
Exposure duration or observation period.
How the study defined wholegrain (verbatim).
Level of wholegrain content in wholegrain foods.
Disclosure of funding source (no disclosure, yes and there is a sponsor, the authors state they received no funding for their work).
Name of the funders of the study (verbatim).
Role of the funders (role of the sponsor not mentioned, sponsor not involved in study design and analyses, sponsor involved, N/A).
Disclosure of author COI (no disclosure, yes, the authors state they had no COI to declare).
Authors COI statement (verbatim).
Outcomes assessed in the study (any CVD death and/or event).
The numerical results of the study (eg, OR, HR).
We stored all extracted data from the included studies in REDcap, a secure web-based application for the collection and management of data.23
Classification of industry sponsorship and author COI
Sponsorship was categorised as (1) industry or (2) non-industry. We defined industry-sponsored studies as those declaring any sponsorship from the food industry, including if the study received ‘mixed funding’ from the food industry, non-profit organisations or other industries (ie, pharmaceutical). Any study with an author with any disclosed financial tie to the food industry was classified as having a COI. Author COI were categorised as (1) presence of a COI with the food industry or (2) no COI. Any studies that did not contain an author COI disclosure statement were classified as no COI. We contacted the authors of one paper24 for clarification on their disclosure of funding source.
Assessment of risk of bias in included studies
We used an adapted version of the Cochrane Collaboration’s ‘Risk of Bias in Non-Randomised Studies-of Interventions’25 tool to measure the risk of bias of included observational studies. The tool assesses bias across seven domains. Each domain is assessed at a low, moderate, serious or critical risk of bias or no information. The domain rating with the highest risk of bias determines the overall risk of bias rating for the study. For example, if a study is rated as being at a serious risk of bias in one domain, the overall risk of bias rating is ‘serious.’
We report frequencies and percentages of study characteristics across all studies and, separately, by funding source. We visually depict the overall risk of bias rating and the ratings for each domain by study.
We calculated RRs or HRs (and 95% CIs) to quantify the association between food industry sponsorship and/or authors with COI with the food industry and favourable results, favourable conclusions and the overall study risk of bias rating. For the risk of bias rating analysis, we dichotomised the overall risk of bias ratings as low (low or moderate) or high (serious or critical). We had planned to calculate an RR for level of concordance; however since in all studies there was concordance between the results and conclusions, we did not undertake this analysis.
We used meta-analysis to examine whether food industry sponsorship and/or authors with COI with the food industry modified the magnitude of association between wholegrains and CVD outcomes. Specifically, we undertook a subgroup analysis within a random effects meta-analysis model that compared the pooled associations across subgroups defined by industry sponsorship. The associations were pooled using inverse variance weighting and DerSimonian and Laird’s method of moments estimator was used to estimate between study heterogeneity. Separate meta-analyses were fitted for studies that had measured the association using HRs and those that had used either RRs or ORs. Given cardiovascular events were rare, the OR approximated risk ratios. We quantified heterogeneity for subgroup differences using the I2 statistic26 and tested for heterogeneity using the χ2 test. Review Manager V.5.3 was used to analyse the data.27
No patients were involved in the completion of this review.
Characteristics of included studies
All studies were published between 1998 and 2015. Three of the studies were case–control and 19 were cohort design. All studies contained a sponsorship disclosure. Five studies disclosed food industry sponsorship, but only one of these had a statement describing the role of the sponsor. Five studies contained an author with a COI with the food industry. Ten studies did not contain an author COI disclosure statement. Nine studies contained either food industry sponsorship or had an author with a COI.
A greater proportion of industry-sponsored studies (67%) than non-industry sponsored studies (31%) used a definition of wholegrain as greater than 25%, and most of these examined breakfast cereals (table 1). Industry sponsored studies were also more likely than non-industry studies to focus on a specific food (44%) than total wholegrain intake (23%) (table 1). Industry-sponsored studies were less likely (56%) to have a serious or critical risk of bias in classification of exposures than non-industry-sponsored studies (85%). Other characteristics were similarly distributed across industry-sponsored versus non-industry-sponsored studies. Details of each individual study are in online supplementary file 4.
Risk of bias in included studies
One study29 was assessed as having an overall moderate risk of bias, four as having a serious risk of bias and seventeen as having a critical risk of bias (figure 2). The majority of studies had a critical risk of bias in the confounding domain. For example, a confounder was fruit and vegetable intake. If this was not appropriately controlled for when assessing the effect of wholegrain intake on a CVD outcome, the study was rated as having a risk of bias for confounding. All but one study was assessed at a low risk of bias on the outcome measurement domain. For all domains, except classification of exposure, the risk of bias ratings were similarly distributed across industry-sponsored versus non-industry-sponsored studies (table 1).
Favourable results: statistical significance: industry ties versus no industry ties
The risk of reporting favourable outcomes was 44% higher in studies with industry sponsorship and/or authors with a COI with the food industry RR=1.44 (95% CI 0.88 to 2.35). However, the CI was wide and included differences in risks that were unimportant or operating in the opposite direction as plausible estimates. When we compared only industry-sponsored (n=5) and non-industry-sponsored studies (n=17), the risk was smaller RR=1.13 (95% CI 0.66 to 1.94).
Favourable results: effect size: industry ties versus no industry ties
There was no difference in the magnitude of RRs (measuring the association between wholegrains and CVD outcomes) between studies with industry sponsorship and/or authors with a COI with the food industry RR=0.77 (95% CI 0.58 to 1.01) and those studies with no industry sponsorship or author COI RR=0.85 (95% CI 0.73 to 1.00) (subgroup test p=0.50, I2=0%) (figure 3). For studies that had measured the association using HRs there was also no difference found in the magnitude of HRs between studies with industry sponsorship and/or authors with a COI with the food industry HR=0.82 (95% CI 0.76 to 0.88) and studies with no industry sponsorship or author COI HR=0.86 (95% CI 0.81 to 0.91) (subgroup test p=0.34, I2=0%) (figure 4).
Our analysis comparing studies with industry sponsorship RR 0.63 (95% CI 0.28 to 1.39) and those with no industry sponsorship RR 0.85 (95% CI 0.74 to 0.97) (subgroup test p=0.46, I2=0%) showed no important difference in the magnitude of RRs. This was again comparable between industry-sponsored HR 0.82 (95% CI 0.77 to 0.87) and non-industry-sponsored studies HR 0.85 (95% CI 0.81 to 0.90) (subgroup test p=0.29), I2=12.2%) that measured the association using HRs.
Favourable conclusions: industry ties versus no industry ties
As there was concordance between the results and conclusions of every included study, the same associations were found for conclusions as for the statistical significance of results. Studies with industry sponsorship and/or authors with a COI with the food industry were more likely to have favourable conclusions compared with those with no industry sponsorship or author COI RR=1.44 (95% CI 0.88 to 2.35); however, the CI was wide. When studies were compared only by industry sponsorship, the risk was again smaller RR=1.13 (95% CI 0.66 to 1.94).
Risk of bias assessment by industry ties
Studies with industry sponsorship and/or authors with a COI with the food industry were less likely (0/9) to have an overall low risk of bias rating compared with those studies with no industry sponsorship or author COI (1/13), RR=0.47 (95% CI 0.02 to 10.32); however, there was large uncertainty in the association.
Observational studies examining the effect of wholegrain consumption on CVD outcomes that were sponsored by the food industry and/or had authors with a COI with the food industry more often had favourable results than research not tied to the food industry. However, this finding was inconclusive with respect to the association between industry ties and favourable results, as the relative risk could be as high as 2.35 or as low as 0.88. We found no evidence of a difference in the magnitude of effect between industry-sponsored and non-industry-sponsored studies. It is difficult to detect differences in effect size by sponsorship as many study design features, such as dose and duration of exposures, and specific CVD outcomes, vary across studies and may influence the effect size. In previous assessments of drug studies that have demonstrated that industry-funded studies are more likely to have results that favour the study sponsors, there was no statistically significant difference found in effect sizes between industry-sponsored and non-industry-sponsored studies.13
Although all the included studies had a sponsorship disclosure, almost half were missing disclosures about author COI. Non-disclosed COIs in nutrition research are a concern.30 Larger samples of industry-funded studies and studies with disclosed author COI could make it possible to establish the association of sponsorship with research outcomes.
Studies that were sponsored by the food industry and/or had authors with a COI with the food industry more often had favourable conclusions than studies with no industry ties, although there was uncertainty in this relationship. There was absence of spin in the included studies as all the results agreed with the conclusions.
The overall risk of bias in every study, other than one non-industry sponsored study,29 was classified as high (meaning either serious or critical). The overall risk of bias rating was based on the domain with the highest risk of bias rating within each study, and most of the studies had a risk of bias related to confounding. Across each domain, we found little difference in the risk of bias between industry-sponsored and non-industry-sponsored studies.
Strengths and limitations of this review
Our review was registered in PROSPERO.28 We conducted a comprehensive search and followed explicit and well-defined inclusion and exclusion criteria for the included studies. Although our sample was small, we searched several databases and reference lists of included studies. Authors of the studies for which we required clarification on funding source were also contacted, but we did not attempt to contact the authors of studies lacking a COI disclosure statement. Thus, we may be underestimating the number of articles that had authors with COI. Our assessment of risk of bias in the included studies was based on a tool that is under development, but changes to the tool are unlikely to affect the risk of bias ratings.25
Agreements and disagreements with other studies or reviews
The relationship that we identified between food industry sponsorship and authors with a COI and favourable study outcomes towards the study sponsor has been previously demonstrated in an assessment of a broad range of nutrition research.20 Only one study has reported an association of food industry funding with effect sizes.31 Of studies examining the association between soft drink consumption and adverse health outcomes, food industry-sponsored studies reported significantly smaller effects than non-food industry-sponsored studies. Compared with our study, this study examined studies with a homogeneous population of industry funders, sugar-sweetened beverage companies, which may have a more consistent influence on study outcomes than the diverse pool of food industry sponsors in our study.
There was also no difference in the level of risk of bias between industry-sponsored and non-industry-sponsored studies. This is consistent with previous assessments of pharmaceutical, tobacco and nutrition research that has shown industry-sponsored studies are of equal or better quality than non–industry-sponsored studies.13 20 32–34
Implications for clinicians, policy makers and future research
The recent critiques to reform the methods used in the development of dietary guidelines have proposed steps to improve the transparency of how evidence is evaluated and synthesised into recommendations.1 2 However, until the influence of industry sponsorship in primary nutrition studies has been further explored and measured with larger samples of industry-sponsored studies, or studies that have author disclosure statements, this bias may still be unaccounted for in dietary guidelines. Although there was uncertainty around the differences in the results and conclusions that we observed between industry and non-industry studies, the differences are unlikely to be explained by methodological risks of bias in these studies.
There are ways that study sponsorship can influence outcomes other than through the design of research. Bias may also be introduced in the way industry-sponsored studies code events and analyse data,35 36 through the selective reporting of study outcomes and through publication bias.37 It has been demonstrated in other areas of medical research that there is a greater propensity to publish studies with statistically significant results.38 Therefore, selective publication of study results or studies in their entirety may limit the availability of all relevant nutrition data and can skew results that are used in dietary guideline development.39 Publication bias could be minimised with the introduction of study registries for nutrition research, as has been established in pharmaceutical research.40 The association of food industry sponsorship with the reporting of nutrition research still needs to be assessed.
Almost half of the studies included in this review had authors that did not disclose if they had a COI with the food industry or not. Compliance with COI disclosure policies is now well documented across many domains of research.41–46 Recent examinations of the levels of disclosure in research assessing the effects of artificially sweetened beverages on weight outcomes found similarly poor disclosure rates.32 Several solutions have been proposed to increase transparency and disclosure rates, including the use of different databases and additional resources to identify conflicted authors, and the introduction of mandatory disclosure requirements in all journals, with the use of penalties for those who do not adhere to the stated policies.20 32
We did not establish that the presence of food industry sponsorship or authors with a COI with the food industry was associated with results or conclusions that favour industry sponsors. The association of food industry sponsorship or authors with a COI with the food industry and favourable results or conclusions is uncertain. However, our analysis was hindered by the low level of COI disclosure in the included studies. This research further strengthens calls for stricter policies relating to the disclosure and management of COI in nutrition research. Without such disclosures, it will not be possible to determine if the results of nutrition research are free of food industry influences and potential biases.
We thank Matthew Page, Monash University for answering specific statistical questions.
Contributors NC, AF, SM, MA-F and LB designed and wrote the review protocol. NC wrote the search strategy and undertook the literature search. NC, SM and JT conducted the title and abstract screening and full article screening for final study inclusion. NC, SM and JT conducted data collection and cleaning, LB supervised. NC and JM undertook all data analysis. LB advised on methods, statistical analyses and interpretation of findings. All authors contributed to the final manuscript. NC and LB are guarantors.
Funding This work was funded by National Health and Medical Research Council Grant #APP1139997. Nicholas Chartres is a recipient of the James Millner PhD Scholarship in Pharmacy from the University of Sydney. Sally McDonald is a scholarship recipient (Charles Perkins Centre summer scholarship) from the University of Sydney. Jessica Turton is a scholarship recipient (Charles Perkins Centre summer scholarship) from the University of Sydney.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Available from The University of Sydney data repository. DOI to be determined.
Patient consent for publication Not required.