Article Text
PDF
Abstract
Objective To examine the certainty of the evidence supporting health claims about probiotics, yoghurt, kefir, kombucha, fibre and prebiotics, and to assess the quality of online information in Spanish.
Design Content analysis.
Methods We compiled a data set of 114 web pages by searching six popular search phrases in Spanish relating to probiotics, yoghurt, kefir, kombucha, fibre and prebiotics on Google.es and coded them for typology and health claims. We examined the certainty of the evidence for health claims from systematic reviews. Information quality was assessed according to 10 criteria, where a web page: mentions scientific publications and reports their conclusions; quantifies relative and absolute effects; acknowledges some limitations; discusses certainty of evidence; reports the potential harms, alternatives and costs; and does not argue based on personal experiences.
Results Gastrointestinal health (86.0%), general health (57.9%), cardiovascular health (53.5%) and immune system health (50.9%) were the most widely mentioned topics. Half of claims (52.6%, 70/133) were supported by evidence from systematic reviews. Probiotics had the highest number of claims supported by evidence and kombucha the lowest. The highest certainty was found for antibiotic-associated diarrhoea, necrotising enterocolitis and otitis (moderate) in probiotics and yoghurt, infectious diarrhoea and hepatic encephalopathy (moderate) in prebiotics, and cardiovascular health (high to moderate) and colorectal cancer (moderate) in fibre. On a scale of 0–10, the median information quality score for all web pages was 3. Only 18.4% reported study conclusions, 7.9% quantified the effects, 28.9% acknowledged some limitations in the research and 42.1% reported potential harms.
Conclusions Most online health claims for dietary interventions intended for improving health through the gut microbiome are supported by low or very low certainty of evidence. Online information does not align with the evidence and is incomplete or unbalanced.
- nutrition & dietetics
- public health
- gastroenterology
Data availability statement
All data relevant to the study are included in a public, open access repository. The data set is available on the following: https://figshare.com/articles/journal_contribution/Supplemental_material_1/20203415, https://figshare.com/articles/journal_contribution/Table_S2/19425824, https://figshare.com/articles/journal_contribution/Supplemental_material_3/20204021, https://figshare.com/articles/journal_contribution/Supplemental_material_4/20204270.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study examines the extent to which online health claims for popular dietary interventions related to the gut microbiome are aligned with certainty of evidence evaluated using the Grading of Recommendations Assessment, Development and Evaluation approach.
We propose 10 criteria (scored from 0 to 10) for assessing information quality, selected on the basis of the first systematic review of the quality of news reports on the effects of health interventions.
The content analysis only focuses on some popular searches and the top 20 search results on Google.
The study is limited in scope, since it only focuses on Spanish-language web pages and does not analyse information available on social media channels.
Introduction
Research into the microbial ecosystem residing in the gastrointestinal tract, which is collectively known as the gut microbiome, is commanding increasing attention among medical audiences and the general public.1–4 While the microbiome is now often thought of as a virtual organ of the body due to its influence in many areas of human health, from immunity to energy metabolism and mental health,5–7 its causal involvement in diseases is mostly unresolved.8 Recent large-scale studies have shown that diet is among the most important environmental factors to which the gut microbiome is exposed and by which it is modified on a daily basis, even outweighing host genetics.9–13
Probiotics, fermented foods, fibre and prebiotics are dietary interventions that influence human health in terms of their effect on the gut microbiome. The health benefits of probiotics include their effect on digestive ailments (ie, treating acute diarrhoea and antibiotic-associated diarrhoea, managing symptoms of lactose intolerance, treating pouchitis, preventing Clostridioides difficile infection and preventing necrotising enterocolitis in preterm infants). They also provide benefits in relation to non-alcoholic fatty liver diseases and some immune-related conditions (ie, preventing or treating infectious diseases and preventing atopic dermatitis).14 15 Fermented foods have also undergone a surge in popularity, although not all have a proven impact on clinical health outcomes. The most widely investigated fermented foods are yoghurt, with evidence for managing symptoms of lactose intolerance and reducing the risk of metabolic syndrome, and kefir, with beneficial effects in both lactose malabsorption and Helicobacter pylori eradication.16 Fibre can aid with gut disorders (ie, irritable bowel syndrome, inflammatory bowel diseases, diverticular disease and functional constipation), reducing the risk of cardiovascular diseases and lowering all-cause mortality rate.17–19 Prebiotics have been studied for reducing constipation and diarrhoea, promoting metabolic health, modulating satiety, helping with symptoms of irritable bowel syndrome, treating hepatic encephalopathy and reducing risk of allergy.14 20
Contemporary audiences are increasingly turning to the internet as a source of information about health and nutrition.21 22 Google is the most widely used search engine23 24 and 1 in 20 Google searches seek health-related information.25 The health and nutrition-related information disseminated by online resources may influence health perception and food practices,26–29 and the online space in particular has fuelled the promotion of microbiome-related interventions for maintaining health and quality of life.30 However, information on the microbiome in online resources or websites (eg, newspapers and Google searches) is often misleading, does not always report limitations and tends to simplify or exaggerate the benefits of microbiome-based interventions.1 4 31–34 That has led to the microbiome being oversold as the main cause of all health and illness, in a phenomenon dubbed ‘microbiomania’.35 Despite the huge amount of health-related information that can be accessed online, there is no universal tool available for evaluating the quality of information on the effects of health interventions. Furthermore, the authors have not found any studies that explore the quality of online information on microbiome-related interventions.
The gut microbiome-related food and dietary supplement industry is largely unregulated in the USA and Europe and marketing of such products is often geared directly at consumers without consistent evidence of efficacy and safety.36 37 On the one hand, regulatory authorities do not allow health claims to be made for probiotics and prebiotics, but on the other hand, there is little regulation of the manufacturing process and marketing actions,38 which can contribute to the spread of misleading information on these products.
As for health and nutrition in general, the internet is a major source of information among the general population about probiotic and fermented food use for the benefit of gut health.16 39 40 During the COVID-19 pandemic, news and commercial websites frequently mentioned the microbiome and gut health in relation to immune boosting strategies, which, nevertheless, were lacking in evidence.41 Two previous content analyses of web pages on probiotics in English showed poor quality and objective information, with commercial websites providing the lowest score.42 43 Whether those findings can extrapolate to online information for other dietary strategies such as fermented foods, fibre and prebiotics, widely promoted as influencing human health through their effect on the gut microbiome, is unknown.
This study addresses both the scientific basis and the quality of the online information on gut microbiome-related interventions to which the public is exposed. Our first objective was to examine the certainty of the evidence from systematic reviews (SRs) that supports health claims regarding probiotics, yoghurt, kefir, kombucha, fibre and prebiotics in the top 20 indexed web pages in Spanish. We focused on such interventions for two reasons. First, most of the elements under focus (ie, probiotics, yoghurt, kefir, fibre and prebiotics) have been studied in at least one human interventional study.6 14–16 Second, it was observed through an analysis on Google Trends44 that those topics had been increasingly subject to consumer interest from 2010 onwards, while becoming relatively stable between 2019 and 2021. Our second objective was to develop an overall score based on 10 criteria for evaluating the quality of information, according to intervention and web page typology.
Methods
Google searches and selection criteria
In line with Neunez et al,43 we conducted searches on https://google.es using the Google Chrome browser and employing phrases based on search term popularity as provided by AnswerThePublic.45 The chosen phrases were: ‘por que tomar probioticos’, ‘qué yogur tiene más probióticos’, ‘por que tomar kefir’, ‘por que tomar kombucha’, ‘fibra beneficios’ and ‘que son prebioticos y para que sirven’. The searches took place in August 2021 in Tarragona, Spain. We decided to choose phrases containing words without accents because, according to Google Trends,44 that is the most common way in which users search. Consequently, the results returned are what most users would find (for the relative popularity of the search terms used, see reference 46). Before searching, we logged out from any Google accounts and cleared caches and browsing histories to limit any personalisation of the search results.
Since consumers’ online information searches are typically limited to initial search results,47 48 we limited our sample to the first 20 uniform resource locators (URLs) returned when searching for the aforementioned six search phrases. As there were six interventions in total, the initial data set consisted of 120 web pages. Based on previous studies on information about health interventions,41 43 49–51 all web pages written in Spanish, which were freely accessible (ie, they did not have paywalls and/or login requirements) and which provided information on each intervention of interest, were considered eligible. The following web pages were excluded: any irrelevant web pages (ie, the main focus was not the searched-for intervention), web pages only featuring video content, retail sites intended for direct purchase and advertisements. After excluding six web pages (three irrelevant web pages, one web page offering only video information and two online shops), a total of 114 web pages were classed as being eligible for analysis.
Web page typology
We coded the content of the web page linked to the URL, but not the content provided in the hyperlinks to other web pages. One author (AP-B) downloaded the web page texts as individual PDF files, deleting any reference to source or authors, and coded the web pages according to Neunez et al’s typology: commercial (C), news (N), health portal (HP), professional (P), governmental (G), non-profit organisation (NP), scientific journal (SJ) and other (O).43 For examples of the classification, see reference 52.
Health claims and the certainty of the evidence that supports them
Two authors (MB and GC) coded the health claims relating to each intervention (gastrointestinal health, immune system health, cardiovascular health, cancer, mental disorders, urogenital disorders and other). ‘Other’ was categorised when the web page stated the intervention was valuable for general health (ie, using general phrasing such as ‘helps maintain health or quality of life’, ‘manages stress’, ‘improves sleep’, etc), skin health (including cosmetic and skin disorders such as eczema and psoriasis) and respiratory disorders. AP-B coded specific indications within each health claim topic mentioned in the web pages. We also noted when an article on a web page made a clear recommendation to consume or avoid the food or supplement and included the advice to consult a healthcare professional.
To identify which health claims were supported by evidence from SRs, we conducted a search of SRs for each intervention in PubMed and the Cochrane Database of Systematic Reviews in December 2021. We did not restrict the search to specific health claims and it was performed after the online health claims were identified. The two authors who identified the SRs (AP-B and MR) were not involved in coding the health claims made on the web pages. SRs were chosen since they gather and analyse all studies that answer the research question and meet inclusion criteria.53
We selected SRs that used systematic methods when searching for and identifying the evidence in two databases and which evaluated certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.54 Briefly, GRADE is a reproducible and transparent methodology widely adopted by organisations such as the World Health Organization and the Cochrane Collaboration for making clinical practice recommendations. It classifies certainty or quality of evidence—that is, the degree of confidence in the results of research on a given outcome of interest (eg, irritable bowel syndrome, cancer or obesity)—as high, moderate, low or very low, according to factors that include the study methodology, consistency and precision of the results, and directness of the evidence supporting health claims on web pages.54 Very low means the true effect is probably substantially different from the estimated effect; low means the true effect might be markedly different from the estimated effect; moderate means the true effect is probably close to the estimated effect; and high means the true effect is similar to the estimated effect.54–56 Regarding an intervention effect, favourable effect means the intervention is associated with a beneficial effect on the outcome of interest; no effect means the intervention is associated with little or no difference to the outcome of interest; and uncertain means the certainty for an outcome was not reported, the results were contradictory or effects could not be estimated.57 The claims about the effect (favourable effect/no effect/uncertain effect) and certainty of evidence (high/moderate/low/very low) were coded by two authors (AP-B and MR). When more than one SR was obtained, we prioritised the most recent, and for two SRs published the same year, we prioritised the Cochrane SR.56 If certainty of evidence differed across outcomes stated in web pages for the same intervention, overall certainty of evidence was understood as the lowest GRADE classification registered.56 For search phrases used in the search for SRs in the Cochrane Library and PubMed, see reference 58.
Quality of information
MB and GC separately analysed the quality of online information based on whether it met the following 10 criteria. The selection of said criteria was based on the only available SR of the quality of information on health interventions59 and two other relevant papers.60 61 The criteria used were: (1) provides references or links to scientific publications; (2) explains the conclusions of scientific publications; (3) quantifies relative effects; (4) quantifies absolute effects; (5) acknowledges some research limitations (eg, preliminary results, small studies, conflicts of interests and differing results between studies); (6) generally discusses certainty of evidence (eg, aligning wording depending on whether the studies are observational or experimental)62; (7) reports potential harms; (8) reports on available alternatives; (9) discusses intervention costs; and (10) does not make arguments based on personal experiences or anecdotes. For each criterion, the story was given a rating of ‘satisfactory’ or ‘unsatisfactory’.
All discrepancies in coding were resolved through discussion with a third author (AP-B) so that the final concordance was 100%. As there were only two raters rating the same sample, Cohen’s kappa was used to calculate inter-rater agreement. Data are reported as kappa and its 95% CI. We considered a kappa between 0.41 and 0.60 as a ‘moderate’ agreement, between 0.61 and 0.80 as a ‘substantial’ agreement and between 0.81 and 1.00 as an ‘almost perfect’ one.63
Statistical analysis
Categorical variables were described by their absolute frequencies and percentages and continuous variables were reported as median and interquartile range (IQR) assuming the data did not fit a normal distribution, which was verified using the Shapiro-Wilk test.
We used Fisher’s exact test to compare web page typologies and the χ2 test applying a Bonferroni correction to compare portrayals of health claims. We used a non-parametric Kruskal-Wallis test to compare information quality score in different interventions and web page typologies. Statistical significance was set at p<0.05 and the actual p value is reported in the Results section for each comparison. V.3.5.2 of the R software (SPSS, Chicago, Illinois, USA) and V.4.7.0.0 of the Joinpoint Regression Program were used for all analysis work.
Patient and public involvement
This research was carried out without patient or public involvement in the design of the study, the interpretation of the results, or the writing or editing of this document.
Results
The two primary types of web pages were commercial (23.7%, 27/114) and news web pages (23.7%, 27/114), followed by professional web pages (hospitals, universities and healthcare professionals) (14.0%, 16/114) and health portals (12.3%, 14/114). All other eligible web page typologies accounted for <10%. Five web pages corresponded to scientific publications relating to fibre (2.6%, 3/114) and prebiotics (1.8%, 2/114).
The certainty of the evidence supporting health claims
All the web pages discussed interventions in relation to at least one health claim. In total, there were 133 different health claims for which probiotics, yoghurt, kefir, kombucha, fibre and prebiotics were portrayed as beneficial (for a complete list, see reference 64). The most frequently reported reason for eating the food or taking the supplement was to reverse an altered gut microbiome (ie, ‘dysbiosis’) secondary to an unbalanced diet or stressful lifestyle, treatment with antibiotics or disease. The four primary and most widely portrayed health claim topics for all interventions were gastrointestinal health (86.0%, 98/114), vague claims about maintaining or improving health without any reference to a specific condition (‘Other’) (57.9%, 66/114), cardiovascular health (53.5%, 61/114) and immune system health (ie, infections, allergies, boosting the immune system) (50.9%, 58/114). The immune system-related health claims for kefir were over-represented compared with fibre (p=0.008). For fibre, the over-representation of health claims related to cardiovascular diseases was higher and statistically significant compared with probiotics (p=0.004) and the over-representation of health claims related to cancer was higher and statistically significant compared with probiotics (p=0.009) and prebiotics (p=0.044). The over-representation of general health claims (‘Other’) for kombucha was higher and statistically significant compared with fibre (p=0.002) and prebiotics (p=0.016) (figure 1).
Online health claim topics portrayed for probiotics, yoghurt, kefir, kombucha, fibre and prebiotics.
Of the total 133 health claims, only half (52.6%, 70/133) were supported by evidence from SRs. Probiotics (54.7%, 29/53), yoghurt (42.6%, 20/47) and fibre (37.1%, 13/35) had the highest number of online health claims supported by evidence from SRs. None of the 55 online health claims for kombucha were supported by evidence from SRs (figure 2).
Number of online health claims for probiotics, yoghurt, kefir, kombucha, fibre and prebiotics supported or not by evidence from systematic reviews (SRs).
The health claims that appeared on the greatest number of web pages were not necessarily the ones with the highest certainty of evidence (figure 3). In the context of gastrointestinal health, the highest certainty of evidence was found for the prevention of antibiotic-associated diarrhoea and necrotising enterocolitis for probiotics and yoghurt (moderate certainty of evidence) and the prevention and treatment of infectious diarrhoea and hepatic encephalopathy for prebiotics (moderate certainty of evidence). The prevention of acute otitis media was the immune system-related health claim supported by moderate evidence for probiotics and yoghurt. Fibre was the intervention with the highest number of online health claims supported by high (reduction of cholesterol and triglyceride levels) to moderate (reduction in obesity, type 2 diabetes, cardiovascular disease mortality, hypertension, coronary heart disease incidence and colorectal cancer incidence) certainty of evidence (for the complete data set, see reference 64).
Effect and certainty of evidence in systematic reviews (SR) supporting online health claims for probiotics, yoghurt, kefir, kombucha, fibre and prebiotics. *Derived from conclusions of SRs. +, favourable; 0, no effect; ?, uncertain effect. aProbiotics in the form of food (fermented milks containing probiotic bacteria) and supplements were analysed together in the SRs consulted. bOutcomes reported by probiotics combined with lactulose.
Quality of information
After assessing the quality of the online information by applying the 10 criteria as described in the Methods section, we obtained a score between 0 and 10 for all web pages. Figure 4 displays the median information quality score by intervention and web page typology. The median quality score by intervention was 3 IQR [2, 4] and was not significantly different across all interventions. Scientific journal web pages had the highest quality score of all typologies, with a significantly higher median than commercial (p=0.009), health portals (p=0.030), news (p=0.026) and professional web pages (p=0.026).
Information quality score by intervention (A) and web page typology (B). Data are reported as median and IQR. *P<0.05 versus scientific journals according to a Kruskal-Wallis test. C, commercial; HP, health portal; N, news; NP, non-profit organisation; P, professional; SJ, scientific journal; G, governmental; O, other.
Table 1 shows how quality criteria ranked among all interventions. While 39.5% of all web pages provided references or links to scientific publications, only a minority (18.4%, 21/114) adequately explained the key messages and conclusions of the paper’s content.
Web pages informing about probiotics, yoghurt, kefir, kombucha, fibre and prebiotics that meet each information quality criterion
Most web pages used verbal descriptions to explain intervention health benefits and did not quantify effects. Only 7.9% (9/114) of web pages quantified relative effects, including the five scientific journal web pages, of which only two included absolute effects.
Overall, only one-third of web pages (28.9%, 33/114) stated some of the limitations of research findings. Mentions of limitations included, for example, acknowledging that research that supports health benefits is still in its early stages; stating that the food can improve a condition for a few people in limited circumstances but it cannot be extrapolated to other people due to the small sample studied; addressing conflicts of interest; and highlighting discrepancies between studies that mean the intervention may not be recommended for all indications. Only 15.8% of web pages (18/114) provided a general discussion of the certainty of the evidence supporting an intervention’s benefits through consistent words and phrases that depended on whether the studies were observational (ie, using cautionary phrases such as ‘The results suggest’ and conditional verb tenses) or experimental (ie, using verbs that indicate causality such as ‘lead to’, ‘reduce’ or ‘increase’). Other means of properly communicating the certainty of the evidence included stating that effects were currently under investigation or more research was needed to consider an intervention in the context of a specific condition. There were web pages, for example, that used a language of uncertainty, mentioning that, ‘The health benefits of the probiotics and prebiotics that are currently available have not been proven conclusively’ or ‘For now, science does not know which of kefir’s components are responsible for its health benefits’. A further phrase mentioned how ‘There is not enough evidence that kombucha tea is as good for your health as some say’.
Only 42.1% of web pages mentioned or adequately discussed the potential harms of the intervention. Harms were reported in more than a half of web pages on kombucha (65.0%, 13/20), kefir (60.0%, 12/20) and fibre (55.0%, 11/20), but only in a quarter of web pages on probiotics (26.3%, 5/19). Similarly, less than half of web pages (43.0%, 49/114) reported available alternatives to the main intervention (ie, in the form of food or food supplements). The reporting of costs only appeared in 3.5% (4/114) of all web pages.
Some commercial (19.3%, 22/114), health portals (9.6%, 11/114), news (7.0%, 8/114) and professional web pages (6.1%, 7/114) included a direct recommendation to consume the food or supplement. Web pages reporting on the potential harms also recommended not consuming the food or supplement under specific circumstances (eg, avoiding probiotics and kombucha in immunocompromised adults). The recommendation of consulting a healthcare professional was included in a third of all web pages (28.1%, 32/114).
While for the criterion of acknowledging some research limitations the inter-rater agreement was 56% with Cohen’s kappa of 0.253 (95% CI 0.095 to 0.411), for the remaining variables, the inter-rater agreement was higher than 70% with Cohen’s kappa between 0.420 (95% CI 0.234 to 0.605) and 0.929 (95% CI 0.849 to 1.008), demonstrating ‘moderate’ to ‘almost perfect’ agreement.63 See reference 52 for inter-rater agreement results.
Discussion
Our study shows that most online health claims for probiotics, yoghurt, kefir, kombucha, fibre and prebiotics are supported by low to very low certainty of evidence. Furthermore, the overall quality of information on the gut microbiome-related interventions studied was low, with a median quality score of 3 on a scale of 0–10 for all interventions when applying our 10 quality criteria.
On web page typology, results were not surprising. The prominent presence of commercial (23.7%) and news (23.7%) web pages in Spanish is in line with previous results on web page content on probiotics in English.42 43 Our findings reflect companies’ interest in therapeutically exploiting the microbiome42 43 and the newsworthiness of the topic.2–4
Regarding the first objective, both the plethora of beneficial health claims for dietary interventions intended to improve health through the gut microbiome and the weak evidence base supporting such health claims were also expected. All in all, our data add valuable details for better understanding the online information to which audiences are exposed.
First, our research finds that probiotics, fermented foods, fibre and prebiotics might be beneficial for 133 health indications. Similarly, Marcon et al found that American and Canadian general newspapers mentioned up to 138 different health topics for which microbiome-related interventions were portrayed as beneficial.4 However, very few of those purported benefits are supported by the evidence and integrated into clinical practice. Thus, while fibre has a long history of use in the clinical setting,17 the degree to which probiotics are recommended to patients by healthcare professionals is variable.39 65 Factors explaining why some specialist doctors do not recommend probiotics include the perceived lack of research evidence and poor knowledge regarding use and cost.66 67 While uncertainty remains around the optimal use of probiotics,15 the perception among patients who seek advice from gastroenterologists is that probiotics improve general health, longevity and gastrointestinal symptoms.68
Second, we found gastrointestinal health and immune system health-related indications are among the most widely mentioned benefits, which is in line with Neunez et al’s findings for probiotics.43 However, the concepts of ‘boosting gut health’ and ‘boosting immunity’, the latter of which spiked on the internet during the COVID-19 pandemic,41 are misleading and scientifically inaccurate.50 69
Third, the evidence-based benefit of probiotics and yoghurt for preventing antibiotic-associated diarrhoea that appeared in most web pages is supported by moderate certainty of evidence,70 while certainty of evidence is low and very low for irritable bowel syndrome71 and infectious diarrhoea,72 respectively, both of which appear in a high number of search results. Conversely, although there is moderate certainty of evidence of probiotics’ role in preventing mortality and infections secondary to necrotising enterocolitis in very preterm infants or infants with a very low birth weight,73 that health benefit only appeared in a few of the web pages that discussed probiotics (21.1%, 4/19). When interpreting SRs that perform a meta-analysis of probiotics, it should be acknowledged that their conclusions can be misleading if different strains or combinations of probiotics at different doses are grouped together inappropriately and studies include different patient populations and measure different outcomes.37 74 That may cloud any potential signalling of the probiotic for preventing or treating diseases and may contribute to explaining why only 54.7% of probiotic-related health claims are supported by evidence from SRs.74 The low number of health claims for yoghurt (42.6%, 20/47) and kefir (1.8%, 1/55), supported by evidence from SRs, coincides with our previous findings using the GRADE approach, which showed that consuming probiotics in the form of fermented milks such as yoghurt and kefir may not be associated with any health benefits, with either low or very low certainty of evidence.75 None of the health claims for kombucha were supported by evidence from SRs, which is expected due to the lack of controlled human studies investigating the potential health effects of this popular fermented drink.16 76
Fourth, not surprisingly given its common use among healthcare professionals in gastrointestinal disorders,17 fibre was the intervention with the most health claims supported by high (reduction of cardiovascular disease risk factors)18 to moderate (protection against colorectal cancer)77 certainty of evidence. The efficacy of prebiotics for preventing constipation supported by low certainty of evidence78 appeared in a high proportion of web pages. In contrast, the more widely studied indication of prebiotics for managing hepatic encephalopathy, which showed moderate certainty of evidence,79 appeared in very few web pages (16.7%, 3/18).
Regarding the second objective, the assessment of information quality carried out using our 10-criteria score shows interesting data on both overall quality and some specific shortcomings.
First, the low quality of online information assessed according to our 10 criteria is not surprising. However, it is even lower than estimated for news reports on health interventions in general, using other indices or scales containing common quality criteria. Thus, in our study, 92.1% of the web pages did not quantify the effects of the intervention, compared with 72% of the news items analysed by Schwitzer60; 84.2% did not discuss the certainty of the evidence, compared with 65%; and 96.5% did not report the costs of the intervention, compared with 77%. On two other common criteria, the results were more similar: 57% of the web pages did not report alternatives to the intervention, compared with 62% of the news items analysed by Schwitzer; and 57.9% did not report potential harms, compared with 67%.
The first SR of the quality of information on health interventions in traditional media outlets and online resources also found room for improvement as regards health news.59 However, nutrition-related information is especially prone to poor quality and may contribute to public misconceptions about dietary strategies targeting the gut microbiome and health.80–84 In our study, retail sites intended for direct purchase and advertisements were excluded from the analysis; nevertheless, a quarter of the analysed web pages were commercial. The regulatory status of commercial information about gut microbiome-related foods and dietary supplements on web pages is not the same as for a pharmaceutical product. In the case of Spain, in spite of current legislation on commercial information related to foods and food supplements,85–87 misleading food marketing prevails. Regulating digital marketing is not straightforward because of its cross-border nature,88 but it is critical for making informed decisions about health. Ongoing voluntary implementation measures involving the food industry, communications agencies and advertisers are insufficient in preventing misinformation about popular gut microbiome-related dietary interventions.89 90 To allow consumers to make informed food choices, stricter regulation of any probiotics, fermented foods, fibre and prebiotics promoted on websites is required, especially to ensure that the dietary advice to which the public is exposed is based on evidence that is either convincing or probable.81 Steering clear of the practice by scientific societies of endorsing prebiotic or probiotic products that have dubious health benefits may also help with avoiding the spread of inaccurate information.91
Another way to improve that situation might be to promote critical thinking among the public. In other words, it might be better to treat the effects of the current overabundance of information than to prevent it, since prevention is an almost impossible task, with exaggerated scientific findings and discoveries always attracting those who produce and recirculate information.59 92 In that regard, we present our suggested 10 criteria for quality of information, aimed at three different groups. First, healthcare providers as a tool for recommending reliable web pages on gut microbiome-related interventions to their patients; second, journalists and communicators involved in disseminating microbiome research findings; and third, the lay public to guide them every time they face a piece of online information related to the gut microbiome.
Second, the strategy of including scientific references embedded in the text or as a list at the end of text (criterion 1) without explaining the conclusions of the scientific publications (criterion 2) is an example of how the ‘health halo effect’ around gut health and the microbiome is used to validate certain unproven alternative therapies.4 41 In addition, web pages also misrepresent the term ‘probiotic’, which is inadequately used to refer to kombucha and kefir and as an umbrella for all probiotic supplements, when, in actual fact, not all probiotics are backed by science and not all fermented foods can be considered probiotics.16 93 Likewise, many web pages use the term ‘dysbiosis’ as a reason to promote interventions with the connotation that an ‘altered’ microbiome in someone with a specific disease is causal or contributory, even though it is not always certain that changing the altered microbiome is beneficial94 and the definition of a healthy microbiome is not known.95
Third, it is also worth noting that the majority of web pages only provide a qualitative description of the health claims without quantifying them (criteria 3 and 4). The few web pages that quantified the effects did so only in relative numbers (7.9%, 9/114), which tend to be more eloquent, are often misleading and can lead to a misguided perception of the reported effects.94 96 Only two out of five scientific journal web pages included absolute effects. Indeed, the microbiome field relies too heavily on relative numbers of microorganisms.94 For instance, one clear example of numerical misinterpretation is the long-assumed ratio, widely disseminated in the media and the scientific literature, that humans have 10 times more microbial cells than body cells.97
Fourth, the observation that only a few web pages acknowledged some research limitations (criterion 5) (28.9%, 33/114) and discussed the certainty of the evidence (criterion 6) (15.8%, 18/114) is common when informing on microbiome-related interventions. For instance, social media content rarely makes critical references to microbiome research findings and the only acknowledgements of limitations found are suggestions around the need for more research.34 Likewise, previous findings show that only 19% of articles in English-language newspapers4 and less than 10% of web pages portraying immune boosting strategies, including the use of probiotics and prebiotics,41 report microbiome-related limitations (eg, suggesting that the health benefits of and current research on the microbiome might be unproven, ineffective or exaggerated). Probiotics was the intervention with the highest proportion of web pages that provided limitations and comments around the certainty of the evidence, which might be explained by probiotics’ status as the most widely studied subject when compared with fermented foods such as kefir and kombucha.14 15
Fifth, only a minority of web pages on probiotics informed of adverse effects (criterion 7) and included advice against consumption by people with severe illnesses or compromised immune systems.98 Previous analyses of online messages about probiotics also found that descriptions of their benefits outnumbered the descriptions of their risks, and the latter appeared significantly less on commercial web pages.42 43 That may be rooted in the lack of safety data in randomised controlled trials for probiotics.99 Safety issues are also a concern for kombucha, with reports of varying degrees of adverse effects in relation to kombucha tea consumption,100 while fibre and prebiotics are limited to mild issues such as abdominal discomfort, bloating and gas.17
As dietary interventions that target the gut microbiome are usually regulated as foods and dietary supplements and not drugs, none of the health claims promoted on the internet need to be backed up by studies in humans. In addition, what it is actually in a probiotic or dairy product does not necessarily coincide with what it is declared on the label.101 In the best-case scenario, the product may be ineffective and the only likely harm is to the consumer’s wallet. In the worst-case scenario, however, a product can have significant side effects. That is the case with the hepatotoxic effects reported from kombucha intake,100 the increased risk of pre-eclampsia with probiotic administration102 and the increased risk of mortality in adult patients with acute pancreatitis who receive probiotics.103 Finally, self-consumption of these kinds of foods and supplements as a non-prescribed alternative treatment due to the consumer’s unfounded expectations, which outpace the scientific evidence, can lead to a delay in the presentation and resolution of a medical diagnosis and the search for effective treatment.
Our study shows two strengths. First, for the evaluation of online health claims, we relied on SRs and assessments of the degree of certainty of evidence using the GRADE approach, which is a systematic, explicit and transparent methodological framework for grading certainty of evidence.104 105 Second, the authors have extensive knowledge of and experience in the fields of nutrition, evidence-based medicine, science journalism and microbiome research communication.
There are also several limitations to this study. First, we used single search phrases to perform the searches. That meant we could not explore differences in results for other search terms, which can vary in the current context of the COVID-19 pandemic. Second, the data set only includes Spanish-language web pages and focuses on the initial search results, although it must be acknowledged that the top 20 search results have a higher chance of being read.47 48 Third, we only focused on 10 parameters for assessing quality of information. Fourth, we did not analyse information published on social media channels, which provide relevant sources for people seeking nutrition advice online. Last, we analysed the certainty of the evidence behind online health claims based only on SRs, which are currently the evidence synthesis tool that offers the highest level of evidence.
Conclusion
Online information on probiotics, fermented foods, fibre and prebiotics does not reflect the available body of scientific evidence and is often incomplete and of poor quality. The observation that the majority of health claims that appeared on the largest number of web pages were not necessarily the ones with the highest certainty of evidence may contribute to distorting the message about the impact of foods on health linked to their effects on the gut microbiome. Furthermore, the fact that research results, the quantification of the effects, limitations and uncertainty of the evidence, and the adverse effects, cost and alternatives of interventions are not usually addressed can distort public perception of the topic. Consequently, online information about the six interventions considered in this study may, in some cases, create a potentially harmful distraction rather than a key element for maintaining health and quality of life.
Data availability statement
All data relevant to the study are included in a public, open access repository. The data set is available on the following: https://figshare.com/articles/journal_contribution/Supplemental_material_1/20203415, https://figshare.com/articles/journal_contribution/Table_S2/19425824, https://figshare.com/articles/journal_contribution/Supplemental_material_3/20204021, https://figshare.com/articles/journal_contribution/Supplemental_material_4/20204270.
Ethics statements
Patient consent for publication
References
Footnotes
Twitter @andreuprados, @MontseRabassa, @mboschpujadas, @gonzalocasino
Contributors AP-B and GC designed the research with input from MR and MB. AP-B, MR, MB and GC performed the research and analysed the data. AP-B and GC interpreted the data. AP-B wrote the manuscript. MR, MB and GC were involved in drafting and revising the manuscript. All authors approved the final version to be published and agreed to be accountable for all aspects of the work. AP-B is the guarantor.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests AP-B works as a health writer for companies commercially involved in the gut microbiota and probiotics.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.