Article Text
PDF
Abstract
Background Patients living with fibromyalgia strongly prefer to access health information on the web. However, the majority of subjects in previous studies strongly expressed their concerns about the quality of online information resources.
Objectives The purpose of this study was to evaluate existing online fibromyalgia information resources for content, quality and readability by using standardised quality and readability tools.
Methods The first 25 websites were identified using Google and the search keyword ‘fibromyalgia’. Pairs of raters independently evaluated website quality using two structured tools (DISCERN and a quality checklist). Readability was assessed using the Flesch Reading Ease score maps.
Results Ranking of the websites' quality varied by the tool used, although there was general agreement about the top three websites (Fibromyalgia Information, Fibromyalgia Information Foundation and National Institute of Arthritis and Musculoskeletal and Skin Diseases). Content analysis indicated that 72% of websites provided information on treatment options, 68% on symptoms, 60% on diagnosis and 40% on coping and resources. DISCERN ratings classified 32% websites as ‘very good’, 32% as ‘good and 36% as ‘marginal’. The mean overall DISCERN score was 36.88 (good). Only 16% of websites met the recommended literacy level grade of 6–8 (range 7–15).
Conclusion Higher quality websites tended to be less readable. Online fibromyalgia information resources do not provide comprehensive information about fibromyalgia, and have low quality and poor readability. While information is very important for those living with fibromyalgia, current resources are unlikely to provide necessary or accurate information, and may not be usable for most people.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
Article summary
Article focus
The purpose of the study was to gain a better understanding of the online information resources available for people with fibromyalgia and to evaluate those resources for content, quality and readability.
To determine the content, website quality and readability of the most readily retrieved information available on the web when searching for fibromyalgia information.
Key messages
Most existing websites do not provide comprehensive information on fibromyalgia.
Websites are highly variable in terms of quality.
Higher quality websites do not present information in language/reading levels appropriate for the general population.
Strengths and limitations of this study
This study examines the quality of online fibromyalgia resources.
Standardised quality and readability tools were used to assess quality and readability.
There is no gold standard for comparison or ways to evaluate the quality of websites.
The quality issue was discussed using critical appraisal tools designed for the lay public.
The readability score may vary for some websites as it may be related to the use of technical terms such as fibromyalgia.
Introduction
More than 70 000 websites offer health information for consumers and the number is increasing daily.1 Many of these websites are accessed by people with fibromyalgia to self-manage their own health. However, it is unknown if these websites are consistent with the literacy and health needs of such users. However, we do know that web-based information has the potential to educate and empower consumers by providing information about their health problems and by helping them make informed decisions about their health.2–5
The extent of the interest in web-based health information is indicated by high and increasing usage.6 7 In Canada about 8.7 million people use the internet to obtain medical and health-related information, with women being more likely to search for health information on specific diseases than men.6 In addition, 54%–79% of those seeking information expressed concerns about the quality of online health information.6 Similarly, in the USA, the number of adults who look for health information online has increased from 46% in 2000 to 61% in 2009.7 Many (66%) of these online health information seekers discuss their concerns about the lack of quality of online health information sources with their healthcare providers.7 Thus, researchers at the Pew Internet and American Life Project anticipate that as more people access the internet for health information, their concern about quality will also continue to grow.7
The internet is now an important resource for people living with fibromyalgia.8–10 Fibromyalgia is described as an invisible chronic condition that has severe impacts on health and quality of life for those living with the illness.11–13 The disease manifests as chronic widespread musculoskeletal pain in different areas of the body.14–16 The need for information is greater due to the controversy surrounding the condition and a lack of specific diagnostics tests and evidence-based treatment guidelines. It has been suggested that people are often left on their own to manage the illness.17 16
Daraz and others studied the information needs and preferences of people living with fibromyalgia.8 9 The majority of patients in those studies expressed their preference for the web as a major source of fibromyalgia-related information. However, they also strongly expressed their concerns about the lack of some types of information about fibromyalgia (content), the need for evidence-based information (quality) and difficulty in understanding medical or technical terminologies (literacy/readability). A similar study by Crooks demonstrated that people living with fibromyalgia like to go online to access information about fibromyalgia to inform themselves about the illness and to assist with shared decision making with their healthcare providers.10 However, the perceived lack of quality of online information was a major factor that was also discussed in the study findings. Others have also suggested that web-based health information can increase people's perception of control, improve their ability to cope with illness, enhance their self-care abilities and improve their quality of life by decreasing anxiety, fear and distress while increasing hope.18 19 A number of studies have evaluated the quality of online health information designed for specific populations and found it to be of variable quality.20–25 2 26 It is imperative that people living with fibromyalgia have access to quality evidence-based information to help them live with their illness, especially since it is a chronic disease. Therefore, it is important to evaluate websites to determine if they can meet the needs of persons with fibromyalgia for accessible, high quality, useful information.
The purpose of this study was to gain a better understanding of the online information resources available for people living with fibromyalgia and to evaluate those information resources for content, quality and readability.
Methods
This study consisted of a keyword search, selection of websites and structured appraisal of the websites using standardised quality and readability tools. Similar methods were used by others who evaluated the quality of websites for specific conditions.20 21 24
Search strategies to find online fibromyalgia information resources
In a previous study, the authors identified search terms and engines that women commonly use when looking for information on fibromyalgia.8 Based on those findings, we performed a keyword search on Google (http://www.google.com) with the keyword ‘fibromyalgia’ on 11 December 2009 to identify the online fibromyalgia resources most likely to be accessed. It has been suggested that lay people seldom search for information beyond the first 20 links retrieved by a search engine, so we used this to dictate our website sample.27
Criteria for selecting online fibromyalgia information resources
Our inclusion criteria for selecting websites were: (1) provision of information on fibromyalgia, (2) provision of information for consumers/patients and their caregivers, and (3) provision of information in English. We excluded duplicate websites or sites with dead links.
Quality appraisal tools
DISCERN is a reliable and valid instrument used to assess the quality of written consumer health information which people can use without content expertise.28 29 The instrument was developed and evaluated by an expert panel and a group of health information providers and self-help members. DISCERN consists of 15 questions (the first eight questions are on publication reliability and last seven questions are on the quality of information on treatment choices) where each question is rated on a 1–5 point scale. We assigned scores according to the DISCERN marking system (topic addressed=5, partially addressed=3, not addressed=1). This instrument has been evaluated for reliability and validity and is being used by many researchers to assess the quality of online health information for specific diseases.20 23 24 However, DISCERN does not include many of the criteria that are important for assessing specific information content and the dissemination of the information, for example, accuracy, completeness, disclosure and readability.27
As a result, we used a quality checklist developed by Daraz and others30 to assess the quality of web health information. This tool was based on a structured review and appraisal of existing web health evaluation tools developed to assess the quality of web health information. Based on their review, the authors determined that the existing web health evaluation tools did not meet the criteria for readability and ease of use for general consumers. Therefore, they recommended a customised tool/quality checklist designed for general consumer use. The Quality Checklist consists of seven categories: (1) authorship, (2) content, (3) currency, (4) usefulness, (5) disclosure, (6) user support and feedback, and (7) privacy and confidentiality. A total of 10 questions are included in the checklist with a ‘yes’ or ‘no’ option.
To determine the overall rating of the websites, we also used the total DISCERN score to categorise the websites as excellent (61–75), very good (60–46), good (45–31), marginal (30–16) or poor (15–1). It was not possible to assign similar categories to assess the overall rating of the websites using the Quality Checklist as the tool does not have a numerical scoring scheme.
For the readability evaluation, the information from each websites was evaluated for (1) reading ease and (2) grade level using the actual content from the website. For the reading ease calculation we used the Flesch Reading Ease31 32 score maps designed to measure the readability of texts; the reading index is 0–100. A score of around 60–70 is equivalent to grade level 6–8. The closer to 100 the text scores, the easier it is to read.31 33
We used the Flesch-Kincaid Grade Level formula to calculate grade level. It is recommended that anyone who aims to provide health information should aim for a grade level of 6–8. The scores using the Flesch Reading Ease formula are explained in box 1.31 32
Flesch Reading Ease scores
90–100: Very easy
80–89: Easy
70–79: Fairly easy
60–69: Standard
50–59: Fairly difficult
30–49: Difficult
0–29: Very confusing
Data extraction and analysis
When the information on websites is consistent with the best research, then that information is high quality and high quality websites are those which are constructed according to certain standards.34 20 35 By ‘content’ we mean to specific information about fibromyalgia, for example, on treatment, diet, finding specialists, etc. By ‘readability’, we mean reading ease and grade level. By ‘quality’ we mean overall website quality, not the quality of specific pieces of information on the website: website quality assesses the efforts made to insure the information on the website is current and accurate based on current evidence/knowledge.
A data extraction tool was devised to allow reviewers to categorise the content on fibromyalgia websites. Categories were developed using concepts derived from both qualitative and quantitative research8 9; open-ended categories were later classified if concepts were reported that were not preconceived by the structured items. The data extraction table included: country of origin, target audience, category of websites and types of content. Websites were categorised as not-for-profit (eg, society, association, charitable group, support group), commercial (eg, private medical site, sponsored site), media (eg, newspaper) and institutional (eg, university, government).
To assess the reliability of evaluation, each site was independently rated by the authors. Although κ scores were not tabulated, the reviewers discussed each question where scoring was different until the scoring conflict was resolved. We used simple descriptive statistics to analyse the data. SPSS v 18 was used in our analysis for calculating frequencies and cross-tabulations. For example, the frequency command was used to determine the percentiles of websites for country of origin or the categories of websites.
Results
Google retrieved 6 720 000 results for the keyword search. The first 25 websites were selected for analysis (table 1). Thirteen (52%) of the websites were from the USA, eight (32%) were from Canada, one was from the UK and the rest had no country specified (table 1). The category of websites varied. Ten (40%) were not-for-profit organisations, six (24%) were commercial, five (20%) were media and four (16%) were institutional. Only five (20%) websites were dedicated to women. Table 1 also gives the scores for DISCERN (column 3) and the percentage of ‘yes’ answers for the Quality Checklist (column 4).
25 Selected sites and their overall scores
Figure 1 shows the types of information provided by the selected websites. Other types of information were also available on the selected websites, for instance on complications (8%), controversies (8%), exercise (8%), lifestyle (8%), education (4%), employment (4%), psychological issues (4%), quality of life (4%) and self-help (4%).
Types of information available on selected websites.
Figure 2 shows the reliability of the websites and the quality of treatment information as measured by DISCERN. The mean score of all 15 questions combined was 2.49 out of 5. No question received a mean score of 4 or more. The questions that received the lowest score were related to sources of information, areas of uncertainty, side effects of treatments, effects of no treatment, effect on quality of life and shared decision-making. Websites were categorised as very good (32%), good (32%) or marginal (36%). The mean overall DISCERN score was 36.88 (good).
Combined scores of the DISCERN reliability and quality of treatment categories.
Figure 3 shows the combined Quality Checklist scores for the websites. The questions with the highest rating related to contact information, confidentiality, ownership and useable/understandable content.
Combined scores of the Quality Checklist questions (percentage of option ‘yes’).
The readability test showed that the reading level for 14 (56%) websites was between grades 10 and 12, seven websites (28%) were between grades 8 and 9, and one website (4%) was between grades 6 and 7. None were between grade 1 and 5 and 12% were college level (table 1).
Table 2 shows the five highest ranked websites according to scores from DISCERN, the Quality Checklist and the Flesch Reading Ease test.
Five top ranked websites based on the DISCERN, Quality Checklist and Flesch Reading Ease scores
Discussion
The results of this study suggest that fibromyalgia websites vary with respect to content, quality and readability. There is considerable variability between the average scores from DISCERN, the Quality Checklist and the Flesch Reading Ease test. Good quality websites often had poor readability. Only three websites (Fibromyalgia Information,53 Fibromyalgia Information Foundation56 and National Institute of Arthritis and Musculoskeletal and Skin Diseases55) had consistently high levels of quality (table 1). Unfortunately, since these also had high reading levels (grade 11), they are not likely to be accessible by people with lower literacy.
The content of the websites most commonly addressed symptoms, treatment and diagnosis. Many websites lacked information about important topics identified by patients as significant, such as causes of fibromyalgia, research, supports, alternative therapies, impact, and specialists who might help them understand and manage their illness.8 9 61 More efforts are needed to include comprehensive information on the websites that provide customised information for people with fibromyalgia.
Quality scoring with the two quality appraisal tools resulted in different rankings for the same website due to different items and scoring systems. Others have shown that there is considerable variability in the critical appraisal tools used for evaluating research34 and it appears that a similar trend is evolving with respect to websites. While DISCERN seems to be the most commonly used tool, it is important for those conducting reviews to evaluate whether the critical appraisal tool used is the most appropriate one for their individual study.
This study focused on assessing the quality of a website from the perspective of a lay person.26 23 Lay ‘quality’ assessment assumes practices that indicate more rigorous development and authorship will lead to more timely and accurate information. That is because the general public cannot verify the accuracy of specific medical or scientific information on the web. This study indicates that websites do not adequately identify the sources or timeliness of the information presented. It has been suggested that providing a date does not necessarily mean that the information is correct or up-to-date.2 However, it is a reasonable proxy. Website currency could be judged by determining if recent evidence has been incorporated, but this is not a reasonable expectation for the lay public. Similarly, providing contact information may be associated with authors taking responsibility for their website, but there may be a variety of other motivations. This study focused on assessment of website quality from the perspective of the consumer. We also observed that two different tools (DISCERN and the Quality Checklist) designed for the lay public provided different scores and rankings. We have no way of knowing whether one tool provided a more valid assessment than the other. However, both scales agreed on three (Fibromyalgia Information,53 Fibromyalgia Information Foundation56 and National Institute of Arthritis and Musculoskeletal and Skin Diseases55) out of five websites in the top five (table 2), suggesting a level of concurrent validity. Studies that assess the extent to which different lay indicators of quality are associated with actual quality and accuracy of information are needed to assess the criterion validity of these scales.
This review demonstrates that a substantial proportion of the most accessible fibromyalgia information websites do not meet the criteria established for website quality, thus undermining confidence in the accuracy of these resources. Healthcare providers and website developers should work together to provide more consistent information for people living with fibromyalgia. There is also a need to determine useful and accurate criteria for lay individuals to assess website quality. For example, none of the tools used for this study can assess websites for accessibility, linking, peer to peer feedback or web standards.
Another major finding of this review is that people need a high level of education to understand online information about fibromyalgia, particularly on high quality websites. For example, the National Institute of Arthritis and Musculoskeletal and Skin Diseases website provides good quality information about fibromyalgia; however, a person needs a grade level of 11 to understand and use that information. Only four websites (Fibromyalgia Treatment Center,36 Fibromyalgia Network,37 Medline Plus,38 Women's Health Matters39) meet the literacy level for the general population. High readability requirements decrease information accessibility and potentially exclude users with low literacy skills.62 Using the web to provide useable quality information remains elusive. A common concern among people living with chronic illnesses including fibromyalgia is that difficult medical terminology is a major barrier to accessing and using online health information efficiently.8 9 20 61 Online information on fibromyalgia needs to be written at or below a grade 8 level so that all are able to read the information and use it for their own health decision-making. This suggests that people with health literacy expertise should be involved in website development.
Overall, there is evidence that there are inconsistencies across websites regarding content, overall quality and readability, as found by those who evaluated websites for other chronic conditions.2 26 People living with fibromyalgia have expressed a strong need for information and a dependency on web-based information as a primary source. This indicates that more effort is needed to ensure that the information provided on fibromyalgia websites meets the information needs, quality and suggested readability criteria.
Limitations of the study
A number of limitations should be considered when interpreting our results. First, our search was not comprehensive as we only used Google and one keyword to search for online fibromyalgia resources. We selected Google and ‘fibromyalgia’ as these were most commonly used by our target audience,8 but recognise that other search engines and keyword combination may have produced different results.
The way that items on the Quality Checklist are formatted also has some limitations. Since some items have multiple questions requiring a single yes/no answer, reviewers sometimes had difficulty selecting an option when partial credit was assigned. In addition, there is lack of evidence to validate the Quality Checklist. Thus, some of the differences between DISCERN and the Quality Checklist relate to scoring methods.
Finally, we have no gold standard for whether these websites were quality websites. We addressed the quality issue using two critical appraisal tools designed for the lay public. Some of the other important criteria such as accessibility, linking, web standards or peer-to-peer feedback, are not included in these quality tools. As a result, a detailed analysis of recommendations on websites would be required to determine whether they are consistent with the highest quality of evidence and therefore whether the information itself is also of high quality. Finally, the readability score may vary for some websites as it may be related to the use of technical terms such as fibromyalgia, since this seems to be a word with low readability.
Conclusion and implications for practice
The internet is changing the way that people gather information about chronic conditions like fibromyalgia and has the potential to facilitate patient self-management. This study has demonstrated that online fibromyalgia resources do not provide comprehensive information about fibromyalgia. The majority of the websites provide information on only a few content areas and are highly variable in terms of quality and readability. Website quality ranking varied by the tool used, although there was general agreement about the top three websites. Higher quality websites do not present information at language/reading levels appropriate for the general population. Thus, it is difficult for people with fibromyalgia to distinguish between good and poor online resources, suggesting the potential for misinformation. Healthcare and social service providers need to be aware of the general lack of quality of online fibromyalgia resources and need to be more involved in the health decision-making of people with fibromyalgia by helping them access quality online health information.
References
Footnotes
To cite: Daraz L, MacDermid JC, Wilkins S, et al. The quality of websites addressing fibromyalgia: an assessment of quality and readability using standardised tools. BMJ Open 2011;1:e000152. doi:10.1136/bmjopen-2011-000152
Funding This work was supported by a Doctoral Research Award (Frederick Banting and Charles Best Scholarship) from the CIHR, a Strategic Training Fellowship in Rehabilitation Research from the CIHR Musculoskeletal and Arthritis Institute, an S. Leonard Syme Training Fellowship from the Institute for Work & Health and an MSK Training Fellowship from the Ontario Rehabilitation Research Advisory Network to Lubna Daraz.
Competing interests None.
Contributors The first author, LD was in charge of all aspects of research including choosing the topic, formulating research questions, reviewing the literature, designing the study, searching on Google, collecting data, evaluating websites, and drafting and revising the manuscript critically for important intellectual content, and provided final approval of the version to be published. JCMcD made substantial contributions to conception and design, evaluated websites, revised the manuscript critically for important intellectual content and provided final approval of the version to be published. SW and JG made substantial contributions to conception and design, evaluated websites, revised the manuscript critically for important intellectual content and provided final approval of the version to be published. LS made substantial contributions to conception and design, revised the manuscript critically for important intellectual content and provided final approval of the version to be published.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.