Article Text
PDF
Abstract
Objectives Recent guideline changes for cardiovascular disease (CVD) prevention medication have resulted in calls to implement shared decision-making rather than arbitrary treatment thresholds. Less attention has been paid to existing tools that could facilitate this. Decision aids are well-established tools that enable shared decision-making and have been shown to improve CVD prevention adherence. However, it is unknown how many CVD decision aids are publicly available for patients online, what their quality is like and whether they are suitable for patients with lower health literacy, for whom the burden of CVD is greatest. This study aimed to identify and evaluate all English language, publicly available online CVD prevention decision aids.
Design Systematic review of public websites in August to November 2016 using an environmental scan methodology, with updated evaluation in April 2018. The decision aids were evaluated based on: (1) suitability for low health literacy populations (understandability, actionability and readability); and (2) International Patient Decision Aids Standards (IPDAS).
Primary outcome measures Understandability and actionability using the validated Patient Education Materials Assessment Tool for Printed Materials (PEMAT-P scale), readability using Gunning–Fog and Flesch–Kincaid indices and quality using IPDAS V.3 and V.4.
Results A total of 25 unique decision aids were identified. On the PEMAT-P scale, the decision aids scored well on understandability (mean 87%) but not on actionability (mean 61%). Readability was also higher than recommended levels (mean Gunning–Fog index=10.1; suitable for grade 10 students). Four decision aids met criteria to be considered a decision aid (ie, met IPDAS qualifying criteria) and one sufficiently minimised major bias (ie, met IPDAS certification criteria).
Conclusions Publicly available CVD prevention decision aids are not suitable for low literacy populations and only one met international standards for certification. Given that patients with lower health literacy are at increased risk of CVD, this urgently needs to be addressed.
- cardiovascular disease
- decision support
- decision aid
- prevention
- shared decision making
- health literacy
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
First systematic search to identify and evaluate freely available online CVD decision aids using the International Patient Decision Aids Standards (IPDAS), the most credible and internationally recognised measure for evaluating patient decision aids.
Patient decision aids were evaluated using the multiple versions of IPDAS as well as the validated Patient Education Materials Assessment Tool for Printed Materials measure relating to health literacy, extracted independently by two reviewers where discrepancies were resolved via discussion to reduce bias.
Google results are not replicable due to the changing nature of the search algorithm and websites; but using known repositories may assist researchers and clinicians to conduct similar searches.
We did not assess the accuracy of the information provided by these decision aids.
Introduction
Cardiovascular disease (CVD) prevention is a key issue for primary care, as one of the most common problems managed in general practice1 and the leading cause of mortality and morbidity in developed nations.2 Clinical guidelines recommend lifestyle interventions with the addition of medication to lower blood pressure and/or cholesterol if CVD risk becomes high.3–5 However, recent guideline changes for CVD prevention medication have increasingly lowered the threshold for treatment: statin initiation has reduced from 20% absolute CVD risk over 10 years down to 10% in the UK and 7.5% in the USA6 7; and the latest US hypertension guidelines recommend a very low threshold of 130 mm Hg for blood pressure medication.8 These changes have led to wide debate in leading medical journals (eg, BMJ, JAMA, Lancet), with calls to implement shared decision-making based on both benefits and harms as well as patient preferences, rather than arbitrary treatment thresholds for CVD prevention.6–10
Shared decision-making is important in this context, because there are many ways to reduce risk and weighing up the benefits and harms of different options is dependent on individual preferences.7 9 For example, a 60-year-old female smoker with elevated blood pressure (130/80 mm Hg) and cholesterol (5/1.8 total/high-density lipoprotein mmol/L) will have a 10% chance of a CVD event in the next 10 years based on the Framingham model (see http://chd.bestsciencemedicine.com/calc2.html for risk and intervention estimates using different models). She may prefer to avoid the side effects and costs of medication and focus on changing her lifestyle,11 which could reduce her risk to 6% if she quit smoking, or 7% if she adopted a Mediterranean diet or increased her physical activity to high levels. Alternatively, she may be unwilling or unable to make these changes,11 and would prefer to reduce her risk to 7% with either low-moderate intensity statins12 or blood pressure lowering medication.13 Although these options have different relative risk reduction benefits, when the baseline CVD risk is only 10% the absolute benefit is very similar, so patient preferences must be taken into account.
Little attention has been paid to existing tools that could facilitate shared decision-making in this context. Decision aids are well established as an effective tool to help patients engage in shared decision-making about their health. International standards have been developed to ensure they provide evidence-based, unbiased information about benefits and harms, using multiple formats to enhance patient understanding (available at http://ipdas.ohri.ca/using.html).14 15 The latest Cochrane review on this topic found 105 RCTs evaluating decision aids, with positive effects on knowledge about options, value clarification and feelings of being better informed.16Patient decision aids for CVD prevention have been shown to improve uptake and self-reported adherence to preventive interventions17; however not all decision aids have reported similar effects on adherence. The Statin Choice decision aid aimed at CVD prevention in diabetes did not report similar adherence to statins but did report that patients accurately perceived their risk for heart attack.18
The availability of high quality, understandable health information is particularly important considering the burden of CVD is greater for people with low health literacy. This means they do not have adequate skills to access, understand or use resources to manage their own health. The majority of the general population falls into this category, and this is associated with less regular healthcare access, lower uptake of prevention services, poorer self-management, greater medication errors, worse CVD outcomes and increased all-cause mortality.19–21 It is therefore important to consider the needs of patients with low health literacy skills when developing online shared decision-making tools, which are likely to be accessed with little support from health professionals.
This study aimed to systematically review the online environment for patient decision aids relating to primary CVD prevention, and evaluate their quality based on international patient decision aid standards and health literacy criteria.
Methods
Inclusion and exclusion criteria
Decision aids were considered if they met all inclusion criteria: (1) focus on primary prevention (ie, not secondary prevention or treatment for established CVD), (2) provides information about blood pressure medication, cholesterol lowering medication and/or aspirin, (3) freely available, and (4) written in English. Exclusion criteria included: (1) could not be viewed due to technical problems after two attempts, (2) developed by a company with a vested interest in medication (eg, pharmaceutical) or (3) targeted at health professionals or clinicians.
Search methods
An environmental scan can be described as an efficient and organised means to collect specific information on a given topic/institution that is pertinent to its internal workings and external influences/surrounding.22 Part of the process involves a purposive approach to a search from which the search is then exploded. For this study, we identified known online decision aid repositories (see table 1) as the most likely sources to contain relevant information pertinent to this study. Our second source was from Google Australia.
List of known decision aid repositories
Two independent searchers (PP and RZ) were instructed to reset their cache in their web browsers before each Google search to minimise the effect of Google search optimisation. The final search terms after piloting included 11 for CVD/medication and 2 for decision aids. The lead researchers (CB, LT) and the two independent searchers agreed on 11 specific terms for CVD/medication: cardiovascular disease, heart disease, stroke, heart attack, hypertension, hypercholesterolemia, hypercholesterolaemia, aspirin, blood pressure medication, cholesterol medication and statin, and two specific terms for decision aids: decision aid and decision support (see online supplementary appendix table A1 for search strategy). Additional terms were pilot tested before settling on the final list. A single CVD/medication term and a single decision aids term were combined for each search, resulting in 22 unique Google searches. The first 50 results for each unique Google search were exported (not including web advertisements), providing a pool of 1100 results to be title scanned for each searcher (2200 results in total). Scanning the first 100 search results for the first few searches found no additional resources after the first 50 results, so the cut-off of 50 was retained.
Supplemental material
The two independent searchers conducted this search as part of a Master of Public Health degree capstone unit during August to November 2016. In March 2017, an independent rater (CB) reconciled these search results at the earliest stage feasible (see figure 1), and the original searchers completed any missing ratings for the final data set. Only websites that were still working when the third independent rater reconciled the lists were included. Duplicates were considered either as identical web addresses or identical PDFs.
Preferred Reporting Items for Systematic Reviews and Meta-Analysis flow diagram of searcher results (searcher 1 PP, searcher 2 RZ).
Evaluation and data extraction
The two independent searchers rated the content of each decision aid using a validated tool to assess whether printed materials are suitable for people with low health literacy, the Patient Education Material Evaluation Tool for Print Materials (PEMAT-P).23 PEMAT-P includes two subscales: (1) understandability, which is a measure of how well a person is able to process and explain the key message of the material; and (2) actionability, which is a measure of how well a person is able to identify what to do based on the information presented. Items were rated on a binary scale (Yes/No) with some items provided a ‘Not Applicable’ option. Final understandability and actionability scores were calculated as a percentage of ‘Yes’ ratings for all items not including ‘not applicable’ ratings; higher percentages indicate better understandability or actionability. Intraclass correlations were calculated using SPSS V.25. For the two independent searchers, the intraclass correlation for final understandability scores was 0.51 and for actionability scores was 0.48. Conflicts for individual PEMAT-P items were therefore resolved by the third rater (MF, after discussion with CB) to finalise the PEMAT-P score for each individual decision aid. A threshold of 70% was used to determine whether the decision aid was understandable or actionable.21
Readability
Each decision aid’s readability was measured using the Gunning–Fog index, which is an index that estimates the formal years of (US) education an individual needs to understand the text.24 Scores range from 0 to 20 which corresponds to the US grade level that the text should be easily understood by, for example, a score of 6 would indicated the test should be easily understood by those educated to the sixth grade level in the US schooling system. The Flesch–Kincaid Reading Ease score was also calculated with higher scores indicating greater ease of comprehension. Scores range from 0 to 100 where a score of about 70–80 is the equivalent to school grade 7.25 The intraclass correlation between two independent ratings was high (Gunning–Fog Index was 0.91 and Flesch–Kincaid was 0.94) so the average of the two scores were used as the final index.
International Patient Decision Aids Standards checklist
The two independent researchers (PP and RZ) each completed a checklist based on version 3 of the International Patient Decision Aids Standards instrument (IPDASi) items, with discrepancies resolved by a third rater (MF, after discussion with CB). IPDASi V.3 has three domains: criteria used to be defined as a patient decision aid (seven items), criteria to lower risk of making a biased decision (nine items), other criteria indicating quality (thirteen items). Criteria used to be defined as a patient decision aid were rated on a yes/no scale and the other two domains were rated on a four-point Likert scale (1=strongly disagree to 4=strongly agree). In addition, two independent raters (PP and MF) used the IPDASi-Short Form (IPDASi-SF) to assess the same decision aids on a quantitative scale.14 Each item is rated on a four-point Likert scale (1=strongly disagree, 4=strongly agree). Total scores are calculated by the sum of all items and then converted into a value out of 100. Higher values indicate closer agreement with meeting the criteria of a decision aid. The IPDASi-SF is a shortened version of the third iteration of the IPDAS. The short form has demonstrated a 0.87 Pearson’s r correlation with the IPDASi 47-item version.14 In April 2018, the evaluation was repeated by two researchers (CB and MF) using IPDAS V.4, an updated version of IPDAS V.3 that reclassified the items into three domains with some revised wording and had collapsed some items into one. The new domains were qualifying, certification and quality. Qualifying criteria, if all met, identify the material as a decision aid. Certification criteria are those deemed essential to avoid harmful bias and all six criteria need to be met (ie, scored 3 or more) for the decision aid to be considered certified. Quality criteria on the other hand were items considered desirable but not essential to avoid harmful bias. All decision aids in the original evaluation were still publicly available at this time. Qualifying criteria are measured on a binary yes/no scale and certification and quality criteria are measured on a four-point Likert scale. To qualify as a decision aid, all six qualifying criteria must be met. To be certified as a decision aid, all six certifying criteria must score at least 3. Agreement for the qualifying items ranged from 64% to 100% and the average intraclass correlation coefficient between independent ratings for quality and certifying items were 0.16 and 0.34, respectively. Questions relating to screening tests were not used.
Patient and public involvement
The development of this research question was informed by the IPDAS, which has involved an extensive consultation process over many years to produce health decision-making tools that are useful and effective for patients. This study did not recruit patients or members of the public, so they were not specifically involved.
Results
The search of 15 known repositories and 2200 Google search results yielded 25 unique CVD prevention decision aids (see figure 1). Table 2 details the overall evaluation of the decision aids; and table 3 presents scores by individual decision aid. Online supplementary appendix table A2 provides the full IPDAS checklist item results and web archive URLs for the included decision aid webpages.
Evaluation for included decisions aids (n=25)
Individual evaluation of included decision aids (n=25)
Evaluation of the quality of the decision aids
For the version 3 IPDASi-SF (short form) scale, the Pearson’s r correlation between the two raters was 0.76 and the mean (SD) score was 64.56 (10.80) out of a maximum 100. For the version 3 IPDAS evaluation (see online supplementary appendix table A3 for individual scoring per item for each decision aid), none of the decision aids met all qualifying criteria to be defined as a patient decision aid and the median was 71% (5 out of 7 criteria met). None of the decision aids met all criteria to lower the risk of making a biased decision, and the median was 33%. None of the decision aids met all criteria to indicate quality, and the median was 85%. For the version 4 IPDAS evaluation (see online supplementary appendix table A4 for individual scoring per item for each decision aid), four decision aids met the criteria to qualify for a decision aid and the median was 83% (5 out of 6 criteria met) ranging from 2 to 6. One decision aid scored 3 or above on all six items to certified as a decision aid and the median was 50% (3 out of 6 items) ranging from 1 to 6. The median quality criteria that scored 3 or above was 30% (7 out of 23 criteria) ranging from 1 to 12 items.
A central component of decision aids is to present all options, risks and benefits in a balanced and unbiased way, with visual representation of numerical information. Nineteen decision aids provided only one intervention option (73%), whereas the remaining six provided 2–7 different options (27%). The presentation of harms versus benefits in icon arrays was highly variable. Icon arrays are graphic representations to show abstract probabilities as more concrete frequencies (eg, 2%=2 coloured dots out of 100 black dots), and are considered best practice for risk communication.26 Of the 12 decision aids that used icon arrays, 5 (42%) presented only benefits in icon arrays and 7 (58%) presented benefits and harms in icon arrays. Of the seven that presented benefits and harms in icon arrays, four (57%) combined benefits and harms in one icon array and three (43%) separated them. Of the four that combined benefits and harms in one icon array, one (25%) separated the benefits and harms and three (75%) overlapped the benefits and harms.
Evaluation of suitability for low health literacy populations
For the PEMAT-P evaluation, the decision aids generally scored well on understandability but lower on actionability. The average understandability score was 87% (SD=7.1%) and actionability was 61% (SD=24.6%). For readability, the average Gunning–Fog index was 9.9 (SD=1.9) and Flesch–Kincaid was 61.8 (SD=10.3), indicating that a US school grade of 9 is required to understand the information. The Pearson’s r correlation between understandability and readability was −0.60 for Gunning–Fog and 0.59 for Flesch–Kincaid.
Discussion
Principal findings
This review found 25 CVD prevention decision aids available to the public online, with the majority of them focussing on a single medication as the primary line of prevention against a potential future CVD event. Overall, the decision aids were very understandable but only had moderate actionability and a high readability level beyond the health literacy level of the general population. Most people would therefore have difficulty taking action based on the information in these decision aids, even though their primary purpose is to help the decision-making process. Of particular concern is that only 1 of the 25 decision aids met the most recent international criteria for certification, but the short form scores and quality checklist were reasonably high indicating decent quality overall. This means many decision aids would only require minor additions to reach certification standards; but the issues for low health literate patients would remain.
Strengths and weaknesses
The strengths of this study include a systematic review and evaluation process with multiple independent searchers/raters. The main limitation is the replicability of conducting a systematic search using online search engines like Google. The dynamic nature of the web with constant variation in website content and metadata means that no search is perfectly replicable even though the cache was cleared between search terms. However, the methods used are likely to have captured the most common and popular search results, since many duplicates were removed between the two searchers. Additional decision aids could have been found by a different searcher, search engine or geographical location and in other languages, which could produce different findings about the overall suitability for low health literate patients. However, this paper provides a list of known repositories of decision aids, including the primary source of IPDAS-assessed decision aids, to guide future researchers. This may improve the consistency of current and future findings. It also highlights the need for a central reputable location for decision aids that consumers could be referred to rather than search for their own.
Comparison to other research
The methods and findings of this study are comparable to two other environmental scans of prenatal decision aids, which also identified issues with presenting unbiased information about both benefits and harms.27 28 Other studies using PEMAT-P for patient education materials have found poor results (CVD decision aids in this study: 87% and 61%; CVD risk calculators: 64% and 19%; online heart failure websites: 56% and 35%; printed lifestyle information for chronic kidney disease: 52% and 37%; for PEMAT-P understandability and actionability scores, respectively).29–31 The IPDAS criteria for decision aid development may have led to higher quality patient education materials, but there is still room for improvement on actionability and readability levels.
Implications and future research
CVD prevention decision aids could be improved to better meet quality criteria and make them suitable for low health literacy populations. In particular, this needs to include: (1) a basic explanation of what CVD is and what CVD risk means, since the mechanism for how this leads to events like heart attack and stroke was rarely explained; (2) inclusion of both medication and lifestyle intervention options to enable a fully informed choice, as there tended to be a focus on single medication options; (3) balanced presentation of risks and benefits using visual communication aids such as icon arrays, since few decision aids used best practice risk communication strategies in an equal way for both the benefit and harm of options (eg, reduced chance of CVD event versus chance of side effects); and (4) more support for what actions to take based on the decision made.26
Several IPDAS items required substantial discussion between raters to decide on the best way to apply them consistently, indicating that further work is needed to provide a reliable tool for certifying decision aids. For example, decision aids that compared a single medication option versus doing nothing were easier to rate highly on balanced presentation than those with multiple options, even though the latter enables a more fully informed choice. In addition, the items did not cover: (1) health literate design issues (eg, use of white space, images that are consistent with text and clear direction for next steps); (2) assessment of the accuracy of the information provided (eg, whether the risks and benefits presented were based on the latest systematic review, if available); (3) ease of access for the intended audience, particularly for low health literacy populations; or (4) how effective the decision aid was, even when an evaluation had been conducted. Work on these issues is ongoing, with particular attention in the US following legislative changes to certify decision aids.32 The IPDAS criteria were more reliable when used by raters who were more familiar with decision aids at a later stage of the project, suggesting a need for structured training in using IPDAS rating scales.
Conclusion
To meet the needs of the general population who are likely to have low health literacy, CVD prevention decision aids need to improve actionability and readability, and better address basic certifying criteria such as explaining CVD and ensure that all options are presented in an unbiased way with visual support for benefits and harms.
Acknowledgments
We thank Pauline Ledermann for assistance with calculating the readability scores.
References
Footnotes
Contributors CB contributed to the conceptualisation, methodology, data analysis and interpretation and drafting the manuscript. PP and RZ contributed to the methodology, data collection and revising the manuscript. MAF contributed to the data analysis and revising the manuscript. LT contributed to the conceptualisation, methodology and revising the manuscript.
Funding The study was supported by grants from the National Heart Foundation of Australia (Vanguard Grant 101326), the Royal Australian College of General Practitioners and Therapeutic Guidelines (TGL/RACGP Research Grant TGL16b).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Website URLs containing the CVD Decision Aids are available in Supplementary Files. Descriptive and evaluative data are also available within Supplementary Files.
Patient consent for publication Not required.