Objective Health literacy (HL) is the degree of individuals’ capacity to access, understand, appraise and apply health information and services required to make appropriate health decisions. This study aimed to establish a predictive algorithm for identifying community-dwelling older adults with a high risk of limited HL.
Design A cross-sectional study.
Setting Four communities in northern, central and southern Taiwan.
Participants A total of 648 older adults were included. Moreover, 85% of the core data set was used to generate the prediction model for the scoring algorithm, and 15% was used to test the fitness of the model.
Primary and secondary outcome measures Pearson’s χ2 test and multiple logistic regression were used to identify the significant factors associated with the HL level. An optimal cut-off point for the scoring algorithm was identified on the basis of the maximum sensitivity and specificity.
Results A total of 350 (54.6%) patients were classified as having limited HL. We identified 24 variables that could significantly differentiate between sufficient and limited HL. Eight factors that could significantly predict limited HL were identified as follows: a socioenvironmental determinant (ie, dominant spoken dialect), a health service use factor (ie, having family doctors), a health cost factor (ie, self-paid vaccination), a heath behaviour factor (ie, searching online health information), two health outcomes (ie, difficulty in performing activities of daily living and requiring assistance while visiting doctors), a participation factor (ie, attending health classes) and an empowerment factor (ie, self-management during illness). The scoring algorithm yielded an area under the curve of 0.71, and an optimal cut-off value of 5 represented moderate sensitivity (62.0%) and satisfactory specificity (76.2%).
Conclusion This simple scoring algorithm can efficiently and effectively identify community-dwelling older adults with a high risk of limited HL.
- public health
- health informatics
- health services administration & management
Data availability statement
Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Our health literacy (HL) prediction model comprising personal, contextual, and socioenvironmental factors can be used to formulate health policies for older adults.
This scoring algorithm not only helps clinicians assess and identify the HL level among older adults but also assists researchers to establish early identification for older adults with limited HL.
Prospective trials should examine the implementation and utility of this simple scoring algorithm for the early detection of older adults with a high risk of limited HL in the community.
Health literacy (HL), which refers to people’s knowledge and competency required to meet the complex demands of health and healthcare (HC), is crucial and increasingly being recognised in modern society. However, no consensus exists regarding the definition, dimensions and conceptual models of HL. The European Health Literacy Consortium has proposed a conceptual model integrating medical and public health perspectives of HL; the model can be used to determine the different dimensions of HL across different clinical and community settings. They defined HL as the ability to access, understand, appraise and apply health information or services for obtaining appropriate HC and for disease prevention (DP) and health promotion (HP).1 The HL measure has been indicated as the sixth vital sign along with temperature, pulse, respiration, blood pressure and pain level. The importance of HL should be emphasised, so that clinicians and public health workers can enable early and precise access to intervention strategies.2 3 However, the construct of HL is complex and dynamic and encompasses many aspects of individuals’ use of health information and the HC system. Therefore, the European Health Literacy Survey (HLS-EU) Consortium has proposed a theoretical model that integrates medical and public health perspectives of HL and that accounts for its various antecedents including personal, contextual, social and environmental determinants. Therefore, the level of HL indicates health service use, health costs, health behaviour, health outcomes, participation and empowerment among individuals.1
Older adults generally have more chronic illnesses and less formal education than their younger counterparts.4 Moreover, older adults experience unique problems related to physical and cognitive functioning that cause difficulty in finding accurate health information and using appropriate HC services.5 Several national surveys have reported that more than half of older adults have limited HL.6–8 Studies have reported that the level of HL was significantly lower in older people than in younger people.9–11 Moreover, limited HL results in poor health outcomes12 and health behaviours,13 increased HC expenditure14 and health service usage,15 and inadequate empowerment16 and participation.17 Therefore, early and accurate prediction of limited HL among older adults is essential to implement prompt and appropriate HC strategies.
Various HL measures have been developed for older adults.18 A US national survey examined the agreement (sensitivity and specificity) and discrimination (c-statistic) of functional HL derived from common sociodemographic data; this functional HL can serve as a readily calculated HL proxy score.19 A study developed a predictive model by performing multiple regression and validating its efficiency in identifying older people with the highest risk of inadequate functional HL.20 However, most of these studies lack underlying theoretical basis and fail to sufficiently cover the comprehensive dimensions of HL across different clinical environments. Furthermore, to the best of our knowledge, no study has developed a prediction model of self-reported general HL for the early and accurate identification of HL levels in older adults. Therefore, developing a simple algorithm that can be applied in clinical settings to accurately identify older adults with a high risk of limited HL is essential. This study conducted a survey among community-dwelling older adults to identify factors predicting limited HL and constructed an optimal scoring algorithm for predicting HL.
Patient and public involvement
Figure 1 presents information on participant selection, data set division and analysis procedures. In this cross-sectional study, by adopting the convenience sampling method, we recruited eligible community participants aged ≥65 years from six senior service centres and three health check-up clinics in northern, central and southern Taiwan between June and September 2018. The individuals were excluded from the study if they were found to be cognitively impaired based on the Mini-Cog instrument screening. The survey questionnaire items and self-reported outcome measures were developed following the conclusions of three consensus meetings among multidisciplinary experts of health policy, HL, health education, social welfare, gerontology and geriatric medicine. We either interviewed participants to obtain responses for the questionnaire or asked them to complete a self-administered HL-related questionnaire.21 Trained interviewers explained the objective of the study to respondents before they expressed their willingness to participate in the study. The survey was anonymous, and respondents were allowed to suspend the interview at any time. There was no difference in expected medical treatment whether the individual join or turn down our study. The participants were allowed to withdraw anytime or complete the interview in different time schedule if they felt tired or stressful. The study results were disseminated to the public through a conference presentation hosted by the Taiwan National Health Research Institutes at the end of the study. The minimum sample size was decided from the 10:1 ratio of participants to variables by Everitt’s recommendations.22
After signing informed consent forms for participation in the study, the participants were either interviewed or asked to complete a self-administered questionnaire including questions on 52 potential predictors including personal, situational, and socioenvironmental determinants and factors related to health service use, health costs, health behaviour, health outcomes, participation and empowerment based on the theoretical model of the HLS-EU Consortium.23
The 47-item HLS-EU Questionnaire (HLS-EU-Q), developed by the HLS-EU Consortium, was used to assess the comprehensive HL of the study participants. The HLS-EU-Q measures four HL competencies (access, understand, appraise and apply health information) required under three health domains: HC (16 items), DP (15 items) and HP (16 items). Each item assesses the self-perceived difficulty in performing selected health-related tasks on a 4-point scale ranging from ‘very easy’ (4) to ‘very difficult’ (1). Higher scores indicate a higher level of HL. For ease of comparison, the score of each domain (ie, HC, DP and HP) was linearly transformed to a score between 0 and 50 by using a scale adopted in the HLS-EU, which h has been validated to have satisfactory psychometric properties.24 On the basis of the scores, HL was divided into four categories: inadequate (0–25), problematic (26–33), sufficient (34–42) and excellent (43–50).25 26 We dichotomised HL into ‘sufficient’ and ‘limited’ based on a cut-off value of 34, as defined by the HLS-EU.25
The dichotomised outcome is defined using the HL level as follows:
To develop a scoring algorithm for predicting a low level of HL, the core data set was divided using stratified random sampling without the replacement method as follows: 85% of the core data set was categorised into the training data set, which was used for training the prediction model to create the scoring rule, and 15% of the data set was categorised as the validation data set, which was used for validating the scoring algorithm (figure 1).27 By Everitt’s recommendations, a minimum of 520 participants should be included in the training data.22 Pearson’s χ2 test was used initially to reduce variables and ensure that variables of crucial dimensions were retained. In addition, Pearson’s χ2 test was used to evaluate the association of the HL level with each of the 52 predictors included in the self-administered questionnaire.28 If the variables were not answered, we would not include the record in the analysis. To select the most relevant predictors, variables with a p<0.1 were included in multiple logistic regressions. Multiple logistic regressions with forward selection were used to examine relationships between limited HL and potential predictors classified into the domains of personal determinants, situational determinants, socioenvironmental determinants, health service use, health costs, health behaviour, health outcomes, participation and empowerment. Furthermore, the potential predictors with a p<0.05 were identified from the multiple logistic regression model.29 30 The formula used for multiple regression was as follows:
where p denotes the probability of limited HL in older adults, α is the intercept of multiple regression, and β is the slope of the specified predictor from 1st, 2nd, …, ith. In multiple regression, the OR was estimated using exp(βi ) to present relationships between dichotomised HL and predictors, and the estimated ORs were applied to calculate the clinical score for predicting health risk.31 32 To develop a simple algorithm, a previous study developed a new score based on the summation of significant predictors identified from multiple logistic regression. Significant predictors that were positively associated with limited HL (OR >1) were assigned a value of +1, whereas those that were negatively associated with limited HL (OR <1) were assigned a value of −1.31
Separately, 15% of the participants were included to validate the proposed scoring algorithm. On the basis of the algorithm obtained from the training data set, the total score was calculated for each older adult in the test dataset. The overall accuracy of prediction of limited HL using this score was evaluated by calculating sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).33 The model fit was assessed on the basis of McFadden’s pseudo R-square (measuring the reduction in the maximised log-likelihood from the intercept only model) and c-statistic (area under (AUC) the receiver operating characteristic (ROC) curve) values.34 35 A two-sided 95% CI for the AUC was used to denote uncertainty,36 and a p>0.05 in the Hosmer-Lemeshow test was used to indicate algorithm performance. As a graphical illustration of sensitivity–specificity trade-offs at each cut-off, the ROC curve demonstrated the degree to which the HL score could discriminate the level of HL, and the best cut-off value corresponded to a perfect scenario (100% sensitivity and 100% specificity). However, in practice, the optimal cut-off value should maintain a balance between sensitivity and specificity,37 which refers to the number of predictors calculated using the optimal classification threshold in our simple scoring algorithm. All statistical analyses were performed using SAS V.9.4 (SAS Institute).
A total of 648 older adults were recruited. We excluded 12 patients who were found to have mild cognitive impairment by using the Mini-Cog instrument. Figure 1 presents age-specific HL levels. Nearly half (41.6%–46.1%) of the participants had problematic HL. A large proportion of the participants (72.6%) aged ≥81 years had limited HL. Table 1 lists the sociodemographic characteristics of the participants in the training data set (n=552/648) and test data set (n=96/648). Sex, age, educational level, marital status, occupation and monthly income showed similar distribution between the training and test data sets.
In the training data set (n=552/648), of the 52 variables included in the original questionnaire (online supplemental appendix 1), 24 (ie, 5 personal determinants, 2 situational determinants, 2 socioenvironmental determinants, 1 factor related to health service use, 1 factor related to health costs, 3 factors related to health behavior, 6 factors related to health outcomes, 1 factor related to participation, and 3 factors related to empowerment of HL) associated with the HL level (p<0.1) were identified using Pearson’s χ2 test (table 2).
These 24 factors including personal determinants, situational determinants, socioenvironmental determinants, health service use, health costs, health behaviours, health outcomes, participation and empowerment were entered in multiple logistic regression, as shown in table 2. Limited HL was significantly associated with less health service use or self-paid vaccination in preventive medicine, such as not having a family doctor (adjusted OR (AOR) 1.46, 95% CI 1.00 to 2.14) or not receiving self-paid vaccination (AOR 1.28, 95% CI 1.05 to 1.57). In addition, older adults with poorer health behaviour regarding the items related to searching online health information (AOR 1.24, 95% CI 1.01 to 1.52), less social participation while attending health classes (AOR 1.38, 95% CI 1.12 to 1.70) and poor empowerment of self-management during illness (AOR 1.28, 95% CI 1.05 to 1.57) had limited HL. Moreover, poorer health outcomes in older adults, such as having difficulty in performing daily living activities (AOR 1.58, 95% CI 1.00 to 2.52) and requiring assistance while seeing a doctor (AOR 1.70, 95% CI 1.16 to 2.48), may be associated with limited HL. In particular, older adults whose dominant dialect was a dialect other than Mandarin had a higher odds of having limited HL (AOR 2.13, 95% CI 1.72 to 2.64; table 3). The indicators of model performance revealed a reasonably good fit in the training data set, including an acceptable pseudo R2 value of 0.27 and nonsignificance (p=0.923) in the Hosmer-Lemeshow test.
Eight variables were selected using the forward stepwise selection method in the multivariate model and were used to calculate the HL score. Cross-validation was performed by including 15% participants (n=96/648), of whom 92 provided all responses for the measurement of prediction accuracy. The overall accuracy in predicting limited HL with various cut-off points is listed in table 4. The optimal cut-off point was considered to be 5, yielding sensitivity and specificity of 62.0% and 76.2%, respectively. By using a score of 5 of 8 to predict the limited HL level, the obtained PPV and NPV were 75.6% and 62.7%, respectively. Figure 2 presents the predictive ability of the scoring algorithm among older adults in the test data set. Figure 3 shows the indicators of model performance revealed reasonably satisfactory performance with an AUC of 0.71 (95% CI 0.61 to 0.81).
Discussion and conclusion
To the best of our knowledge, this is the first study to develop a model for predicting the HL level of community-dwelling older adults. This algorithm-based model was well calibrated by integrating HL-related factors from the model of the HLS-EU Consortium and was useful for HL risk prediction among older adults. In addition, this model exhibited a moderate ability for discriminating between older adults with sufficient HL and limited HL.
In this study, we integrated variables associated with both medical and public health perspectives from the aforementioned HL model of the HLS-EU Consortium and proposed a simple scoring algorithm. The scoring system dichotomises older adults into high-risk (cutoff ≥5) and low-risk (cutoff <5) populations to maximise the sensitivity and specificity of the prediction of limited HL. Therefore, early identification using five or more factors from a total of eight significant HL predictors among older adults can be an effective strategy that can be implemented in future clinical practice. On the basis of proposed cut-off points, among the 92 older adults included in the test data set, 63 (68.5%) with a cut-off value of ≥5 were recommended to undergo further HL intervention, although only 31 (62.0%) actually had limited HL, resulting in a PPV of 75.6%. Given the importance of early identification and strategy provision for community-dwelling older adults with a high risk of limited HL, the proposed scoring algorithm can be considered useful in community practice.
A previous study proposed using 80%–90% of data to build a prediction model and 10%–20% of data to validate the model.38 In our study, we followed two previous studies and accordingly divided our data into a training data set (by using 85% of the data) and a test data set (by using 15% of the data).39 40 In addition, we randomly selected 10% (n=64) and 20% (n=128) of the data to examine the reliability of dividing our data into 85% and 15% in our study. Similar validation results were obtained when we included 10% (n=64), 15% (n=96), and 20% (n=128) of the participants. The overall accuracy was 75.0% (48/64), 68.5% (63/96) and 64.8% (83/128) for 10%, 15%, and 20% of the participants, respectively, indicating that the score of 5 was the optimal cut-off point.
This conceptual framework integrating medical and public health perspectives developed by the HLS-EU Consortium is suitable for determining the most relevant determinants of the HL level in older adults. Eight predictors were identified to be significantly associated with the HL level: one socioenvironmental determinant (ie, dominant spoken dialect) and seven HL-related factors including health services (ie, having a family doctor), health cost (ie, self-paid pneumonia vaccination), health behaviours (ie, searching online health information), health outcomes (ie, assistance while visiting a doctor and performing activities of daily living), participation (ie, attending health classes) and empowerment (ie, self-management during illness). The results for the seven identified predictors of HL-related factors are consistent with those of previous studies, for example, having a family doctor,8 costs for self-paid vaccination,41 searching online health information,10 functional status such as difficulty in performing daily activities and assistance while visiting doctors,41 42 participation in health classes43 and self-efficacy in disease management.44 However, our study result revealed that personal and situational factors did not affect HL among older adults. Previous studies have reported that personal determinants, namely age, educational level and working status, as well as situational and environmental determinants, including marriage and residential area, were significantly associated with the HL level.15 19 20 45 This difference might be because personal and situational determinants were the proximal factors of HL that are affected and displaced by a more distal and upstream factor (societal and environmental determinants).46
Some studies have developed a weighted score based on multiple regression to determine the HL level. These studies have used originally estimated beta values or have transformed the beta values into a score of multivariate regression.19 20 47 For example, Miller et al used 20 variables to establish a nonequal weighted HL scoring system for older adults, and their scoring system correctly classified the HL of 73.2% of the participants.20 Our simple scoring algorithm referred to associations (positive and negative) between the HL level and risk factors with equal weighs (+1 or −1) to identify individuals with limited HL.31 Our algorithm is rapid and straightforward to use and can be widely implemented for determining the HL of older adults in clinical practice. Our results from the eight questions (correctly classified the HL of 65.8% of the participants) can help rapidly determine the target intervention tailored for a specific older adult.
Users can rapidly predict limited HL through an evaluation of their HL-related personal, situational and environmental factors as well as health behaviour and outcomes by using our algorithm and thus identify community-dwelling older adults who may require further health assistance. Hospitalisation and mortality due to poor HL in older adults can be prevented through early identification and intervention. Therefore, this assessment tool should be promptly extended to broader communities.
Our study has some limitations. First, in this cross-sectional study, participants were recruited from northern, central and southern Taiwan by using the convenience sampling method. Therefore, potential selection bias might exist. In addition, although we included comprehensive HL-related factors on the basis of a conceptual model of HL, some crucial situational or socioenvironmental factors related to the situational demand or organisational environment might be difficult to measure because information on the organisational environment was not available in our individual interview study. Second, this study relied on the 47-item HLS-EU-Q self-reported questionnaire for determining HL. Additional objective HL assessments might be required to recognise functional HL for preventing the potential for outcome misclassification bias. Third, the high prevalence rate of limited HL (54.9%) in our sample may affect the prediction ability (ie, PPV) of this algorithm when applied in other populations. Therefore, when this algorithm is applied to a population with a lower prevalence of limited HL, older adults with positive results of limited HL may in fact have sufficient HL. Furthermore, we excluded older adults who could not pass the Mini-Cog screening or follow instructions to complete the assessment. Therefore, our model may not be generalisable to the entire population of older adults. Thus, this model is not recommended to be used in individuals with cognitive impairment or dementia, who may have difficulty understanding instructions. Larger population studies with prospective longer-term outcome measures are necessary to validate our study findings.
We proposed a simple clinical scoring algorithm with substantial sensitivity and satisfactory specificity to assess the risk of limited HL among community-dwelling older adults.
This scoring algorithm can not only help clinicians to assess and identify the HL level among older adults but also assist researchers to establish intervention strategies for older adults with limited HL. However, for further population-based application for the early detection of older adults with a high risk of limited HL, prospective trials should examine the implementation and utility of this algorithm in the community.
Data availability statement
Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information.
Patient consent for publication
This study was approved by Taipei Medical University-Joint Institutional Review Board (N201804046) and National Taiwan University Hospital (201804057RIND).
We thank Wu-Shou Chang from the Asian Health Literacy Association for authorising the use of the Mandarin Chinese version of the HLS-EU-Q47.
H-NS and C-YL contributed equally.
Contributors W-HH, Y-MC and KNK designed the study. C-TS, D-SH and D-CC acquired of data. H-NS, M-JC and H-WT analysed the data. W-HH, H-NS, and H-WT drafted the manuscript. C-TS, D-SH, D-CC, KNK and C-YL reviewed and revised the manuscript. The author (s) read and approved the final manuscript. C-YL is the overall guarantor.
Funding This research was supported by research grants from the Taiwan National Health Research Institutes (MOHW107-TDU-M-212-133001-107-FR-04), Chi Mei Medical Center (108CM-TMU-01), and Taipei Meidcal University Hospital (109TMUH-H-01).
Disclaimer The funder was not involved in study design, and will not be involved in the collection, analysis or interpretation of data.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.