Article Text

PDF

What is the test-retest reliability of the Malay version of the Hypertension Self-Care Profile self efficacy assessment tool? A validation study in primary care
  1. Kai Cong Seow1,
  2. Diana Mohamed Yusoff2,
  3. Yi Ling Eileen Koh2,
  4. Ngiap Chuan Tan2,3
  1. 1 National University Singapore Yong Loo Lin School of Medicine, Singapore, Singapore
  2. 2 SingHealth Polyclinics, Singapore, Singapore
  3. 3 Duke NUS Medical School, Singapore, Singapore
  1. Correspondence to Dr Ngiap Chuan Tan; tan.ngiap.chuan{at}singhealth.com.sg

Abstract

Objectives Self-efficacy and self-care measures are key attributes to optimal control of essential hypertension. Self-efficacy can be measured by the Hypertension Self-Care Profile (HTN-SCP) tool but its utility is dependent on the literacy and understanding of the subjects. A Malay version of the HTN-SCP Tool was developed to assess self-efficacy of Malay-literate patients with hypertension in the multi-ethnic Asian population in Singapore. The study aimed to determine the test-retest reliability of this tool which has been translated in Malay language.

Methods and Material 145 Malay-literate patients, aged 41-70 years, with essential hypertension were recruited in a polyclinic (primary care clinic) in Singapore. Forty-three percent of them completed both the first and second HTN SCP tool online, with a period of two weeks in between. The Cronbach’s alpha and Intra-class correlation coefficient (ICC) were computed to assess its test-retest reliability and internal consistency.

Results The Cronbach’s alpha/ICC for “Behavior” (0.851/0.664)), “Motivation” (0.928/0.655) and “Self-efficacy” (0.945/0.682) domains showed high internal consistency, fair to good reliability and stability. No floor or ceiling effect was found for the "behavior" and "motivation" domains. However, the borderline ceiling effect (15.2) for "self-efficacy" suggested limited discriminating power of the tool for patients with high self-efficacy. Positive association was shown between the HTN-SCP score and reported self-care measures but it was not statistically significant.

Conclusion Overall, the translated HTN-SCP tool showed satisfactory test-retest reliability and internal consistency amongst the Malay-literate study population. Further research is needed for its application in general practice to identify patients with low self-efficacy for possible intervention.

  • hypertension
  • self-care
  • tool
  • malay

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Strengths and limitations of the study

  • This study used a web-based method in administrating the test, showing its potential application in clinical practice.

  • Without incentive, the web-based method resulted in significant dropouts in the re-test segment of the study.

  • Reliability assessment of the ‘self-efficacy’, ‘behaviour’ and ‘motivation’ domains may be limited by ceiling and floor effects.

  • The study excluded subjects who were not internet-savvy and those who were not proficient in written Malay language.

Introduction

Hypertension is often the most prevalent risk factor for vascular diseases in developed and developing countries.1 Maintaining optimal blood pressure among patients with hypertension is key to avert vascular complications such as stroke and cardiac diseases.2However, achieving stable blood pressure (BP) control requires these patients to undertake multiple measures consistently, such as salt and fat restriction in diet, regular physical exercises, reduction in consumption of alcohol, self-monitoring of BP and weight and adherence to prescribed BP lowering medications.3 4

The capacity to conduct these self-care activities is related to an individual’s self-efficacy.5 6 Thus assessing self-efficacy of patients with hypertension provides the healthcare provider an insight into their vascular risk status, and decision support in designing individualised care plan. Questionnaire-based tools have been developed to facilitate the multi-faceted assessment of self-efficacy. One example is the English-based Hypertension Self-Care Profile (HTN-SCP, also known as HBP SCP) questionnaire developed by Han HR et al, which covers three key domains of behaviour, motivation and self-efficacy.7 Her team had used two validated theoretical approaches, Orem’s self-care model8 and Motivational Interviewing (MI)9 to develop the questionnaire. The Orem’s model depicts the enablement of individuals to embark on deliberate actions such as taking medication and physical activity as indicators of self-care.8Using similar constructs such as those from the Social Cognitive Theory and Health Belief Model, MI indicates a commitment to change (ie, motivation) and reflects the development of confidence for behaviour change.9 Self-efficacy predicts medication adherence, physical activity, diet and weight control.7 Han HR et al reported high reliability estimates and strong evidence of validity of the HBP SCP when the study was carried out in an inner city American population. Their results suggest that the questionnaire can be used to assess and identify gaps in self-care behaviour, motivation, and self-efficacy in patients with hypertension.

The test-retest reliability assessment of this English-based questionnaire had also been performed in Singapore, where English is the main language of communication of its population on the developed island state.10 Among the 5.61 million multi-ethnic Asian population in Singapore, 13.4% are of Malay ethnicity.11 According to the Singapore 2010 Population Census, 37% of the local Malay population had up to ‘below secondary’ education or equivalent of grade six educational level.12 The questions of the original version of HTN-SCP were developed to pitch at grade six level.7 Uncertainty arises if over a third of the overall Malay population could understand the English-based questionnaire. Moreover, hypertension is more prevalent among the more senior residents. Based on the same census, only 5.5% of Malays of age 55 years and older, and 4.4% of Malays with ‘below secondary education’ are proficient and use English at home.12 Hence, approximately one in twenty Malay patients with hypertension would have potential difficulties in using the locally validated HTN-SCP tool.

Singapore is also facing a rapidly ageing population and the prevalence of hypertension is expected to rise significantly by the next decade. A Malay version of the HTN-SCP tool is postulated to ease the assessment of self-efficacy among the older Malay patients with hypertension. To ensure that they understand the content, the translated tool needs to be contextualised to the local setting and tested on stability and reliability among the local Malay-literate patients with hypertension. The tool will allow its application not only in local Malay patients with hypertension, but will also cater to the larger Malay-literate population in Malaysia and other parts of South-East Asia.

Methods

Aim

The study aimed to determine the test-retest reliability of the Malay version of the HTN-SCP among Malay hypertensive patients who were managed in a public primary care clinic in Singapore.

Development of the malay version of the HTN SCP tool

The original English-based HTN-SCP tool has been enhanced in selected items to contextualise to the dietary habits of the local Asian patients with hypertension.7 The enhanced version was subsequently validated in an earlier study, which showed satisfactory test-retest reliability, as reflected in the internal consistency coefficient (ICC) (0.671, 0.762, and 0.720 for the three domains) and Cronbach-alpha indicators (0.857, 0.948, and 0.931 respectively). Next, the investigators engaged a commercial agency, whose two independent professional and certified bi-linguists in English and Malay carried out the forward and backward translation of the content respectively. The draft Malay version of the HTN-SCP was then pilot-tested among independent bilingual staff within the institution department of research and nurses at the study site. Minor amendments were carried out to the tool after this preliminary assessment. The Malay version (HTN-SCP-M) was finalised after approval by the investigator team members, who are also bilingual in English and Malay.

Subjects

The target subjects were patients of Malay ethnicity with essential hypertension based on their diagnosis code in the electronic medical records at the study site. They were either Singaporean citizens or Permanent Residents.

Inclusion criteria

Subjects must be between the age of 41 and 70 years on enrolment. They were treated with blood pressure lowering medication for their essential hypertension for at least 1 year.

Subjects were screened by the investigators and research assistants to be able to read and understand the written language as a minimal standard of Malay proficiency. They had to be internet-savvy in order to access the web-based administration of the tool.

Exclusion criteria

Subjects who lacked proficiency in the written Malay language and access to internet, were unwilling to execute the test, or were screened to have cognitive, visual or auditory impairment were excluded from the study.

Recruitment of subjects at study site

The study was conducted from May 2016 to Dec 2016 in a typical public primary care clinic located in an estate in north-eastern Singapore serving a population of about 140 000 multi-ethnic Asian population (2015). The investigators and research assistants identified potential subjects by their Malay names or attire and approached them before or after their medical consultation at the study site. The subjects were next screened to confirm their ethnicity and fulfilment of their eligibility criteria. They were provided with information on the study procedure in Malay and their doubts were clarified before obtaining their written informed consent according to stipulations by the institution review board.

Ethics approval

Ethics approval was granted from the Centralised Institutional Review Board (CIRB) for this study. Ref no: 2016/2332.

Administration of the tool

To ensure anonymity, each subject was given a unique study identification number after enrolment, which was used to log in to a free web-based online tool (Qualtrics) for both the test and retest. They were instructed on the step-by-step procedure to administer the web-based HTN-SCP-M during the test. The web design ensured mandatory filling of the response to each question before the subject was allowed to progress to the next question. This format eliminated any missing data.

After the completion of the test, the subjects were provided with information sheet, instructing them to access the online platform for the retest using their personal electronic devices (smart mobile phone, tablet, laptop or computers) on a specified date 2 weeks later in order to complete the study. The investigators would send reminders via calls, phone messages and electronic mails to defaulters after the stipulated retest dates. Those that failed to complete the retest beyond a 1 week grace period were excluded from the reliability analysis. Participation by the subjects were voluntary, with no incentive provided for their study participation.

Sample size calculation

The COSMIN checklist recommended a good sample of between 50 to 99 study subjects for assessing their test-retest reliability.14 In view of a significant drop-out rate for the retest using a web-based approach from an earlier study by the principal investigator, the target recruitment size was increased to 150 to account for attrition.

Data management and statistical analysis

The demographics of study participants were reported. Differences between the overall HTN-SCP-M scores and recommended self-management activities were assessed using the independent t-test.

Ceiling and floor effects were derived from the percentage of respondents with the highest and lowest scale scores. Ceiling and floor effects were considered present when it is higher than 15%.15 Internal consistency indicates the extent to which the items in a subscale are correlated, in order to evaluate homogeneity using Cronbach’s Alpha statistic.16 A value of 0.7 was considered acceptable while above 0.8 to 0.9 was considered good to excellent internal consistency.17

The agreement between repeated measurements (Test-retest) was evaluated through the use of the intra-class correlation coefficient (ICC). An ICC coefficient of>=0.75 was considered as evidence of measurement stability. ICC between 0.4 and 0.75 indicates fair to good reliability, and ICC <0.4 indicates poor reliability.18 This analysis excluded those who dropped out from the study for the retest 2 weeks after their recruitment.

A p-value of less than 0.05 was considered as statistically significant. All analyses were performed using the IBM SPSS Statistics for Windows, Version 23.0, IBM Corp, released 2013.

Results

A total of 768 subjects were approached by the investigators and research assistants, of which 204 patients satisfied the eligibility criteria. With 59 refusal of consent, 145 of them were recruited for the test segment of the study, resulting in a response rate of 71%. Subsequently, 63 (43%) of the recruited subjects completed the retest of the HTN-SCP-M on-line. Reasons for drop-out include patients being unable to be contacted for reminder, patients' refusal to complete even after reminder, and patients’ completion of the retest after an extended period beyond two to 3 weeks.

The demographics of the respondents were presented in table 1. There were more females (66.9%) than males (33.1), and the majority of them (66.2%) had up to 10 years of education. Among the subjects, 19.3% of them had hypertension only, while the majority had other co-morbidities, such as dyslipidemia (59.5%) and type two diabetes mellitus (34.6%).

Table 1

Baseline characteristics of study population

Table 2 reveals self-care measures reported by the subjects, which are relevant to hypertension management. Majority of them (71.7%) measured their BP at home but only 57.9% documented these measurements regularly. The HTN-SCP-M scores of the subjects who recorded their BP readings were significantly higher than those without BP documentation. The same finding was observed in those who measured their weight regularly.

Table 2

Association between Self-Reported Self-Care Activities and the Overall Mean Scores of the Hypertension Self-Care Profile (HTN-SCP)

The HTN-SCP-M scores of those who performed self-care measures, such as home BP measurements, keeping a food diary, documentation of weight and using health-related mobile phone application, were higher than those who without such activities, although these differences did not attain statistical significance (table 2).

Floor and ceiling effect

Table 3 The mean score for the ‘Self-Efficacy’ domain within the HTN-SCP-M was highest, compared with those in the ‘Behaviour’ and ‘Motivation’ domains. The ‘self-efficacy’ domain presented a borderline ceiling effect of 15.2% (table 3). The other two domains show minimal ceiling effect (<15%).

Table 3

Reliability indices of the HTN-SCP-M tool

Internal consistency

All three domains showed excellent internal consistencies: the Cronbach’s alpha for ‘Behaviour’ was 0.851, 0.928 for ‘Motivation’ and 0.945 for ‘Self-efficacy’ domains respectively.

Test-Retest reliability

The Intra-class Correlation Coefficient (ICC) for all domains ranged from 0.655 to 0.682, which suggested fair to good reliability and stability.

Discussion

The study demonstrated efforts to validate a tool in a specific Asian language, which has been originally developed in English in North America.7 It followed a successful test-retest reliability assessment of the HTN-SCP tool in English, which is contextualised to the multi-ethnic Asian community in a primary care setting in Singapore.10 The over-arching aim is to expand the scope of the tool to appraise the self-efficacy of patients with hypertension in diverse populations. In spite of incorporating innovative approach in this study riding on the rapidly and extensively internet-connected community, the validation framework is aligned to the STROBE guidelines.19

Overall, the Malay version of the HTN-SCP tool (HTN-SCP-M) has attained satisfactory test-retest reliability and internal consistency among the study population based on the Cronbach’s alpha and ICC indices. While the flooring effect was minimal, the results showed borderline ceiling effect for the self-efficacy domain. It suggests potential limitation to the discriminating power of the tool among those with high scores in the self-efficacy domain. However, this subset of the study population with higher capacity for self-efficacy to control their blood pressure is not the target patients of concern to clinicians. Attention should focus on the group of patients with lower self-efficacy capacity, reflected by the lower HTN-SCP-M scores, who are at risk of poor blood pressure control.

The next step will be to test the application of the HTN-SCP-M on the local Malay-literate patients with hypertension to determine the correlation between its scores and self-care activities. The total aggregated score in this study was significantly associated with patients’ self-reporting of their documentation of their home blood pressure monitoring and weight measurement. For other self-care measures such as keeping a food diary, the positive correlation between the total scores and reported self-care measures was not statistically significant.

Nonetheless, the sample size of the study population was not computed to determine the differences in self-care measures based on the score. Further adequately powered study with larger number of subjects and incorporating reliable and objective assessment of self-care measures is needed to assess if the tool can be used to stratify Malay-literate patients into varying capacities for health behaviour, motivation and self-efficacy.

The study has its strength and limitations. It excluded patients who lacked access or were not competent internet users. The use of a web-based approach in implementing the test-retest reliability evaluation enabled the patients to self-administer the tool remotely using their smart-phone, tablets, computer and laptop at their preferred timing and venue. Despite the convenience of this method, the uptake of the retest segment was suboptimal, even with the use of reminders by the investigators. In retrospect, the uptake could potentially be improved with incentives and nominal reimbursement for the effort and time by the participants, even though the cost in carrying out the on-line retest was minimal. Nevertheless, the demographic profiles, in terms of gender, ethnic group, age and number of medications, of those who dropped out of the study were similar to those who completed the study.

The study also excluded patients with poor literacy of written Malay language, and those with cognitive, auditory and visual impairment. Future research will target expanded application of the tool to bridge these gaps, leveraging on voice-annotated administration based on info-communication technology and related tool to capture observations of self-efficacy by caregivers and proxies.

The study shows viable web-based administration of self-efficacy assessment, and indicates its potential application in routine clinical practice using this approach. However, the significant dropouts in the subsequent re-test highlight the challenge in serial measurements if the intention is to chart improvement in self-efficacy after interventions. Either spacing out the intervals between repeat administrations of the HTN-SCP or the use of a shorter, user-friendly version of the tool can be possible solutions.

The findings in this study can potentially be extrapolated to the larger ethnically similar Malay population in the neighbouring countries. This Malay version of the tool can be utilised to evaluate the self-efficacy of Malay patients with hypertension in the community, including those who are managed by the public polyclinics in Malaysia (Kesihatan Klinik).

Conclusion

The Malay version of the HTN-SCP has satisfactory test-retest reliability and internal consistency. The total scores of the Malay-based tool have shown potential association with patients’ self-reporting of their self-care behaviour, which require further research for its validation.

Acknowledgments

The investigators would like to acknowledge SingHealth Polyclinics for sponsoring the cost of the translation service in this study using the institution research grant. We would like to thank A/P Han HR for her support and permission to use the original HTN SCP and Caris Tan for her role as protocol administrator and retrieving data from the web portal. We are also grateful to Felicia Tan for her role in recruiting participants and administrating the survey.

Acknowledgments

None

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
View Abstract

Footnotes

  • Contributors NCT designed the study protocol. DBMY and KCS were involved in the recruitment and administration of the questionnaire. YLEK was involved in data analysis. NCT drafted the manuscript while all authors reviewed and improved the final manuscript before submission.

  • Funding None.

  • Competing interests None declared.

  • Ethics approval Centralised Institutional Review Board.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Anonymous data will be shared upon request if researchers/reviewers are interested.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.