Article Text
PDF
Abstract
Background/objectives Remote monitoring as a component of chronic heart failure (CHF) management programmes has demonstrated utility in reducing the risk of rehospitalisation and mortality. There is little evidence on mobile health app facilitated remote monitoring in India. We conducted a pilot usability and feasibility assessment of a smartphone-based application (Suhriday) to remotely monitor patients with CHF.
Methods We used a mixed-methods design. Usability testing consisted of the think-aloud approach followed by semistructured in-depth interviews (SSIs) and a satisfaction questionnaire. Feasibility testing was done using acceptability and user satisfaction questionnaires in addition to SSIs. We trained five purposively sampled patients with CHF (based on health literacy and gender) and their caregivers (n=10) in self-care monitoring and app use. Usability was assessed using metrics such as task completion, time required for task completion and user satisfaction using Brooke’s System Usability Scale (SUS). Content analysis of the transcripts with deductive coding was performed for both usability and feasibility interviews. The number and types of medical alerts transmitted through the app were captured and escalated to the treating team.
Results Critical tasks involving (1) opening the app and identifying task list, (2) reporting blood pressure, weight, heart rate and fluid intake and (3) reporting symptoms were completed within 60 s by four patients. Median (IQR) SUS score was 85 (75–92.5) indicating high level of usability. There were 62 alerts from four patients over 4 weeks, with 36 (58.1%) excess fluid intake alerts and 16 (25.8%) blood pressure variations being the most frequent. One participant had challenges using the app and was monitored through active phone calls.
Conclusion Overall usability and satisfaction with Suhriday were good and we were able to remotely manage patients. However, patients with limited health literacy and those facing technological challenges required active structured telephone support.
- Heart failure
- Information technology
- Cardiology
Data availability statement
Data are available on reasonable request. Currently, we have not deposited the raw data in an open access repository. However, we are willing to share the raw data with all interested investigators. Interested investigators may send an email to corresponding author with the brief write up of the research question, objectives and ethics approval for the proposal.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
To the best of our knowledge, this is the first Indian study among patients with chronic heart failure (CHF) to assess usability and feasibility of smart phone-based application for remote monitoring.
The sampled participants were from diverse socioeconomic backgrounds which helped us identify usability and feasibility problems.
This study is a stepping stone that has informed the design of a large trial with a complex intervention centred on mobile health and task sharing to improve self-care and outcomes in patients with CHF. Based on the characteristics of participant 4 in the pilot study who was unable to use the app successfully, we incorporated structured telephone and WhatsApp-based support as alternate options for remote monitoring.
Think-aloud approach was conducted for all five participants, however, semistructured in-depth interviews for usability and feasibility were only possible in three patients out of five. A usability study of the nurse interface was not done. Ideally, this interface would also need to be evaluated to ensure that it is simple and not time-consuming. Feasibility interviews had to be conducted telephonically due to the COVID-19 pandemic.
Although suggestions regarding the incorporation of a chat or video call option were made by a few participants, these features could not be included in the app due to financial constraints.
Introduction
Heart failure (HF) is a rapidly growing cardiovascular disorder, which affects about 38 million individuals worldwide.1 The INDia Ukieri Study (INDUS) study estimated the prevalence of HF in India in 2016 as 1% of the total population; that accounts for nearly 8–10 million patients.2 Self-care is essential for patients with HF and is comprised of treatment adherence in addition to health maintenance behaviours. Patients need to learn to take medications as prescribed, understand how to monitor signs and symptoms of worsening HF, as well as what to do in response to such symptoms when they occur.3 A cluster randomised controlled trial conducted in Ethiopia concluded that self-care education significantly improved self-care adherence score among patients with HF.4
The 2022 ACC/AHA/HFSA (American College of Cardiology/American Heart Association/Heart Failure Society of America) guidelines for the management of HF documented the role of interventions that aim to improve self-care knowledge and skill, and those that focus on enhancing medication adherence or reinforcing self-care with structured telephone support (STS) as being effective in patients with HF. In addition to improving HF self-care significantly, such strategies also reduce the risk of HF-related hospitalisation, all-cause hospitalisation and all-cause mortality. There is some uncertainty regarding improvement of self-care in patients with HF through educational interventions delivered through mobile health applications.3 A systematic review and meta-analysis of the outcomes of STS or remote telemonitoring as the primary component of chronic HF (CHF) management in 8323 patients demonstrated a 34% risk reduction in all-cause mortality with telemonitoring. Additionally, STS and telemonitoring reduced HF-related hospitalisation by 23% and 21%, respectively5.
There is a dearth of evidence on mobile health app facilitated remote monitoring in India. Hence, we developed a mobile-based application named Suhriday (‘Well Heart’), which has multiple capabilities including facilitating remote monitoring of patients with HF (details described below under the app section Functions). We intend to use it as a part of a complex intervention in a larger randomised controlled trial. Hence, we conducted a study to assess the usability and feasibility of smartphone-based application (Suhriday) in remotely monitoring patients with HF involving caregivers. The information from this pilot study will inform the mHealth component of a complex intervention to improve self-care in patients ith HF.
Methods
We conducted a usability and feasibility testing of Suhriday, using a mixed-methods study. The mixed-methods approach which includes qualitative and quantitative methods provides a detailed understanding of user view of the app with regard to immediate engagement as well as attitude and perceptions with the continued use of the app.6
Setting
This study was carried out in the Cardiology and Internal Medicine departments in both inpatient and outpatient wards of St. John’s Medical College Hospital, a tertiary care teaching hospital in South India from March to July 2020.
Eligibility criteria, sampling
The participants for the study include patients with a clinical diagnosis of HF and at least 1 of their caregivers, who consent to use the smartphone app for 4 weeks at home. We excluded patients who in the opinion of the treating cardiologist had a survival prognosis at baseline of less than 3 months and those for whom an intervention procedure had been planned in the next 1 month. We conducted purposive sampling to ensure that at least one patient was female, one had inadequate health literacy and two were from semiurban or rural areas. The health literacy scale has three validated questions related to ability to read and understand medical records.7 Based on the score obtained, health literacy can be classified as low, marginal or adequate.
Suhriday APP
The mHealth application has been developed by One Health Solutions, a software company based in Bangalore, India. This application works on both Android and iOS. It uses JavaScript, Hypertext Markup Language and Cascading Style Sheets, and can be deployed for both desktop and mobile versions.
Functions
The application is intended for care providers (nurses and treating physicians) to remotely monitor key parameters of patients with a primary diagnosis of HF who are on treatment. The application will also be able to generate reminders for medication adherence based on information entered by a healthcare provider nurse into the application at discharge. The patients or their caregivers, after measurement of key parameters (blood pressure (BP), heart rate, body weight and fluid intake) using validated instruments, will enter the same data into the app. The app also has a validated questionnaire for symptom/sign reporting.8 It can identify and alert the study nurse when there are outlying values. The nurse will be able to view these alerts and escalate them so that the treating team can then take appropriate action (eg, order a titration of doses of high ceiling diuretics). The security features of this system include encryption of data on the device, user authentication and a secure Hypertext Transfer Protocol-based data transmission system. We used the Apache CouchDB database on a secure web server and the backend data were stored securely in the Cloud.
Study procedures
Step 1: training a study nurse
We recruited a nurse with a master’s degree. She was trained for 2 months on HF by three physician investigators using lectures and bedside demonstration to recognise worsening signs and symptoms as well as medications for HF. She was educated on the importance of self-care in HF including monitoring, maintenance and management. Furthermore, she was empowered to train patients and caregivers to measure BP, fluid intake, check weight and use the Suhriday app (detailed in online supplemental appendix 1 table 1).
Supplemental material
Step 2: training for patients/caregivers
Patients and their caregivers were educated on salient aspects of HF self-care. Participants received training in measuring BP, fluid intake and weight from the study nurse. They were also trained to recognise worsening signs and symptoms of HF. Following this, detailed app training to perform the tasks assigned was provided to the participants (Tasks detailed in online supplemental appendix 1 (table 1)).
Step 3: setting for usability testing at the patient interface
The patients/caregivers were asked to measure their BP, heart rate and body weight in the presence of the nurse. Then they were instructed to open the link from the message received, to download the app and open the application. The patients or their caregivers were asked to perform the think-aloud exercise while performing and completing the in-app tasks. (Usability measures definitions detailed in online supplemental appendix 1)
Step 4: usability testing at the patient interface
To assess usability, we equipped five patients or their caregivers with the smartphone-enabled mHealth application, Suhriday and used the following methods:
Think-aloud approach
The purpose of this approach was to capture ease of use and an understanding of potential interface issues. Patients/caregivers were instructed to ‘think aloud’ (ie, verbalise their thoughts) as they interact with the Suhriday app while the mobile screens were recorded through the screen recorder option of Android phones. The study team observed and made notes about completeness of tasks with patients/caregivers. We measured effectiveness of task completion by noting whether tasks were (1) completed with ease, (2) completed with difficulty requiring intervention from the study nurse or (3) not completed. We also measured efficiency by noting time taken to perform tasks using the mobile screen and/or audiorecordings.9–11
Qualitative semistructured in-depth interviews
At the end of think-aloud approach, we interviewed the patient and caregiver as a dyad to capture the acceptability and barriers of the app and suggestions for improving its features. Details about qualitative data collection have been described below.
Satisfaction measurement
Patient and caregiver’s satisfaction was measured utilising Brooke’s System Usability Scale (SUS). The SUS has been evaluated for validity, reliability and sensitivity.12–16 Scores were calculated according to Brooke’s guidelines12 and is detailed in online supplemental appendix 1.
Step 5: feasibility study
In the context of technology development is an analytical method used to determine if different components of a project can perform together in order to create a technically and operationally viable concept.17 The guidance from the Medical Research Council, UK on the development and evaluation of complex interventions recommends an early phase of assessing feasibility prior to a full evaluation. Patients and their principal caregivers who participated in the usability testing were provided a smartphone enabled with the Suhriday app, a BP monitoring device and an Liquid-Crystal Display (LCD) weighing scale. They were asked to measure BP and body weight every morning for a minimum of 4 weeks and to report measured values using the application. In addition, they were asked to monitor and report symptoms or other signs through the app.
We provided the study nurse with a smartphone onto which the Suhriday application was installed. The study nurse monitored the patients for a minimum of 4 weeks and made telephone calls to address alerts received for variance in values of measurements and symptoms/signs (detailed in online supplemental appendix 1 (table 2)) in addition to weekly structured telephone follow-up calls. During the course of the study, the nurse maintained a paper dairy and an electronic diary (MS Excel Issue Tracker) to capture type of issue (medical or app-related), details of medical issues, person the issue was escalated to and description of resolution.
Alerts and resolution process
The study nurse was the primary recipient of the alerts through the Nurse interface of the Suhriday app. She would call patients and ask them additional questions related to their symptoms from a preprepared list by the investigators. Salient data retrieved including present complaints, history, key investigation parameters and the current list of prescribed medications would then be informed to the study physicians. General medical escalations were attended by the Internist or the clinical pharmacologist (medically qualified in India), while HF symptom-related queries were escalated to the on-call Cardiology resident. If more information was required, the nurse would be asked to make an additional call to gather the same. Based on the physicians’ assessment, the escalation would either be (1) resolved over the phone, (2) advised an outpatient visit for follow-up, (3) advised an emergency room visit or (iv) advised hospitalisation following an emergency room or outpatient visit.
A feasibility study conducted with qualitative research methods can help identify fundamental problems with the intervention workflow process or trial conduct.18 Hence, after 4 weeks of continuous use of the application, we conducted semistructured feasibility interviews with the patients or their caregivers. We used a feasibility interview guide for this purpose (online supplemental appendix 1 —Feasibility interview guide). The interviews were audiorecorded, transcribed and translated to English. Satisfaction was measured using the SUS instrument.
Sample size, data collection and analysis
We carried out the usability and feasibility testing in five participants as it has been demonstrated that this can detect over 80% of usability problems.10 For usability analysis, we noted effectiveness (task completion), efficiency (time required for task completion, noted through the screen recorder and/or audio recording) and user satisfaction (SUS score).6 9–12 Interviews were conducted (for usability—in the Cardiology Research Office, for feasibility—telephonic) by DYK and BKB with the help of ISJ, and recorded using an audiorecorder. The interviews were transcribed verbatim and then translated into English by a research assistant. This was verified for content accuracy by either DYK or BKB. Transcripts were manually analysed using content analysis approach and deductive coding for acceptability and barriers related to app use, as well as suggestions for improving app functionality. For feasibility testing, while we intended to interview all five participants, we were only able to interview three due to the COVID-19 disruptions. Content analysis codes from these interviews were categorised as those indicating the overall experience, interaction with study staff, impact of training on HF self-monitoring, satisfaction with the team and suggestions to improve the app. We measured user satisfaction using the SUS instrument at the end of the study. Components of acceptability (not validated) were measured using a Likert scale.
Role of public and patient involvement
While the public was not involved in the study, patients and their caregiver’s feedback on the utility and ease of app usability were investigated in detail as described in the results.
Results
We conducted this study from March 2020 to July 2020 and recruited five patients.
Patient demographics and models of app use
Of the five patients, three were from urban areas, one from semiurban and one from rural India. Clinical features such as type of HF-reduced (HFrEF) or preserved ejection fraction, New York Heart Association class at recruitment and aetiology were captured (table 1).
Patient demographics clinical features and models of app use
Usability, task completion
The task completion results are presented in figure 1. The critical tasks involving (1) opening the app and identifying task list (task 1), (2) BP, weight, heart rate and fluid intake reporting (tasks 3, 4, 5, 6, respectively) and (3) symptom reporting—understanding and reporting (task 7) were done easily. Majority of the tasks (tasks 1, 3, 4, 5, 6 and 7) were completed with ease by four (80%) of the participants. Task 8 (viewing shared medical records) and task 9 (sharing medical records) were reported as the most difficult to complete. Among five participants, two (40%) and one (20%) completed tasks 8 and 9, respectively, both of whom had adequate health literacy.
Task completion rate for usability—effectiveness.
Errors
One participant made the error of swiping across instead of tapping on task 2 (acknowledgement for medication reminder). Majority of the errors were faced with a single participant (participant 4, male, rural, inadequate health literacy), who completed task 2 with difficulty, as the drug names were not translated to his native language, Tamil. This participant also took seven attempts to complete task 3 (entry of BP values) due to the inability to locate the number mode after locating the area to enter the value as he forgot the instruction. He completed task 6 (entry of fluid intake the previous day) after three attempts due to difficulty in locating and placing the decimal point.
Usability, time taken per task
The time taken per task results are presented in figure 2. For one participant (participant 3), we did not record the think-aloud approach through the screen recorder as the caregiver of the patient was not comfortable with it. We were not able to record the time taken to complete tasks through the audio recording, as the tasks were not performed systematically. The study team observed and made notes on completeness of tasks for this participant. Participant 4 who made the most errors took the longest time.
Box and whisker plot for usability efficiency measure, IQR.
Usability, satisfaction
The median (IQR) SUS score for usability of all the five participants was 85 (75–92.5), indicating high level of usability. Participant 4 however had a score of 32.5 indicating poor usability.
Qualitative interview
Among five participants who completed think aloud process with the app, interviews were conducted with three participants (reasons explained in the Discussion section) to determine acceptability, barriers of use and suggestions for improvement (table 2).
Acceptability and barriers of the app, suggestions for improving app features at baseline
Remote monitoring through Suhriday app
Overall, patients were managed remotely throughout the 4-week evaluation period with no unplanned rehospitalisations due HF or deaths.
Alerts, number and type
There were a total of 62 alerts (detailed in online supplemental appendix 2 (table 1)) from four participants (snapshots of alerts in online supplemental appendix 2). Alerts related to fluid intake (58.1%), variance in diastolic BP (19.4%), HF symptom worsening (16.1%) and variance in systolic BP (6.5%) were received at nurse interface. Patients were counselled over the phone for fluid intake alerts. For alerts related to BP, the nurse ascertained whether patients were measuring it correctly. If the value was deemed accurate and uncontrolled, it was escalated to the study investigators. Patient 4 hardly used the app, but reported issues through STS. This was monitored actively by the study nurse and there were no HF related escalations over a 4-week period. Overall, five issues were remotely managed for three out of five patients. HF-related escalations led to uptitration of loop diuretics three times, and general medical queries were addressed for constipation and iron-deficiency anaemia.
Supplemental material
Resolution process and time (for alert-led issues and other medical issues)
The study nurse made 21 telephone calls to cardiologists and internists/clinical pharmacologist to resolve issues regarding HF symptom/sign related alerts, general medical queries, prescription confirmation and drug dose queries, as well as BP variations. Majority of the medical issues were resolved within 60 min.
App issues at nurse interface
These were totally 29 in number. Difference in getting alerts with two different phones (8), log in issues (5), alert sync issues between two different phones (3), alert sync lag from patient to nurse (2) were the predominant issues at the nurse’s interface. App issues at patient interface are summarised in online supplemental appendix 2 (table 2).
Feasibility interview
We conducted telephonic interviews with three out of five participants (reasons explained in the Discussion section). Findings are in table 3.
Feasibility of the app and impact of pilot intervention
Feasibility, SUS, overall satisfaction, acceptability
Feasibility results among the three participants showed a median (minimum, maximum) SUS score of 92.5 (87.5, 100), which represents high usability with an overall satisfaction adjective rating of ‘excellent’ (2 participants) and ‘best imaginable’ (1 participant). In the context of acceptability, all three participants felt that they were confident in using the Suhriday app and would be able to teach others to do the same to a large extent (detailed in online supplemental appendix 1 (figure 1)).
Discussion
This is the first report from an Indian setting on how patients with HF respond to a remote monitoring application, pain points experienced, symptoms/signs that resulted in escalations and how they were resolved.
The usability assessment (think-aloud approach, in-depth interviews and SUS) of Suhriday demonstrated satisfactory usability for remotely monitoring among four of the five participants in our study. Most of the critical tasks were completed with ease (figure 1) which was comparable to the usability assessment of the HeartMapp study.19 IQR for tasks 1, 2 and 3 were within 1 min and for tasks 4, 5 and 6 were well within half a minute. Our findings from table 2, figures 1 and 2 reflects that the app is easy to use.
However, participant 4 completed majority of the critical tasks with difficulty, took the longest time, made many errors, and had poor usability in contrast to rest of the participants. This may be due to inadequate health literacy levels and educational attainment. Both patient and caregiver had initial apprehensions and were reluctant to use technology. Though they had smartphones, they were only accustomed to making calls. Although we trained them patiently and tried to build confidence, they could not sustain performing daily tasks and instead maintained manual records. Active STS was imperative in this case. The insights gained helped us plan contingencies for the randomised controlled trial, where we planned to incorporate active structured telephonic and WhatsApp-based support among patients who preferred not to use the app.
A decision-making algorithm for symptom management was developed based on the queries and alerts received. Escalations led to changes in prescriptions for participants 1 and 3. During the COVID-19 pandemic lockdown and heavy restriction imposed on movements, all patients were managed remotely through the app and by STS to manage issues. During remote monitoring, we did not have any unplanned hospital readmission or unplanned emergency/outpatient visit related to HF symptom/sign worsening when compared with a study by Heiney et al having one hospital admission and an emergency OPD visit during a 6-week study.20
During the feasibility assessment, interviewed participants expressed that the app was user-friendly, became part of their routine, helped maintain health and reduce hospitalisation. They also expressed that training led to awareness and improved self-monitoring. Participants opined that they were satisfied with the team’s turnaround time to resolve issues quickly. Having video call option and chat tool within the app were a few suggestions made towards improving the app (findings from table 3).
The limitation of the study was that in-depth interviews for usability and feasibility were conducted in only three out of five participants. The usability interview for participant 4 could not be conducted initially due to in-hospital constraints, and later due to COVID-19-related disruptions. On the other hand, the feasibility interview was not conducted as he had not used the app. Another limitation was that no caregiver details were collected.
Users are the first citizens of a technology; hence their personal technological experiences are principally important.21 Variations in age, gender, affluence and profession among patients and caregivers emerge as critically important factors in technological experience and engagement.22 23 The issues that participant 4 faced, for instance, might be attributed to his social variables. Such cases are more likely to be encountered in communities where patients and caregivers are aged or ageing, and consequently, digitally unfamiliar. It is well evidenced in literature that ageing affects familiarity and fluency with digital devices,24 and the digital divide hinders the elderly from using technology to enhance their quality of life.25 26 Other studies27–29 have shown evidence that factors such as self-efficacy, cognitive decline, declining motor skills and disorientation with hypertext structure heavily influence information and communication technologies (ICTs) among the aged.
This pilot study helped inform a randomised controlled trial designed to improve self-care and remote monitoring in HF patients using a smartphone application in a lower-middle-income country setting.
Conclusion
This study demonstrated that Suhriday was acceptable and easy to use among most patients. Health literacy and preferences need to be considered while enrolling Indian patients into mHealth-based intervention programmes. This study has informed the design of an ongoing multicentre trial with a complex intervention centred on mobile health and task sharing to improve self-care and outcomes in patients with CHF.
Data availability statement
Data are available on reasonable request. Currently, we have not deposited the raw data in an open access repository. However, we are willing to share the raw data with all interested investigators. Interested investigators may send an email to corresponding author with the brief write up of the research question, objectives and ethics approval for the proposal.
Ethics statements
Patient consent for publication
Ethics approval
We obtained ethical approval from the Institutional Ethics Committee of St. John’s Medical College Hospital (Reference number: 124/2017) and written informed consent from all participants. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would like to acknowledge Ms. Sangeetha and Ms. Radha from the Division of Clinical Research and Training, St. John’s Research Institute for their assistance with transcribing interviews.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors BKB, DYK, ISJ, JS, DX and SK contributed to the conception and design of the study. KV, JS, ISJ and BKB were involved in acquisition of data from recruited participants. BKB, PR and DYK performed the analysis. BKB, DYK, ISJ, JS, AK, SK and KV were involved in interpretation. BKB drafted the manuscript, with revisions from DYK, ISJ, JS, AK, PR, SK, KV and DX. DYK and BKB are the guarantors for the overall content of this article.
Funding This study was funded by the India Alliance-Department of Biotechnology (DBT)/ Wellcome Trust as an early career fellowship to DYK, Ref.no. IA/CPHE/15/1/502053.
Disclaimer The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.