Article Text
PDF
Abstract
Objectives To investigate mobile health product use in Australia and societal and clinician perceptions towards smartphone based visual acuity (VA) assessment tools.
Design Quantitative analysis of a cross-sectional survey delivered to the general public and thematic analysis of in-depth interviews of eye health clinicians.
Setting Online survey within Australia and face-to-face in-depth interviews of clinicians.
Participants 1016 adults were recruited via Survey Monkey Audience, social media (Facebook and Twitter), Rotary Australia and Lions Clubs Australia. Six clinicians were recruited from private and public settings in Melbourne, Australia.
Primary and secondary outcome measures The study assessed socio-demographic characteristics, history of mobile health product use and perceived advantages and potential drawbacks of smartphone based VA assessment tools.
Results A total of 14.4% of the study population had previously used a mobile-based health product. After adjusting for covariates, younger age (p=0.001), male gender (p=0.01) and higher income (>$45 000) were associated with increased likelihood of having used a mobile health product (p=0.005). Seventy-two per cent of participants would use an automated smartphone based VA assessment tool, provided that the accuracy was on par to that of human assessors. Convenience (37.3%) and cost-savings (15.5%) were ranked as the greatest perceived advantages. While test accuracy (50.6%), a lack of personal contact with healthcare providers (18.3%) and data security (11.9%) were the greatest concerns. Themes to emerge from clinician qualitative data included the potential benefits for identifying refractive error in patients, as well as the ability to self-monitor vision. Concerns were raised over the potential misuse of self-testing vision apps and the inability to detect pathology.
Conclusion Our findings suggest that a substantial proportion of the Australian population do not use mobile health products. Furthermore, there remains notable concerns, including test accuracy and data privacy, with smartphone-based VA assessment tools by both clinicians and the public.
- mobile health
- visual acuity
- perceptions
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study, to the best of our knowledge, is the first to investigate the utilisation and perceptions of mobile health products in both eye healthcare clinicians and a large and diverse sample of the Australian public.
Given smart device-based vision testing apps offer substantial potential to improve accessibility of vision testing clinically and in patients’ homes, and their availability is becoming more prevalent, we explored the perceptions of two diverse groups towards these tools.
While our cross-sectional survey included a large representative sample of Australian adults, the survey responses relied on self-report and therefore we cannot discount the potential influences of self-reporting bias. Response rates were also unable to be ascertained which may cause uncertainty over the validity of the data.
General practitioners were not included in the in-depth interviews which may have added strength to the findings as they play an integral role in vision assessment in primary care settings.
Introduction
Visual acuity (VA) is the most commonly performed measure of visual function globally.1 Clinically, VA assessment is important to quantify changes in vision over time and is a good predictor of the presence of refractive error.2 Additionally, in research settings, VA is a critical measure to assess the efficacy of therapies in clinical trials and to quantify the burden of vision impairment and blindness in population-based studies.1
Developed in the 1860s, the Snellen chart remains the most common method of VA assessment in clinical practice.3 While this chart offers an inexpensive means to measure VA, it has a number of flaws including a variable number of letters per line and non-geometric progression of letter size.4 5 The retro-illuminated LogMAR acuity chart overcomes these limitations and is currently considered to be the gold standard in vision assessment. Despite this, the LogMAR chart has not been widely adopted for clinical use, which is likely due to a longer testing speed required, larger chart size and cost.1
In recent years, there has been a rapid emergence of smart phone applications (apps) that enable VA assessment. Given the fact that there are an estimated 2.6 billion smartphone subscriptions globally,6 these apps offer substantial potential to improve accessibility of VA testing, particularly in undeserved population and developing regions. However, to date, very few of these smartphone based VA assessment tools have been robustly validated7–9 and there remains limited data on acceptance and understanding of these technologies by both the public and clinicians, posing a significant barrier to their uptake within clinical, research and screening settings. In the present study, we sought to investigate the perception and attitudes of potential end-users (patients and clinicians) towards smartphone-based VA assessment tools and explore factors associated with these views.
Materials and methods
This study employed a mixed methods approach to analyse societal and clinician acceptance and understanding of smartphone-based VA assessment tools. The study included two components: (1) a cross-sectional survey of the general public and (2) in-depth interviews with eye health clinicians. The design, sample and data analysis of each sub-study is described separately in the following sections.
Cross-sectional survey of the Australian public
Study design and sample
The majority of participants were recruited via Survey Monkey Audience, with a small proportion also recruited via social media (Facebook and Twitter), and the community service groups of Rotary Australia and Lions Clubs Australia. The link to the survey was available for a 3-month period from July to September 2017 and all questions were presented in the same order for all participants. Survey Monkey Audience quota sampling capability was employed in an attempt to achieve a diverse demographic distribution. Fifty cents (AUD) was donated to a charity of the respondents’ choice for those who completed the survey via the Survey Monkey Audience platform. Once the respondents clicked on the survey link, they were screened to ensure they were 18 years or older and spoke English.
Survey items
An online survey was developed by the qualitative research team at the Centre for Eye Research Australia to investigate previous utilisation of mobile health products and perceptions, advantages and potential drawbacks of smartphone based VA assessment tools (online supplementary table 1). Survey items were peer-reviewed and piloted on a small group of recipients prior to implementation. The questionnaire ascertained the type of smart device/s owned by respondents, demographic details (age, gender, level of education and income level), ocular histories and previous utilisation of eye health services. The remainder of the survey questions related to:
Previous utilisation of mobile health product/s.
A scenario that explored the likelihood of using an automated, smartphone based VA assessment tool, provided it was as accurate as human assessors (pre-coded responses; ‘Yes’ or ‘No’).
Perceived advantages of mobile phone-based vision testing (respondents ranked the most important factor based on pre-coded responses of; convenience, time-saving, cost-saving, improved access, more frequent testing).
Greatest concerns about mobile phone-based vision testing (respondents ranked the most important factor based on pre-coded responses of; difficulty of use, accuracy, possibility of missed follow-up care, security of personal information, lack of personal contact with doctor).
Who would need to endorse such a test (if anyone) for respondents to adopt it (pre-coded responses of; family/friend, health practitioner, would use without endorsement, would not use this technology)?
Supplementary file 1
Data analysis
The de-identified data were downloaded directly from the Survey Monkey secure server and imported into the STATA V.14.0 (College Station, Texas, USA) for statistical analysis. A p value of 0.05 was used for significance testing. Descriptive statistics were used to summarise demographic characteristics of the study population and χ2 tests were utilised to compare questionnaire responses. Logistic regression analysis was applied to assess the relationship between demographic variables and respondent’s perceptions.
Clinician in-depth interviews
Study design and sample
Eye healthcare clinicians were prospectively recruited from private and public settings in Australia. Expressions of interest were circulated via e-mail through the Australian College of Optometry and Royal Victorian Eye and Ear Hospital via e-mail. Interested clinicians were purposively sampled to represent an even spread across three groups: gender, professional role (ophthalmologist and optometrist) and clinic type (public vs private). Face-to-face interviews were conducted by an experienced qualitative researcher (EH) during August 2017. Interviews were conducted in a private setting within the participants’ workplace or at the Centre for Eye Research Australia and each interview lasted 30–60 min. The in-depth interview included open-ended questions and probes, and its development was informed by a thorough literature review and input from clinicians and the study team. Example interview questions are provided in online supplementary table 2.
Data analysis
The interviews were audio-recorded, transcribed verbatim, anonymised and imported into NVivo V.11 to facilitate data management. A thematic analysis was conducted using an inductive ‘bottom-up’ approach, meaning that the themes identified were strongly linked to the data.10 Analysis commenced with researchers familiarising themselves with the data and generating an initial coding framework, grounded in the data. Codes were gradually built into broader categories through comparison across transcripts and higher-level recurring themes were developed. Emergent themes were discussed by two researchers (EH and AM) until a consensus was reached. Reliability was ensured through continuous discussion about the data with the research team and reflexivity was employed throughout the research. Written informed consent was obtained from all participants. The study was approved by the Royal Victorian Eye and Ear Hospital Human Research Ethics Committee and adhered to the tenets of the Declaration of Helsinki.
Patient and public involvement
No participants were involved in the development of the research question, outcome measures, nor the design, recruitment or implementation of the study. There are no plans to disseminate the results of the research to the survey participants or clinicians involved.
Results
Cross-sectional survey of the Australian public
We analysed the cross-sectional survey responses from 1016 participants from Australia. About 1002 (98.6%) provided complete demographic data and there was minimal attrition (<2%) between all survey items. Key characteristics of the study population are provided in table 1. Participants were equally distributed across the four pre-defined age categories (range: 24.9% [45–59 years] to 25.2% [30–44 years]) and 50.1% (502/1002) were female. The current income level was >$45 000 in 49.4% (502/1016) of respondents and 45.3% (460/1016) had completed an undergraduate or post-graduate university degree. Sixty-seven per cent (684/1016) of participants reported that they wore spectacle correction and 71.4% (725/1016) reported that they have their eyes examined at least every 2 years.
Socio-demographic characteristics of the sample (n=1016)
A total of 14.4% (146/1016) of the study population had previously used a mobile-based health product. Multivariate logistic regression analysis was conducted to examining associations between demographic factors and previous use of a mobile health product (table 2, Model 1). After adjusting for covariates (age, gender, education and income), male gender (Coefficient=0.51, p=0.01) and higher income (>$45 000) were associated with increased likelihood of having used a mobile health product (Coefficient=0.73, p=0.005). Level of education was associated with acceptance, with participants who had completed a postgraduate degree being more likely to have used this technology than those with apprenticeship/Technical and Further Education (TAFE) qualifications (Coefficient=0.86, p=0.005). In addition, respondents aged 30–44 years were less likely to adopt mobile health products than those aged 18–29 years (Coefficient=−0.79, p=0.001).
Multivariate logistic regression analysis examining associations between demographic factors and previous use of a mobile health product (Model 1) and acceptance of smartphone-based visual acuity assessment (Model 2)
Seventy-two per cent (731/1016) of participants responded that they would use an automated smartphone based VA assessment tool, provided that the accuracy was on par to that of human assessors. After multivariate adjustments, participants who had completed a postgraduate degree were more likely to accept this technology than those with apprenticeship/TAFE qualifications (Coefficient=0.64, p=0.03) (table 2, Model 2). In addition, respondents aged 45–59 years were less likely to adopt smartphone based VA testing than those aged 18–29 years (Coefficient=−0.52, p=0.02).
Convenience (37.3%, 379/1016) was ranked as the greatest perceived advantage of smartphone based VA testing, followed by cost-savings (15.5%, 157/1016) and ability to undertake more frequent testing (14.8%, 150/1016). Overwhelmingly, test accuracy (50.6%, 514/1016) was ranked as the greatest concern about mobile phone-based vision testing, this was followed by a lack of personal contact with healthcare providers (18.3%, 186/1016) and concerns over security of personal information (11.9%, 121/1016). To adopt this technology, the majority of respondents stated that it would need to be endorsed by a health practitioner (52.6%, 534/1016), with only approximately one quarter (25.5%, 259/1016) suggesting that they would consider using a smartphone based VA testing app without any recommendation.
In-depth interviews
A total of 6 eye health clinicians participated in the qualitative study (ophthalmologists n=3; optometrists n=3). Clinician characteristics are summarised in table 3 and the key themes identified in the qualitative study are presented in the following sections. Detailed themes arising from the clinician interviews capturing the advantages and concerns of the test are presented in online supplementary tables 3 and 4.
Supplementary file 2
Clinician characteristics
Opportunities and advantages of an automated VA test
Clinicians perceived that the main advantage of an automated VA tool was in its potential to allow patients to self-monitor their VA between eye examinations (n=4, freq=11). They also commented that these tools may lead to an increased number of individuals presenting for an eye examination, thus improving early detection of eye disease (n=4; freq=7). Overall, 50% of participants (freq=9) perceived this tool to be useful for detecting uncorrected refractive error. All optometrists perceived the test to be useful for primary healthcare providers (both optometrists and general practitioners) when performing VA tests outside of the clinic (eg, home visits). One-third of clinicians (freq=9) reported that smart device-based vision testing could be a useful strategy for increasing public awareness of the importance of eye health and obtaining regular eye examinations.
Concerns and barriers
Clinicians reported concerns over the limited utility of smart device-based vision testing, given that VA can be normal in the presence of many eye pathologies, particularly in their early stages (n=6; freq=33). This theme was linked to concerns about a normal (or unchanged VA) test result deterring users from having a regular eye examination (n=4; freq=13). Other key concerns raised by clinicians related to the ability of these tools to reach at risk segments of the population, including individuals who do not engage in regular eye examinations (n=5; freq=6) and older adults or those with low vision who may be less likely to own or use a smartphone (n=3; freq=8).
Clinician recommendations
All clinicians expressed the importance of having a clear message that the automated test did not replace the need for a regular eye examination and that a change in VA should be followed up with an examination (freq=14). It was also suggested that the app should include a frequently asked questions section providing the user with an explanation of their results in plain language (eg, If I have concerns about my vision, who should I call? What does my result mean?) and possible generic links to optometry or General Practitioner (GP) networks or to provide the user with a choice (n=4; freq=18) as to whether contact details for providers in their local area are provided.
Most clinicians suggested developing the vision test for an iPad (n=4; freq=9) would be beneficial, suggesting that older adults would be more likely to engage with this technology due to the larger screen size and healthcare providers (optometrists and general practitioners) could still utilise the vision test for home and rural visits when access to traditional testing instruments are limited. Other suggestions for the app features included incorporating culturally agnostic graphics, simple instructions (avoiding those which are multi-level), fun and engaging alerts to ensure the test is being performed correctly, no time limit to complete the test, the ability to repeat the test if an error was made and providing audio instructions ‘some of your subjects are not going to be very well-sighted …. so, I would recommend an audio option to describe the steps’.
Discussion
Smartphone-based applications are increasingly being used to support health behaviours, with an estimated 500 million users of mobile health apps worldwide.11 Currently, there are over 100 smart device-based vision testing apps available in the Google Play Store.1 This type of technology offers the potential for patients living in rural and remote areas who have limited access to healthcare providers, or those who require more frequent monitoring of their vision. While medical and academic communities have considerable influence on the development and uptake of these technologies in the clinical and research settings,1 12 it is the general public that drives the majority of uptake and usage patterns as most vision testing apps are freely downloadable and delivered independent of hands-on clinician support. Given the role that both clinicians and the public play in the development and successful deployment of these apps, we conducted an online survey of the Australian population to investigate utilisation of mobile health products as well as public and clinician perceptions towards smart device-based VA testing tools.
Our finding from the public survey that only 14% of participants had previously used a mobile-based health product is similar to that of a survey by Fox and Duggan13 who reported that 19% of mobile phone users had a health app on their phone. However, it is notably lower than the reported 58% from a recent national survey from the USA.14 We can only speculate that variations in population demographics between studies, and/or differences may exist in awareness or acceptance of mobile based health products between countries. In addition, differences in recruitment methods (telephone interview13 and online survey14) may have contributed to the variable frequency of health app usage. The key concern of a lack of personal contact with healthcare providers that was identified in the present study, may provide evidence for lower levels of acceptance found. Older participants were less likely to adopt mobile based health products and this concern was raised by clinicians which is not surprising given the general perception of less comfort, efficacy and control over computers compared with their younger counterparts.15 However, given 85% of vision loss occurs in those aged 50 years or older,16 older adults would likely be one of the greatest beneficiaries of mobile vision apps. Interestingly, a recent report by Deloitte17 found that 82% of 55–64 years old and 78% of 65–75 years old own a smartphone, with the biggest growth being in the 65–75-year-old bracket (up from 69% in 2016).17 Given this, future systems may benefit from making technology more user-friendly for all age groups. In the present study, clinicians suggested that ensuring the app is compatible with an iPad and providing instructions and results are clear with adjustable font size may be beneficial features to promote increased usage among older age groups.
Our findings suggest that the convenience of self-testing, ability to monitor VA and cost-savings were identified by patients as the most important potential advantages of smart device-based VA tools. From a public health perspective, it is also obvious that, if accurate, these systems offer great potential as an inexpensive means to improve accessibility to VA screening, particularly low recourse areas. However, this research also identified a number of critical concerns relating to smartphone-based VA testing. In line with Krebs and Duncan,14 we report that the test accuracy was overwhelmingly the strongest concern among survey respondents. This is an encouraging finding given there could be significant consequences of displaying unreliable VA data to patients. In addition, over half (53%) of the respondents stated that the mobile vision testing product would need to be endorsed by a health practitioner for them to adopt it. This level of societal caution towards smartphone-based VA tools appears to be well founded and mirrors recent commentary from the medical community1 as, to date, only a very few of the hundreds of apps available have been robustly validated for repeatability and reliability.7–9 These findings highlight that Australian healthcare professionals may have a central role in informing and guiding patients to those apps that have undergone formal assessment. Another noteworthy concern identified by survey respondents involved data security (or lack thereof) of mobile health platforms. This finding is also consistent with that of Krebs and Duncan, reinforcing that the security of patient information should be at the forefront of the minds of software developers and these systems should be compliant with approved standards of data sharing. It should be noted, however, that our findings in relation to potential cost savings is in contrast to Krebs and Duncan, who reported that cost associated with downloading and using health apps was a strong concern for the public.
Clinicians raised concerns regarding the potential for misuse of smart device-based vision apps, suggesting that these tools may potentially deter some patients from having a regular eye examination, particularly if they believe their vision is stable. In line with this, the potential for a delayed diagnosis of eye disease was highlighted by all clinicians, given the preservation of VA is often seen in the early stages of many common blinding diseases, such as glaucoma and diabetic retinopathy.18 19 To avoid such issues patients must have a clear understanding of the importance of receiving regular, comprehensive eye examinations.20 The results the qualitative clinician study identified potential strategies to tackle these issues, including the use of clear messaging that the smart device-based vision test does not replace the need for regular eye examinations, the provision of generic links to eyecare providers even, and a frequently asked questions section providing a plain language explanations of the results.
Typically, online surveys under-represent specific population groups, including older people. A key strength of the current study includes the large and diverse sample of Australians across all age groups. A number of limitations must also be considered. First, selection bias towards those who use the internet more frequently may have impacted the positive perception towards the use of mobile health technology and smart devices. While it can be argued that those who don’t use the internet were under-represented, our survey sample was equally represented across all age groups and participants were engaged through community groups as well as via online avenues. Second, all survey responses relied on self-report and the majority of participants regularly participate in surveys managed by the survey company (Survey Monkey). Third, we were unable to ascertain non-response rates and therefore there is some uncertainty over the validity of the data. F, the inclusion of qualitative data from general practitioners may have added strength to our findings given they are routinely involved in assessing vision within primary care settings.
The results from this study has implications for future research, including strategies to better educate and engage the public about smartphone-based VA tests. Additionally, insight from patients and clinicians will help guide the strategic development, usability and validity of future VA testing apps. Allowing users to customise settings to their own specific needs has been shown to enhance usefulness and levels of enjoyment with the technology.21–23 The results of qualitative studies such as this will improve the app developers understanding of potential barriers to user engagement and what features need to be incorporated prior to release.
In summary, while it appears that the younger population in Australia are relatively accepting of smartphone based VA assessment tools and mobile health products in general, reservation remains among the older Australian population. While clinicians felt that vision testing apps would be useful for detecting refractive error and self-monitoring visual changes between examinations, there remains notable concerns relating to test accuracy, usability and data privacy. We hope that this data can provide practical insight into strategies for the direction, development and validation of future vision testing apps.
Acknowledgments
This work was supported by funding from Google Australia.
References
Footnotes
Contributors SK, JS, EH, WY and AM performed data collection and interpretation. EH, JS and XH conducted data analysis. SK, JS and EH wrote the main manuscript text and prepared figures. MH directed and supervised the project. All co-authors were actively involved in reviewing the manuscript.
Funding This work was supported in part by a Google Australia grant. Professor receives support from the University of Melbourne Research Accelerator Program and the CERA Foundation. The Centre for Eye Research Australia receives Operational Infrastructure Support from the Victorian State Government. The sponsor or funding organisation had no role in the design or conduct of this research.
Competing interests None declared.
Ethics approval Royal Victorian Eye and Ear Hospital Human Research Ethics Committee (HREC-17/1336).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The datasets generated during and/or analysed during the current study may be available from the corresponding author on reasonable request.
Patient consent for publication Obtained.