You are here

Article Text

Article menu

Download PDFPDF

Clinical review
Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements
Free
  1. John Torous1,
  2. Jennifer Nicholas2,
  3. Mark E Larsen2,
  4. Joseph Firth3,
  5. Helen Christensen2
  1. 1 Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA
  2. 2 Black Dog Institute, University of New South Wales, Sydney, New South Wales, Australia
  3. 3 NICM Health Research Institute , School of Science and Health, Western Sydney University, Sydney, New South Wales, Australia
  1. Correspondence to Dr John Torous, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA 02115, USA; jtorous{at}bidmc.harvard.edu

Abstract

The potential of smartphone apps to improve quality and increase access to mental health care is increasingly clear. Yet even in the current global mental health crisis, real-world uptake of smartphone apps by clinics or consumers remains low. To understand this dichotomy, this paper reviews current challenges surrounding user engagement with mental health smartphone apps. While smartphone engagement metrics and reporting remains heterogeneous in the literature, focusing on themes offers a framework to identify underlying trends. These themes suggest that apps are not designed with service users in mind, do not solve problems users care most about, do not respect privacy, are not seen as trustworthy and are unhelpful in emergencies. Respecting these current issues surrounding mental health app engagement, we propose several solutions and highlight successful examples of mental health apps with high engagement. Further research is necessary to better characterise engagement with mental health apps and identify best practices for design, testing and implementation.

https://doi.org/10.1136/eb-2018-102891

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Currently, great enthusiasm exists that technology will revolutionise health and healthcare service delivery. Smartphone applications (apps) offer the potential of unobtrusive mental health monitoring, combined with the ability to deliver mental health services in the palms of people’s hands, delivered in a scalable manner and at low cost. Given the global mental health crisis with depression now the leading cause of worldwide disability1 and suicide rates increasing in countries like the USA,2 interest in mental health smartphone apps continues to surge.

    A recent special edition of Evidence-Based Mental Health highlighted promising advances in digital mental health3 including innovative uses of smartphones.4 With meta-analyses suggesting benefits of smartphone apps for conditions like depression5 and anxiety,6 one might expect apps to be commonly used clinical tools. In reality, the clinical uptake and utilisation of these smartphone apps remains limited, despite the tremendous clinical potential, interest and early supporting evidence. One factor limiting mental health smartphone apps is low engagement, and in this paper, we seek to explore both reasons for and solutions to this challenge.

    Methods

    In this paper, we aim to detail how ‘low engagement’ with mental health apps among service users presents a key barrier to the widespread use of these technologies. While there are no standard metrics to compare app engagement across different published papers (and app companies often do not release such data), this paper conducts a narrative review of the literature to identify causes of low engagement with mental health apps, followed by case examples of how apps can be made more engaging. We selected a narrative review because of the high heterogeneity of engagement metrics themselves as well as use cases, apps and mental health app user populations that makes quantitative analysis less informative and potentially even misleading. Another reason for the narrative review is that current engagement data for health apps live not only in the peer-reviewed literature, but also in grey literature, industry reports, patient forums and clinical experience. Our goal is not to summarise every paper published on mental health apps but instead offer a selective review to help identify themes and suggest next steps for increasing engagement. The theories of low engagement identified in this paper were derived from an adapted mini Delphi consensus process in which the corresponding author collected responses from all authors on barriers to app engagement and summarised those results in a second round of online discussion and selection from which seven theories emerged. During a third round of discussions conducted with all authors present in person, at the Black Dog Institute in October 2017, a consensus was reached on the five theories described below. Our goal in identifying these theories is to offer a useful scaffold for discussion of app engagement. As the word itself suggests, we hope these theories will generate further exploration leading towards more definitive themes and knowledge on mental health app engagement.

    Background

    ‘Low engagement’, in the context of health services and this paper, refers to a lack of uptake and/or poor adherence to an intervention among service users. Low engagement is not unique to apps for mental health, but rather has proven to be a critical factor in both the implementation and efficacy of both traditional therapies (such as in-person cognitive behavioural therapy)7 and computerised therapies8 for mental disorders. Furthermore, industry market research data report that 74% of users stop engaging with a health app after only 10 uses.9 Long-term engagement is especially difficult. For instance, a recent population-level study of an iPhone app to track asthma symptoms successfully enrolled nearly 8000 participants. However, by 6 months, only 175 (ie, just 2%) of those participants had engaged enough to take a survey.10 Even one of the world’s most engaging and popular apps, Pokémon GO, which encourages users to be active and walk in order to collect virtual items spread throughout their community, suffered from limited engagement in terms of health outcomes. A study by Microsoft Research noted that within 30 days the increased step count related to using Pokémon GO had returned to baseline levels experienced before playing the game.11

    Mental health apps also suffer from similar engagement issues. However, given that decreased motivation is often a core feature of many mental illnesses such as depression and schizophrenia—engagement may be even more challenging. Indeed, the initial step of downloading a recommended app can often be a challenge. One study reported that 57.9% of participants in a remote research study of apps for depressive symptoms never downloaded the study app.12 Once downloaded, sticking with the app for more than a few days may be even less likely. For example, the popular PTSD Coach app, developed by the U.S. Veterans Administration, was reported to have been downloaded 166 861 times. However, only 14% of individuals had used the app the day after downloading.13 Interestingly, PTSD Coach is one of the few apps to report download and usage data. This paucity of engagement data for mental health apps makes characterising engagement difficult, although a study of user engagement with a schizophrenia symptom tracking app noted the drop off in use followed a power log distribution14 indicating that users were unlikely to keep using the app more than a few times.

    Theories of low engagement

    To date, there is little direct evidence about why engagement with mental health apps remains low. However, there are several potential theories. These include that apps (i) are not user-friendly, (ii) are not designed in a user-centric manner, (iii) do not respect privacy, (iv) are seen as an untrustworthy source of mental health information and (v) are of less use when needed the most in emergency situations. These factors are discussed below.

    Poor usability

    Many mental health apps suffer from poor usability related to being difficult or unenjoyable to use. While the term usability can be difficult to define, examples of apps with poor usability are common. In a recent study of an app designed to help with recovery post presentation to the hospital for self-harm, the authors noted, ‘Especially common were descriptions of the [app] interface as “buggy” or “clunky”, with one participant expressing a belief that the application “didn’t really work” in the sense of basic technical function and another who wondered if the application was deliberately designed to malfunction as part of a deception-based study’.4 More quantitatively, a small feasibility study comparing use of a diet app in those with chronic illness compared with healthy controls noted that overall adherence was 16% for those with chronic illnesses versus 76% for those without.15 Reasons for low use by those with chronic illness included device fatigue, technical difficulty and difficulty with initial set-up. A national US survey study looking at those who had installed a health app onto their phone found the most common reason for discontinuation was the time required to enter data, followed by losing interest and then discovering hidden costs.16 Difficulty and the excessive time to use apps for depression was specifically confirmed by a study noting that >50% of participants suffering from major depressive disorder had some degree of difficulty in both entering and accessing their mood data across four common and popular mood tracking apps.17 Finally, looking beyond individual use cases—there is growing evidence for ‘the hidden cost of personal quantification’—for example, tracking symptoms on an app may feel like work and thus lead to decreased engagement.18

    Lack of user-centric design

    Another reason for low engagement beyond the design of apps is that the actual function of many apps may not meet the needs of users. Recent research on self-management apps for bipolar disorder found that while consumers were interested in a wide range of apps to support self-management, the app stores were devoid of apps that performed the desired functions such as sleep management, understanding early warning signs or triggers, and wellness plans.19 An aptly titled paper ‘You Get Reminded You’re a Sick Person: Personal Data Tracking and Patients With Multiple Chronic Conditions’20 also summarises this theory well, highlighting that symptom tracking has ‘emotional and moral consequences’ and is therefore not universally helpful and desired. There is a need for research investigating the impact of frequency of smartphone-based mood monitoring itself as an intervention in order to determine if and how ecological momentary assessment may become an inadvertent ecological momentary intervention which offers both benefits and risks of harm. Related to this point, in a study using thematic analysis to explore consumer opinions and concerns about apps for mental health, Martinez de Alva et al found that mental health apps often fail to provide the emotional support users seek and at times may distract from real-world challenges.21 These insights from user feedback emphasise the importance of consumer input as it is essential that apps function in a manner that helps users solve real-world problems or progress on their personal goals. When technology is helping patients solve problems they care about, engagement levels often become high.22 23 For example, an individual with schizophrenia was willing to track each of his auditory hallucinations, at times over 100 per day, with a customised tracker when he felt the resulting information was useful for his needs in finding the optimal medication dose.22 While there are examples of involving those with mental health conditions in the conception and design of apps,24 these are currently in the minority.

    Concerns about privacy

    A more systematic reason for low engagement may be related to privacy concerns about many digital health tools like smartphone apps. A survey conducted by a San Francisco-based incubator for technology start-up companies found that only 8% of people surveyed were willing to share their health data with a technology company.25 The 2018 global uproar over the unintended sharing of millions of Facebook users’ personal data after completing what was seemingly a psychology research study has further raised concerns about online privacy.26 Given that the majority of health apps, including mental health apps, exist outside of healthcare regulation—they are not subject to healthcare privacy legislation.27 In this context, many health apps offer users no protections for any personal health information given to or collected by the app and instead will often actively market and sell that data. Illustrating this, Sunyaev et al.’s28 investigation of health app privacy policy availability found that only 30.5% of health apps provided users with a privacy policy. The lack of consumer protections is also reflected by mental health apps, with only 24% of apps for bipolar disorder29 and 29% of suicide prevention apps30 making a privacy policy available to inform users of how their data are used. While a privacy policy does not mean users’ data will be treated as a user hopes—there is no excuse for not having a privacy policy present for any health app. Of particular concern, a recent study found that those with the lowest levels of health literacy were most likely to misinterpret documents like app privacy policies and believe they are beneficial to their rights.31 There is a clear need for digital health technology literacy campaigns to level the playing field and reduce this emerging digital divide.

    Lack of trust

    Despite the potential of mental health apps, the often unsubstantiated bold claims made in marketing and advertisements of these apps have given rise to a healthy scepticism regarding their use. A published newspaper quote from an individual with lived experience offers a useful summary: ‘Very little information is available about which [apps] can help and which might even hinder recovery. I’ve not yet come to rely on the small rectangle of glass and metal in my pocket to maintain my mental health.’32 This sentiment was echoed in a survey of 431 young individuals who raised questions about the ‘perceived instrumentality’ or effectiveness of mental health apps as their primary concern about using them.33 A recent meta-analysis on the evidence for apps for depression and anxiety confirmed that clinical data on these mental health apps/solutions remain nascent although are now rapidly growing.5 6 But while clinical evidence for apps may be small, the number of apps directly available for download today is tremendous, with >10 000 apps for mental health.34 Reviews of the clinical quality and utility of these available apps for mindfulness,35 substance abuse,36 bipolar disorder,29 suicide prevention30 and other conditions all return a similar result: most publicly available mental health apps are not evidence based and some are even dangerous. This may also contribute to lack of trust not only by users but also clinicians and others who may be in a position to recommend these apps.

    Unhelpful in emergencies

    Yet another barrier is that apps may offer the least help when they are needed the most: during a crisis or an emergency. In an assessment of the conversational agents built into smartphones, one research group underscored the current limitations of smartphone responses to statements about mental health, self-harm or suicide.37 Results indicated that conversational agents such as Siri, Google Now and S Voice were unable to consistently identify statements about mental health and interpersonal violence as concerning and respond appropriately. While technology companies have since made improvements to their conversational agents, the vast majority of apps are not designed to help users through emergencies such as suicidal thoughts/actions or overdoses. Most apps instead display a pop-up screen advising the user to seek professional help and notifying the user the app is actually not a medical device. A recent review of publicly available apps to address suicide found that none followed best practice by always having crisis support information visible and available within the app.30 Within the research setting, even apps specifically designed to help users’ urges to self-harm have encountered challenges including a lack of efficacy38 in one study and serious technical bugs in another study that led some research participants to distrust the app.4

    Improving app engagement

    The goal in identifying theories for lack of app engagement is not to deny the potential of digital mental health efforts, but rather to better understand important areas of improvement and future focus. While creating mental health apps that will be more engaging is a challenge, there are numerous opportunities including education, consumer involvement in design/testing, clinical and peer engagement, developing apps responsive to emergencies, building trust through standards and developing collaborations with professional designers and game developers.

    There is emerging evidence that health information technology adoption is correlated with users’ health literacy.31 Therefore, providing health education may be an effective way to increase engagement with technologies such as mental health apps. Both service users and clinicians can benefit from educational support about health information technology, as keeping up to date with apps is challenging given the rapid pace of technological, regulatory and clinical change happening in this space. While increasingly more people have access to smartphone devices, ownership does not translate into comfort using apps for physical or mental health. This is analogous to assuming that access to exercise facilities in a community means that everyone will exercise. Rather targeted teaching on health and how apps can be used to monitor and improve health may be a smart investment in driving engagement. Services like e-mental health in practice (eMHprac) that offer free training and lessons to clinicians may serve as a model and can be accessed at: http://www.emhprac.org.au/

    Another potentially useful approach is to involve end users in the conception, design and testing of apps. Instead of bringing in service users to comment on an app far in the development, some have tried a different approach of working closely with the community to learn what their needs are and formulate how an app may even be of use.39 Such close partnerships throughout the app creation cycle can yield impressive results. Underscoring the potential of engaging users as more than just research participants, the iBobbly suicide prevention app study involved end users in the conception, design and implementation of the app and achieved 97% adherence 23. Of note, the app was not a stand-alone service and integrated into a comprehensive set of services including face-to-face support, suggesting the potential of building apps around communities instead of vice versa. While the mental health field has vast experience around engagement given the nature of treatments offered and history of a strong peer support movement, there is also benefit turning to experts in design and engagement who can offer further insights.

    A related pathway to increasing user engagement is to ensure that mental health clinicians or peer support are an active component of the app. Acknowledging the power of the therapeutic relationship in mental health,40 ideally apps will strengthen that relationship rather than disrupt or replace it. Early evidence suggests that for online platforms increasing therapist support may lead to more user engagement compared with efforts to make the technology platform’s presentation more appealing.41 Bringing peers into the loop to help support and staff mental health apps also offers potential to increase engagement with recent evidence suggesting high rates of acceptability and engagement for peer-to-peer interactions on digital platforms targeting psychotic disorders.42

    A complementary pathway to increase engagement is to better understand the settings in which the apps are being deployed. This means more than just engaging clinicians. Engagement and adherence can be enhanced by embedding apps into the diverse systems and settings in which the participant interacts. In schools, this may occur through health and personal development classes. In primary care, it may involve integration of kiosks or iPads into the clinician’s dashboard in real time.

    Making apps more useful when they may be most needed is another route to increasing engagement. There are now new efforts to develop apps that will offer users help in emergencies. Recent research on real-time smartphone monitoring of suicidal ideation in those recently discharged from inpatient psychiatric units highlights the potential of apps to offer actionable insights on the real-time and personalised risk factors for suicidal ideation.43 Some start-ups are also exploring how peer crowd sourcing-based smartphone apps may help direct users in need to immediate help via suicide hotlines.44

    Even the most engaging app will never be used if it cannot be found. With >10 000 mental health-related apps on the commercial marketplaces, it is difficult for service users and clinicians to identify which are high quality and useful. The current lack of health app regulation means that there is scant guidance or standards to rely on in looking for an app. In an effort to rectify this, organisations like the American Psychiatric Association are actively building frameworks to improve informed decision making around mental health apps.45 New privacy regulations such as the European Union’s General Data Protection Regulation, which takes effect in May 2018, also offer a potential solution with the promise of harmonising data privacy laws across Europe and enforceable standards to protect consumer data collected from mobile devices.46 The opportunity to help steer users towards better apps through standards and regulation in the app space is likely to improve app engagement as consumers and clinicians discover more useful apps while avoiding dangerous and unengaging ones.

    It is also important to acknowledge that apps are not the only sphere where engagement can be difficult. Adherence and engagement with information, monitoring, medications and psychotherapy are challenging problems encountered with both the face-to-face or via a digital interface. One idea we have recently been developing to improve engagement is offering a ‘challenge’ with apps. This challenge may improve engagement over short periods (periods long enough to engage for achievement of some psychotherapeutic advantage), and in some cases to turn the use of the app into a habit, thereby creating tangible behavioural change. In two digital programs, we have used the device of ‘challenges’ over short bursts as a method of providing brief interventions help. See http://www.biteback.org.au/MentalFitnessChallenge/FindOutMore and https://www.headgear.org.au/#/. We believe this challenge approach may help by adapting the time frame of app use just like brief Cognitive Behavioral Therpay (CBT) adapts the time frame of CBT, although we are still collecting data to validate this hypothesis.

    Conclusions

    The potential of smartphone apps to advance and even transform clinical care for psychiatric illness is increasingly clear. But translating novel research study findings into real-world use cases for these apps requires they are engaging and well-used by service users. The five theories for low engagement presented in this paper offer researchers seeking to make more effective apps, clinicians seeking to recommend useful apps and service users seeking to identify helpful apps a framework to approach engagement. While app translational research is certainly not represented as bench to bedside, bridging the gap from code to clicks is the new challenge for mental health apps. The potential solutions and successful engagement cases outlined above suggest best practices for closing the gap between successful app research and app clinical uses.

    References

    1. 1.↵
      World Health Organization. Depression and other common mental disorders: global health estimates. http://apps.who.int/iris/bitstream/10665/254610/1/WHO-MSD-MER-2017.2-eng.pdf
    2. 2.↵
      Centers for Disease Control and Prevention’s (CDC). Increase in suicide in the United States, 1999–2014. https://www.cdc.gov/nchs/products/databriefs/db241.htm
    3. 3.↵
      2. Marzano L ,
      3. Hollis C ,
      4. Cipriani A , et al
      . Digital technology: coming of age? Evid Based Ment Health 2017;20:97.doi:10.1136/eb-2017-102821
      OpenUrlFREE Full Text
    4. 4.↵
      2. Mackie C ,
      3. Dunn N ,
      4. MacLean S , et al
      . A qualitative study of a blended therapy using problem solving therapy with a customised smartphone app in men who present to hospital with intentional self-harm. Evid Based Ment Health 2017;20:118–22.doi:10.1136/eb-2017-102764
      OpenUrlAbstract/FREE Full Text
    5. 5.↵
      2. Firth J ,
      3. Torous J ,
      4. Nicholas J , et al
      . Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017;218:15–22.doi:10.1016/j.jad.2017.04.046
      OpenUrl
    6. 6.↵
      2. Firth J ,
      3. Torous J ,
      4. Nicholas J , et al
      . The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017;16:287–98.doi:10.1002/wps.20472
      OpenUrl
    7. 7.↵
      2. Westra HA ,
      3. Dozois DJA ,
      4. Boardman C
      . Predictors of treatment change and engagement in cognitive-behavioral group therapy for depression. J Cogn Psychother 2002;16:227–41.doi:10.1891/jcop.16.2.227.63996
      OpenUrlCrossRef
    8. 8.↵
      2. Gilbody S ,
      3. Littlewood E ,
      4. Hewitt C , et al
      . Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ 2015;351:h5627.doi:10.1136/bmj.h5627
      OpenUrlAbstract/FREE Full Text
    9. 9.↵
      Localytics. 24% of users abandon an app after one use. 2017 http://info.localytics.com/blog/24-of-users-abandon-an-app-after-one-use
    10. 10.↵
      2. Chan YY ,
      3. Wang P ,
      4. Rogers L , et al
      . The asthma mobile health study, a large-scale clinical observational study using ResearchKit. Nat Biotechnol 2017;35:354–62.doi:10.1038/nbt.3826
      OpenUrl
    11. 11.↵
      2. Althoff T ,
      3. White RW ,
      4. Horvitz E
      . Influence of pokémon go on physical activity: study and implications. J Med Internet Res 2016;18:e315.doi:10.2196/jmir.6759
      OpenUrl
    12. 12.↵
      2. Arean PA ,
      3. Hallgren KA ,
      4. Jordan JT , et al
      . The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Internet Res 2016;18:e330.doi:10.2196/jmir.6482
      OpenUrl
    13. 13.↵
      2. Owen JE ,
      3. Jaworski BK ,
      4. Kuhn E , et al
      . mHealth in the wild: using novel data to examine the reach, use, and impact of PTSD coach. JMIR Ment Health 2015;2:e7.doi:10.2196/mental.3935
      OpenUrl
    14. 14.↵
      2. Torous J ,
      3. Staples P ,
      4. Slaters L , et al
      . Characterizing smartphone engagement for schizophrenia: results of a naturalist mobile health study. Clin Schizophr Relat Psychoses 2017.doi:10.3371/CSRP.JTPS.071317
    15. 15.↵
      2. Shaw RJ ,
      3. Steinberg DM ,
      4. Bonnet J , et al
      . Mobile health devices: will patients actually use them? J Am Med Inform Assoc 2016;23:462–6.doi:10.1093/jamia/ocv186
      OpenUrlCrossRefPubMed
    16. 16.↵
      2. Krebs P ,
      3. Duncan DT
      . Health app use among us mobile phone owners: a national survey. JMIR Mhealth Uhealth 2015;3:e101.doi:10.2196/mhealth.4924
      OpenUrl
    17. 17.↵
      2. Sarkar U ,
      3. Gourley GI ,
      4. Lyles CR , et al
      . Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 2016;31:1417–26.doi:10.1007/s11606-016-3771-6
      OpenUrl
    18. 18.↵
      2. Etkin J
      . The hidden cost of personal quantification. J Consum Res 2016;42:967–84.doi:10.1093/jcr/ucv095
      OpenUrlCrossRef
    19. 19.↵
      2. Nicholas J ,
      3. Boydell K ,
      4. Christensen H
      . Beyond symptom monitoring: Consumer needs for bipolar disorder self-management using smartphones. Eur Psychiatry 2017;44:210–6.doi:10.1016/j.eurpsy.2017.05.023
      OpenUrl
    20. 20.↵
      2. Ancker JS ,
      3. Witteman HO ,
      4. Hafeez B , et al
      . "You get reminded you’re a sick person": personal data tracking and patients with multiple chronic conditions. J Med Internet Res 2015;17:e202.doi:10.2196/jmir.4209
      OpenUrl
    21. 21.↵
      2. de Alva FE ,
      3. Wadley G ,
      4. Lederman R
      . It feels different from real life: users' opinions of mobile applications for mental health. In: Proceedings of the annual meeting of the australian special interest group for computer human interaction: ACM, 2015:598–602.
    22. 22.↵
      2. Torous J ,
      3. Roux S
      . Patient-driven innovation for mobile mental health technology: case report of symptom tracking in schizophrenia. JMIR Ment Health 2017;4:e27.doi:10.2196/mental.7911
      OpenUrl
    23. 23.↵
      2. Tighe J ,
      3. Shand F ,
      4. Ridani R , et al
      . Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: a pilot randomised controlled trial. BMJ Open 2017;7:e013518.doi:10.1136/bmjopen-2016-013518
    24. 24.↵
      2. Ben-Zeev D ,
      3. Kaiser SM ,
      4. Brenner CJ , et al
      . Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatr Rehabil J 2013;36:289–96.doi:10.1037/prj0000019
      OpenUrlCrossRefPubMed
    25. 25.↵
      Rock Health. Digital health consumer adoption: 2015. http://rockhealth.com/reports/digital-health-consumer-adoption-2015/
    26. 26.↵
      U.S. House of Representatives. Hearing: facebook: transparency and use of consumer data. https://docs.house.gov/Committee/Calendar/ByEvent.aspx?EventID=108090
    27. 27.↵
      2. Torous J ,
      3. Keshavan M ,
      4. Gutheil T
      . Promise and perils of digital psychiatry. Asian J Psychiatr 2014;10:120–2.doi:10.1016/j.ajp.2014.06.006
      OpenUrl
    28. 28.↵
      2. Sunyaev A ,
      3. Dehling T ,
      4. Taylor PL , et al
      . Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc 2015;22:e28–e33.doi:10.1136/amiajnl-2013-002605
      OpenUrlCrossRefPubMed
    29. 29.↵
      2. Nicholas J ,
      3. Larsen ME ,
      4. Proudfoot J , et al
      . Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res 2015;17:e198.doi:10.2196/jmir.4581
      OpenUrlCrossRefPubMed
    30. 30.↵
      2. Larsen ME ,
      3. Nicholas J ,
      4. Christensen H
      . A systematic assessment of smartphone tools for suicide prevention. PLoS One 2016;11:e0152285.doi:10.1371/journal.pone.0152285
    31. 31.↵
      2. Mackert M ,
      3. Mabry-Flynn A ,
      4. Champlin S , et al
      . Health literacy and health information technology adoption: the potential for a new digital divide. J Med Internet Res 2016;18:e264.doi:10.2196/jmir.6349
      OpenUrl
    32. 32.↵
      The Guardian. Can you trust an app with your mental health? http://www.theguardian.com/commentisfree/2016/apr/07/app-mental-health-illness-treatment
    33. 33.↵
      2. Schuster L ,
      3. Drennan J ,
      4. Lings I
      . Understanding consumers’ decisions to adopt technology-enabled transformative services. The Service Industries Journal 2015;35:846–64.doi:10.1080/02642069.2015.1090979
      OpenUrl
    34. 34.↵
      2. Torous J ,
      3. Roberts LW
      . Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry 2017;74:437–8.doi:10.1001/jamapsychiatry.2017.0262
      OpenUrl
    35. 35.↵
      2. Mani M ,
      3. Kavanagh DJ ,
      4. Hides L , et al
      . Review and evaluation of mindfulness-based iPhone apps. JMIR Mhealth Uhealth 2015;3:e82.doi:10.2196/mhealth.4328
      OpenUrl
    36. 36.↵
      2. Bakker D ,
      3. Kazantzis N ,
      4. Rickwood D , et al
      . Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health 2016;3:e7.doi:10.2196/mental.4984
      OpenUrl
    37. 37.↵
      2. Miner AS ,
      3. Milstein A ,
      4. Schueller S , et al
      . Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Intern Med 2016;176:619–25.doi:10.1001/jamainternmed.2016.0400
      OpenUrl
    38. 38.↵
      2. Franklin JC ,
      3. Fox KR ,
      4. Franklin CR , et al
      . A brief mobile app reduces nonsuicidal and suicidal self-injury: Evidence from three randomized controlled trials. J Consult Clin Psychol 2016;84:544–57.doi:10.1037/ccp0000093
      OpenUrlPubMed
    39. 39.↵
      2. Goodwin J ,
      3. Cummins J ,
      4. Behan L , et al
      . Development of a mental health smartphone app: perspectives of mental health service users. J Ment Health 2016;25:434–40.doi:10.3109/09638237.2015.1124392
      OpenUrl
    40. 40.↵
      2. Lambert MJ ,
      3. Barley DE
      . Research summary on the therapeutic relationship and psychotherapy outcome. Psychotherapy: Theory, Research, Practice, Training 2001;38:357–61.doi:10.1037/0033-3204.38.4.357
      OpenUrlCrossRef
    41. 41.↵
      2. Alfonsson S ,
      3. Olsson E ,
      4. Linderman S , et al
      . Is online treatment adherence affected by presentation and therapist support? A randomized controlled trial. Comput Human Behav 2016;60:550–8.doi:10.1016/j.chb.2016.01.035
      OpenUrl
    42. 42.↵
      2. Biagianti B ,
      3. Quraishi SH ,
      4. Schlosser DA
      . Potential benefits of incorporating peer-to-peer interactions into digital interventions for psychotic disorders: a systematic review. Psychiatr Serv 2018;69:377–88.doi:10.1176/appi.ps.201700283
      OpenUrl
    43. 43.↵
      2. Kleiman EM ,
      3. Turner BJ ,
      4. Fedor S , et al
      . Digital phenotyping of suicidal thoughts. Depress Anxiety 2018.doi:10.1002/da.22730
    44. 44.↵
      2. Kshirsagar R ,
      3. Morris R ,
      4. Bowman S
      . Detecting and explaining crisis. arXiv preprint arXiv 2017:1705.09585.
    45. 45.↵
      2. Torous JB ,
      3. Chan SR ,
      4. Gipson SYT , et al
      . A hierarchical framework for evaluation and informed decision making regarding smartphone apps for clinical care. Psychiatr Serv 2018;69:498–500.doi:10.1176/appi.ps.201700423
      OpenUrl
    46. 46.↵
      EUGDPR.Org. GDPR portal: site overview. https://www.eugdpr.org/

    Footnotes

    • Funding JT is supported by a carreer development award from the NIMH: 1K23MH116130-01 and a NARSAD Young Investigator Award from the Brain and Behavior Foundation.

    • Competing interests None declared.

    • Patient consent Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.