Article Text
PDF
Abstract
Introduction Only a minority of people living with mental health problems are getting professional help. As digitalisation moves on, the possibility of providing internet/mobile-based interventions (IMIs) arises. One type of IMIs are fully automated conversational software agents (chatbots). Software agents are computer programs that can hold conversations with a human by mimicking a human conversational style. Software agents could deliver low-threshold and cost-effective interventions aiming at promoting psychological well-being in a large number of individuals. The aim of this trial is to evaluate the clinical effectiveness and acceptance of the brief software agent-based IMI SISU in comparison with a waitlist control group.
Methods and analysis Within a two-group randomised controlled trial, a total of 120 adult participants living with low well-being (Well-being Scale/WHO-5) will be recruited in Germany, Austria and Switzerland. SISU is based on therapeutic writing and acceptance and commitment therapy-based principles. The brief intervention consists of three modules. Participants work through the intervention on 3 consecutive days. Assessment takes place before (t1), during (t2) and after (t3) the interaction with SISU, as well as 4 weeks after randomisation (t4). Primary outcome is psychological well-being (WHO-5). Secondary outcomes are emotional well-being (Flourishing Scale), psychological flexibility (Acceptance and Action Questionnaire-II), quality of life (Assessment of Quality of Life -8D), satisfaction with the intervention (Client Satisfaction Questionnaire-8) and side effects (Inventory for the assessment of negative effectsof psychotherapy). Examined mediators and moderators are sociodemographic variables, personality (Big Five Inventory-10), emotion regulation (Emotion Regulation Questionnaire), alexithymia (Toronto Alexithymia Scale-20), centrality of events (Centrality of Events Scale), treatment expectancies (Credibility Expectancy Questionnaire) and technology alliance (Inventory of Technology Alliance–Online Therapy). Data analysis will be based on intention-to-treat principles. SISU guides participants through a 3-day intervention.
Ethics and dissemination This trial has been approved by the ethics committee of the Ulm University (No. 448/18, 18.02.2019). Results will be submitted for publication in a peer-reviewed journal and presented at conferences.
Trial registration The trial is registered at the WHO International Clinical Trials Registry Platform via the German Clinical Trials Register (DRKS): DRKS00016799 (date of registration: 25 April 2019). In case of important protocol modifications, trial registration will be updated. This is protocol version number 1.
- mental health
- telemedicine
- public health
- depression & mood disorders
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
To our knowledge, this is the first full-scale randomised controlled trial on a chatbot delivering a brief psychological intervention to uplift psychological well-being.
Results on user acceptance will help to gain further insights for requirements due to the fully automated presentation form of psychological internet interventions.
Technology alliance and side effects will be monitored.
Dropout rate is to be kept small by automated guidance and prompts.
Introduction
The global direct and indirect economic costs of mental disorders are estimated at US$2.5 trillion.1 Thus, untreated mental disorders are a public health concern worldwide. However, the majority of individuals living with mental disorders do not receive any healthcare supply.2–4 In Europe, only about 25% of people with mental disorders receive professional treatment.5
On one hand, there are societal barriers to receiving adequate mental healthcare offers. On the other hand, there are barriers on the side of individuals, keeping them from seeking professional help.6 The latter aspect comprises fear of stigmatisation,7 8 restrictions of time and location,9 10 negative attitudes towards pharmacological and psychotherapeutic treatments,11 negative experiences with professionals12 13 or missing conscientiousness for diseases.14 In order to overcome some of these barriers and to improve mental healthcare at a large scale, digital means are frequently discussed options.
Digitalisation sets societal changes in motion in various fields.15 Other than in the areas of work, economy and science, new technologies slowly emerge in the field of mental healthcare. Internet-based and mobile-based interventions (IMIs) can provide low-threshold, flexible interventions that are resource, time and location independent,9 10 and can be as effective as traditional face-to-face psychotherapy.16 As such, they might help to reduce societal and individual barriers to mental healthcare and expand supply offers.9 16 17 At this point, their effectiveness and cost-effectiveness could be established for the prevention18 and treatment of mental disorders9 19–24 and chronic somatic diseases25 as well for positive mental health promotion purposes.26–29
IMIs are highly standardised computer programs. They are often manualised, which means that they are incorporating instructions, theory-based key elements and concepts as well as how-to approaches regarding the evidence-based implementation of a certain delimited psychological programme, which can be seen as digitised therapeutic interventions.9 30 While they have without doubt substantial merits, some limitations still restrict their scalability and widespread roll-out. As yet, for example, IMIs seem to work best if they provide any form of human guidance alongside the digital programme.21 31 However, fully unguided interventions could be a more cost-effective way of providing digital interventions (eg, 32). Thereby, professional guidance does not only limit the cost-effectiveness, but also necessitates healthcare infrastructures that might not always be at place at a large (enough) scale. In addition to the possibility of an increased cost-effectiveness, unguided fully automated interventions like mHealth interventions have shown potential to effectively targeting mental health symptoms.33
Evidence shows that the effectiveness of IMIs might be in part attributable to other effect factors than in face-to-face therapy.34 In comparison with face-to face therapy, the therapeutic alliance might not be as relevant as effect factor.35 Instead, other factors, for example, an agreement on tasks and goals35 or the fostering of self-efficacy,34 have been discussed. Software agents could combine the best of both worlds, as they seem to have the potential to human–machine alliance.36 Delivering IMIs by software agents could compensate for some of the disadvantages of conventional computer program-based IMIs (eg, 31) Among others, they could show human-like, immediate responses with regard to user input.37
A software agent or ‘chatbot’ is a computer program that can hold a fully automated text-based conversation in real time with people via a chat interface (eg, smartphone application) by using a natural language style.37 The growing interest and body of research about software agents38 39 is realised in various populations and contexts, such as problem-solving and stress.40–42 In the context of clinical psychology and psychotherapy, research on software agents is sparse43 but could create opportunities for the field regarding the provision of mental health services. Software agents could be used to convey therapeutic contents and brief interventions.44 45 Establishing contact to a software agent might not be as stigmatising as using formal mental health services like starting a face-to-face therapy or asking a general practitioner for possibilities of mental healthcare.46 Furthermore, they are flexible regarding location and time,47 can be used anonymously48 49 and provide personalisation through implicit customisation.50 Therefore, software agents could help to overcome barriers and provide psychological and health behaviour change interventions on a large scale in the future.
Current mental health software agents are primarily based on cognitive–behavioural therapy.43 However, other popular approaches with proven effectiveness in face-to-face settings could also readily be realised in a digital form, such as writing interventions51 and acceptance and commitment-based approaches.52
Writing with the aim of improving health has a long history.53 In the current literature, the labelling of this kind of intervention varies. Terminology includes expressive writing,54 55 benefit-finding or positive writing,56 57 and therapeutic writing (eg, 58). Regardless of terminology, the writing intervention to be investigated in this study will refer to the process of freely and emotionally writing about a positive personal life event without paying attention to spelling or grammar. The call to write about personal life events, to tell a story, seems to go straight at the centre of subjective experiences,59 which in turn is the main medium in traditional face-to-face therapy. In that, the term therapeutic writing will be used in this context to acknowledge that the intervention refers to some kind of therapeutic work.60 It has been shown that writing interventions can be highly time‐efficient and cost-efficient.61 A recent meta-analysis shows that writing interventions can help to improve general psychological health (standardised mean difference (SMD)=−0.46, 95% CI −0.86 to –0.06).62 Finally, a meta-analysis from Bolier et al63 found an effect of Cohen’s d=0.34 (95% CI 0.22 to 0.45) for positive interventions to uplift cognitive and/or affective appraisal of one’s life as a whole and d=0.20 (95% CI 0.09 to 0.30) for optimal functioning including mastery, hope and purpose in life.
Acceptance and commitment therapy (ACT)64 aims at acceptance, mindfulness and value-based living, and has been found to be effective in the prevention of stress and the increase of well-being.27 65 The efficacy of ACT-based interventions in general and ACT-based IMIs in particular has been indicated in a number of studies and systematic reviews. Within a randomised controlled trial (RCT), Fledderus et al29 investigated an ACT-based IMI for people living with depression. The authors found significant reductions in depression, anxiety, fatigue, experiential avoidance and improvements in positive mental health, compared with a waitlist control condition (effect sizes Cohen’s d=0.51–1.00).29 In their meta-analysis, Brown et al66 examined 10 RCTs investigating the effectiveness of ACT in the treatment of depressive or anxiety symptoms and well-being in adult populations. ACT interventions were compared with passive control groups (N=3), active control groups (N=4) or both (N=3). The authors found small effect sizes regarding the improvement of depression (g=0.24, 95% CI: 0.04 to 0.45), while the heterogeneity of conditions and outcome measures on anxiety and well-being was too high to draw firm conclusions. Spijkerman et al28 examined 15 RCTs in adults with various mental problems and healthy populations. Mindfulness interventions, of which the authors include ACT, were compared with passive control groups (N=10), active control groups (N=5) or both (N=2). The authors found small to medium effect sizes concerning the improvement of depression (g=0.29, 95% CI: 0.13 to 0.46), anxiety (g=0.22, 95% CI: 0.05 to 0.39) and well-being (g=0.23, 95% CI: 0.09 to 0.38).28
We developed a gender neutral software agent called SISU (Software agent providing an Intervention for Self-help to Uplift psychological well-being and Finnish word (‘sisu’) for inner strength) with the aim to provide an easily deployable software agent that improves people’s well-being. Therefore, SISU combines therapeutic writing and acceptance-based and commitment-based principles. Results of a feasibility trial on SISU67 showed that SISU is feasible in terms of user acceptance and the potential of the software agent to deliver a brief writing intervention. Thus, SISU is feasible to be implemented within a confirmatory clinical trial. Hence, the present study is designed to investigate the clinical effectiveness and acceptance of the software agent SISU thereby focusing on the following specific research aims:
To estimate the effects of SISU on psychological well-being compared with the waitlist control group (WL) at post-treatment at day 3 (t3) (primary outcome).
To estimate the effects of SISU regarding the secondary outcomes flourishing, quality of life and psychological flexibility compared with the WL at t3.
To explore, which factors are associated with, moderate or mediate the effects of SISU.
To investigate if the intervention is associated with measured side effects.
To investigate the level of acceptance (satisfaction, adherence) with the intervention.
Methods
Study design
This is a two-arm, parallel RCT with the intervention group SISU (IG) and a WL. The IG receives the online-based intervention guided by the SISU software agent. The WL receives the intervention 4 weeks later. Primary and secondary outcomes will be assessed over a period of 4 weeks. Assessments will take place at screening (t0), baseline at day 1 (t1), intermediately at day 2 (t2), post-treatment at day 3 (t3), as well as 4 weeks’ follow-up (t4).
The present study is conducted and will be reported in accordance with the Consolidated Standards of Reporting Trials 2010 guidelines for RCTs68 and the guidelines for executing and reporting IMI research.69 The study protocol follows recommendations of the Standard Protocol Items: Recommendations for Interventional Trials 2013 Checklist for clinical trial protocols.70
Recruitment
Recruitment has started in May 2019 and will be continued until the targeted sample size of N=120 has been reached. We recruit in German-speaking countries: Germany, Austria and Switzerland. Recruitment strategies comprise a dynamic, broad online and offline recruitment strategy. Offline recruitment will be conducted via posters and flyers at different universities, psychosocial counselling services, city libraries and other publicly accessible sites. Online recruitment strategies will comprise postings in online self-help groups on social media (eg, facebook), displays on eBay and Xing as well as the StudiCare website. StudiCare is a project that offers a broad assortment of internet-based interventions for psychological and behavioural issues.71 Interested persons will get access to the screening (t0) at an online survey tool (unipark.com) via QR code, link or via email on request. Directly after the screening, eligible participants will automatically receive informed consent for signing via email. Apart from the recruitment, the study will be fully conducted online.
Eligibility criteria
Participants will be eligible for inclusion in the present trial if they are (a) 18 years or older, (b) willing to take part in this study, (c) have internet access and an email address, (d) have a low psychological well-being (Well-being Scale (WHO-5) ≤52) and (e) possess sufficient German language skills.
Study procedures
If eligibility criteria are fulfilled, applicants will receive an online information letter including detailed information about study procedure and informed consent. They will be informed that they can withdraw from the intervention and/or study at any time without any negative consequences. After signing the informed consent, participants will be randomised to the IG or WL condition. Following, they will receive their individual ID and get an invitation for the baseline questionnaire (t1) at unipark.de via email. Afterwards, participants will learn about their group membership. The IG will get in contact with SISU and the intervention using the end-to-end encrypted online messaging app ‘Wire’ after finishing baseline (t1). SISU guides participants through a writing intervention on 3 consecutive days using a standardised conversation script. Each writing intervention is automatically followed by an assessment. Participants who are part of the WL will receive access to SISU 4 weeks after randomisation. If participants complete questionnaires for t3 and t4, they will each time get the chance to win a €10 gift card from Amazon as a monetary incentive to promote retention and follow-up completion. All participants with a low WHO-5 score (<28) in the screening receive an automatised email with further information about offers of the healthcare system.
Randomisation and blinding
Participants will be randomised to either IG or WL. An academic assistant (JM) from the Department of Clinical Psychology and Psychotherapy at the Ulm University, not otherwise involved in the trial and blinded towards all further procedures, will perform the allocation. A permuted block randomisation with 4, 6, 8 and 12 block-size and an allocation ratio of 1:1 will be used. The randomisation list will be created by a well-accepted website (https://www.sealedenvelope.com). Whereas blinding of participants is not possible, data collectors and data analysts are blinded regarding group membership.
Intervention
The software agent (SISU) provides a brief 3-day intervention. The writing instruction provided by SISU is based on the paradigm of therapeutic writing as well as ACT.72 The version of SISU used for this study was improved through participant feedback collected in the feasibility trial.67 Revisions included the enrichment of the instruction for writing about positive life events with elements of ACT (more mindfulness exercises, authenticity of the dialogue through reduction of repetitions, interactions on reported life events) and elements for the reconstruction of narrative identity.
SISU mimics a human conversational style. Participants are guided to write each day at the same time for 10–20 min about a self-chosen autobiographical, positive life event. On day 1 there is psychoeducation in the beginning. Then, instructions for the writing tasks are followed by the narratives of the participants. Participants are instructed to write about a meaningful, outstanding positive life event on day 1 and about an outstanding positive event from adulthood on day 2. On day 3 participants are guided to write about their best possible future. After the writing task, SISU encourages participants to experience the positive emotions due to the reported event in the present moment. Mindfulness exercises are provided by an audio file right after the writing intervention, while ACT metaphors are integrated into the conversational content. Participants are encouraged to practise on a daily basis. To increase adherence, SISU reminds participants at 24-hour intervals. More details on intervention contents can be derived from table 1. For an illustration of content and chronological structure, see figure 1.
Content and chronological structure of the study.
Content and techniques of the writing tasks as delivered by SISU
Using the online messaging Wire Services SDK enables programmatic end-to-end encrypted communication. Thanks to this encryption, messages sent by SISU or participants are not accessible by third parties, including the service provider. We further protect participation data by hosting SISU on premises and by encrypting the data at rest, thus limiting the access to our research group. The communication logic is implemented as a finite-state machine. Our SISU implementation parses incoming messages based on a fixed set of rules and responds with an appropriate answer. In addition, SISU can react to external triggers. That is, external triggers can lead to a status change of SISU. For example, the termination of a survey at Unipark can cause a status change of SISU from ‘user is active’ to ‘user finished the interaction for the day’. External triggers can be (a) conversation timeouts (ie, the participant has not responded in a set time frame), (b) Unipark events (ie, participant has completed an external survey) and (c) scheduled events (eg, daily participation reminder at predefined time frames).
The (ultra)brief intervention rational of 3 days was chosen because we wanted to provide participants with a brief possibility to do something for their mental well-being, despite their busy everyday lives. Indeed, evidence suggests that brief writing interventions of, for example, only 1 week can increase emotional well-being even 6 months after the intervention,73 particularly in case of interventions focusing on improving mental health rather than treating mental disorders.
Waitlist control group
Participants of the WL get access to the writing intervention provided by SISU 4 weeks after randomisation. The intervention has the same content for both groups.
Administrative and technical support
In case participants forget their individual ID or have other technical issues, they can make use of the study team via email for technical support at every point during the training.
Outcome assessment
Screening for eligibility takes place at t0. Data for relevant outcomes will be collected prior to the intervention on day 1 (t1), on day 2 (t2) and day 3 (t3; intervention completed) and 4 weeks after randomisation (t4; follow-up). Demographic data and personality traits are measured once (t1). A flow chart of the study can be seen in figure 2. The outcomes, their measurement instrument and points of assessment are shown in table 2.
Flow chart of the planned study procedure. WHO-5, Well-being Scale; WL, waitlist control group.
Constructs, measurement instruments and points of assessment
Screening, T0
The short WHO-5 is administered to assess the subjective psychological well-being of participants in the last 2 weeks.74 Participants can answer on a 6-point Likert scale (5=‘all of the time’, 4=‘most of the time’, 3=‘more than half the time’, 2=‘less than half the time’, 1=‘some of the time’, 0=‘at no time’). The sum of raw scores (range: 0–25) is multiplied with 4 and produces a total score (range: 0–100) with 0 representing the worst imaginable well-being to 100 representing the best imaginable well-being.74 Scores ≤52 indicate a low and scores ≤28 indicate a very low psychological well-being. Topp et al74 mention a comparable cut-off score of ≤50. The WHO-5 shows a sensitivity of 0.93 and a specificity of 0.83 in the detection of depression.74 Additionally, the screening includes age, sex, contact information and the sufficient knowledge of German language.
Demographic data
The following information will be collected from each participant at t1: sex, age, education, nationality, German-speaking skills, relationship status, profession and highest educational attainment.
Primary outcome
Psychological well-being
Primary outcome is psychological well-being at t3 measured by the WHO-574 already described in the section for screening.
Secondary outcomes and covariates
Emotional well-being
The German version of the Flourishing Scale75 is a measure of psychosocial well-being and personal growth and development (ie, flourishing). Each of the eight items is rated on a 7-point Likert scale ranging from 1=‘strongly disagree’ to 7=‘strongly agree’. A sum score is computed with higher scores indicating higher flourishing. With a Cronbach’s α of 0.87, the scale shows good internal consistency.75
Psychological flexibility
The German version of the Acceptance and Action Questionnaire-II76 is a general measure for psychological inflexibility and consists of seven items. On a 7-point Likert scale that ranges from 0=‘never true’ to 6=‘always true’, the questionnaire assesses a person’s willingness to experience unwanted thoughts and feelings and a person’s ability to act despite the presence of undesirable thoughts and feelings. In this study items were reverse coded to assess psychological flexibility. Sum scores (range: 0–42) are computed with higher scores indicating higher psychological flexibility. The questionnaire shows good to excellent psychometric properties in a German sample.76
Quality of life
With the help of the inventory Assessment of Quality of Life (AQoL-8D), participants’ quality of life is recorded.77 Each of 35 items loads on one of eight dimensions of quality of life and is rated on 4-point to 6-point Likert scales. For analysis there is an algorithm which can be used for quality of life in general as well as for particular subdimensions. In total, scores between 0 and 1 are possible. Standard values are available. Reliability of AQoL-8D is very good with Cronbach’s α of 0.96.77
Side effects
Subjective adverse events of the intervention are recorded with the 15-item inventory for the assessment of negative effects of psychotherapy.78 Items are rated on a 4-point Likert scale (0=‘no agreement’ to 3=‘total agreement’) or a bipolar 7-point scale. Adverse effects in social life, intrapersonal factors or work-related situations are taken in consideration. The original inventory with 32 items has an internal consistency of α=0.95.79
Satisfaction with the intervention
To assess the global satisfaction with the intervention, a revised version of the German version of the Client Satisfaction Questionnaire (CSQ-880) was used. Participants rate their satisfaction on a 4-point Likert scale for each of the eight items. A sum score is computed. Higher scores indicate higher satisfaction. Internal consistency of the CSQ-8 is very good with α=0.90.81 A study on reliability and validity of assessing user satisfaction with internet-based interventions indicates good overall psychometric quality of the measure.82
Post-Writing Questionnaire
To assess therapeutic writing after every writing session, the participants answer four questions about their feelings and thoughts during and after the writing experience. Answers are rated on a 5-point Likert scale (1=‘not at all’, 3=‘few’, 5=‘very much/extremely’). The questionnaire was adapted from the English version of Pennebaker and Beall.55
Open questions
For the final survey (t3), four open questions inspired by the open questions from Fitzpatrick et al83 about the interaction with SISU are provided. The answers are individually evaluated and thematically summarised.
Questions for the future of software agents
The final survey (t3) will assess the behavioural intention to use a software agent in the future or recommend one to friends as well as the future performance expectancy of software agents providing psychological interventions to uplift psychological well-being in three open questions. Participant responses will be analysed on a qualitative basis.
Moderators/mediators
Centrality of events
The Centrality of Events Scale84 assesses the centrality of an event to a person, differentiating three independent characteristics, whether the event is seen as (1) a reference point for everyday inferences, (2) a turning point in the life story and (3) an element of the personal identity. Participants rate the seven items of the short version on a 5-point Likert scale from 1=‘totally disagree’ to 5=‘totally agree’. With a Cronbach’s α of 0.88, the scale shows high internal consistency.84
Personality
To assess the big five personality traits of participants, the short version of the Big Five Inventory85 is used. Each of the five personality dimensions is measured with two items depicting either the positive or the negative pole of the spectrum. Participants rate the items on a 5-point Likert scale from 1=‘fully disagree’ to 5=‘fully agree’. The questionnaire shows average retest reliabilities ranging from 0.56 to 0.60.85
Alexithymia
The German version of the Toronto Alexithymia Scale86 assesses alexithymia of participants. Each of the 20 items is rated on a 5-point Likert scale ranging from 1=‘strongly disagree’ to 5=‘strongly agree’. The German version assesses three factors87: ‘difficulties in identifying and describing feelings’, ‘external oriented thinking’ and ‘importance of emotional introspection’. For each dimension, sum scores are computed with higher scores each indicating higher manifestations of alexithymia. Internal consistency of the scale is good with an α=0.80.87
Emotion regulation
The Emotion Regulation Questionnaire88 is a 10-item questionnaire measuring positive and negative feelings as well as their regulation. Items refer to two different emotion regulation strategies: reappraisal and suppression. Participants rate the items on a scale from 1=‘strongly disagree’ to 7=‘strongly agree’. Means show the preference for each strategy indicating higher preference at higher mean scores. Internal consistencies are acceptable to good and differ from α=0.75 to α=0.82.88
Treatment expectancy
Treatment expectancy is measured with the Credibility/Expectancy Questionnaire89 with six items. Participants rate four items on a 9-point and two items on a 10-point Likert scale with varying descriptions. The scale can be separated into two factors: credibility and expectancy. Cronbach’s α for credibility differs from 0.79 to 0.90, for expectancy from 0.81 to 0.86 and for the total scale from 0.84 to 0.85, indicating acceptable to high internal consistency.89
Technology alliance
The Inventory of Technology Alliance–Online Therapy (TAI-SF) was used to evaluate the technological alliance between the participants and the online intervention, thus the software agent. The TAI-SF is a 12-item questionnaire developed by Labpsitec (http://www.labpsitec.uji.es/eng/index.php) that assesses the degree to which the participant perceives the online intervention as helpful. Items are rated on a 7-point Likert scale from 1=‘never’ to 7=‘always’.
Data privacy and ethics
Data will be pseudonymised and analysed in the Department of Clinical Psychology and Psychotherapy of the Ulm University via individual ID and an internal participant ID for every participant to encode the individual datasets. Messages exchanged between participants and SISU are encrypted in-transit by the end-to-end encryption of the ‘Wire’ application. Thus, only the study team will have access to the collected data. Participants will have the opportunity to have all of their collected data deleted. External researchers may get access to the final trial dataset (from HB) on request depending on the specified data security and data exchange regulation agreements. To ensure confidentiality, data dispersed to any investigator or researcher will be blinded of any identifying participant information. Anonymised results will be published in peer-reviewed journals and presented on international conferences.
The participation in this study should not be associated with any specific risks. However, temporary changes in mood could arise directly after the writing task.57 Furthermore, therapeutic writing can lead to emotional–cognitive (change) processes60 with which the participants could have difficulties in dealing with. Therefore, participants will have the opportunity to contact the study team at every point during the trial. Additionally to the interventions, participants with a very low WHO-5 score (<28) in the screening will be sent an automatised email with further information about offers of the healthcare system.
Sample size
A meta-analysis by Bolier et al63 found an effect size of d=0.34 for positive psychological interventions aiming at uplifting well-being. Riddle et al62 reported an effect size of d=0.46 for writing interventions to enhance well-being. However, for internet-based mindfulness interventions, Spijkerman et al28 found a somewhat smaller effect of g=0.23.
Based on these previous findings, a small effect size of d=0.30 is expected. Power analysis for an analysis of variance with repeated measures with G*Power (http://gpower.hhu.de/) recommends a sample size of at least 60 participants per group (N=120) on the assumption of two-tailed testing, an alpha error α=0.05 and power 1−β=0.90.
Statistical analysis
Patterns of missing data will be investigated, and analyses will be adjusted accordingly (multiple imputation). Regarding the imputation method and predictor selection, we will follow the recommendations of Buuren and Groothuis-Oudshoorn.90 It will be assumed that missing values are missing at random. Analyses will be conducted on a two-sided level of significance (α=0.05). Participant characteristics will be described descriptively.
All statistical analyses will be performed based on the intention-to-treat (ITT) principle. Additional per-protocol analyses will be conducted in order to examine associations in case of patients adhering to the intervention protocol. Participants who completed at least 66% of the intervention are defined as intervention completer (=per protocol).
The primary outcome will be analysed using linear regression models at t3 as dependent variable and the baseline value as covariate, adjusting for sex and age. The necessity of multilevel models will be explored by interclass correlations (ICCs). On substantial ICC (>0.10), multilevel models will be specified to account for the dependency in the data.91 To analyse between-group effect sizes, SMDs with 95% CIs will be calculated for post-treatment (t3) and follow-up (t4). Secondary outcomes will be analysed accordingly.
Exploratory mediation and moderator analyses involving the primary and secondary outcomes as well as demographic data will be conducted. Moderator and subgroup analyses are aimed for in case of a sufficiently large sample size.
For the planned exploratory moderator analyses, regression models will be employed. Initially, each potential moderator described under ‘covariates’ will be analysed in a separate regression model. The primary outcome psychological well-being at t3 will be the dependent variable. Predictors will comprise group, the moderator variable and the interaction of group and moderator. In a next step, a comprising model of all identified moderators will be tested.
Mediation analyses will be conducted according to the principles of time-lagged mediation.92 Psychological well-being at t3 will be the outcome variable. Group will be chosen as independent variable, whereas the variables defined in the section ‘Moderators/mediators’ will constitute the respective mediating variables. No interim analyses will be applied to the data.
Patient and public involvement
Patient and public involvement (PPI) representatives provide input to the present study in several stages. Results of the feasibility trial on SISU were used to further develop and optimise study design and procedures. PPI representatives were included in the intervention development to improve content, usability and design of SISU. However, acceptance of SISU from the participants’ perspective is a crucial outcome of the study and both quantitative and qualitative methods are applied to capture acceptance and side effects. The dissemination plan of the study results includes presentations on international conferences and publications in peer-reviewed journals.
Discussion
To the best of our knowledge, this study will be the first to investigate an intervention on therapeutic writing combined with mindfulness-based exercises provided via a software agent. It is a two-parallel arm controlled trial with the aim of evaluating SISU, a software agent as an innovative form of providing a scalable mental health interventions43 to uplift people’s well-being.
The proposed study can be characterised by several strengths. First, our software agent SISU was successfully tested within a feasibility trial of Bendig et al submitted67 and provides elements of established approaches.57 72 Therefore, we consider SISU to provide an eligible intervention and the potential to uplift psychological well-being in participants. To our knowledge, there are no known risks or negative effects for IMIs in the context of self-help interventions to uplift psychological well-being. Still, we will systematically record via questionnaire (Inventory for the assessment of negative effects of psychotherapy) if and which negative effects of SISU might appear. This will contribute to the still understudied area of research on risks and side effect93 and therefore help make future IMIs safer.
Second, besides the relevance and necessity of our intervention, the methodical quality of our study is another strength. This is especially relevant in the relatively young field of research on therapeutic software agents, where highly qualitative studies are still sparse. First, we will use a randomised controlled design and we will apply ITT analysis to avoid a possibly overestimated effect of the intervention. Second, the writing intervention is highly standardised due to the completely automated instructions and feedback given by SISU. Third, we will collect data on many variables and time points to enable moderator and mediator analysis on an explanatory level. The knowledge of how and for whom interventions work best is an important prerequisite improving their content and target groups.94
Third, although effectiveness with the same range of expected effect size (and at the same cost) can be expected from other fully automated unguided intervention formats (eg, 95), this is the very first study to evaluate a software agent-delivered intervention. As it can be assumed that not everybody or every population prefers the same kind of delivery format, it is important to evaluate a broad variety of formats to enable adaptability. In this respect, the present study makes an important contribution.
Another strength concerns our recruitment strategy. We will be able to reach a wide range of participants by broad online and offline recruitment in Germany, Austria and Switzerland. Recruitment strategies might help to gain knowledge on feasibility and effectiveness of SISU in a broad range of adult people living with low psychological well-being. However, people living with high psychological well-being, which for example, want to further invest in their mental health, will be excluded. Thus, it is not possible to say whether SISU is useful in people with already high psychological well-being. Furthermore, new technologies like chatbots could be especially attractive to the youth, who were excluded as a population. Thus, it remains unclear if SISU could be useful in younger people living with low psychological well-being. Self-selection bias could lead to a population which has an internet affinity. Only participants with internet access and email address can be included in the intervention. Whereas this is probably not relevant for younger people, it might still be a potential reason for selection and limited generalisability, especially with regard to elder people. To rule out a potential gender bias due to a male or female software agent, SISU was conceptualised gender neutral so that members of all sexes feel equally addressed.
Usually, moderate to high dropout rates are a problem within online interventions, which needs to be addressed in the planning of a study.96 In our feasibility trial, 39% of the participants dropped out during study progress (assessment dropout), which could be (partly) explained by organisational effort providing informed consent and unfulfilled expectations concerning the intervention or the interaction with SISU. Nonetheless, the dropout rate of 14% during the intervention with SISU (intervention dropout) is comparably low, which could be traced back to the responsiveness/guidance by SISU. Those have been shown to improve intervention adherence.97 For the present trial we maintained these successfully tested techniques.
Another possible limitation is the use of a WL. This can be associated with overestimation of effects compared with psychological placebo or no intervention.98 If SISU shows its effectiveness compared with a WL, a next step should be to compare it with an active control group like, for example, participants receiving a pamphlet with instructions for doing mindfulness exercises at home. Furthermore, a potential methodological confound concerns blinding. Participants are not blinded towards the primary outcome and could possibly answer in a socially desirable way. However, as participants are unlikely to know the study team personally, test manager effects might be low. Another methodological problem could arise from assessment reactivity. Frequent assessments can trigger self-reflection which can lead to an incremental effect regardless of the intervention.99 However, this is a general problem which can be particularly noticeable in control groups and in groups which receive low-threshold intervention offers.
Last but not least, the planned analyses are based on classic inferential statistics to test the significance of group differences. A sample size calculation (G*Power) was performed to plan the sample size accordingly. However, recent evidence emphasises that it might be fruitful not to test for differences from zero. Instead, Bayesian methods could be used. They allow discovering uncertainties of the effects of treatments instead of solemnly focusing on dichotomising evidence into significant and not significant.100 If this trial points towards the usefulness/effectiveness of SISU, future trials could substantiate results using Bayesian methods.
Ethics and dissemination
This trial has been approved by the ethics committee of the Ulm University (No. 448/18, 18.02.2019) and registered in the German Clinical Trials Register (DRKS-ID: DRKS00016799) on 25 April 2019. Written informed consent for participation in the study will be obtained from all participants prior to their involvement. Participants will receive written information on study conditions, data security, publication of anonymised results, voluntariness of participation and the right to leave the study at all times. They will also be informed that in case of study withdrawal, they will be able to decide whether they want their data to be included in the analysis or to be deleted. Additionally, participants will be asked for permission for the research team to share relevant data with people from regulatory authorities, where necessary. This trial will only involve the collection and storage of self-report data, not of biological specimens. Data collection will be pseudonymised and data will only be accessed by authorised study personnel obliged to secrecy. After data collection is completed, personalised information will be deleted and all data will be completely anonymised. All participant information will be stored securely in locked file cabinets and/or password protected in a secured cloud storage with restricted access. All reports, data collection and administrated forms will be identified by a coded ID number only to maintain participant confidentiality. All records that contain names or other personal identifiers, such as informed consent forms (online supplemental file 1), will be stored separately from study records identified by ID number. Listings that link participant ID numbers to other identifying information will be stored in separate password-protected files with limited access. According to German law, data will only be shared with parties outside the project team in anonymised form. Trial results will be submitted for publication in a peer-reviewed journal and presented at conferences.
Supplemental material
Acknowledgments
We kindly thank Lisa Schischke and Christina Wirth for helping to test the revised version of SISU and the surveys as well as for helping with study recruitment. We kindly thank Jana Moos for performing the allocation procedure.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @EileenBendig, @b_erb
Contributors EB had the idea of SISU. SISU was developed by the Department of Clinical Psychology and Psychotherapy and the Institute of Distributed Systems at Ulm University (lead developer DM, BE and EB). EB and HB designed and planned the study. EB and HB supervised the study. EB and LW operatively performed the study. EB drafted the manuscript, all other authors critically revised the work for important intellectual content. All authors (HB, BE, DE, A-MK, NB, LW, DE) approved the final version to be published and agree to be accountable for all aspects of the work.
Funding No external financing. Budget funds of the Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, Ulm University, Albert-Einstein-Allee 47, 89069 Ulm, Germany.
Competing interests HB reports to have received consultancy fees and fees for lectures/workshops from chambers of psychotherapists and training institutes for psychotherapists in the e-mental health context. A-MK has received fees for lectures/workshops from chambers of psychotherapists and health insurance companies. DE reports to have received consultancy fees/served in the scientific advisory board from several companies such as Minddistrict, Lantern, Schoen Kliniken and German health insurance companies. He is a stakeholder of the Institute for Health Training online (GET.ON), which aims to implement scientific findings related to digital health interventions into routine care. All other authors declare not to have competing interests.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.