Introduction Although guided forms of internet-based cognitive–behavioural therapy (iCBT) result in a substantial reduction in depression, it seems that the most scalable way to deliver iCBT is without guidance. However, direct evidence on the comparison between guided and self-guided iCBT is scarce. Moreover, it is unclear which types of patients may benefit more from each of these two forms of iCBT. Network meta-analysis (NMA) using individual participant data (IPD) offers a way to assess the relative efficacy of multiple (>2) interventions. Moreover, it maximises our power to detect patient-level characteristics (covariates) that have an important effect on the efficacy of interventions. This protocol describes the procedures of an IPD-NMA, which aims at examining the relative efficacy of guided compared with self-guided iCBT and at identifying predictors and moderators of treatment outcome.
Methods and analysis We will use an existing database on psychotherapies for adult depression to identify eligible studies. This database has been updated up to 1 January 2018, through literature searches in PubMed, Embase, PsycINFO and Cochrane Library. The outcome of this IPD-NMA is reduction in depressive symptoms severity. We will fit the model in a Bayesian setting. After fitting the model, we will report the relative treatment effects for different types of patients, and we will discuss the clinical implications of our findings. Based on the results from the IPD-NMA model, we will develop and validate a personalised prediction model, aiming to provide patient-level predictions about the effects of the interventions.
Ethics and dissemination An ethical approval is not required for this study. The results will be published in a peer-review journal. These results will guide clinical decisions about the most efficient way to allocate iCBT resources, thereby increasing the scalability of this innovative therapeutic approach.
- individual patient data
- network meta-analysis
- prediction models
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
We will conduct an individual patient data network meta-analyses to examine all available evidence on the comparison between guided and self-guided internet-based cognitive–behavioural therapy (iCBT) for depression.
We will develop and validate a personalised prediction model indicating who may benefit from guided and who from self-guided iCBT for depression.
This will be the largest individual patient data meta-analysis, which will directly compare the effects of guided and self-guided iCBT using an innovative methodological approach.
It is possible that we will not receive all primary datasets from all eligible trials. However, we expect to gather the majority of existing data.
There is also a possibility that studies have not collected all variables of interest. Thus, we will not be able to examine all candidate moderators.
Depression is one of the most common mental health disorders, with debilitating consequences for individuals and society.1 Lifetime prevalence rates of depression range from 6.5% in Japan to 21% in France,2 while according to the WHO, over 300 million people suffer from this disorder worldwide.3 Depression is a highly prevalent and a very disabling disorder, which is expected to be the largest contributor to the global burden of disease by 2030.4 These alarming observations underline the necessity to provide effective treatment to all individuals with depression in a timely manner.
Although there are effective treatments for depression, a considerable number of those affected do not seek or receive help for their condition.5 6 Barriers to treatment include, among others, the high cost of treatment, the limited availability of trained clinicians and the fear of stigmatisation.7 8 Therapeutic approaches, such as internet-based interventions, have the potential to overcome many of the treatment barriers, thereby increasing access to treatment. A growing body of literature suggests that internet-based cognitive–behavioural therapy (iCBT) leads to substantial improvements on depression.9–13 It can either be delivered with some form of therapeutic guidance (guided interventions) or without any support related to the therapeutic content (self-guided interventions).14 15
It has been shown that for the treatment of depression, guided forms of iCBT have larger effects than controls13 and similar effects with face-to-face psychotherapy.16 Moreover, it has been shown that individuals with depression are more likely to respond and remit after receiving guided iCBT13 while they have a lower probability of deteriorating.17 Despite these promising outcomes, the provision of guided iCBT depends on the availability of therapists or coaches. Thus, scaling-up guided iCBT remains a challenge, especially in low-resource settings.18 Furthermore, it may also not be necessarily cost-effective in high-income countries. Thus, it might be the case that the most efficient way to scale-up iCBT would be to provide it without therapeutic guidance.18
Clinical trials that have examined self-guided forms of iCBT have shown a wide range of possible therapeutic effects on depression varying from zero up to moderate effects.19–22 These different outcomes have been summarised in a recent individual patient data (IPD) meta-analysis, which has shown that self-guided iCBT results in small effects in treating depression when compared with control conditions.23 These effects, however, are much smaller in magnitude to those observed in trials comparing guided iCBT to controls.12 In light of this difference in the effect sizes, it has been suggested that guided iCBT is more effective compared with unguided iCBT.24 25 However, this assumption is mostly based on indirect comparisons, while evidence derived from direct comparisons is scarce.26 Therefore, it remains unclear whether guided and self-guided iCBT result in significantly different outcomes in treating depression.25
Furthermore, no study has ever examined who may benefit from guidance and who may not. For instance, individuals with severe depression may benefit from guidance, while those with mild symptoms may be more capable of following the intervention content without support. Understanding for whom guided or self-guided iCBT is more effective will help us in allocating therapy resources in the most efficient way, thereby increasing iCBT’s scalability. It is, thus, important to further examine the effects of guided versus self-guided iCBT and to gain a deeper understanding of who may benefit more from which form of this therapeutic approach.
Individual randomised controlled trials (RCTs) are expected to be underpowered to answer either of these two questions. Conventional pair-wise meta-analyses that use aggregated (study-level) data may be able to overcome the issue of statistical power if enough direct comparison trials are available, but even in this case, they cannot identify which patient-level characteristics affect the efficacy of the interventions. Novel meta-analytic approaches enable us to synthesise IPD and use both direct and indirect treatment comparisons, that is, comparisons via a common comparator. IPD network meta-analyses (IPD-NMA) offer a way to maximise the statistical power to detect treatment-covariate interactions and to provide an overall estimate of treatment effects, using information from trials involving different treatment comparisons.27 28 In the current study, we aim to conduct an IPD-NMA to examine the relative efficacy of guided compared with self-guided iCBT in treating adult depression and to identify who may benefit more from each type of iCBT. We also aim to develop and validate a personalised prediction model. This will use information on a particular patients’ characteristics and will provide a prediction of the progression of the depression severity under guided or self-guided iCBT, so as to facilitate a personalised choice of the treatment.
We will use an existing database of psychological treatments for depression that has been described in detail elsewhere29 and that has been used in a series of published meta-analyses (www.evidencebasedpsychotherapies.org). This database is annually updated through comprehensive literature searches in the bibliographic databases of PubMed, PsycINFO, Embase and the Cochrane Library. The latest search was performed from database inception to 1 January 2018. The search strings use a combination of Index and free terms of psychological treatments and depression (full strings of PubMed are provided in online supplementary appendix A). In these searches, two independent reviewers (PC and EK) examined 14 290 titles and abstracts. From these 14 290 abstracts, 2092 full-text papers were retrieved and examined for possible inclusion in the database. In addition to this database, we also checked the primary studies from meta-analyses of psychological treatments for depression to ensure that no published studies will be missed. We will also ask the primary authors of the eligible studies if they are aware of any other study that has been conducted in the examined field.
In the present IPD-NMA, we will include RCTs that examined at least two of the following interventions: guided iCBT, self-guided iCBT or a control comparison condition for treating adults with depression. No language restrictions will be applied. The participants, interventions, comparisons and outcomes will be explained in detail in the following.
Participants will be adults (≥18 years of age) with depression based on either a diagnosis established by a clinical interview (eg, the Mini International Neuropsychiatric Interview30) or elevated depressive symptoms rated by any self-report outcome measure (eg, Beck Depression Inventory (BDI); based on the cut-off scores originally used by the eligible studies).31 Studies including participants with comorbid common mental health disorders will not be excluded as long as they include people with depression primarily. However, we will exclude studies on comorbid depression and cognitive impairment/dementia or severe mental illness (eg, bipolar disorder and psychosis). We will also exclude studies with a primary focus on a medical illness.
We will include only studies that examined a guided and/or a self-guided CBT delivered via the internet. We have based the definition of CBT on an expert taxonomy of psychotherapy for depression.32 According to this taxonomy, CBT is defined as a psychotherapeutic intervention that employs cognitive restructuring as either a core element or an important component of the treatment.32 Therefore, interventions falling under the broader CBT umbrella (eg, problem solving therapy and behavioural activation) will be excluded. This study also focuses on interventions delivered via the internet, so that older studies that used stand-alone computerised CBT will not be included. Moreover, we will exclude studies with face-to-faced contact (eg, blended interventions or teleconferences).
In the present IPD-NMA, guided iCBT is defined as an intervention based on CBT self-help material and delivered via the internet with some form of guidance related to the therapeutic content. This guidance can be provided either by a professional (psychiatrist or clinical psychologist) or e-coaches (eg, trained clinical psychology master students or trained paraprofessionals), and it is provided by electronic means (eg, email or brief telephone calls). Self-guided iCBT is defined as an intervention provided without any support related to the therapeutic content. Studies focusing on self-guided iCBT with technical support (related to the online platform use) will be classified as self-guided iCBT interventions with technical support. Email or telephone support by administrative staff will be classified here. The effect of self-guided iCBT with or without technical support separately will be examined in a subgroup analysis. Face-to-face contact with therapists is allowed for baseline interview only. Studies focusing on blended iCBT (combined internet and face-to-face CBT in one treatment protocol) will be excluded. In a blended iCBT, patients have face-to-face sessions with psychotherapists. Thus, we decided to exclude blended iCBT as it differs substantially from stand-alone online interventions with minimal therapeutic contact.
Studies will be included in the present IPD-NMA if they compare guided or self-guided iCBT to each other and/or to an inactive control condition, such as waiting list, attention placebo, no treatment and treatment as usual (TAU).
Therefore, the network of treatments in our IPD-NMA will include guided iCBT, self-guided iCBT, waiting list, attention placebo no treatment and TAU.
Our primary outcome will be post-treatment depression symptom severity on a continuous scale. We will include studies using a depression outcome measure, such as the BDI, the Center of Epidemiological Studies Depression Scale33 or the Patient Health Questionnaire-9.34 If a study uses more than one depression measure, preference will be given to the measure reported by the majority of the included studies. In case a study reports two or more outcome measures, none of which are used by the rest of the included studies, preference will be given to the measure listed as primary in this study. For the IPD-NMA, if the studies use different outcome measures, they will be converted into the most commonly used scale using the established conversion algorithms.35 36 Outcomes will be extracted at pretreatment/post-treatment and follow-up (at ≥6 months postrandomisation). To assess long-term outcomes, we will examine outcomes at ≥6 months postrandomisation. If a study includes multiple follow-up time points, we will use the longer follow-up assessment provided.
Prognostic factors and effect modifiers of treatment outcome
In this study, we will start from a wide range of patient-level variables, and we will explore their role as either prognostic factor (ie, variable that affects the disease progression equally for all the treatments in the network) or effect modifiers (ie, variables that have an impact on the relative effects of interventions). We will initially select candidate covariates based on previous literature findings and the availability of these variables in the included studies. For instance, previous IPD meta-analyses have shown that people with higher depressive symptoms at the baseline have more chances to remit after guided iCBT.13 However, baseline depression severity is not associated with greater symptom improvement after self-guided iCBT.23 37 Moreover, it has been shown that ethnic minorities are less likely to respond to guided iCBT compared with natives.13 Therefore, we will collect data related to baseline depression severity and ethnicity. Nevertheless, given that existing literature on effect modifiers is limited, we will also collect data on variables that are available across the majority of the eligible trials. Next, we will collect and merge into the IPD dataset all participants’ sociodemographic and clinical characteristics reported in the primary included trials. More specifically, we will include demographics such as age, gender, ethnicity, educational background, marital status and employment.
Furthermore, according to existing literature, there are several factors associated with poor prognosis of depression and differential response to treatment.38 For instance, it has been suggested that depression severity, comorbidity and number of previous episodes are associated with depression prognosis.39 Thus, in the present IPD-NMA, we will collect and merge variables related to clinical characteristics of participants, such as depression severity at the baseline, number of previous depressive episodes, duration of depression, comorbid symptoms of other mental disorders (anxiety disorders, eating disorders, alcohol misuse, insomnia, personality disorders and so on), comorbid physical illness (eg, diabetes, heart failure and so on), history of treatment and current use of antidepressant medications.
Data collection and aggregation
We will build on an existing database of internet-based interventions for depression. This database already includes the majority of the eligible trials, and it has been used by two previous IPD meta-analyses on guided13 and self-guided23 internet-based interventions for depression. Trials’ corresponding authors will be contacted for permission to use their primary datasets. In case a corresponding author cannot be reached, a follow-up email will be sent to the senior author of the trial. Reminders will be sent after 2 weeks, and if necessary, after 1 month. If no response is received after 1 month, the trial will be excluded as unavailable. Authors will be asked to provide their trial outcome raw data as well as data related to potentially important covariates (as described above). We will also collect study-level variables based on the published reports.
After collecting all primary datasets, two independent reviewers will perform an initial data check to ensure that each dataset matches with its published report. This initial data check will involve checking sociodemographic (eg, percentage of males and females) and clinical characteristics (eg, mean of depression scale scores at the baseline) and comparing those characteristics to the numbers reported on the published papers. In case the numbers do not match, we will contact the authors of the trials for clarifications. After each dataset is checked, two independent reviewers will merge the variables into the IPD dataset. Disagreements in coding will be resolved through discussion.
Risk of bias assessment
We will assess the risk of bias of the included studies in the following domains: (A) the random sequence generation, (B) the concealment of allocation to conditions, (C) the selective outcome reporting and (D) other possible sources of bias (eg, extreme baseline differences).40 The criteria for judgement will be based on the risk of bias assessment tool of the Cochrane Collaboration, which systematically assesses possible risk of bias in RCTs related to selection, performance, detection, attrition, reporting and other sources.40 In addition to the four domains mentioned above, this tool originally evaluates (E) the prevention of knowledge of the allocation by participants, personnel (performance bias) and assessors (assessment bias) and (F) the handling of incomplete outcome data. However, blinding of participants and personnel (performance bias) is mostly not possible in psychotherapy research. Thus, we will not rate this item since all psychotherapeutic trials are at high risk of performance bias. Regarding incomplete outcome data, a study will consider at high risk of bias if it has overall >50% study dropout and >30% imbalance in missing values between groups. Two independent reviewers will first perform the risk of bias assessment based on the published reports. Disagreements will be solved through discussion. When the judgement is unclear risk of bias, we will go back to the original authors for clarification.
Patient and public involvement
There was no patient or public involvement in the development of this manuscript.
We will employ a one-stage IPD network meta-regression model to synthesise our data, set in a Bayesian background. Let us assume that patient i in study j received treatment treatij and that for this patient we have vectors of prognostic factors (PFij ) and effect modifiers (EMij ). To facilitate convergence of the model, all continuous covariates will be standardised by subtracting the overall mean and dividing by their corresponding standard deviation across all trials. When presenting the final results, we will back-transform the covariates to their original scales. For some patients, some of the covariates may be missing. For each covariate we will use a study-specific distribution to stochastically impute missing values, assuming that covariates are missing at random. For example, if for patient i in study j we have no information on his/her age, we will impute it assuming . For continuous covariates, we will use normal distributions; for binary covariates, we will use a Bernoulli distribution and so on.
Next, let us assume that for each patient we observe a (continuous) outcome Yij . The model will be as follows:
In this expression, Tref is the reference treatment of the network, which we will take it to be no treatment. S j 2 is the study-specific variance of the outcome. β is the vector of regression coefficients associated to prognostic factors. γtreatij is the vector of regression coefficients for effect modification, for the comparison treatij versus the reference. In essence, γtreatij captures the treatment–covariate interactions.
dtreatij,i is the average relative effect of treatij versus the reference treatment of the network in study i, that is, the relative effects for the case when effect modifiers are equal to zero. Note that the relative treatment effects are estimated separately in each study, that is, information from patients from different studies is not pooled directly together. The consistency equations required for NMA are automatically incorporated in the model. We will assume random effects, that is, , where τ will be the random effects SD. This will be set common for all comparisons versus the reference treatment—a usual assumption in NMA.41 For the case of multiarm studies, we will include multivariate distributions instead, taking into account the correlation of random effects. The parameter δtreatij corresponds to the pooled average treatment effects of treatment treatij versus the reference treatment of the network (no treatment).
Note that the imputation models for the missing covariates as well as the estimation model will be fitted simultaneously.
If we will not be able to fit a one-stage IPD NMA model, for example, due to computational constraints, we will fit a two-stage model IPD-NMA model instead. In case there is information on many patient-level covariates available in all the studies (or in the great majority of the studies), we will employ a variable selection method (such as Bayesian Lasso)42 to identify the most prominent ones. In our results, we will report the estimated relative treatment effects for various combinations of the covariate values.
We will fit our IPD-NMA model in OpenBUGS.43 We will use multiple independent chains for the analysis, and we will check convergence using the Brooks-Gelman-Rubin criterion.44 For all model parameters, we will use vague prior distributions. We will report the posterior median and 95% credible intervals. We will make all software codes used for our analyses freely available by uploading them in github.com
Building a prediction model
Using the estimated parameters of the IPD-NMA model, we will develop and validate a prediction model that will aim to provide personalised predictions about the efficacy of the two interventions.45 The inputs of the model will be patient-specific values for prognostic factors and effect modifiers, and the output will be a prediction of the outcome under guided or self-guided iCBT. In order to cross-validate this model (ie, to assess its predictive performance), we will follow an internal–external cross-validation method.46 More specifically, we will take one study out at a time, develop our model using the rest of the studies and test the model in the left out study by calculating the mean bias and the mean squared error of the predictions. We will repeat this procedure for all studies. We will report the mean bias and the mean squared error of the predictions across all studies. Via this approach, we will have a better understanding of the external validity of our prediction model, and we will visualise the expected heterogeneity regarding our model’s performance. If the overall performance of the model is deemed satisfactory, we will develop our final model using all studies. In that case, we will also develop an online tool that will implement the model, so as to facilitate its use in clinical practice.45 Note that the main advantage of the internal–external approach to cross-validation is that it does not waste any data, that is, all data will be used for the development of the final model.
The model imputes missing values on the covariates assuming they are missing at random. Also, it assumes that missing outcomes are missing at random. If the overall rate of missing outcome values in one study is large (>50%), we will exclude the study from our analysis. If a study has a large imbalance in missing values between the examined conditions (>30%), we will perform a sensitivity analysis where we will assume that the outcomes are missing not at random (MNAR), that is, we will fit a model where the probability of dropping out is associated with the unobserved outcome.28 If we deem that our results are not robust, that is, if we observe clinically important differences between the primary analysis and the MNAR analysis, we will exclude studies with large imbalances.
Heterogeneity and inconsistency
We will measure heterogeneity in the included studies by estimating a common heterogeneity parameter τ2. We will then compare this estimate with its empirical distribution.47 48 We will assess the residual inconsistency in the network by estimating the difference between direct and indirect evidence in each loop of the network. We will do this by adding a number of inconsistency factors (IFs) to the model, depending on the number of closed loops in the network.
Publication bias and data availability bias
We will first examine small study effects by visually inspecting the contour enhanced funnel plots of pairwise meta-analyses for our outcome measure. We will also produce a comparison-adjusted funnel plot. If asymmetry is observed and if we have 10 or more studies per comparison, we will test for publication bias using the Egger’s test49 for the continuous outcomes and Peters’s test for the binary ones.50 Moreover, it is possible that we will not receive all the datasets from the eligible studies. In this case, we will calculate the effects of the available and the unavailable trial to test whether they differ significantly. If we identify discrepancies, we will report them and take them into account in the interpretation of our outcomes.
Evaluation of certainty of evidence for the network estimates
We will assess the certainty of evidence of each network estimate of our two primary outcomes with the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) framework, which characterises the quality of a body of evidence on the basis of the study limitations, imprecision, inconsistency, indirectness and publication bias.51
It has been claimed that guided iCBT is more effective than self-guided iCBT, based on indirect literature evidence, but this is largely an unexplored hypothesis.24 25 It also remains unclear which patients may benefit from therapeutic guidance. In the present paper, we have outlined an IPD-NMA, which aims to examine the relative efficacy of guided compared with self-guided iCBT and identify who may benefit more from which form of iCBT. IPD meta-analyses maximise statistical power and precision, thereby allowing to detect a true effect size and to test prognostic factors and relative effect modifiers of treatment outcome. Furthermore, network meta-analyses provide reliable evidence combining trials, which examine direct and indirect comparisons. The IPD-NMA is an innovative meta-analytic tool, which will enable us to test long-standing assumptions about the relative efficacy of guided versus self-guided iCBT for depression.
There are several limitations that should be expected. First, there is a possibility that we will not receive all primary datasets from all eligible trials. For instance, some of the trials on iCBT have been conducted more than 10 years ago. It is, therefore, possible that primary authors of these trials do not have access to the datasets anymore due to ethical restrictions regarding the maximum time that data can be stored. However, most of the iCBT trials are relatively recent; thus, we expect to receive the majority of the eligible datasets. Second, there is a possibility that studies have not examined all variables of interest. As a result, we will not be able to examine all moderators. Third, most of trials on iCBT have recruited participants through community, thereby limiting the generalisability of the outcomes to clinical samples.23
Nevertheless, we believe that the aforementioned benefits of the IPD-NMA approach will counterbalance potential limitations. Therefore, we hope that this study will shed light on the comparison between guided and self-guided iCBT and help us to gain a better understanding of who may benefit from these treatments. This knowledge will guide clinical decisions about the most efficient way to allocate iCBT resources, thereby increasing the scalability of this innovative therapeutic approach.
Contributors EK and OE drafted the study protocol. All the authors provided input into the study design and helped in the protocol writing. EK contributed to the original data acquisition. PC and HR helped with data acquisition and administration for IPD-NMA. TAF, HR and PC supervised the overall conduct of the study. All the authors read and approved the final protocol. EK is the guarantor of the review.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Patient consent for publication Not required.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.