Searches

We will use an existing database of psychological treatments for depression that has been described in detail elsewhere29 and that has been used in a series of published meta-analyses (www.evidencebasedpsychotherapies.org). This database is annually updated through comprehensive literature searches in the bibliographic databases of PubMed, PsycINFO, Embase and the Cochrane Library. The latest search was performed from database inception to 1 January 2018. The search strings use a combination of Index and free terms of psychological treatments and depression (full strings of PubMed are provided in online supplementary appendix A). In these searches, two independent reviewers (PC and EK) examined 14 290 titles and abstracts. From these 14 290 abstracts, 2092 full-text papers were retrieved and examined for possible inclusion in the database. In addition to this database, we also checked the primary studies from meta-analyses of psychological treatments for depression to ensure that no published studies will be missed. We will also ask the primary authors of the eligible studies if they are aware of any other study that has been conducted in the examined field.

Eligibility criteria

In the present IPD-NMA, we will include RCTs that examined at least two of the following interventions: guided iCBT, self-guided iCBT or a control comparison condition for treating adults with depression. No language restrictions will be applied. The participants, interventions, comparisons and outcomes will be explained in detail in the following.

Participants

Participants will be adults (≥18 years of age) with depression based on either a diagnosis established by a clinical interview (eg, the Mini International Neuropsychiatric Interview30) or elevated depressive symptoms rated by any self-report outcome measure (eg, Beck Depression Inventory (BDI); based on the cut-off scores originally used by the eligible studies).31 Studies including participants with comorbid common mental health disorders will not be excluded as long as they include people with depression primarily. However, we will exclude studies on comorbid depression and cognitive impairment/dementia or severe mental illness (eg, bipolar disorder and psychosis). We will also exclude studies with a primary focus on a medical illness.

Interventions

We will include only studies that examined a guided and/or a self-guided CBT delivered via the internet. We have based the definition of CBT on an expert taxonomy of psychotherapy for depression.32 According to this taxonomy, CBT is defined as a psychotherapeutic intervention that employs cognitive restructuring as either a core element or an important component of the treatment.32 Therefore, interventions falling under the broader CBT umbrella (eg, problem solving therapy and behavioural activation) will be excluded. This study also focuses on interventions delivered via the internet, so that older studies that used stand-alone computerised CBT will not be included. Moreover, we will exclude studies with face-to-faced contact (eg, blended interventions or teleconferences).

In the present IPD-NMA, guided iCBT is defined as an intervention based on CBT self-help material and delivered via the internet with some form of guidance related to the therapeutic content. This guidance can be provided either by a professional (psychiatrist or clinical psychologist) or e-coaches (eg, trained clinical psychology master students or trained paraprofessionals), and it is provided by electronic means (eg, email or brief telephone calls). Self-guided iCBT is defined as an intervention provided without any support related to the therapeutic content. Studies focusing on self-guided iCBT with technical support (related to the online platform use) will be classified as self-guided iCBT interventions with technical support. Email or telephone support by administrative staff will be classified here. The effect of self-guided iCBT with or without technical support separately will be examined in a subgroup analysis. Face-to-face contact with therapists is allowed for baseline interview only. Studies focusing on blended iCBT (combined internet and face-to-face CBT in one treatment protocol) will be excluded. In a blended iCBT, patients have face-to-face sessions with psychotherapists. Thus, we decided to exclude blended iCBT as it differs substantially from stand-alone online interventions with minimal therapeutic contact.

Comparisons

Studies will be included in the present IPD-NMA if they compare guided or self-guided iCBT to each other and/or to an inactive control condition, such as waiting list, attention placebo, no treatment and treatment as usual (TAU).

Therefore, the network of treatments in our IPD-NMA will include guided iCBT, self-guided iCBT, waiting list, attention placebo no treatment and TAU.

Outcomes

Our primary outcome will be post-treatment depression symptom severity on a continuous scale. We will include studies using a depression outcome measure, such as the BDI, the Center of Epidemiological Studies Depression Scale33 or the Patient Health Questionnaire-9.34 If a study uses more than one depression measure, preference will be given to the measure reported by the majority of the included studies. In case a study reports two or more outcome measures, none of which are used by the rest of the included studies, preference will be given to the measure listed as primary in this study. For the IPD-NMA, if the studies use different outcome measures, they will be converted into the most commonly used scale using the established conversion algorithms.35 36 Outcomes will be extracted at pretreatment/post-treatment and follow-up (at ≥6 months postrandomisation). To assess long-term outcomes, we will examine outcomes at ≥6 months postrandomisation. If a study includes multiple follow-up time points, we will use the longer follow-up assessment provided.

Prognostic factors and effect modifiers of treatment outcome

In this study, we will start from a wide range of patient-level variables, and we will explore their role as either prognostic factor (ie, variable that affects the disease progression equally for all the treatments in the network) or effect modifiers (ie, variables that have an impact on the relative effects of interventions). We will initially select candidate covariates based on previous literature findings and the availability of these variables in the included studies. For instance, previous IPD meta-analyses have shown that people with higher depressive symptoms at the baseline have more chances to remit after guided iCBT.13 However, baseline depression severity is not associated with greater symptom improvement after self-guided iCBT.23 37 Moreover, it has been shown that ethnic minorities are less likely to respond to guided iCBT compared with natives.13 Therefore, we will collect data related to baseline depression severity and ethnicity. Nevertheless, given that existing literature on effect modifiers is limited, we will also collect data on variables that are available across the majority of the eligible trials. Next, we will collect and merge into the IPD dataset all participants’ sociodemographic and clinical characteristics reported in the primary included trials. More specifically, we will include demographics such as age, gender, ethnicity, educational background, marital status and employment.

Furthermore, according to existing literature, there are several factors associated with poor prognosis of depression and differential response to treatment.38 For instance, it has been suggested that depression severity, comorbidity and number of previous episodes are associated with depression prognosis.39 Thus, in the present IPD-NMA, we will collect and merge variables related to clinical characteristics of participants, such as depression severity at the baseline, number of previous depressive episodes, duration of depression, comorbid symptoms of other mental disorders (anxiety disorders, eating disorders, alcohol misuse, insomnia, personality disorders and so on), comorbid physical illness (eg, diabetes, heart failure and so on), history of treatment and current use of antidepressant medications.

Data collection and aggregation

We will build on an existing database of internet-based interventions for depression. This database already includes the majority of the eligible trials, and it has been used by two previous IPD meta-analyses on guided13 and self-guided23 internet-based interventions for depression. Trials’ corresponding authors will be contacted for permission to use their primary datasets. In case a corresponding author cannot be reached, a follow-up email will be sent to the senior author of the trial. Reminders will be sent after 2 weeks, and if necessary, after 1 month. If no response is received after 1 month, the trial will be excluded as unavailable. Authors will be asked to provide their trial outcome raw data as well as data related to potentially important covariates (as described above). We will also collect study-level variables based on the published reports.

After collecting all primary datasets, two independent reviewers will perform an initial data check to ensure that each dataset matches with its published report. This initial data check will involve checking sociodemographic (eg, percentage of males and females) and clinical characteristics (eg, mean of depression scale scores at the baseline) and comparing those characteristics to the numbers reported on the published papers. In case the numbers do not match, we will contact the authors of the trials for clarifications. After each dataset is checked, two independent reviewers will merge the variables into the IPD dataset. Disagreements in coding will be resolved through discussion.

Risk of bias assessment

We will assess the risk of bias of the included studies in the following domains: (A) the random sequence generation, (B) the concealment of allocation to conditions, (C) the selective outcome reporting and (D) other possible sources of bias (eg, extreme baseline differences).40 The criteria for judgement will be based on the risk of bias assessment tool of the Cochrane Collaboration, which systematically assesses possible risk of bias in RCTs related to selection, performance, detection, attrition, reporting and other sources.40 In addition to the four domains mentioned above, this tool originally evaluates (E) the prevention of knowledge of the allocation by participants, personnel (performance bias) and assessors (assessment bias) and (F) the handling of incomplete outcome data. However, blinding of participants and personnel (performance bias) is mostly not possible in psychotherapy research. Thus, we will not rate this item since all psychotherapeutic trials are at high risk of performance bias. Regarding incomplete outcome data, a study will consider at high risk of bias if it has overall >50% study dropout and >30% imbalance in missing values between groups. Two independent reviewers will first perform the risk of bias assessment based on the published reports. Disagreements will be solved through discussion. When the judgement is unclear risk of bias, we will go back to the original authors for clarification.

Patient and public involvement

There was no patient or public involvement in the development of this manuscript.

IPD-NMA model

We will employ a one-stage IPD network meta-regression model to synthesise our data, set in a Bayesian background. Let us assume that patient i in study j received treatment treat_ij and that for this patient we have vectors of prognostic factors (PF_ij ) and effect modifiers (EM_ij ). To facilitate convergence of the model, all continuous covariates will be standardised by subtracting the overall mean and dividing by their corresponding standard deviation across all trials. When presenting the final results, we will back-transform the covariates to their original scales. For some patients, some of the covariates may be missing. For each covariate we will use a study-specific distribution to stochastically impute missing values, assuming that covariates are missing at random. For example, if for patient i in study j we have no information on his/her age, we will impute it assuming  . For continuous covariates, we will use normal distributions; for binary covariates, we will use a Bernoulli distribution and so on.

Next, let us assume that for each patient we observe a (continuous) outcome Y_ij . The model will be as follows:

In this expression, T_ref  is the reference treatment of the network, which we will take it to be no treatment. S _j ² is the study-specific variance of the outcome. β is the vector of regression coefficients associated to prognostic factors. γtreat_ij  is the vector of regression coefficients for effect modification, for the comparison treat_ij  versus the reference. In essence, γtreat_ij captures the treatment–covariate interactions.

d_treatij,i is the average relative effect of treat_ij  versus the reference treatment of the network in study i, that is, the relative effects for the case when effect modifiers are equal to zero. Note that the relative treatment effects are estimated separately in each study, that is, information from patients from different studies is not pooled directly together. The consistency equations required for NMA are automatically incorporated in the model. We will assume random effects, that is,  , where τ will be the random effects SD. This will be set common for all comparisons versus the reference treatment—a usual assumption in NMA.41 For the case of multiarm studies, we will include multivariate distributions instead, taking into account the correlation of random effects. The parameter δ_treatij corresponds to the pooled average treatment effects of treatment treat_ij  versus the reference treatment of the network (no treatment).

Note that the imputation models for the missing covariates as well as the estimation model will be fitted simultaneously.

If we will not be able to fit a one-stage IPD NMA model, for example, due to computational constraints, we will fit a two-stage model IPD-NMA model instead. In case there is information on many patient-level covariates available in all the studies (or in the great majority of the studies), we will employ a variable selection method (such as Bayesian Lasso)42 to identify the most prominent ones. In our results, we will report the estimated relative treatment effects for various combinations of the covariate values.

Model fitting

We will fit our IPD-NMA model in OpenBUGS.43 We will use multiple independent chains for the analysis, and we will check convergence using the Brooks-Gelman-Rubin criterion.44 For all model parameters, we will use vague prior distributions. We will report the posterior median and 95% credible intervals. We will make all software codes used for our analyses freely available by uploading them in github.com

Building a prediction model

Using the estimated parameters of the IPD-NMA model, we will develop and validate a prediction model that will aim to provide personalised predictions about the efficacy of the two interventions.45 The inputs of the model will be patient-specific values for prognostic factors and effect modifiers, and the output will be a prediction of the outcome under guided or self-guided iCBT. In order to cross-validate this model (ie, to assess its predictive performance), we will follow an internal–external cross-validation method.46 More specifically, we will take one study out at a time, develop our model using the rest of the studies and test the model in the left out study by calculating the mean bias and the mean squared error of the predictions. We will repeat this procedure for all studies. We will report the mean bias and the mean squared error of the predictions across all studies. Via this approach, we will have a better understanding of the external validity of our prediction model, and we will visualise the expected heterogeneity regarding our model’s performance. If the overall performance of the model is deemed satisfactory, we will develop our final model using all studies. In that case, we will also develop an online tool that will implement the model, so as to facilitate its use in clinical practice.45 Note that the main advantage of the internal–external approach to cross-validation is that it does not waste any data, that is, all data will be used for the development of the final model.

Missing data

The model imputes missing values on the covariates assuming they are missing at random. Also, it assumes that missing outcomes are missing at random. If the overall rate of missing outcome values in one study is large (>50%), we will exclude the study from our analysis. If a study has a large imbalance in missing values between the examined conditions (>30%), we will perform a sensitivity analysis where we will assume that the outcomes are missing not at random (MNAR), that is, we will fit a model where the probability of dropping out is associated with the unobserved outcome.28 If we deem that our results are not robust, that is, if we observe clinically important differences between the primary analysis and the MNAR analysis, we will exclude studies with large imbalances.

Heterogeneity and inconsistency

We will measure heterogeneity in the included studies by estimating a common heterogeneity parameter τ². We will then compare this estimate with its empirical distribution.47 48 We will assess the residual inconsistency in the network by estimating the difference between direct and indirect evidence in each loop of the network. We will do this by adding a number of inconsistency factors (IFs) to the model, depending on the number of closed loops in the network.

Publication bias and data availability bias

We will first examine small study effects by visually inspecting the contour enhanced funnel plots of pairwise meta-analyses for our outcome measure. We will also produce a comparison-adjusted funnel plot. If asymmetry is observed and if we have 10 or more studies per comparison, we will test for publication bias using the Egger’s test49 for the continuous outcomes and Peters’s test for the binary ones.50 Moreover, it is possible that we will not receive all the datasets from the eligible studies. In this case, we will calculate the effects of the available and the unavailable trial to test whether they differ significantly. If we identify discrepancies, we will report them and take them into account in the interpretation of our outcomes.

Evaluation of certainty of evidence for the network estimates

We will assess the certainty of evidence of each network estimate of our two primary outcomes with the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) framework, which characterises the quality of a body of evidence on the basis of the study limitations, imprecision, inconsistency, indirectness and publication bias.51