Eligibility criteria

We will include all RCTs that compared any form of iCBT against another form of iCBT or a control intervention in the treatment of adults with depression.

We will include adult patients of both sexes aged 18 years or older, with a primary diagnosis of depression, either diagnosed as unipolar major or minor depression according to operationalised diagnostic criteria including Diagnostic and Statistical Manual 3rd Edition (DSM-III), 3rd revised edition (DSM-III-R), 4th edition (DSM-IV), 5th edition (DSM-5), International Classification of Diseases 10th revisioin (ICD-10) or any similar criteria, or judged so by elevated scores on any self-report depression scales. The effect of including studies without formal diagnosis of major depression will be tested in a sensitivity analysis. Inpatients as well as participants with bipolar depression or with psychotic depression will be excluded. Studies including participants with depression comorbid with another mental disorder will be included as long as their study inclusion criteria do not specifically include such comorbidities; however, we will exclude studies focusing on depression comorbid with another mental disorder, because in such cases, the CBT interventions would usually have components that target at such comorbidities in addition to depression. If studies include patients with depression or another mental disorder, we will include them if and only if we can focus on participants with depression based on individual data. Studies focusing on depression with another physical disorder (eg, diabetes and parkinsonism) or in special populations (eg, elderly, pregnancy, mother of autistic children and ethnic minority) will be included; however, the effect of including such studies will be examined in a sensitivity analysis. If a minority of the participants satisfy any of the above exclusion criteria, we will do our best to exclude such participants by employing IPD: when we do not have IPD or cannot exclude them using IPD because details are not available, then we will include such studies in our analysis when they constitute less than 20% of the total population.

iCBT must be a web-based or app-based programme using the internet to deliver the CBT contents. Computerised CBT will be included if it allows interaction between the programme and the individual. Telephone CBT will be excluded. If the use of the internet is limited to teleconferencing/videoconferencing, emails or text messaging, such programmes will also be excluded. When a web-based or app-based programme is used in the context of face-to-face sessions (so-called blended treatment), such programmes will be excluded because then the delivery of the CBT contents is no longer as assured as in pure iCBT. Encouragement to proceed with iCBT by telephone or face-to-face contact limited to an initial evaluation or orientation session will be allowed and considered to be a component of iCBT programme. Studies will be excluded if the iCBT programme allows participants to choose among the available components because such studies will not answer our clinical question.

The control conditions of interest will include the waiting list control, no treatment control, attention/psychological placebo control and treatment as usual. Different studies call different conditions as treatment as usual.21 In this cNMA study, treatment as usual must include pharmacotherapy; watchful waiting or follow-up by community nurses will be classified as attention/psychological placebo even when it is ‘treatment as usual’ in some settings. Pill placebo control will not be included in the present network as it is not decomposable into the components of our interest.

This study focuses on the acute phase treatment of the above-defined depression. The intervention can be of any duration; however, the influence of duration will be examined in a meta-regression.

We conceptualise CBT broadly as a psychotherapy involving any one of the following cognitive or behavioural skills’ training. Table 1 presents the different components of interest and their definitions. We will include all intervention or control conditions as long as they could be regarded as a combination of these components. Pharmacological coadministration will be allowed so long as there are no systematic differences in drug administration between study arms, and its presence will be denoted by dt (for definition see table 1). Table 2 provides how various forms of iCBT can be conceptualised from the component perspective.

Study identification and selection

We will use an existing database of psychological treatments for depression that is updated annually through comprehensive literature searches in the bibliographic databases of PubMed, PsycINFO, Embase and the Cochrane Library.22 The search strings use a combination of index and free terms of psychological treatments and depression.

Two independent researchers will check this database for relevant studies according to the eligibility criteria. Any disagreement will be resolved by discussion and, where necessary, in consultation with a third reviewer.

A further literature search will be conducted for studies published since the last update of the database in PubMed and the Cochrane Library. In addition, we will also check the primary studies from recent meta-analyses of internet treatments for depression to ensure that no published studies will be missed. We will also ask the primary authors of the eligible studies if they are aware of any other study that has been conducted in the examined field.

Data collection and integrity checks

Authors of the eligible studies will be contacted and requested to contribute their individual-level data. The corresponding author will be contacted first; if unreachable, a follow-up email will be sent to the senior author of the study. Reminders will be sent after 2 weeks and if necessary after 4 weeks. If no response is received after additional 4 weeks, this trial will be classified as ‘IPD unavailable’ and will be included in the analyses at the aggregate data level.

Authors will be asked to provide the individual-level raw data for their primary depression measures at baseline and at end of the acute phase treatment defined by the original study authors, as well as other potentially important covariates (PFs and EMs of treatment outcome; see section below).

After collecting all primary individual-level data, two independent reviewers will cross-examine the obtained data against the summary statistics (numbers and percentages or means and SDs) of the baseline demographic and clinical variables as reported in the publications of each study. In case the numbers do not match, we will contact the authors of the trials for clarification.

Identification of components

Two independent reviewers will determine the classification of all identified arms and their constituent components according to the definitions in tables 1 and 2, based on all available information including the publications, the trialled iCBT programmes and inquiry with the original investigators. Any disagreement will be solved by the two reviewers and, where necessary, in consultation with a third member of the review team.

Outcome measures

Our primary outcome is:

1. Depression severity as measured on a continuous scale for depression at the end of the acute phase treatment. We will include change in scores from pretreatment to post-treatment on any validated depression outcome measure, such as the Beck Depression Inventory, Beck Depression Inventory-II, Center of Epidemiological Studies Depression Scale, Patient Health Questionnaire-9 or Hamilton Rating Scale for Depression. If a study uses more than one depression measure, preference will be given to the measure reported by the majority of the included studies. In case a study reports two or more outcome measures, none of which are used by the rest of the included studies, preference will be given to the measure listed as primary in this study. If the studies use different outcome measures, they will be converted into the most commonly used scale using the established conversion algorithms.23 24 If this approach cannot cover a substantial proportion of the obtained data, scale scores will be standardised (transformed into z-scores) to create a common metric for depression severity.25–27

We will also examine the following two secondary outcome measures.

1. Dropouts from the end-of treatment assessment for any reason, as a proxy measure of treatment acceptability.

2. Dropouts from the treatment, defined as completing less than 80% of the contents of the programme. If the original authors used a different threshold/definition for ‘completion’ of the programme, we will use their definition.

PFs and EMs of treatment outcome

In this study, we will start from a wide range of patient-level variables and explore their role as either PFs or EMs. We will initially select candidate covariates based on previous literature findings and the availability of these variables in the included studies. The following is the list of candidate variables based on the published literature.28

Risk of bias assessment in individual studies

Two independent raters will assess the risk of bias in the included studies using the tool described in the Cochrane Collaboration Handbook44 as being at high risk of bias, low risk of bias or unclear risk of bias in the following domains: generation of allocation sequence, allocation concealment, blinding of study personnel and participants, blinding of outcome assessor, attrition, selective outcome reporting and other domains. Disagreements between the two independent assessors will be solved through discussion; when there still remains doubt, we will go back to the original authors for clarification.

Note that blinding of study personnel and participants is usually impossible in psychotherapy research; however, in studies of iCBT, it may be at least theoretically feasible if alternative active treatments are provided simply as ‘active treatments’. Note also that attrition bias will be evaluated in the case of studies with individual-level data, not according to the published report, but according to the provided dataset after missing data are imputed as per the statistical methods below. Some studies may still be rated at high risk of bias after imputation if large and unbalanced dropouts exist in the provided raw data. In the case of studies without individual data, attrition bias will be rated according to the publication.

In the above risk of bias assessment, we chose not to evaluate treatment fidelity as it will not be an issue with iCBT, which is fully structured. We also chose not to evaluate allegiance as its measurement is still controverted and will be especially difficult to measure in the comparisons against control conditions45 and may be less important in high-quality studies.46

Patient and public involvement

There was no patient or public involvement in the development of this manuscript.

Synthesis methods

We will start by performing an aggregated data network meta-analysis (NMA) on the treatment level to gain a first insight of the relative treatment effects.47 We will do this analysis both with and without trials for whom we do not have IPD.

Then we will continue to our main analysis. This will be done in two steps. In the first step, we will perform a variable selection procedure to decide which of the candidate covariates to include in the evidence synthesis model of the second step. To this end, we will fit a penalised linear regression model with an elastic net penalty48 to our primary outcome (continuous efficacy). We will explore all candidate covariates as well as their interactions with the components, aiming to identify the most important patient characteristics in terms of both prognosis and effect modification. Aiming to facilitate convergence of the models and to help the interpretation of the results of the second step, all continuous covariates will be standardised, that is, by subtracting the mean and dividing by the corresponding SD. The output of this first step of the analysis will be a list of PFs and EMs. Note that different EMs may be associated with different components, for example, patient age may be found to strongly interact with cognitive restructuring but not with behavioural activation.

At the second step, we will fit a one-stage IPD-NMA model.16 17 Let us assume that study j  compares two interventions and  , where comprises components and and comprises components and . Let us assume that patient i  in this study had an observed outcome  . Let us also assume that for this patient we have the corresponding PFs in the form of a vector  . Similarly, let us assume that the EMs for this patient, for each component, are in vectors  , ,  and  . The model can now be written as follows:

In this expression,  β is the vector of regression coefficients associated to PFs.  is the study-specific variance of  .  is the vector of regression coefficients for effect modification (component–covariate interaction), for component X . The ‘baseline’ effect, , will be assumed exchangeable across studies, that is,  is the study-specific estimate of relative effect for  versus , at the zero values of the (possibly standardised) EMs, that is, when all  . We will assume that  , where  τ denotes the random effects SD in the network. We will assume τ  to be common for all treatment comparisons in the network. Finally,  will be expressed as a linear combination of the effects of the components, for example, for this particular example we will set  . This model assumes additivity of treatment effects (Welton et al 14). In order to include in our analysis patients with missing values for one or more of the covariates (both PFs and EMs), we will use a study-specific imputation scheme. For example, if for patient i  in study j  there is no information regarding age, we will stochastically impute it by drawing from a study-specific distribution, that is,  . Here denotes the mean age of patients in this study and the corresponding variance.

For the secondary outcome ‘dropout from end-of-treatment assessment’, we will use a binomial likelihood for the observed data. We will synthesise data on the OR scale. The rest of the model will be as for the primary outcome.

For the secondary outcome ‘dropout from the treatment’, we will use a different modelling approach. This is because, by definition, for the inactive control conditions, treatment dropout is not observed. This implies that in studies with inactive controls, we cannot infer about relative effects. Such studies are expected to represent the vast majority of all eligible studies. Thus, for the analysis of this outcome, we will synthesise absolute treatment effects. We will model the probability of dropout for each active treatment, using a generalised linear model with a binomial likelihood

We will implement the variable-selection procedure of our analysis (step one) in R,49 using the glmnet package.50 The variable-selection procedure will only be performed for the primary outcome. We will implement the evidence-synthesis part (step 2) in OpenBUGS.51

Limitations of the proposed statistical model

The basic model assumes additivity of components and does not take account of possible interactions among components, that is, when some components may be particularly effective or ineffective in combination with some other components. The dataset will likely lack statistical power to test for such interactions. However, for components that are well represented in the network (ie, those which have been studied in many trials involving many participants), we will run exploratory analyses of some representative interactions.

The proposed model cannot discern the effect of the ordering of the components, but we expect that we will not have enough relevant data from the studies to explore this effect. If enough data become available, we will modify our model to explore the ordering effect among the most well-represented components of the network.

Heterogeneity and inconsistency

We will measure heterogeneity in the included studies by estimating a common heterogeneity parameter  τ , as described in the previous section. We will compare this estimate with its empirical distribution,52 53 for the dropout outcome. For the continuous outcome, we will compare τ  with an empirical distribution (Rhodes et al 52) only if we use standardised scores (see section Outcome measures).

We expect that the inclusion of the covariates in the model will lower the observed inconsistency, that is, by explaining some of the observed differences between the studies. In order to assess this, we will also fit a model without patient covariates and report any changes in  τ .

We will assess inconsistency in the network at the treatment level by estimating the difference between direct and indirect evidence. For this, we will use the design-by-treatment inconsistency model (Higgins et al 44). We will then also check inconsistency at the component level and report on any important differences.

Data availability bias

As discussed above, we anticipate that for at least some of the identified trials, we might not be able to obtain IPD. If studies providing IPD are systematically different from studies not providing IPD, there may be doubts regarding the validity of our findings. We will formally assess the agreement between these two sets of trials by analysing them separately and subsequently comparing the corresponding results. We will report any important discrepancies, and we will take them into consideration when we evaluate the quality of the evidence provided by our analysis.

Publication bias

We will first examine small study effects by visually inspecting the contour-enhanced funnel plots of pairwise meta-analyses for efficacy when 10 or more studies per comparison are available. We will test for small study effects using Egger’s test for the continuous outcome.54

Sensitivity analyses

The following five sensitivity analyses will be conducted for the primary outcome.

1. We will examine the impact of studies without formal diagnosis of depression by excluding such studies from the analyses.

2. We will examine the impact of studies focusing on patients with depression and a physical disorder by excluding such studies from the analyses.

3. We will examine the impact of miscellaneous skills not covered under any category in our classification in table 1 by excluding studies that included such skills.

4. We will run a sensitivity analysis by limiting to studies where at least 60% of the participants have completed at least 80% of the programme in order to exclude the influence of trials where the completion rate may have been particularly low due to some external circumstances that are not inherent to the components themselves.

5. Our main analyses regarding depression severity will only use information from patients for whom the outcome was reported. This corresponds to a ‘missing completely at random (MCAR)’ assumption.55 In this sensitivity analysis, we will explore a ‘missing not at random’ scenario, where we will assume that the probability of dropping out from assessment is affected by the (unobserved) depression severity. More specifically, we will use a selection model described by Debray et al,16 where patients with worse outcomes are more likely to drop out. We will perform this sensitivity analysis only if the dropout rates are high (>50%).