Introduction Psychotherapy is a complex intervention, consisting of various components and being implemented flexibly in consideration of individual patient’s characteristics. It is then of utmost importance to know which of the various components or combinations thereof are more efficacious, what their specific effect sizes are and which types of patients may benefit more from different components or their combinations.
Methods and analysis Internet-delivered cognitive–behavioural therapy (iCBT) offers a unique opportunity to systematically review and quantitatively disentangle the efficacy of various components because, unlike face-to-face cognitive–behavioural therapy, it allows identification of constituent components that are actually delivered to patients. We will systematically identify all randomised controlled trials that compared any form of iCBT against another form or a control intervention in the acute phase treatment of adult depression. We will apply component network meta-analysis (cNMA) to dismantle efficacy of individual components. We will use individual participant data in the cNMA to identify participant-level prognostic factors and effect modifiers for different components.
Ethics and dissemination The investigators of the primary trials will have obtained ethical approval for the data used in the present study and for sharing the data, if this was necessary, according to local requirements and was not covered from the initial ethic assessment. Results from this study will be published in peer-reviewed journals and presented at relevant conferences.
PROSPERO registration number CRD42018104683.
- depressive disorder
- internet-delivered cognitive-behavioral therapy
- network meta-analysis
- individual participant data meta-analysis
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- depressive disorder
- internet-delivered cognitive-behavioral therapy
- network meta-analysis
- individual participant data meta-analysis
Strengths and limitations of this study
Internet-delivered cognitive–behavioural therapy will allow identification of constituent components used in each intervention and effectively delivered to participants.
Component network meta-analysis (cNMA) will allow estimation of a specific incremental effect size for each component.
Individual participant data cNMA (IPD-cNMA) will allow identification of prognostic factors and effect modifiers for different components.
IPD-cNMA, while a powerful methodology, is limited by the availability of individual participant data, their quality and their comprehensiveness.
Psychotherapy is a complex intervention, comprising multiple components in various combinations.1 Psychotherapy may be even more complex in practice, as its implementation is variable and flexible even in research settings. It is generally believed, though seldom empirically demonstrated,2 that efficacy of psychotherapy is moderated by individual patients’ characteristics.
Cognitive–behavioural therapy (CBT) is the most widely studied type of psychotherapy for depression. CBT, however, should be considered more as an umbrella term, because under the general label of CBT, we can find psychotherapies that include various combinations of many different components. Moreover, these are administered in a flexible manner, presumably in concordance with individual patient characteristics. Viewed from the clinicians’ and consumers’ points of view, it is then of utmost importance to determine which of the various components or combinations of components proposed for CBT are more efficacious, what their corresponding effect sizes are and which types of patients may benefit more from different components or their combinations.
Apart from a few exceptions known as dismantling studies,3 research into CBT has usually studied it as a package of various components. The complexity of psychotherapeutic interventions noted above at the levels of intervention and population applies to such research and adds much uncertainty to their analyses and interpretations.4 5 In traditional systematic reviews of psychotherapeutic interventions, identification of specificity in psychotherapy has proved extremely difficult.6 7 Detection of differences among treatments and treatment components would require extremely large samples.3 In addition, the influence of patient-level characteristics on the efficacy of the interventions could not be adequately explored in analyses that use aggregate (study-level) information, due to the risk of ecological bias that occurs when the association at the group level does not reflect the underlying association at the individual level.8 9 Individual data, either at the level of a trial10 or of a meta-analysis,11 are necessary to examine effect modification by individual characteristics.
The recent upsurge of research in internet-delivered CBT (iCBT) provides a unique opportunity to systematically review and quantitatively disentangle the efficacy of the various components of CBT. First, in iCBT, constituent components of CBT are easily identifiable, and it is guaranteed that they were made available to the participants, unlike in face-to-face CBT. Second, new methods to analyse complex interventions and synthesise their findings12 13 can then be applied to randomised controlled trials (RCTs) in iCBT. In this study, we will apply component network meta-analysis (cNMA), a newly developed meta-analysis method, where various components of different therapies can be dismantled and compared while taking advantage of the whole network of randomised evidence.14 15 We will extend existing cNMA models to include individual participant data (IPD) in the meta-analysis.16 17 This will allow us to identify and explore the impact of participant-level prognostic factors (PFs; variables that affect the disease progression equally for all the treatments in the network) and effect modifiers (EMs; variables that have an impact on the relative effects of interventions).
This study therefore aims to uniquely overcome the complexity in psychotherapy research at the levels of intervention and population by identifying the more effective components of iCBT and by pinpointing the individual patient’s characteristics that modify their effects. Such findings will help develop more effective and more efficient forms of iCBT by focusing on the best-performing components and facilitate personalised applications of iCBT, aiming to maximise the therapeutic effect while better matching the administered treatment to the individual patients’ characteristics, needs and preferences.
The protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension statement for network meta-analysis,18 individual participant data meta-analysis19 and study protocol.20
We will include all RCTs that compared any form of iCBT against another form of iCBT or a control intervention in the treatment of adults with depression.
We will include adult patients of both sexes aged 18 years or older, with a primary diagnosis of depression, either diagnosed as unipolar major or minor depression according to operationalised diagnostic criteria including Diagnostic and Statistical Manual 3rd Edition (DSM-III), 3rd revised edition (DSM-III-R), 4th edition (DSM-IV), 5th edition (DSM-5), International Classification of Diseases 10th revisioin (ICD-10) or any similar criteria, or judged so by elevated scores on any self-report depression scales. The effect of including studies without formal diagnosis of major depression will be tested in a sensitivity analysis. Inpatients as well as participants with bipolar depression or with psychotic depression will be excluded. Studies including participants with depression comorbid with another mental disorder will be included as long as their study inclusion criteria do not specifically include such comorbidities; however, we will exclude studies focusing on depression comorbid with another mental disorder, because in such cases, the CBT interventions would usually have components that target at such comorbidities in addition to depression. If studies include patients with depression or another mental disorder, we will include them if and only if we can focus on participants with depression based on individual data. Studies focusing on depression with another physical disorder (eg, diabetes and parkinsonism) or in special populations (eg, elderly, pregnancy, mother of autistic children and ethnic minority) will be included; however, the effect of including such studies will be examined in a sensitivity analysis. If a minority of the participants satisfy any of the above exclusion criteria, we will do our best to exclude such participants by employing IPD: when we do not have IPD or cannot exclude them using IPD because details are not available, then we will include such studies in our analysis when they constitute less than 20% of the total population.
iCBT must be a web-based or app-based programme using the internet to deliver the CBT contents. Computerised CBT will be included if it allows interaction between the programme and the individual. Telephone CBT will be excluded. If the use of the internet is limited to teleconferencing/videoconferencing, emails or text messaging, such programmes will also be excluded. When a web-based or app-based programme is used in the context of face-to-face sessions (so-called blended treatment), such programmes will be excluded because then the delivery of the CBT contents is no longer as assured as in pure iCBT. Encouragement to proceed with iCBT by telephone or face-to-face contact limited to an initial evaluation or orientation session will be allowed and considered to be a component of iCBT programme. Studies will be excluded if the iCBT programme allows participants to choose among the available components because such studies will not answer our clinical question.
The control conditions of interest will include the waiting list control, no treatment control, attention/psychological placebo control and treatment as usual. Different studies call different conditions as treatment as usual.21 In this cNMA study, treatment as usual must include pharmacotherapy; watchful waiting or follow-up by community nurses will be classified as attention/psychological placebo even when it is ‘treatment as usual’ in some settings. Pill placebo control will not be included in the present network as it is not decomposable into the components of our interest.
This study focuses on the acute phase treatment of the above-defined depression. The intervention can be of any duration; however, the influence of duration will be examined in a meta-regression.
We conceptualise CBT broadly as a psychotherapy involving any one of the following cognitive or behavioural skills’ training. Table 1 presents the different components of interest and their definitions. We will include all intervention or control conditions as long as they could be regarded as a combination of these components. Pharmacological coadministration will be allowed so long as there are no systematic differences in drug administration between study arms, and its presence will be denoted by dt (for definition see table 1). Table 2 provides how various forms of iCBT can be conceptualised from the component perspective.
Study identification and selection
We will use an existing database of psychological treatments for depression that is updated annually through comprehensive literature searches in the bibliographic databases of PubMed, PsycINFO, Embase and the Cochrane Library.22 The search strings use a combination of index and free terms of psychological treatments and depression.
Two independent researchers will check this database for relevant studies according to the eligibility criteria. Any disagreement will be resolved by discussion and, where necessary, in consultation with a third reviewer.
A further literature search will be conducted for studies published since the last update of the database in PubMed and the Cochrane Library. In addition, we will also check the primary studies from recent meta-analyses of internet treatments for depression to ensure that no published studies will be missed. We will also ask the primary authors of the eligible studies if they are aware of any other study that has been conducted in the examined field.
Data collection and integrity checks
Authors of the eligible studies will be contacted and requested to contribute their individual-level data. The corresponding author will be contacted first; if unreachable, a follow-up email will be sent to the senior author of the study. Reminders will be sent after 2 weeks and if necessary after 4 weeks. If no response is received after additional 4 weeks, this trial will be classified as ‘IPD unavailable’ and will be included in the analyses at the aggregate data level.
Authors will be asked to provide the individual-level raw data for their primary depression measures at baseline and at end of the acute phase treatment defined by the original study authors, as well as other potentially important covariates (PFs and EMs of treatment outcome; see section below).
After collecting all primary individual-level data, two independent reviewers will cross-examine the obtained data against the summary statistics (numbers and percentages or means and SDs) of the baseline demographic and clinical variables as reported in the publications of each study. In case the numbers do not match, we will contact the authors of the trials for clarification.
Identification of components
Two independent reviewers will determine the classification of all identified arms and their constituent components according to the definitions in tables 1 and 2, based on all available information including the publications, the trialled iCBT programmes and inquiry with the original investigators. Any disagreement will be solved by the two reviewers and, where necessary, in consultation with a third member of the review team.
Our primary outcome is:
Depression severity as measured on a continuous scale for depression at the end of the acute phase treatment. We will include change in scores from pretreatment to post-treatment on any validated depression outcome measure, such as the Beck Depression Inventory, Beck Depression Inventory-II, Center of Epidemiological Studies Depression Scale, Patient Health Questionnaire-9 or Hamilton Rating Scale for Depression. If a study uses more than one depression measure, preference will be given to the measure reported by the majority of the included studies. In case a study reports two or more outcome measures, none of which are used by the rest of the included studies, preference will be given to the measure listed as primary in this study. If the studies use different outcome measures, they will be converted into the most commonly used scale using the established conversion algorithms.23 24 If this approach cannot cover a substantial proportion of the obtained data, scale scores will be standardised (transformed into z-scores) to create a common metric for depression severity.25–27
We will also examine the following two secondary outcome measures.
Dropouts from the end-of treatment assessment for any reason, as a proxy measure of treatment acceptability.
Dropouts from the treatment, defined as completing less than 80% of the contents of the programme. If the original authors used a different threshold/definition for ‘completion’ of the programme, we will use their definition.
PFs and EMs of treatment outcome
In this study, we will start from a wide range of patient-level variables and explore their role as either PFs or EMs. We will initially select candidate covariates based on previous literature findings and the availability of these variables in the included studies. The following is the list of candidate variables based on the published literature.28
Life and social history
History of present illness
Present illness: symptomatology
Baseline psychomotor symptoms.34 40
Baseline anxiety symptoms.40 41
Baseline somatic anxiety.34
Comorbid personality disorder.2
Comorbid alcohol or substance use/abuse.40
The following study-level characteristics will also be examined as PF or EM.
It can be expected that different studies use different scales or different categorisations to measure the same or similar constructs. Some measures (eg, social adjustment and baseline anxiety) may be standardised to arrive at a common metric; others may need be dichotomised (eg, employment status, marital status) to harmonise the covariates in the analyses.
Risk of bias assessment in individual studies
Two independent raters will assess the risk of bias in the included studies using the tool described in the Cochrane Collaboration Handbook44 as being at high risk of bias, low risk of bias or unclear risk of bias in the following domains: generation of allocation sequence, allocation concealment, blinding of study personnel and participants, blinding of outcome assessor, attrition, selective outcome reporting and other domains. Disagreements between the two independent assessors will be solved through discussion; when there still remains doubt, we will go back to the original authors for clarification.
Note that blinding of study personnel and participants is usually impossible in psychotherapy research; however, in studies of iCBT, it may be at least theoretically feasible if alternative active treatments are provided simply as ‘active treatments’. Note also that attrition bias will be evaluated in the case of studies with individual-level data, not according to the published report, but according to the provided dataset after missing data are imputed as per the statistical methods below. Some studies may still be rated at high risk of bias after imputation if large and unbalanced dropouts exist in the provided raw data. In the case of studies without individual data, attrition bias will be rated according to the publication.
In the above risk of bias assessment, we chose not to evaluate treatment fidelity as it will not be an issue with iCBT, which is fully structured. We also chose not to evaluate allegiance as its measurement is still controverted and will be especially difficult to measure in the comparisons against control conditions45 and may be less important in high-quality studies.46
Patient and public involvement
There was no patient or public involvement in the development of this manuscript.
We will start by performing an aggregated data network meta-analysis (NMA) on the treatment level to gain a first insight of the relative treatment effects.47 We will do this analysis both with and without trials for whom we do not have IPD.
Then we will continue to our main analysis. This will be done in two steps. In the first step, we will perform a variable selection procedure to decide which of the candidate covariates to include in the evidence synthesis model of the second step. To this end, we will fit a penalised linear regression model with an elastic net penalty48 to our primary outcome (continuous efficacy). We will explore all candidate covariates as well as their interactions with the components, aiming to identify the most important patient characteristics in terms of both prognosis and effect modification. Aiming to facilitate convergence of the models and to help the interpretation of the results of the second step, all continuous covariates will be standardised, that is, by subtracting the mean and dividing by the corresponding SD. The output of this first step of the analysis will be a list of PFs and EMs. Note that different EMs may be associated with different components, for example, patient age may be found to strongly interact with cognitive restructuring but not with behavioural activation.
At the second step, we will fit a one-stage IPD-NMA model.16 17 Let us assume that study j compares two interventions and , where comprises components and and comprises components and . Let us assume that patient i in this study had an observed outcome . Let us also assume that for this patient we have the corresponding PFs in the form of a vector . Similarly, let us assume that the EMs for this patient, for each component, are in vectors , , and . The model can now be written as follows:
In this expression, β is the vector of regression coefficients associated to PFs. is the study-specific variance of . is the vector of regression coefficients for effect modification (component–covariate interaction), for component X . The ‘baseline’ effect, , will be assumed exchangeable across studies, that is, is the study-specific estimate of relative effect for versus , at the zero values of the (possibly standardised) EMs, that is, when all . We will assume that , where τ denotes the random effects SD in the network. We will assume τ to be common for all treatment comparisons in the network. Finally, will be expressed as a linear combination of the effects of the components, for example, for this particular example we will set . This model assumes additivity of treatment effects (Welton et al 14). In order to include in our analysis patients with missing values for one or more of the covariates (both PFs and EMs), we will use a study-specific imputation scheme. For example, if for patient i in study j there is no information regarding age, we will stochastically impute it by drawing from a study-specific distribution, that is, . Here denotes the mean age of patients in this study and the corresponding variance.
For the secondary outcome ‘dropout from end-of-treatment assessment’, we will use a binomial likelihood for the observed data. We will synthesise data on the OR scale. The rest of the model will be as for the primary outcome.
For the secondary outcome ‘dropout from the treatment’, we will use a different modelling approach. This is because, by definition, for the inactive control conditions, treatment dropout is not observed. This implies that in studies with inactive controls, we cannot infer about relative effects. Such studies are expected to represent the vast majority of all eligible studies. Thus, for the analysis of this outcome, we will synthesise absolute treatment effects. We will model the probability of dropout for each active treatment, using a generalised linear model with a binomial likelihood
We will implement the variable-selection procedure of our analysis (step one) in R,49 using the glmnet package.50 The variable-selection procedure will only be performed for the primary outcome. We will implement the evidence-synthesis part (step 2) in OpenBUGS.51
Limitations of the proposed statistical model
The basic model assumes additivity of components and does not take account of possible interactions among components, that is, when some components may be particularly effective or ineffective in combination with some other components. The dataset will likely lack statistical power to test for such interactions. However, for components that are well represented in the network (ie, those which have been studied in many trials involving many participants), we will run exploratory analyses of some representative interactions.
The proposed model cannot discern the effect of the ordering of the components, but we expect that we will not have enough relevant data from the studies to explore this effect. If enough data become available, we will modify our model to explore the ordering effect among the most well-represented components of the network.
Heterogeneity and inconsistency
We will measure heterogeneity in the included studies by estimating a common heterogeneity parameter τ , as described in the previous section. We will compare this estimate with its empirical distribution,52 53 for the dropout outcome. For the continuous outcome, we will compare τ with an empirical distribution (Rhodes et al 52) only if we use standardised scores (see section Outcome measures).
We expect that the inclusion of the covariates in the model will lower the observed inconsistency, that is, by explaining some of the observed differences between the studies. In order to assess this, we will also fit a model without patient covariates and report any changes in τ .
We will assess inconsistency in the network at the treatment level by estimating the difference between direct and indirect evidence. For this, we will use the design-by-treatment inconsistency model (Higgins et al 44). We will then also check inconsistency at the component level and report on any important differences.
Data availability bias
As discussed above, we anticipate that for at least some of the identified trials, we might not be able to obtain IPD. If studies providing IPD are systematically different from studies not providing IPD, there may be doubts regarding the validity of our findings. We will formally assess the agreement between these two sets of trials by analysing them separately and subsequently comparing the corresponding results. We will report any important discrepancies, and we will take them into consideration when we evaluate the quality of the evidence provided by our analysis.
We will first examine small study effects by visually inspecting the contour-enhanced funnel plots of pairwise meta-analyses for efficacy when 10 or more studies per comparison are available. We will test for small study effects using Egger’s test for the continuous outcome.54
The following five sensitivity analyses will be conducted for the primary outcome.
We will examine the impact of studies without formal diagnosis of depression by excluding such studies from the analyses.
We will examine the impact of studies focusing on patients with depression and a physical disorder by excluding such studies from the analyses.
We will examine the impact of miscellaneous skills not covered under any category in our classification in table 1 by excluding studies that included such skills.
We will run a sensitivity analysis by limiting to studies where at least 60% of the participants have completed at least 80% of the programme in order to exclude the influence of trials where the completion rate may have been particularly low due to some external circumstances that are not inherent to the components themselves.
Our main analyses regarding depression severity will only use information from patients for whom the outcome was reported. This corresponds to a ‘missing completely at random (MCAR)’ assumption.55 In this sensitivity analysis, we will explore a ‘missing not at random’ scenario, where we will assume that the probability of dropping out from assessment is affected by the (unobserved) depression severity. More specifically, we will use a selection model described by Debray et al,16 where patients with worse outcomes are more likely to drop out. We will perform this sensitivity analysis only if the dropout rates are high (>50%).
TAF and EK contributed equally.
Contributors TAF and OE conceived the study. TAF, EK, PC and OE designed the study. TAF, EK and OE drafted the protocol, and AS, AP, EGO, AC and PC critically revised it. EK and PC have conducted the original study search, and AS, AP and EGO will search results for inclusion and conduct data extraction. TAF and AC will assist with data extraction. EK, PC and TAF will build the individual participant dataset. OE will conduct the analyses. AS will draft the final manuscript, and all authors will critically revise it. All authors have contributed to and have approved the final protocol paper.
Funding This work is supported in part by JSPS Grant-in-Aid for Scientific Research (Grant Number 17K19808) and by a grant-in-aid from Japan Agency for Medical Research and Development (AMED) (Grant Number JP18dk0307072) to TAF.
Competing interests TAF has received lecture fees from Meiji, Mitsubishi-Tanabe, MSD and Pfizer. He has received research support from Mitsubishi-Tanabe. He has a patent 2018-177688 pending. All the other authors report no conflict of interest.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.