Design

This study is a systematic review and meta-analysis of IPD that is registered in the PROSPERO International prospective register of systematic reviews (registration number: CRD42017056029). Important protocol amendments will be documented in this register too. The project started on 1 December 2016 and is expected to be completed by 31 July 2019.

Search strategy

We will use an extensive search strategy including six different search methods in order to retrieve as many relevant studies as possible. These searches have already been performed in 2007 and 2014 for two previous conventional meta-analyses concerning the efficacy of STPP for depression4 13 and will be updated in 2017.

First, we will systematically search the bibliographic databases PubMed, PsycINFO (via EBSCO), Embase.com, Web of Science (via Elsevier) and Cochrane’s Central Register of Controlled Trials (via Wiley). Search terms will include a wide range of synonyms, both in index terms and free-text words, for (1) psychodynamic psychotherapy (eg, psychotherapy, psychoanalytic), (2) therapy (eg, psychotherapy), (3) psychodynamic (eg, dynamic*) and (4) depression (eg, depressive disorder). These four sets of search terms will be combined as follows: (#1 OR (#2 AND #3)) AND #4. The exact terms for the search in PubMed are presented in table 1. Complete search terms for all electronic databases are available on request from the corresponding author. No language or date restrictions will be applied in the searches.

Second, in order to identify relevant studies from the so-called ‘grey literature’, we will search GLIN, a Dutch electronic database for grey literature, and UMI database ProQuest for digital dissertations. Third, a prospective trial register will be searched for unpublished ongoing research (http://www.controlled-trials.com). The grey literature and prospective trial register searches will be conducted using the search strategy described above. Fourth, we will search an Internet database of controlled and comparative outcome studies on psychological treatments of depression (http://www.psychotherapyrcts.org 14) for studies examining STPP. Fifth, reviews and meta-analyses concerning the efficacy of psychodynamic treatments for depression or for psychiatric disorders in general retrieved from the first search method will be screened for relevant references not located by means of the other search methods. Sixth, we will contact an email list of researchers in the field of psychodynamic therapy to ask for ongoing or unpublished studies.

Selection of studies

We will include studies if they report (a) outcomes on standardised measures of (b) depressed (c) adult patients (d) receiving STPP. Participants are considered depressed if they meet specified criteria for major depressive disorder or another unipolar mood disorder as assessed by means of a semistructured interview or clinicians’ assessment, or if they present an elevated score above the ‘no depression’ cut-off on a standardised measure of depression. Participants need to be at least 18 years old, and studies concerning older adults (mean age>55) will be included as well. We will include studies in which STPP (a) is based on psychoanalytic theories and practices, (b) is time-limited from the onset (ie, not a therapy that is brief only in retrospect) and (c) applies verbal techniques (eg, therapies applying art as expression form are not considered STPP). Studies need to include at least 10 subjects. Case studies will therefore be excluded. We will also include naturalistic studies with a heterogeneous study sample, if these studies include more than 10 participants diagnosed as depressed, as these subgroups also meet the inclusion criteria specified previously. For these studies, the authors will be contacted with a request for subgroup data.

The screening process will consist of three phases. At first, the selection criteria will be applied to the citations generated from the searches independently by two raters. Disagreements will be discussed and resolved by consensus. Unless they can be definitely excluded, titles identified as potentially relevant will be requested in full text. During the second screening phase, two independent raters will apply the selection criteria to the full-text papers. Disagreements will be discussed and resolved by consensus. During the third phase, two expert STPP raters will confirm that the included papers meet criteria for STPP. Again, disagreements will be discussed and resolved by consensus. When disagreements cannot be resolved in this way, a third rater will be consulted.

Data collection

Authors of the included studies will be contacted and invited to contribute the participant-level data of their studies. Researchers who share their data will be offered coauthorship for all publications that are based on their study’s data (group-authorship), given that they meet standard criteria for authorship of scientific publications according to internationally accepted criteria (www.icmje.org). In addition, the collected data will be made available to investigators who contribute data to examine other research questions in the combined dataset. This strategy has been used in previous IPD meta-analyses concerning depression treatments15 16 and has been successful in convincing researchers to share their data.

Contact details of all first authors will be collected from the relevant publications, or if not reported there, through Internet searches or personal contacts with other researchers. First authors will be contacted by email with a letter of invitation outlining the project’s goals and asking if they would be willing to collaborate by sharing the participant-level data of their trial. If an author does not respond after 3 weeks, a second and third email will be sent. In case of non-response to email, a letter will be sent (again with three attempts). If still no response is received, we will try to contact the author by telephone. If all these attempts fail, the last, second, third, fourth, etc. author of the study (in this order) will be contacted in the same way. If none of the authors respond to these efforts, other ways will be sought to contact one of the authors (eg, via colleagues or anyone who might know them). Study data will be considered unavailable only if all these attempts fail, or in the event that an author indicates that the participant-level data have not been retained or declines sharing these data.

If the author is willing and able to share the IPD of his/her trial, both parties will sign a data-sharing agreement. The author will then transfer the participant-level dataset, including all potential predictors and moderators assessed before the start of treatment, as well as all outcome variables assessed during and after treatment, both in the STPP condition as well as in any comparison condition included in the study. The author will anonymise the data, so that the dataset is transferred without containing personal information that can lead back to individuals. Data can be submitted to the project in any format, and will then be converted to Stata.

Data integrity check

After the dataset has been transferred, the file will be checked to examine whether the data received match the data reported in the publication. For all treatment conditions included in the study, sample size, number of females, mean age, observed mean pretreatment depression scores, observed mean post-treatment score for the primary depression outcome and the number of missing cases for the latter will be calculated from the dataset received and checked against the published article for this purpose. Discrepancies will be resolved with the authors. In addition, the data will be checked for invalid, out-of-range, or inconsistent items. Furthermore, we will check the integrity of the randomisation for randomised studies by inspecting the balance of the potential predictor/moderator variables across treatment arms.

For each study, we will list all predictor/moderator variables that were assessed, as well as all outcome variables, intermediate and follow-up assessments. We will also extract multiple STPP characteristics (eg, number of sessions, treatment format, STPP mode) and study design characteristics (eg, therapist training, treatment integrity check, use of a treatment manual; for a complete overview see4). Finally, we will extract study validity criteria according to the Cochrane risk of bias assessment tool.17

After checking the data, the datasets will be standardised. For this purpose, a copy of each trial’s raw data file will be recoded into a data file that matches the IPD meta-analysis database in terms of variables. Next, the individual study data files will be concatenated in one database structured by study and individual participant ID. After all data files have been recoded and entered, the data for each study will be checked with the original data file received for accuracy. A codebook document will be made that includes the coding of the individual studies as well as the coding of the combined study database. Coding for the database will be finalised when all data have been received from the study authors.

Measures

The primary outcome for this study is treatment efficacy as assessed by a continuous depression outcome measure at post-treatment. We have chosen depressive symptom status as the main outcome for this study as we consider this to be the primary target of STPP for depression. We chose continuous symptom level as the primary outcome, because we expect that the increased variance relative to dichotomous outcomes might facilitate the search for predictors and moderators. We have chosen post-treatment assessment as the primary end point as it is difficult to control additional treatment in the follow-up period in psychotherapy efficacy studies (eg, reference 18).

For each trial, we will identify the primary continuous depression outcome as defined by the study authors. All instruments explicitly measuring depression qualify in this regard. Different depression measures have probably been used and, therefore, we will standardise the depression outcomes by converting the depression scores into z-scores. Sensitivity analyses will be conducted using unstandardised scores for each depression measure that is assessed in the majority of studies included in the meta-analysis (eg, Hamilton Depression Rating Scale,19 Beck Depression Inventory20).

The secondary outcomes for this study are dichotomous depression outcomes for (1) response (a 50% reduction in symptoms from pretreatment to post-treatment) and (2) remission (maximum absolute post-treatment scores reflecting normalisation). In addition, outcome measures other than depression will be collected (eg, anxiety symptoms, quality of life, interpersonal functioning). These can be considered tertiary outcomes.

Potential predictors and moderators include sociodemographic variables (eg, gender, age, education level, marital status, employment status, ethnicity), clinical variables (eg, number of previous depressive episodes, previous exposure to treatment, comorbid Axis I and II psychopathology, global assessment of functioning) and psychological patient characteristics (eg, personality organisation, attachment, interpersonal styles, childhood maltreatment, alexithymia). These are likely assessed differently in individual studies and will be standardised as well, for instance by converting scores into z-scores for continuous variables or by recoding variables into similar categories for categorical variables. In the latter case, we will consult study authors to confirm correctness of the recoding.

Missing data

All datasets received that contain IPD, at least one relevant outcome measure and at least one predictor/moderator variable will be considered for quantitative synthesis. We will examine the possibility of complete-case analyses by evaluating the extent of missing data as well as the possible reasons for missing data, and by comparing patients with complete data to patients with missing data. We will also assess whether the missing data can be considered missing at random.21 If complete-case analysis is not justifiable and data can be considered missing at random, we will impute missing data using multiple imputation, which is currently considered to be the most sophisticated method for handling missing data.22

We will generate one imputed dataset for each percent of missing data (eg, when 30% of the data are missing, we will create 30 imputed datasets). We will impute missing data by means of hierarchical imputation with fully conditional specification for the generalised linear model (FCS-GLM),23 which allows for preserving the heterogeneity across studies, non-normal distributions of variables and imputing systematically missing variables. FCS-GLM has shown to be a reliable procedure with advantageous properties for IPD meta-analyses with limited numbers of studies or studies with small sample sizes.24 FCS-GLM will be conducted in R. To ensure congeniality, imputation will be based on all variables that will be included in the meta-analysis model including their interactions as well as any variables that were identified to be predictive of missing values.24 If multiple imputation is pursued, we will conduct sensitivity analyses restricted to complete cases and compare the results. We will not pursue efforts to combine IPD with aggregate data from studies for which no IPD is available,25 since the requisite treatment-covariate interactions are seldom reported in publications.

Data analysis

We will conduct IPD meta-analyses according to the one-stage approach, because that accounts for the correlation among model parameters when modelling interactions, offers the highest degree of flexibility for making the necessary assumptions when detecting treatment-covariate interactions,26 and provides a more exact likelihood in the case of small studies.27 We will conduct IPD meta-analyses using mixed models with restricted maximum likelihood to estimate between-study heterogeneity, which is recommended when there are few studies in the meta-analysis or studies have small sample sizes.28 Analyses will be conducted in Stata ‘mixed’. We will apply the Kenward-Roger denominator-degrees-of-freedom adjustment for CIs,29 because it more adequately accounts for uncertainty than the standard Dersimonian-Laird estimator when the number of studies is small or when heterogeneity is present.30 To account for clustering of patients within studies and to preserve randomisation, we will include a random intercept for study and a random slope for baseline depression. We assume that these random effects will be normally distributed and this assumption will be assessed. We will assess heterogeneity by examining the SD of the predictor or moderator parameter estimates.

Separate analyses will be conducted for the identification of predictors and the identification of moderators of STPP efficacy. With regard to predictors, IPD from STPP conditions across all studies will be combined, regardless of study type (randomised controlled study, non-random comparative study, naturalistic study). For each of the potential predictors, a model will be estimated with baseline depression and the predictor’s main effect. A random slope will be added to the predictor to examine if this results in a model improvement. If so, this predictor variable will be included with a random slope in subsequent analyses. Next, all predictors will be modelled simultaneously. Backward selection based on P value will be conducted until a final prediction model is obtained consisting of statistical significant predictors only. This prediction model will be validated in a held-out sample of the data.

With regard to moderators, analyses will be conducted separately for each comparison (eg, STPP vs control conditions, STPP vs other psychotherapy and STPP vs antidepressant medication). These analyses will only include IPD from randomised studies. For each of the potential moderators, a model will be estimated with baseline depression as well as the moderator’s main effect and interaction with treatment. Next, to examine ecological bias, within-study and across-study interaction effects will be separated.25 If within-study and across-study interactions differ, ecological bias may be at play, and only the within-trial interaction will be interpreted in subsequent analyses. Finally, all significant treatment-covariate interactions, their main effects and higher order interactions will be modelled simultaneously. The resulting prediction model will be validated in a held-out sample of the data.

A number of sensitivity analyses will be conducted to examine the robustness of the findings. To examine the impact of study quality, we will conduct sensitivity analyses including only studies that score high on all quality criteria. In addition, we will conduct analyses in which we add the risk of bias items as covariates to the mixed-effects models, centred at a value indicating freedom of bias.31 We will also conduct sensitivity analyses to control for the effects of additional non-study treatment during the trial and in the follow-up period by adding collected data in this regard as covariates to the mixed-effects models. Finally, we will examine the impact of STPP characteristics (eg, STPP type, delivery mode) and study design characteristics (eg, therapist training, use of a treatment manual) by adding these variables to the mixed-effects models too.

Furthermore, to examine possible data availability bias, t-tests and Χ² analyses will be conducted comparing the included studies with studies for which no participant-level data was obtained with regard to the extracted study characteristics. Conventional meta-analysis techniques will be used to examine differences in effect sizes between studies that contributed data and studies that did not. We will assess potential publication bias by assessing asymmetry in contour-enhanced funnel plots for meta-analyses including 10 or more trials, as is recommended by Sterne et al.32 The confidence in the cumulative body of evidence will be assessed according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system.33

In addition, we will conduct explorative analyses using tree-based statistical learning. More specifically, we will use the generalised linear mixed-effects regression trees algorithm (glmertree34) in R to detect predictors and moderators of continuous and binary treatment outcomes. These analyses offer several advantages over more traditional mixed-effects modelling approaches, as they allow for the detection of non-linear and higher level interactions, allow for specifying a large number of potential predictor and moderator variables and involve less stringent assumptions about the distribution of the data. Furthermore, the result of applying glmertree consists of a decision tree that graphically depicts the effects of the relevant predictor and moderator variables. Such a tree can easily be interpreted, and applied in clinical decision-making. However, it should be noted that such tree-based analyses are exploratory in nature. We will assess the expected predictive accuracy on new data using k-cross validation35 and the stability of the resulting decision trees using subsampling methods.36 The glmertree analyses will be performed using observed, non-imputed data only, as it is currently unclear how to best specify the model for generating imputed data in tree-based analyses for clustered data.