Types of studies

All RCTs reported as double-blind comparing one active drug with another or with placebo in the acute phase treatment of major depression will be included. Only monotherapy studies will be included; thus RCTs in which antidepressants were used as an augmentation strategy will be excluded. Quasi-randomised trials (such as those allocating by using alternate days of the week) will be excluded. Cross-over and cluster randomised trials will be included. We will not include studies where sequence generation was at high risk of bias, or where the allocation was clearly not concealed.

Types of participants

Patients aged 18 years or older, of both sexes, with a primary diagnosis of major depression will be included. Studies adopting any standard operationalised diagnostic criteria to define patients suffering from unipolar major depression will be included, such as Feighner criteria, Research Diagnostic Criteria, DSM-III, DSM-III-R, DSM-IV, DSM-5 and ICD-10. Studies in which 20% or more of the participants may be suffering from bipolar or psychotic depression will be excluded. A concurrent secondary diagnosis of another psychiatric disorder will not be considered as exclusion criterion, but RCTs in which all participants have a concurrent primary diagnosis of another mental disorder will be excluded. Studies in which all participants have a diagnosis of resistant depression will be excluded. Antidepressant trials in depressive patients with a serious concomitant medical illness will be excluded. RCTs of women with postpartum depression will be also excluded, because postpartum depression appears to be clinically different from major depression.13 Trials which allow rescue medications will be included so long as these are equally provided among the randomised arms.

Types of interventions

We are interested in comparing the following active agents: agomelatine, amitriptyline, bupropion, citalopram, clomipramine, desvenlafaxine, duloxetine, escitalopram, fluoxetine, fluvoxamine, levomilnacipran, milnacipran, mirtazapine, nefazodone, paroxetine, reboxetine, sertraline, trazodone, venlafaxine, vilazodone and vortioxetine. We will include all the second generation antidepressants, and of older agents, we have selected the two tricyclics included in the WHO3 list of essential medicine: (1) amitriptyline, recommended for major depression and (2) clomipramine, although a typical tricyclic antidepressant, as it has a different biochemical, mainly serotonergic, action. We also selected trazodone and nefazodone because these are believed to have very distinct effect and tolerability profiles.14 We will include only studies randomising patients to the drug within its licensed dose range.4 If a study included arms with both unapproved and approved doses, we include the study but only the arms that used the therapeutic doses.15

We will obtain information about the interventions of interest either from head-to-head or placebo controlled trials. Hence the synthesis comparator set consists of all the interventions listed above and placebo controlled trials. Figure 1 shows the network of all possible pairwise comparisons between the eligible interventions. We anticipate that any patient who meets all inclusion criteria is, in principal, equally likely to be randomised to any of the interventions in the synthesis comparator set.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Network of all possible pairwise comparisons between the eligible interventions.

Outcome measures

Considering that clinical trials of antidepressant drugs are usually small and that data distribution is difficult to assess for studies with small samples, in this review priority will be given to the use and analysis of dichotomous variables both for efficacy and acceptability.

• Primary outcomes

  • (1) Efficacy (as dichotomous outcome)—response

  •  Measured by the total number of patients who had a reduction of at least 50% on the total score between baseline and week 8 (range 4–12 weeks) on a standardised observer-rating scale for depression. We will employ Hamilton Depression Rating Scale (HDRS) or, if HDRS was not used, another standardised and validated observer-rating scale. Any version of HDRS will be accepted.

  • (2) Acceptability of treatment

  •  Treatment discontinuation (acceptability) is defined as the proportion of patients who leave the study early for any reason during the first 8 weeks of treatment (range 4–12 weeks).

• Secondary outcomes

  • (3) Efficacy (as continuous outcome)

  •  Measured by the end point score on the HDRS or Montgomery-Åsberg Depression Rating Scale (MADRS), if HDRS was not used, after 8 weeks (range 4–12 weeks). If none of the former scales is used, we will consider other standardised rating scales. When end point scores are not reported but change scores are, we will use the latter scores.16 See figure 2 for full details about the data extraction process (decision tree).

  • (4) Efficacy (as dichotomous outcome)—remission

  •  Measured by the total number of patients who had a remission of depressive symptoms between baseline and week 8 (range 4–12 weeks) on a standardised rating scale for depression (HDRS or another standardised rating scale, if HDRS was not used). Remission will be defined as score of less or equal to 7 or 8 on the 17-item HDRS (or the corresponding threshold for longer versions of HDRS),17 or of less or equal to 10 or 11 on the MADRS scale at week 8 (range 4–12 weeks).18

  • (5) Tolerability of treatment

  •  The proportion of patients who leave the study early due to adverse events during the first 8 weeks of treatment (range 4–12 weeks).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Decision-tree for data extraction of continuous efficacy outcome. HDRS: Hamilton Depression Rating Scale; MADRS: Montgomery-Åsberg Depression Rating Scale.

Dichotomous outcomes

We opt for the number of successes and failures per treatment arm as defined in Outcome measures section. When these numbers are not reported but baseline mean and end point mean and SDs of the depression rating scales (such as HDRS or MADRS) are provided, we will calculate the number of responding patients at 8 weeks (range 4–12 weeks) by employing a validated imputation method.20 Below we also discuss our strategy when means and/or SDs are not reported in the articles.

Continuous outcomes

We will extract means, SDs, and numbers of patients randomised in each study arm. When means and their SDs are not recorded, authors will be asked to supply the data. When SEs, t-statistics or p values are reported, these will be transformed to SDs. If SDs are not reported and not provided by the authors, the mean value of known SDs will be calculated from the group of included studies according to Furukawa and colleagues.21 When mixed method repeated measures or other appropriate imputation methods are used,22 we will prefer these results. When data on dropouts are carried forward and included in the evaluation (Last Observation Carried Forward, LOCF), these will be analysed according to the primary studies.

Missing outcome data

Outcomes of patients who leave the study early are typically imputed by the trialists, often using LOCF.23 It is very rare for an article to report the outcome separately for fully observed and imputed data, and the summary statistics that we will collect are bound to refer to both completers and patients who dropped out. The appropriateness of the imputation method to account for early dropouts will be considered in the Risk of bias assessment section. During the protocol development process, we have carried out some exploratory analyses to assess the comparability between studies with placebo arm and studies with only active treatments. Considering that the number of dropouts is usually higher in placebo controlled trials,24 we anticipate that the imputation of missing outcome data using LOCF can be problematic when comparing head-to-head with placebo trials within the same network of treatments. In case of material differences between these types of studies, we will carefully investigate this methodological issue, and try to address it properly from a statistical point of view (refer Risk of bias assessment section).

After imputations at the individual participant level by the original authors, the outcome might be unknown (and not imputed by the original authors) for a very small proportion of study participants. For the dichotomous efficacy outcome, we will first assume that participants with an unknown outcome are non-responders. Although this corresponds to naive imputations24 an extensive sensitivity analysis using more appropriate methods to account for missing outcome data in antidepressant trials has shown that imputing outcomes for a very small percentage of patients (as in our case) has no material impact on the results.25 For continuous outcomes, participants with missing outcome data will be excluded from the analysis.

Unit of analysis issues

We will extract data from cross-over studies using only the first period because carry-over effects can be important in antidepressant trials.26 In cluster randomised trials, we will extract data that account for the clustering in the results (eg, from multilevel models). If such adjusted results are not available, we will extract unadjusted data and will adjust the sample size (in the continuous outcomes) and both the sample size and number of events (in the dichotomous outcomes) by dividing it with the design effect.27

Length of trial

Clinically, whether efficacy is assessed after 8 weeks of treatment or after 16–24 weeks or more may lead to differences in terms of assessed treatment outcome. Clinicians need to know whether (and to what extent) treatments work within a clinically reasonable period of time. Unfortunately, there is no consensus on what the appropriate duration of an acute phase trial is. In the present review, acute treatment will be defined as an 8-week treatment in both the efficacy and acceptability analyses.4 If 8-week data are not available, we will use data as close to 8 weeks as possible (ranging between 4 and 12 weeks). Longer term studies will be included in the systematic review, but excluded from the statistical synthesis of data if they do not provide data for the 4–12 weeks period.

Comparability of dosages

We will include only study arms randomising patients to drugs within the licensed dose. Both fixed-dose and flexible-dose designs will be allowed.4 There is a possibility that some trials compare one agent at the upper limit of its therapeutic range with another agent at the lower limit of its therapeutic range within the same study. We plan to capture this study characteristic by adding a dichotomous variable indicating whether dosages are comparable, and use this information for a sensitivity analysis.

Characteristics of included studies and information flow in the network

We will generate descriptive statistics for the trial, and study population characteristics across all eligible trials, describing the types of comparisons and some important variables, either clinical or methodological (such as year of publication, age, severity of illness, sponsorship and clinical setting).

The available evidence will be presented in the network diagram. The size of the nodes will reflect the amount of evidence accumulated for each treatment (total number of patients), the breadth of each edge will be proportional to the inverse of the variance of the summary effect of each direct treatment comparison, and the colour of each edge will represent risk of bias (low, moderate or high, refer Risk of bias assessment section). To understand which are the most influential comparisons in the network and how direct and indirect evidence influences the final summary data, we will use the contribution matrix that describes the percentage contribution of each direct meta-analysis to the entire body of evidence.29 ,30

Pairwise meta-analyses

For each pair-wise comparison, we will synthesise data to obtain summary standardised mean differences (SMD, Cohen's d) for continuous outcomes or ORs for dichotomous outcomes, both with 95% Credible Intervals (CrI). We will use a random effects model to incorporate the assumption that the different studies are estimating different, yet related, treatment effects.27 For each outcome, we will first assume that each pairwise meta-analysis comparing treatments X and Y has its own heterogeneity variance parameter and then assume that there are two heterogeneity parameters; one common for all placebo-controlled trials and one for all active versus active comparisons . Visual inspection of the forest plots and monitoring of the posterior distributions of , and will be used to investigate the possibility of statistical heterogeneity. The posterior distributions of the heterogeneity parameters will be compared to their predictive distributions, as described elsewhere.31 ,32 Finally the I² statistic and its 95% CrI will be calculated to convey the amount of heterogeneity.

Assessment of the transitivity assumption

Transitivity, which is the key underlying assumption of NMA, will be investigated carefully. Joint analysis of treatments can be misleading if the network is substantially intransitive. We will need to investigate the distribution of clinical and methodological variables that can act as effect modifiers across treatment comparisons.33 The clinical features, which have been demonstrated to date to moderate efficacy of antidepressants include bipolarity,34 psychotic features,35 and subthreshold depression.36 We have assured transitivity in our network with regard to these variables by limiting our samples to participants with non-psychotic unipolar major depression. Other clinical or methodological variables that may influence our primary outcomes of antidepressant efficacy or acceptability include: age, depressive severity at baseline,37 ,38 and the dosing schedule.39 We will investigate if these variables are similarly distributed across studies grouped by comparison. The inclusion of placebo and concerns about its potential to violate the transitivity assumption have been highlighted in general7 ,8 and particularly in depression studies.40 ,41 Consequently, the comparability of placebo-controlled studies with those that provide head-to-head evidence will be examined carefully.

Network meta-analyses

We assume that patients who fulfil the inclusion criteria outlined in Criteria for considering studies for this review section are equally likely to be randomised to any of the antidepressants that we plan to compare. If the collected studies appear to be sufficiently similar with respect to the distribution of effect modifiers (refer Assessment of transitivity assumption section), we will conduct a random effects NMA to synthesise all evidence for each outcome, and obtain a comprehensive ranking of all treatments. We will use arm-level data and the binomial likelihood for dichotomous outcomes. We will account for the correlations induced by multiarm studies by employing multivariate distributions. We will assume a single heterogeneity parameter for each network. We will present the summary ORs or SMD for all pairwise comparisons in a league table. We will also estimate the prediction intervals to assess how much the common heterogeneity affects the relative effect with respect to the extra uncertainty anticipated in a future study. To rank the various treatments for each outcome, we will use the surface under the cumulative ranking curve (SUCRA) and the mean ranks.42

Assessment of inconsistency

The strategical and conceptual evaluation of transitivity will be supplemented with a statistical evaluation of consistency, the agreement between direct and indirect evidence. We will employ local as well as global methods to evaluate consistency.43 Local methods detect ‘hot spots’ of inconsistency, evidence loops that are inconsistent or comparisons for which direct and indirect evidence disagree. We will employ the loop-specific approach to evaluate inconsistency within each loop of evidence,44 and a method that separates direct evidence from indirect evidence provided by the entire network.45 We will also evaluate consistency in the entire network by calculating the I² for network heterogeneity, inconsistency, and for both.46 ,47

Tests for inconsistency are known to have low power,48 and empirical evidence has suggested that 10% of evidence loops published in the medical literature are expected to be inconsistent.49 Therefore, interpretation of the statistical inference about inconsistency will be carried out with caution and possible sources of inconsistency will be explored even in the absence of evidence for inconsistency.

Exploring heterogeneity and inconsistency and sensitivity analyses

We expect small amounts of heterogeneity and inconsistency to be present given the variety of study settings we plan to include. We will explore whether treatment effects for the two primary outcomes are robust in subgroup analyses and network meta-regression using the following characteristics: (1) study year; (2) sponsorship; (3) depressive severity at baseline; (4) dosing schedule; (5) response to placebo; (6) proportion of participants allocated to placebo; number of recruiting centres (single-centre vs multicentric studies).50 ,51 The sensitivity of our conclusions for the two primary outcomes will be evaluated by analysing (1) only studies with reported SD rather than imputed; (2) only studies with balanced doses in all arms (ie, we will exclude studies with unfair dose comparisons); (3) only studies with unpublished data (ie, we will exclude studies providing published data only); (4) only studies with low risk of bias (as defined in Risk of bias assessment section); (5) only head-to-head studies.

Selection bias

The risk of selection bias is high in antidepressants trials, in particular with placebo-controlled trials.15 We will use the comparison-adjusted30 and contour-enhanced52 funnel plots to investigate whether results in imprecise trials differ from those in more precise trials. We will also run network meta-regression models to detect associations between study size and effect size.53 If an important association is found and publication bias is suspected, we will attempt to explore the possibility that funnel plot asymmetry is due to publication bias by employing a selection model.54

Model implementation

We will fit our model using OpenBUGS55 and Stata (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP). For the Bayesian implementation we will employ the binomial likelihood for dichotomous outcomes and will use uninformative prior distributions for the treatment effects, that is, N(0,1000), and a minimally informative prior distribution for the common heterogeneity SD depending on the outcome, that is, U(0,5). Also, we will assume uninformative priors, that is, N(0,1000) for all meta-regression coefficients. To check convergence, we will run multiple chains and monitor their mixing; we will use the Brooks-Gelman-Rubin diagnostic.

Analyses for statistical evaluation of the inconsistency and production of network graphs and result figures will be carried out in Stata using the mvmeta command56 and a collection of routines described elsewhere.30 All analyses of the primary outcomes will be duplicated using the netmeta package in R.57