Article Text
PDF
Abstract
Introduction There have been important advances in the use of electroconvulsive therapy (ECT) to treat major depressive episodes. These include variations to the type of stimulus the brain regions stimulated, and the stimulus parameters (eg, stimulus duration/pulse width). Our aim is to investigate ECT types using a network meta-analysis (NMA) approach and report on comparative treatment efficacy, cognitive side effects and acceptability.
Method We will conduct a systematic review to identify randomised controlled trials that compared two or more ECT protocols to treat depression. This will be done using the following databases: Embase, MEDLINE PubMed, Web of Science, Scopus, PsycINFO, Cochrane CENTRAL and will be supplemented by personal contacts with researchers in the field. All authors will be contacted to provide missing information. Primary outcomes will be symptom severity on a validated continuous clinician-rated scale of depression, cognitive functioning measured using anterograde verbal recall, and acceptability calculated using all-cause drop-outs. Secondary outcomes will include response and remission rates, autobiographical memory following a course of ECT, and anterograde visuospatial recall.
Bayesian random effects hierarchical models will compare ECT types. Additional meta-regressions may be conducted to determine the impact of effect modifiers and patient-specific prognostic factors if sufficient data are available.
Discussion This NMA will facilitate clinician decision making and allow more sophisticated selection of ECT type according to the balance of efficacy, cognitive side effects and acceptability.
Ethics This systematic review and NMA does not require research ethics approval as it will use published aggregate data and will not collect nor disclose individually identifiable participant data.
PROSPERO registration number CRD42022357098.
- Depression & mood disorders
- MENTAL HEALTH
- Adult psychiatry
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
This network meta-analysis (NMA) will synthesise evidence from randomised controlled trials of electroconvulsive therapy (ECT) reporting on efficacy, cognitive and acceptability outcomes.
The NMA will investigate the impact of ECT electrode placements (right unilateral, bifrontal and bitemporal), pulse width (ultrabrief and brief) and dosing protocols including those based on seizure titration, fixed doses and formula-based dosing.
The findings from this study will provide a comprehensive ranking of ECT interventions according to their comparative antidepressant efficacy and cognitive side effect profile to guide clinical treatment decisions.
The Confidence In Network Meta-Analysis approach will be used to assess the quality of evidence of primary studies and estimate the confidence in the NMA results.
As a limitation, considering the large parameter space for ECT, there may be insufficient data available to comprehensively assess the impact of all potential prognostic factors and effect modifiers for all outcome measures.
Introduction
Depression is a prevalent mental illness characterised by a loss of interest in pleasurable activities, or persistent low mood.1 Global estimates suggest that 3.2% of individuals are currently depressed, and that rates of depression have increased by 27.6% due to the COVID-19 pandemic.2 In addition to the subjective suffering caused by this mental illness, patients with severe depression represent a collective socioeconomic cost of US$29–US$48 billion in the USA alone,3 due to increased unemployment, disability and reduced work performance.4 Over 10% of depressed individuals report suicidal ideation with some intention to act,5 producing a heightened lifetime risk of suicide of 3.8%–7.8%.6 Approximately one-third of patients with depression do not attain remission after multiple adequate courses of evidence-based pharmacotherapy,7 suggesting that there are limits to standard drug interventions—although novel interventions, including esketamine show promise.8 Therefore, there is an imperative to apply high-efficacy, non-pharmacological based interventions to treat depression. Electroconvulsive therapy (ECT) is a highly efficacious therapy for clinical depression, including treatment resistant depression and melancholic, psychotic, and bipolar subtypes.9 10
ECT involves the administration of a single transcranial electrical pulse train to stimulate the brain and induce a generalised seizure.10–12 Empirical studies of neurobiological predictors and correlates of response have led to several proposed mechanisms of action, including enhancement of monoaminergic transmission,13 neurotrophic changes14–17 and metabolic changes.18 19 ECT is safe,20 21 and has been shown to be more effective than pharmacotherapy,22 23 and other forms of non-invasive brain stimulation,24 including transcranial magnetic stimulation and transcranial direct current stimulation, for severe and treatment-resistant presentations of depression. Treatment remission rates may be as high as 60%–70%.25 26 Further, ECT rapidly reduces suicidal ideation,27–29 death by suicide30 and significantly improves quality of life.31
The efficacy, safety and acceptability of modern ECT practices are the result of several major advances in treatment delivery since the technique’s development in the 1930s. Improvements in ECT technique have focused on preserving and improving efficacy while minimising cognitive sequelae such as memory dysfunction, which includes both anterograde (verbal and visual) and retrograde memory side effects.32 33 While anterograde memory side effects with ECT tend to be more transient and typically resolve within a month following acute treatment, retrograde memory side effects, for example, for autobiographical information, can be more lasting.34 35 For instance, in the 1980s, ECT stimulus parameters changed from a sine wave to a brief rectangular-wave biphasic stimulus of pulse duration 0.5–2.0 ms, reducing the incidence of cognitive side effects while retaining therapeutic efficacy.36 More recently, pulse width has been narrowed even further, to 0.25–0.3 ms, giving rise to ultrabrief (UB) pulse ECT.37–39
In addition to ECT stimulus parameters, electrode placement on the scalp can also alter efficacy and cognitive outcomes due to changes in the pattern and spatial extent of cortical activation.40–42 The neural correlates associated with antidepressant response following ECT, and those associated with cognitive side effects, have been demonstrated to be dissociable.43 This suggests that brain regions responsible for antidepressant effects may be distinct from those that mediate cognitive impairment, and that ECT may be improved by the restriction of electrical currents to the former. For example, bifrontal (BF) ECT has been shown to have similar overall efficacy to bitemporal (BT) ECT, with possibly less cognitive impairment.40 44 45 Alternative electrode placements and convulsive therapy types that are capable of restricting current pathways while eliciting adequate seizure activity are in development for example, left anterior right temporal electrode placement,46 focal electrically administered seizure therapy47 48 and magnetic seizure therapy,49 50 although large randomised controlled tirals (RCTs) are not yet available to confirm their utility.
The clinical decision of which ECT protocol to pursue must be made with an understanding of the risks and benefits associated with the choice of electrical dose, stimulus parameters (eg, pulse width) and electrode placement (eg, bilateral or unilateral). Considering that approximately one million patients receive ECT for the treatment of mental disorders each year globally,51 or between 0.11 and 0.50 individuals per 10 000,52–54 it is vital that clinicians have access to the information needed to make an appropriate evidence-based selection of ECT parameters. However, variability in methodology and findings between studies make this choice difficult, and have contributed, in part, to significant global variations in ECT practices.52
One means of addressing this issue is to conduct multiple head-to-head RCTs comparing all interventions of interest, including more recent stimulus parameters and seizure induction methods. Such an endeavour, however, is unlikely to eventuate due to the number of possible pairwise comparisons between all treatment variations. An alternative solution is to leverage the available direct, as well as indirect, comparisons in the current ECT literature using a network meta-analysis (NMA).55 An NMA allows for comparisons of different interventions, even though they may not have been examined in head-to-head RCTs. Moreover, an NMA can be used to facilitate clinical decision making by ranking ECT interventions across a range of clinical outcomes, including their comparative antidepressant efficacy and cognitive side effect profile. Although prior NMAs have confirmed the overall safety and efficacy of ECT relative to other interventions,24 56–59 they have not investigated crucial differences between treatments according to electrode placement, pulse-width and dosing protocol.
Objectives
The aim of this systematic review and NMA is to inform and facilitate clinical practice decisions by comparing and ranking ECT types for the acute treatment of depression. ECT types will be compared according to three outcomes obtained from the acute post-treatment period:
Efficacy of the treatment to reduce depressive symptomatology.
Cognitive functioning, including the severity and pattern of any cognitive dysfunction.
Acceptability operationalised as the all-cause drop-out rate.
Methods
The methodology of this NMA will be conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension statements for NMAs.60 This protocol has been prepared according to the recommendations of Cochrane.61 This review does not require ethics approval.
Eligibility criteria
Types of studies
The systematic review will identify and include RCTs of at least two eligible ECT interventions under investigation for the acute treatment of depression, including unipolar and bipolar disorders. Trials may be double-blind, single-blind or unblinded. Studies without blinding will not be excluded because most modern ECT RCTs compare active interventions. For cross-over trials, we will extract data from before crossover.
Exclusion criteria include: (1) naturalistic observations of mood and cognitive outcomes following ECT; (2) investigations using ECT for the primary treatment of neuropsychiatric conditions other than a major depressive episode, for example, schizophrenia; (3) and finally, we will exclude the use of historical and now outmoded sine wave forms due to evidence of deleterious neurocognitive effects and subsequent decline in popularity for clinical use.35 62
Types of participants
Studies investigating adult patients aged 18 years and older will be included in the review, including patients diagnosed with either unipolar or bipolar depression experiencing a major depressive episode. Diagnoses must have been made using standardised diagnostic criteria, such as the Feighner criteria,63 Research Diagnostic Criteria,64 Diagnostic Statistical Manual (any version),1 or the International Statistical Classification of Diseases and Related Health Problems. Comorbidities and secondary diagnoses of other neuropsychiatric disorders will not be considered as exclusion criteria.
Types of interventions
We will include RCTs of head-to-head comparisons, as well as studies with a sham-controlled trial arm. Nodes in the NMA will be defined using a combination of electrode placement, pulse width and dose (see figure 1 for a network plot of included ECT types).
Provisional network plot of electroconvulsive therapy nodes. BF, bifrontal; BT, bitemporal; Fix, fixed; Form, formula; Mod, moderate; RUL, right unilateral; UB, ultrabrief.
Electrode placements include right unilateral (RUL), BF and BT. Left unilateral (LUL) electrode placements are infrequently used in clinical practice, although a recent narrative review has suggested comparable antidepressant and antipsychotic effects to RUL and BF placements.65 Considering the lack of adequately powered RCTs of LUL in the literature, this montage is not included as a node in the NMA. If sufficient studies are identified during the systematic review, LUL will be included as a node.
Pulse width will be dichotomised as either UB for values <0.5 ms, and brief for values ≥0.5 ms but ≤2 ms. Appropriate dosing of the ECT stimulus is known to impact treatment outcomes.66 67 In recent years, studies have suggested that ECT should optimally be delivered as a multiple of the patient’s individual seizure threshold (ST).68 69 Importantly, adequate dosing relative to ST differs between ECT types. For example, efficacy for brief-pulse RUL ECT delivered at six times ST does not differ from BT ECT given at 1.5–2.5-times ST.26 38 70 71 Similarly, UB-RUL ECT delivered eight times ST has shown comparable efficacy to brief-pulse RUL ECT given at five times ST.72 To ensure consistency across comparisons, the adequate dose range relative to ST will, therefore, be specified according to electrode placement. BF and BT placements will be categorised as low dose for values ≤1.5 × ST, as moderate dose for values >1.5 × ST and ≤3.0 × ST, and as high dose for values >3.0 × ST. Given the wider range of studied values, RUL will be categorised as low dose for values ≤3.0 × ST, as moderate dose for values >3.0 × ST and <5.0 × ST, and as high dose for values ≥5.0 × ST.73
Fixed or formula-based dosing protocols,74 which remain in widespread use globally,75 will be included as separate nodes for each of the available placements (BF, BT and RUL). However, we are conscious that substantial heterogeneity may arise within nodes for fixed and formula-based dosing protocols, due to differences in the choice of fixed charge or the specific dosing algorithm in use between studies, as well as due to the limited number of RCTs adopting these methods. We aim to reduce heterogeneity in treatment nodes to limit the possibility of inconsistency within network loops and improve precision of effect effects.60 Therefore, depending on the inconsistency and quantity of data available for these nodes, they may be excluded from NMA analyses.
If an included study examined multiple ECT types, only data from those defined as nodes will be extracted. In instances where a study reports participant switching between ECT types, for example, to improve efficacy or reduce adverse events, we will correspond with the study authors and obtain data strictly prior to any change in ECT type. During the literature review if new treatment interventions are identified the study investigators will decide whether to include them as a node in the NMA through consensus. The working group will ensure that participants within the identified studies meeting inclusion criteria would have an equal likelihood, in theory, of being randomly allocated to any of the interventions included as nodes in the final NMA. That is, it is feasible to conceive of a multi-arm trial consisting of all included ECT types, such that they can be jointly randomisable.
Outcome measures
Primary outcomes
Antidepressant efficacy: symptom severity (continuous outcome)
Depression severity scores will be extracted using the total score on the Montgomery Asberg Depression Rating Scale (MADRS) or Hamilton Depression Rating Scale (HDRS). If these are not reported, other standardised observer-rated scales for depression may be used. Group means and SD will be extracted at baseline and the primary study end point. When end point scores are not reported but change scores are, we will use the latter.
Cognitive functioning: anterograde verbal recall
The primary outcome for effects of ECT on cognitive functioning in patients with MDD will be a standardised neuropsychological measure of anterograde verbal recall. This was chosen as verbal recall is commonly most affected in the immediate period after a course of ECT,34 76 despite such effects generally being transient. Verbal recall will be measured using scores obtained from standardised neuropsychological tests such as the Rey Auditory Verbal Learning Test, Buschke Selective Reminding Test, Verbal Learning and Memory Test, the Hopkins Verbal Learning Test or others.
Acceptability
The proportion of patients exiting the study for any reason from baseline to the end of the course of treatment (ie, all-cause drop-out rate). If sufficient data are available, we will perform an additional analysis of drop-outs due specifically to tolerability in receiving the intervention.
Secondary outcomes
Antidepressant efficacy: response rate
The proportion of responders to ECT for the treatment of depression will be used as a secondary measure of efficacy. Response will be defined as an improvement of at least 50% in depression scores from baseline to study end point on a standardised observer-rated scale for depression. The MADRS and HDRS will be used as primary sources to determine response. However, if these are not reported, other standardised observer-rated scale for depression may be used. The number of patients meeting response criteria, and the number of non-responders, will be extracted.
Antidepressant efficacy: remission rate
Remission will be defined as a reduction in depression scores below a scale-specific threshold on a standardised scale of depression. A depression score less than 10 will be considered remission using the MADRS,77 less than 8 on the HDRS 17-item scale, and less than 11 on the 24-item HDRS scale.78 If other standardised observer-rated scales of depression are used, the scale-specific threshold for remission will be used, if defined. If undefined then, where possible, a validated conversion of remission threshold from HDRS or MADRS (whichever is available) will be performed.79 80 The total number of patients who experienced a remission of depressive symptoms between baseline and study end point will be extracted for each study.
Cognitive functioning: autobiographical memory
Retrograde autobiographical memory impairments are the most durable side effects of ECT34 and are associated with the greatest concern from patients. Notwithstanding, unlike standardised neuropsychological tests, measures of retrograde autobiographical memory have greater variability in their respective outcomes, which increases heterogeneity when collating results from different studies. For this reason, anterograde verbal recall was prioritised as a primary cognitive outcome. Retrograde autobiographical memory changes after ECT will be obtained from the Autobiographical Memory Interview (AMI), the AMI-Short Form, or a similar standardised scale.
Cognitive functioning: anterograde visuospatial recall
Visuospatial recall will be assessed as a secondary cognitive outcome using assessments of delayed visuospatial recall (eg, Brief Visuospatial Memory Test) or complex figure tasks (eg, Medical College of Georgia Complex Figure) extracted from baseline and the study end point.
Study selection
Two researchers will independently conduct searches in the previously mentioned databases using the terms: [Depression (MeSH Terms) OR Depression (All Fields) OR Depress* (All Fields) OR Melancholia (All Fields) OR ‘Bipolar disorder’ (All Fields) OR ‘Bipolar spectrum’ (All Fields)] AND [‘Electroconvulsive therapy’ (MeSH Terms) OR ‘Electroconvulsive shock’ OR ‘Electroconvulsive shock therapy’ OR ‘Electroshock therapy’ OR Electroconvulsive OR ECT]. They will sequentially review article titles, abstracts and full-text articles to determine whether studies meet eligibility criteria. Any disagreements will be resolved through consensus. If necessary, authors of disputed studies will be contacted for clarification.
Data extraction
Information sources
Embase, MEDLINE, PubMed, Web of Science databases are recommended to obtain adequate coverage of search terms for a systematic review, and so will be used for the present review.81 In addition to the databases listed above, the search will also include Scopus, PsycINFO and the Cochrane Central Register of Controlled Trials (CENTRAL).
Reference lists of studies meeting inclusion criteria, and of prior systematic reviews examining therapeutic efficacy and cognitive impairments associated with ECT, will also be examined to locate additional relevant trials.
Data collection and management
First or corresponding authors from studies identified during the systematic review will be contacted by email and asked to provide additional information using our data extraction form. Aggregate data will be extracted from the included studies’ manuscripts or supplementary materials. Additionally, we will check for data on public repositories, such as the National Institute of Mental Health Data Archive. A structured data extraction form will be used to obtain qualitative and quantitative data from all included studies. This form will collect information regarding study characteristics including, lead author, publication year, study title, study link (eg, DOI), country where the majority of data collection occurred, study type (eg, cross-over or parallel design); participant demographics including, sample size, gender, age, proportion with psychotic depression, proportion with bipolar depression, and treatment resistance; intervention details including ECT device, pulse width, number of convulsive therapy treatments, treatment frequency (ie, number of treatments per week), dosage as proportion of ST, electrode placement and anaesthetic used; and outcome measures including depression rating scale, baseline depression score, end of acute phase treatment depression score, response and remission rates, drop-outs, cognitive scale/s used for verbal recall, visuospatial recall, and retrograde autobiographical memory, baseline cognition score, and end of acute phase treatment cognition score.
Continuous outcomes
Means and SD will be extracted for ECT types included in prespecified NMA nodes. If these values are not reported, authors will be contacted to supply any missing information. If needed, 95% CIs, SEs of the mean, manuscript figures and other approaches will be used to estimate SD.
Dichotomous outcomes
Dichotomous outcomes, such as response and remission rates and all-cause drop-outs will be the secondary outcome measures for antidepressant efficacy and acceptability. These will be examined in addition to continuous outcomes because they are commonly reported in studies of depression and therefore allow easier generalisability to other treatment modalities, for example, antidepressant medications. For measures of antidepressant efficacy, patients that drop-out will conservatively be labelled as non-responders and non-remitters, unless explicitly stated as otherwise in the study manuscript. An NMA of pharmacological treatments for depression used a similar rationale to include dichotomous outcomes as a variable of interest.82
Length of a trial
There is no consensus regarding the length and session frequency of an adequate course of acute phase ECT treatment. For the purposes of this NMA, we will consider an acute phase of treatment as a protocol of 2–3 sessions of ECT per week and a minimum of six sessions based on findings suggesting that the majority of patients experience a response to treatment by this stage.83 Data will be extracted from the primary endpoint as defined by the study, using the earliest assessment ≤7 days after the final ECT session and excluding assessments performed immediately after ECT during emergence from anaesthesia.
Risk of bias assessment within individual studies
Once the search is complete, independent raters will evaluate the risk of bias of included studies using the Cochrane tool.84 This assesses several methodological criteria, including: (1) random sequence generation (selection bias); (2) allocation concealment; (3) blinding of participants and study personnel rating antidepressant efficacy and cognitive outcomes (performance bias), though blinding of study treaters may not be possible when comparing ECT montages; (4) blinding of study personnel rating antidepressant efficacy and cognitive outcome assessments (detection bias); (5) incomplete outcome data (attrition bias); (6) selective outcome reporting (reporting bias) and (7) other bias. Each study’s overall risk of bias will be summarised using a label of low, high or unclear using the methodology described in the Cochrane Collaboration Handbook.
Risk of bias assessment across studies
Primary and secondary outcomes will be examined for small-study effects through visual inspection of contour-enhanced funnel plots,85 and statistical methods such as the Harbord-Egger bias test of asymmetry,86 if more than 10 studies are available. This will be repeated for each comparison between included ECT interventions. Lack of significance on the Harbord-Egger bias test will not be interpreted as indicative of lack of bias but will be reported as no evidence of bias. Evidence of bias will be incorporated into the interpretation of the results. Comparison-adjusted funnel plots of the entire network will be used to assess the presence of small-study effects.87 Small-study effects will be assumed to be in the direction of exaggerated outcomes. If funnel plot asymmetry is detected, we will perform network meta-regression models to test whether small studies systematically tend to favour specific interventions.87
Reporting bias
Unpublished studies alter the effect size of meta-analyses, generally in the direction of reduced effects,88 though not in all cases.89 Clinical trials registries, including ClinicalTrials.gov and the European Union Clinical Trials Register, will be searched to identify unpublished studies. If any are identified, study authors will be contacted to obtain access to outcomes of interest, thereby minimising the risk of publication bias. This approach can significantly increase the number of studies included in the analysis, though may not necessarily lead to a qualitative change in the interpretation of results.90
Transitivity
Transitivity is the assumption that the interventions included in the analysis do not differ with respect to the distribution of effect modifiers (ie, variables associated with treatment outcomes). To check the transitivity assumption additional sensitivity, subgroup and meta-regression analyses will be conducted to confirm the robustness of findings. If the distribution of effect modifiers is balanced across comparisons, we will conclude that there is insufficient evidence of intransitivity, not that there is evidence against intransitivity. If intransitivity is identified within an outcome measure, we will not proceed with statistical analysis for that outcome.
Summary measures
Clinical and demographic information
A table containing clinical and demographic data extracted from included studies will be provided in a supplementary document.
Network geometry
For each outcome, a network graph will be reported showing all included studies in the NMA. Nodes and edges will be scaled according to the number of participants, and the number of trials, respectively.
Treatment effects
Mean effect sizes, 95% credible intervals (CrIs) and 95% predictive intervals (PrIs) will be reported. The 95% PrI indicates the range within which a treatment effect is to be expected, with 95% probability, if a new trial were to be conducted comparing a specific pair of treatments, and is generally more conservative than the 95% CrI.91
Statistical analysis and model implementation
Bayesian random effects NMAs will be conducted using rjags,92 and nmjags,93 which will run on open-source R software (R Foundation for Statistical Computing). Random effects models do not assume a priori that the true treatment effect is the same between studies and thus provide a more conservative estimate of outcomes. Markov Chain Monte Carlo (MCMC) methods will be implemented to fit the model and obtain posterior distributions of summary treatment effects, thereby allowing estimation of rank probabilities. For each model, the first 50 000 iterations will be discarded as the burn-in period to achieve convergence, and the subsequent 100 000 iterations will be used to fit the data. Convergence will be assessed via inspection of MCMC history plots, as well by using the Brooks Gelman-Rubin diagnostic tool and the potential scale reduction factor statistic representing the ratio of between-chain to interchain variability.94 95 Model fit will be quantified using the deviance information criterion.96 Non-informative or vague priors will be used for the overall mean effect and between-study SD.97 Multiarm trials will be incorporated into the NMA by substituting the random effects distribution with a multivariate normal distribution of intervention effects, thus statistically accounting for the covariate structure between arms of the same trial.98
Rank order analysis
To facilitate clinical decision making, we will place the included interventions into a treatment hierarchy for the primary outcomes of efficacy and cognitive functioning using surface under the cumulative ranking (SUCRA) curve analyses.99 A SUCRA value of 100% indicates that a treatment is certain to outperform other treatments included in the NMA, whereas a value of 0% indicates it will be the worst. Additionally, posterior probabilities obtained from the Bayesian NMA model can also be used to rank ECT types, with higher posterior values indicating a higher likelihood that the intervention is a better performer.
Missing data
Outcomes of patients who leave the trial early are often imputed. The most common approach is last observation carried forward100; however, other methods may include interpolation, the group mean at the time of drop-out, prediction of missing data using regression, and matching (using values from similar cases). The type of imputation used will be reflected in the risk of bias assessment. Where possible, study authors will be contacted and asked to provide more information.
Assessment of heterogeneity
Residual heterogeneity will be calculated using τ2, and 95% PrI will be used to investigate the impact of heterogeneity. Standard pairwise meta-analyses and forest plots will be generated for all possible pairwise node comparisons to check that there is no relevant heterogeneity between trials. A random effects model will be used as it produces a more conservative estimate of heterogeneity. A high level of heterogeneity may indicate that pooling of studies within that comparison is unadvisable.
Assessment of inconsistency
Within the context of NMAs, inconsistency is a statistical term that refers to the discrepancy between direct effects (ie, direct comparison between nodes) and indirect effects (ie, indirect comparisons between nodes via at least one additional node). Both local and global tests will be used to examine network inconsistency. For the local test, we will conduct a preliminary assessment using the loop-specific approach.101 Subsequently, a node-splitting method called Separating Indirect from Direct Evidence102 103 will be used, focusing on the subset of nodes highlighted in the loop-specific method as showing a difference between indirect and direct forms of evidence, if any. Finally, we will use a design-by-treatment interaction/inconsistency model, similar to a goodness-of-fit-test,104–106 as a global measure to assess whether the network as a whole demonstrates any inconsistency.
Non-significance on either local or global tests of inconsistency will not be interpreted as proof of absence of inconsistency, but rather as a lack of evidence of statistical difference between direct and indirect treatment effects.107
In the event that we detect statistically significant inconsistency we will investigate potential underlying causes. This will include inspection of the data for extraction errors as well as an exploration of effect modifiers identified a priori as possible sources for differences between direct and indirect comparisons. These effect modifiers are described in further detail in the Additional analyses section. Finally, if inconsistency cannot be resolved, we will consider avoiding synthesis of the problematic network loop.
Additional analyses
Sensitivity analyses
Studies with a high risk of bias will be excluded as part of sensitivity analyses to check the robustness of the model.
To further explore heterogeneity between studies, we will perform subgroup and meta-regression analyses on the following variables, if reported in a sufficient number of studies.
Subgroup and meta-regression analyses
Exploratory analyses will investigate the influence of gender, as well as the type of depression, including psychotic versus non-psychotic depression, as well as bipolar versus unipolar disorder, on outcome measures. These variables will be assessed by including the proportion of participants in a specified reference category (eg, percentage of bipolar participants) for each included study in the NMA model. Regarding acceptability outcomes, measured using all-cause drop-out rates, if sufficient data are available, we will perform an additional analysis of drop-outs occurring specifically due to tolerability issues arising during the treatment course.
Additional meta-regression analyses will be used for continuous variables, including age at the time of ECT treatment, baseline depression severity, baseline measures of cognitive functioning and treatment resistance defined as the average number of failed courses of antidepressants given at an adequate therapeutic dosage in the current episode.
Quality assessment of all comparisons in the network
The open-source software package CINeMA (Confidence In Network Meta-Analysis) will be used to assess the quality of evidence contributing to network estimates of primary and secondary outcomes,108 and, in so doing, estimate the confidence in the NMA results. CINeMA is similar to the GRADE methodological framework and considers six domains to evaluate the confidence in the NMA findings. These domains are within-study bias, across-studies bias, indirectness, imprecision, heterogeneity and incoherence.
Patient and public involvement
No patients or public representatives were involved in the design of the NMA protocol.
Discussion
Limitations
Though dropouts will be well documented, psychological test batteries and cognitive adverse events experienced during the treatment course are not expected to be consistently reported. Therefore, it may not be possible to conduct a full assessment of cognitive outcomes as outlined in the protocol.
Due to the large parameter space for ECT, and the subsequent variability in treatment protocols observed in the literature, there may be insufficient data available to comprehensively assess the impact of all potential prognostic factors and effect modifiers for all outcome measures. Where there is insufficient data to perform additional analyses, this will be transparently reported.
Finally, the aim of this NMA is to investigate immediate outcomes of efficacy, cognitive effects and acceptability following ECT. Of equivalent concern is the longevity of efficacy outcomes in the months and years following successful treatment, as well as the persistence of cognitive side effects. However, these issues are beyond the scope of the present analysis and will require separate study.
Strengths
Despite the limitations listed above, we hope this study will be a valuable clinical tool. This NMA will determine the factors that influence efficacy, cognitive effects and patient acceptability of ECT. Ranking of treatments according to these outcomes will facilitate clinical decision making when determining the appropriate convulsive therapy type to prescribe for patients.
Ethics statements
Patient consent for publication
Acknowledgments
We are grateful to Brendan Mauch and Sereibanndith Seang for conducting a preliminary review of the ECT literature to establish the feasibility of this NMA.
References
Footnotes
Twitter @NikolinStevan
Contributors SN conceived the study. SN, RF-T and KO, drafted the manuscript. SN, KO, RF-T, AC, DoMM, MB, HAS, DeMM, PS, CHK and CL, critically revised the protocol manuscript. All authors approved the final version of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests HAS serves as a scientific advisor to Cerebral Therapeutics, Holmusk Technologies, LivaNova, Neurolief, Neuronetics, Parow Entheobiosciences and SigmaStim (formerly MECTA Corporation); he receives honoraria and royalties from Elsevier and Oxford University Press; he is the inventor on non-remunerative US patents for Focal Electrically Administered Seizure Therapy (FEAST), titration in the current domain in ECT, and ECT devices with current adjustment, each held by SigmaStim; and he is the originator of magnetic seizure therapy. DoMM has received travel support from Miyuki Giken, speaker’s honorarium from Lundbeck and consultancy fees from Duncan Pharmaceuticals, all unrelated to the current work. DeMM, has received speaker's honoraria from MECTA and Otsuka and an honorarium from Janssen for participating in an esketamine advisory board meeting. All of these were unrelated to the current work. CHK, receives royalties from Cambridge University Press for Handbook of ECT, fees from UpToDate for ECT topics, and fees from Northwell Health for teaching in an ECT course. CL has received speaker's honoraria from MECTA, an honorarium from Janssen for participating in an esketamine advisory board meeting, and fees from Wesley Hospital and South Eastern Sydney Local Health District for teaching ECT courses. These were unrelated to the current work. The other authors declare no competing interests.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.