Introduction It is unclear how pharmacological and non-pharmacological interventions compare with each other in terms of efficacy and tolerability for core symptoms and additional problems in adults with attention-deficit/hyperactivity disorder (ADHD). We aim to conduct the first network meta-analysis (NMA) comparing pharmacological and non-pharmacological interventions (or their combinations) in adults with ADHD.
Methods and analysis We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement for NMAs. We will search a broad set of electronic databases/registries and contact drug companies and experts in the field to retrieve published and unpublished randomised controlled trials (RCTs) (parallel or cross-over) of medications (either licensed or unlicensed) and any non-pharmacological intervention in adults (≥18 years) with ADHD. Primary outcomes will be: (1) change in severity of ADHD core symptoms, and (2) acceptability (all-cause discontinuation). Secondary outcomes will include tolerability (drop-out due to side effects) and change in the severity of emotional dysregulation, executive dysfunctions and quality of life. The risk of bias in each individual RCT included in the NMA will be assessed using the Cochrane Risk of Bias tool-version 2. We will evaluate the transitivity assumption comparing the distribution of possible effect modifiers across treatment comparisons. We will perform Bayesian NMA for each outcome with random-effects model in OpenBUGS. Pooled estimates of NMA will be obtained using the Markov Chains Monte Carlo method. We will judge the credibility in the evidence derived from the NMA using the CINeMA tool (which includes assessment of publication bias). We will conduct a series of sensitivity analyses to assess the robustness of the findings.
Ethics and dissemination As this is the protocol for an aggregate-data level NMA, ethical approval will not be required. Results will be disseminated at national/international conferences and in peer-reviewed journals.
PROSPERO registration number CRD42021265576.
- adult psychiatry
- impulse control disorders
- clinical trials
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Strengths and limitations of this study
The study will be conducted by a team with extensive expertise in the clinical assessment and treatment of attention-deficit/hyperactivity disorder (ADHD), as well as in advanced network meta-analysis (NMA) statistics.
The protocol was designed and the study will be carried out with the involvement of patients and members of the public in the review team.
We will include both published and unpublished data, systematically gathered by drug manufacturers and study authors.
We will include, as outcomes, both ADHD core symptoms and related clinical problems, thus increasing the ecological validity of the study.
The main limitation is that the proposed NMA will include aggregate-level data, rather than individual patient level data.
Attention-deficit/hyperactivity disorder (ADHD) is the most common neurodevelopmental condition in children,1 2 and its impairing symptoms persist into adulthood in up to ~75% of childhood cases.3 4 Adult ADHD has a prevalence estimated at ~2.5%5 and is commonly comorbid with other disorders (eg, depression or anxiety)6 and with problems such as emotional dysregulation, which are often the main trigger for a referral to clinical services.6 If untreated, adult ADHD is associated with substantial societal burden, including significantly increased risk of unemployment, substance abuse, criminal acts, accidents and mortality.6 The personal and societal costs of untreated ADHD in adults are estimated at around £20 000/person/year.7
Both pharmacological and non-pharmacological (eg, psychological) treatments are available for adults with ADHD.8 Pharmacological and non-pharmacological interventions should be considered to be complementary, rather than mutually exclusive options. For instance, while stimulants are considered highly effective in decreasing the severity of adult ADHD core symptoms over the short-medium term (effect size, ES: ~0.8),9 their efficacy in the treatment of emotional dysregulation is lower (ES: ~0.3–0.5),10 suggesting the need for additional pharmacological or non-pharmacological options.
The current ADHD guidelines from the National Institute for Health and Care Excellence (NICE) recommend pharmacotherapy (stimulants followed by the selective norepinephrine reuptake inhibitor atomoxetine) as first-line treatment options for adult ADHD, with psychological therapies as second option.11 However, the recommendation on the sequencing of pharmacological and non-pharmacological options was based on one randomised clinical trial (RCT) only,12 comparing head-to-head pharmacotherapy and psychological treatment, retrieved from a literature search (up to 28 April 2017) that is now outdated. Since the NICE guidelines were published, a number of RCTs have been published pointing to significant efficacy and good tolerability of a variety of non-pharmacological interventions—including cognitive behavioural therapy, dialectic behavioural therapy, mindfulness, cognitive training and neurofeedback—for ADHD core symptoms and/or associated dysfunctions.13 Additionally, due to the paucity of RCTs, the NICE committee was not able to make any evidence-based recommendation on which type(s) of non-pharmacological treatments are preferred. This is highly problematic in particular for those patients who do not opt for or are unable to tolerate a pharmacologic treatment and need to be informed on the comparative efficacy/tolerability of currently available non-pharmacologic options. Furthermore, recent studies have assessed internet delivered non-pharmacological interventions, to possibly maximise efficiency and cost-effectiveness (eg, 14 15). This is particularly relevant considering the need for remote assessment/treatment prompted by the current pandemic-related restrictions and the likely push towards digital interventions in the post-COVID-19 era.
Therefore, there is a need for updated evidence synthesis regarding how non-pharmacological interventions—and different ways to deliver them —compare with each other, to pharmacologic treatments or to combinations of pharmacological and non-pharmacological interventions in terms of efficacy and tolerability on specific relevant outcomes (eg, ADHD core symptoms, emotional dysregulation and executive functions) in adults with ADHD.
A well-powered RCT would be suited to compare pharmacological and non-pharmacological options, but there are obvious financial and logistic constraints in conducting well-powered RCTs comparing all interventions for ADHD in adults. Network meta-analysis (NMA), which allows for the comparison of two or more interventions even when they have not been compared head-to-head in the studies included in the meta-analysis,16 provides a cost-effective, practical option to address this gap.
A scoping search in PubMed/Medline (PubMed Central and Europe PubMed Central), PsycInfo and Embase (up to 1 October 2021) using search terms for ‘ADHD’, ‘adults’ and ‘network meta-analysis’ did not find any NMA including, in the same network, pharmacologic and non-pharmacologic interventions for adults with ADHD. Based on our searches, no protocol for such NMA has been registered in PROSPERO or other registries at the time of writing.
Therefore, we aim to conduct the first systematic review/NMA of published and unpublished RCTs to assess the comparative efficacy and tolerability of UK licensed and unlicensed medications for ADHD, non-pharmacological treatments, or their combination, on ADHD core symptoms severity and related dysfunctions (eg, emotional dysregulation) in adults with ADHD.
Methods and analysis
The methods of the proposed study are based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and its extension for NMA.17 The methods are in line with those of another NMA18 of pharmacological and non-pharmacological interventions for depressive disorder.
The protocol of the present NMA is preregistered in PROSPERO.19
We plan to start the study on 1 March 2022 and to complete it by 1 March 2024.
We will update the search for RCTs of medications for ADHD from an NMA9 published in 2018 and conduct a de novo search for the non-pharmacological interventions. The search will be conducted with the support of Systematic Review Solutions Ltd. (SRS), an independent health research service company specialising in evidence-based medicine methods and meta-research training, production of systematic reviews and Health Technology Assessment reports, and development of clinical practice guidelines. SRS conducted the search for the previous NMA.9 Using a similar search strategy, we will search a broad set of electronic databases, including PubMed, BIOSIS Previews, CINAHL, the Cochrane Central Register of Controlled Trials, EMBASE, ERIC, MEDLINE, PsycINFO, OpenGrey, Web of Science Core Collection, ProQuest Dissertations and Theses (UK and Ireland), ProQuest Dissertations and Theses and the WHO International Trials Registry Platform, including ClinicalTrials.gov, with no language/type of document restrictions. For the specific syntax for each database, see online supplemental 1.
We will also search the US Food and Drug Administration, European Medicines Agency and relevant drug manufacturers’ websites for RCTs of medications. We will also endeavour to gather relevant unpublished data for pharmacological and non-pharmacological interventions by contacting drug companies, study authors and the members of key scientific organisations of ADHD worldwide. Specifically, we will contact the European Network of Hyperkinetic disorder (Eunethydis), the World ADHD Federation, the he American Professional Society of ADHD and Related Disorders (APSARD) and the Canadian ADHD Resource Alliance (CADDRA) to advertise the study and query about the existence of any eligible unpublished study.
We will include parallel or cross-over RCTs of at least 1-week duration for pharmacological treatment, in line with prior work,9 and of at least four sessions for psychotherapy. For cross-over studies of medications, to address concerns around possible ‘carry over’ effects, we will use data from the precross-over phase, when reported in the paper. When data for the precross-over phase are not reported, we will contact study authors to gather them. If precross-over data are not reported and not available on request, we will use data at the end point (after crossing over), only if there was a washout period (as reported in Cortese et al)9 between the two phases (precross-over and postcross-over) of the RCT. For trials of neurotherapies (eg, neurofeedback), we will include RCTs of any length deemed appropriate for these approaches. For trials of medications, cognitive training or neurotherapies alone, we will include only double-blind RCTs (patients and raters blinded). For trials of psychotherapy alone or the combination of medications and psychotherapy, we will include trials in which observers and/or raters were masked and/or participants were assessed by self-rating ADHD scales, because participants and therapists cannot be blinded, but we will then conduct a sensitivity analysis including only double blind RCTs (please see below). We will exclude studies with enrichment designs (eg, trials selecting responders only after a run-in phase), because these types of trial can potentially inflate efficacy and tolerability estimates.20
We will retain RCTs including adults (≥18 years) with a formal diagnosis of ADHD according to DSM-III, DSM III-R, DSM-IV (TR), DSM-5, ICD-9, ICD-10 or ICD-11. We will not restrict our search by ADHD subtype or presentation, sex, ethnicity, IQ, socioeconomic status or comorbidities.
As in prior work,9 pharmacological interventions will include stimulants (methylphenidate and amphetamines, including lisdexamphetamine); atomoxetine; guanfacine XR, clonidine, bupropion and modafinil. We will also search for eligible studies of viloxazine, which has been recently approved by the FDA for children and adolescents (aged 6–17) with ADHD.21 In the analyses, we will lump methylphenidate and amphetamines as: (1) a previous NMA9 did not find any significant difference, in terms of efficacy, between methylphenidate and amphetamines in adults with ADHD; (2) accordingly, current NICE guidelines11 recommend methylphenidate or lisdexamphetamine (or other amphetamines) as first-line pharmacological treatment for adults. Any type of non-pharmacological intervention will be considered.
The pharmacological control condition will be a pill placebo; non-pharmacological controls will include waiting list, treatment as usual, clinical management, active control in psychotherapy, and psychological placebo (sham).
Primary outcomes: (1) change in severity of ADHD core symptoms, according to a standardised rating scale.9 We will consider separately self-rated ADHD core symptoms and observer as well as clinician-rated symptoms; (2) acceptability (all-cause discontinuation measured by the proportion of patients who withdrew from the study for any reason). Secondary outcomes: tolerability (drop-out due to side effects); change in the severity of emotional dysregulation, measured with any of the scales listed in Lenzi et al;10 executive dysfunctions, based on any of the scales in Tamminga et al,22 and quality of life, measured with any of the scales listed in Tsujii et al.23
We will select studies, and extract/collect data in a two-step process. First, two independent investigators will screen the titles and abstracts we identified. Second, two independent investigators will obtain and read the full texts of all potentially relevant studies and determine the final list of studies to include. Any disagreement will be resolved by senior investigators. We will extract data into prespecified data extraction forms.24 For each study, we will extract information on study characteristics (eg, setting, study design and sample size), participant characteristics (eg, mean age and range, presence of comorbidities and concomitant therapies), interventions and controls (eg, dose, frequency of treatment) and outcomes. We will systematically contact study when needed to gather unpublished information/data.
Study risk of bias assessment
The risk of bias in each individual RCT included in our NMA will be assessed using the Cochrane Risk of Bias tool-version 2 (RoB-2), as recommended in The Cochrane Handbook of Systematic Reviews of Interventions.25 The tool includes five domains through which bias might be introduced into the result. For individually randomised trials (including cross-over trials), these include:
Bias arising from the randomisation process.
Bias due to deviations from intended interventions.
Bias due to missing outcome data.
Bias in measurement of the outcome.
Bias in selection of the reported result.
We will use the appropriate templates for randomised parallel-group and cross-over trials, respectively.
Measures of treatment effects
For continuous outcomes, we will use mean difference (MD) as a measurement of treatment effect, with the relative 95% CI, when studies assessed the outcome with the same instruments; standardised mean difference (SMD, Cohen’s d) when studies used different instruments.26 We will use published mean values and standard deviations (SDs). If SDs are not available, they will be estimated by conversion from SEs, p-values, CIs or t-values.27 If none of the above values is available from the published paper, we will contact the authors of the study to obtain information. If the information is not provided by the study author, we will employ a validated method for imputation to derive missing SDs.28
For dichotomous outcomes, we will calculate the OR and relative 95% CI. Missing dichotomous outcome data will be handled according to the intention-to-treat principle. Participants who drop out after randomisation will be considered as having a negative outcome.
Assessment of clinical and methodological heterogeneity within treatment comparisons
In each pairwise comparison, patient characteristics, treatments and outcome deﬁnitions of included studies should be similar.26 We will produce descriptive statistics for studies and study population characteristics across included trials to assess clinical and methodological heterogeneity. Within each pairwise comparison, we will compare these characteristics to assess the presence of clinical heterogeneity.
Assessment of transitivity across treatment comparisons
It is appropriate to use NMA if the assumption of transitivity can be defended. Transitivity holds when the distributions of the potential effect modifiers, like study and patient-level covariates, are balanced across all pairwise comparisons.29 30 To assess the transitivity assumption, we will compare the distribution of clinical and methodological variables (eg, ADHD severity at baseline, comorbidities, adherence and treatment duration) that could act as effect modifiers across treatment comparisons.
First, we will conduct conventional pairwise meta-analyses with a random-effects model in STATA V.16.1 for all outcomes and comparisons with at least two studies. Then, we will perform Bayesian NMA for each outcome with random-effects model in OpenBUGS,31 accounting for correlations induced by multiarm studies.32 Pooled estimates of NMA will be obtained using the Markov Chains Monte Carlo method. We will employ the binomial (dichotomous outcomes) and normal (continuous outcomes) likelihood functions and will use vague prior distributions for the treatment effects and a minimally informative prior distribution for the common heterogeneity SD depending on the outcome. We will examine Gelman-Rubin trace plots to check that multiple chains achieve convergence. All results will be reported as treatment effects (MD, SMD or OR) and their 95% credible intervals. NMA results will be presented in league tables and forest plots.33
We will estimate heterogeneity variances for each pairwise comparison in standard pairwise meta-analyses and assess the presence of statistical heterogeneity by visually inspecting the forest plots and calculating the I-squared statistic.34 In the NMA, we will assume a common estimate for heterogeneity variance across comparisons and base our assessment of statistical heterogeneity in the whole network by comparing the magnitude of the common heterogeneity variance (τ2) with the empirical distribution as derived by Rhodes and Turner.35 36 We will also calculate the total I-squared statistic.
Statistical disagreement between direct and indirect effect sizes (incoherence) will be evaluated globally, by comparison of the fit and parsimony of consistency and inconsistency models, and locally, by calculation of the difference between direct and indirect estimates in all closed loops in the network.37 The node splitting method, which separates evidence on a particular comparison into direct and indirect evidence, will be used to calculate the inconsistency of the model.38 To determine whether the results are affected by possible effect modifiers, we will conduct network meta-regression and subgroup analysis according to the following variables: study sponsorship, treatment duration, comorbid psychiatric disorders, study risk of bias, mean baseline severity and percentage of participants treated with stable doses of medications in non-pharmacological RCTs.
We will then use the Surface Under the Cumulative RAnking (SUCRA) curve to measure, for any outcome, the probability each treatment is the best option among all treatments included in the network treatment and express the SUCRA measure as a percentage.39 We will use a comparison-adjusted funnel plot for active treatments versus control to determine the possibility of small study-effects.33 40
We will assess the certainty of evidence derived using CINeMA (http://cinema.ispm.ch/).41 42 CINeMA is a software which uses the netmeta R-package for performing NMA of the data. The tool considers the following domains: within-study bias, publication bias, indirectness, imprecision, heterogeneity and incoherence. It classifies overall confidence in evidence for each comparison as high, moderate, low or very low. In particular, for publication bias, we will use the new tool implemented within CINeMA, ROB-MEN, that allow to evaluate the impact of this bias on the results of NMA of interventions.43
We will finally conduct sensitivity analyses for primary outcomes by excluding trials without unpublished data, trials with imputed data, trials with overall sample size smaller than 20 and trials with non-blinded assessments. We will also conduct a sensitivity analysis combing guanfacine and clonidine in the same node.
Patient and public involvement
Our patient and public involvement coauthor, Ms. Bilbow, CEO of The National Attention Deficit Disorder Information and Support Service (ADDISS), one of the largest UK charities of patients with ADHD, has played a central role in the development of this protocol since its initial design. During the preparation of the present proposal, Ms. Bilbow: liaised with representatives (patients) of ADDISS, and ADHD Europe to gather their feedback on the proposal; based on the feedback form patients, critically commented on the overarching plan of the application, highlighting that it covers an important gap perceived as crucial by patients with ADHD and their families; noted the importance of comparing different types of non-pharmacological interventions given the patchy provision across the UK and uncertainties around the evidence base supporting at least some types of non-pharmacological approaches; recommended inclusion of quality of life as a secondary outcome measure.
As this is a protocol, no patients were directly involved in this study.
Ethics and dissemination
As this is the protocol of an aggregate-data level NMA, no ethical approval will be needed.
On publication, the full data set of the NMA and the codes for the analyses will be available online freely in Mendeley Data, a secure online repository for research data. The results of the study will be disseminated nationally, via conferences organised by groups of people with lived experience (eg, National Attention Deficit Disorder Information and Support Service and ADHD Foundation) and professional organisations (eg, UKAAN, Royal College of Psychiatrists). Results will also be disseminated internationally via conferences (for service users: eg, annual meeting of ADHD Europe; for professionals, eg, meetings of the WFA, Eunethydis, APSARD) and publications in peer-reviewed journals in the field of psychiatry/psychology and general medicine.
Patient consent for publication
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Twitter @CorteseSamuele, @And_Cipriani
Contributors SCo and AC designed the study. SCh contributed to the design, in particular in relation to the pharmacological interventions. AP and SY contributed to the design, in particular in relation to the non- pharmacological interventions. CDG contributed to the design, in particular in relation to the statistical analyses. AB contributed to the design, in particular in relation to PPI input. SCo drafted the first version of the manuscript. SCh, AP, SY, CDG, AB and AC critically revised the first version. Conduct, acquisition of data, analysis and interpretation of data are not applicable here as this is a protocol of a meta-analysis.
Funding This project is funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number NIHR203035). (PI: SCo).The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. SCh’s role in this study was funded by a Wellcome Trust Clinical Fellowship (110049/Z/15/Z & 110049/Z/15/A /A). AC is supported by the National Institute for Health Research (NIHR) Oxford Cognitive Health Clinical Research Facility, by an NIHR Research Professorship (grant RP-2017-08-ST2-006), by the NIHR Oxford and Thames Valley Applied Research Collaboration and by the NIHR Oxford Health Biomedical Research Centre (grant BRC-1215-20005).
Competing interests SCo declares honoraria and reimbursement for travel and accommodation expenses for lectures from the following non-profit associations: Association for Child and Adolescent Central Health (ACAMH), Canadian ADHD Alliance Resource (CADDRA), British Association of Pharmacology (BAP) and from Healthcare Convention for educational activity on ADHD. SCh receives honoraria from Elsevier for editorial work. He previously consulted for Promentis. SCh receives honoraria from Elsevier for editorial work. He previously consulted for Promentis. AC has received research and consultancy fees from INCiPiT (Italian Network for Paediatric Trials), CARIPLO Foundation and Angelini Pharma, outside the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.