Article Text

Protocol
Placebo effects in mental health disorders: protocol for an umbrella review
  1. Nathan TM Huneke1,2,
  2. Jay Amin1,2,
  3. David S Baldwin1,2,3,
  4. Samuel R Chamberlain1,2,
  5. Christoph U Correll4,5,6,7,
  6. Matthew Garner1,8,
  7. Catherine M Hill1,9,
  8. Ruihua Hou1,
  9. Oliver D Howes10,11,12,
  10. Julia MA Sinclair1,
  11. Marco Solmi5,8,13,14,15,
  12. Samuele Cortese1,8,16,17,18
  1. 1Clinical and Experimental Sciences, University of Southampton Faculty of Medicine, Southampton, UK
  2. 2Southern Health NHS Foundation Trust, Southampton, UK
  3. 3University Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
  4. 4Psychiatry, The Zucker Hillside Hospital, Glen Oaks, New York, USA
  5. 5Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin Berlin, Berlin, Germany
  6. 6Department of Psychiatry and Molecular Medicine, Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, USA
  7. 7Center for Psychiatric Neuroscience, Feinstein Institute for Medical Research, Manhasset, New York, USA
  8. 8Centre for Innovation in Mental Health, School of Psychology, Faculty of Environmental and Life Sciences, University of Southampton, Southampton, UK
  9. 9Department of Sleep Medicine, Southampton Children’s Hospital, Southampton, UK
  10. 10Department of Psychosis Studies, King's College London Institute of Psychiatry Psychology and Neuroscience, London, UK
  11. 11H Lundbeck A/s, Iveco House, Watford, UK
  12. 12Institute of Clinical Sciences (ICS), Faculty of Medicine, Imperial College London, London, UK
  13. 13Department of Psychiatry, University of Ottawa, Ottawa, Ontario, Canada
  14. 14Ottawa Hospital Research Institute (OHRI) Clinical Epidemiology Program, University of Ottawa, Ottowa, Ontario, Canada
  15. 15School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
  16. 16Solent NHS Trust, Southampton, UK
  17. 17Division of Psychiatry and Applied Psychology, School of Medicine, University of Nottingham, Nottingham, UK
  18. 18Hassenfeld Children’s Hospital at NYU Langone, New York University Child Study Center, New York, New York, USA
  1. Correspondence to Dr Nathan TM Huneke; n.huneke{at}soton.ac.uk

Abstract

Introduction Given the high prevalence of mental health disorders and their significant socioeconomic burden, there is a need to develop improved treatments, and to evaluate them through placebo-controlled trials. However, the magnitude of the placebo response in randomised controlled trials to test medications may be substantial, affecting their interpretation. Therefore, improved understanding of the patient, trial and mental disorder factors that influence placebo responses would inform clinical trial design to better detect active treatment effects. There is a growing literature exploring the placebo response within specific mental health disorders, but no overarching synthesis of this research has been produced to date. We present a protocol for an umbrella review of systematic reviews and/or meta-analyses in which we aim to understand the effect size and potential predictors of placebo response within, and across, mental health disorders.

Methods and analysis We will systematically search databases (Medline, PsycINFO, EMBASE+EMBASE Classic, Web of Knowledge) for systematic reviews and/or meta-analyses that report placebo effect size in clinical trials in patients with mental health disorders (initial search date 23 October 2022). Screening of abstracts and full texts will be done in pairs. We will extract data to qualitatively examine how placebo effect size varies across mental health disorders. We also plan to qualitatively summarise predictors of increased placebo response identified either quantitatively (eg, through meta-regression) or qualitatively. Risk of bias will be assessed using the AMSTAR-2 tool. We aim to not only summarise the current literature but also to identify gaps in knowledge and generate further hypotheses.

Ethics and dissemination We do not believe there are any specific ethical considerations relevant to this study. We will publish the results in a peer-reviewed journal.

  • PSYCHIATRY
  • Systematic Review
  • SLEEP MEDICINE
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This umbrella review will be conducted according to current best practice.

  • We plan to prevent duplication of data by only including meta-analyses with the most studies/information.

  • As with other umbrella reviews, our findings will be limited to the variables and outcomes investigated and reported within the included systematic reviews and meta-analyses.

Introduction

A placebo is an ‘inactive’ substance or a ‘sham’ procedure that is administered as a type of psychological comfort or as a control for evaluating the efficacy of an active treatment. However, since the introduction of placebo-controlled research, it has been demonstrated that patients in a placebo control group can experience significant symptom improvements (a ‘placebo response’).1–3 The placebo response can be partially accounted for by statistical artefact or non-specific effects. For instance, many patients seek care and are recruited into clinical trials when their symptoms are at a peak. Over time, their symptoms will fall closer to their average severity (regression to the mean), which can appear to be a placebo response.4 Further, it has been suggested that baseline symptom severity might be selectively inflated if raters are aware of severity criteria for entry to a trial, exacerbating this effect.5 6 Other potential sources of apparent placebo responses include sampling biases due to withdrawal of the least improved patients in the placebo arm, non-specific beneficial effects arising from interactions with healthcare staff or other unaccounted factors such as change in diet or exercise habits during the trial.7–9 Nevertheless, there is also evidence that administration of placebo leads to ‘true’ (or non-artefactual) placebo effects, that is, detectable changes in biological systems including, but not limited to, the immune,10 11 dopaminergic12 13 and endogenous opioid14 15 systems. The placebo effect size can be increased through the use of verbal suggestions and conditioning procedures, thus demonstrating the importance of psychological mechanisms including learning and expectations.16 17

Current treatments for mental health disorders do improve symptoms across age groups, treatment modalities and different mental health disorders,18–21 but only a portion of patients reach a clinically significant response or remission. Notably, less than half of patients with anxiety or depressive disorders achieve remission following first-line pharmacotherapy, and current treatments also cause unwanted side effects.22–28Given the high prevalence of mental health disorders and their significant socioeconomic burden,29–31 there is a need to develop more effective and safer treatments. However, the magnitude of the placebo response in randomised controlled trials (RCTs) may be substantial, which can affect their interpretation.32 33 As an example, approximately 35%–40% of patients in antidepressant trials respond to placebo.34 Over the past 40 years, placebo response has increased in RCTs with antipsychotics, while medication effect sizes have remained consistent.35 36 This results in decreased sensitivity of the clinical trial to distinguish between active treatment and placebo in terms of efficacy.37 As a result, large placebo responses have been implicated in hindering psychotropic drug development.38 39

Therefore, there is a significant interest in understanding the placebo response in mental health disorders. Improved understanding of patient, trial, and mental disorder factors important in producing placebo responses might allow better clinical trial design to detect active treatment effects. There is a growing literature of individual studies and systematic reviews/meta-analyses exploring the placebo response within certain specific mental health disorders.33 However, to date, no overarching synthesis of this literature across mental health disorders has been produced. Moreover, comparisons of placebo effect sizes and predictors of placebo response across mental health disorders are lacking. To address this need, we will carry out an umbrella review of systematic reviews and/or meta-analyses with the aim of answering the following questions:

  1. What is the effect size of placebo response in clinical trials for mental health disorders?

    1. Does this differ across mental health disorders?

    2. How does the effect size of placebo compare with active treatments?

    3. Has the effect size changed over time?

  2. What are the potential predictors of placebo response rate in mental health disorders?

We aim to not only summarise current understanding of the literature but also to identify gaps in the knowledge and generate hypotheses to be tested by future research.

Methods and analysis

This protocol has been preregistered on the open science framework (https://osf.io/75ptj).

Searches

We will search the following electronic databases: PubMed (including MEDLINE), Ovid databases (PsycINFO, EMBASE+EMBASE Classic (which include grey literature), Ovid Medline) and Web of Knowledge (Web of Science Core Collection, Biological Abstracts, BIOSIS Citation Index, Current Contents Connect, Data Citation Index, Derwent, Innovations Index, FSTA—the food science resource, KCI-Korean Journal Database, MEDLINE, Russian Science Citation Index, SciELO Citation Index). All databases will be searched from inception. An initial search was performed on 23 October 2022 and will be updated at yearly intervals (see online supplemental material for full search strategy). There will be no restrictions on dates or language. All types of record (full text, conference proceedings, abstracts or other format) will be included. Where a published full text version of grey literature is not available, corresponding authors will be contacted to enquire about the publication status of their systematic review/meta-analysis and their willingness to share unpublished data. We will also manually check the references of systematic reviews or meta-analyses retained in the present umbrella review, to detect any relevant record not retrieved with the electronic search.

Condition or domain being studied

We aim to summarise the evidence regarding efficacy of placebo within a range of mental health disorders. We will focus on mental health disorders defined in the ICD-11 within the following categories: neurodevelopmental health disorders (including attention-deficit/hyperactivity disorder and autism spectrum disorder)), schizophrenia spectrum disorders, bipolar and related disorders, depressive disorders, anxiety disorders, obsessive–compulsive and related disorders, trauma-related and stressor-related disorders, substance-related and addictive disorders, sleep disorders, and neurocognitive disorders (eg, dementia).

Types of study to be included

We will search for all systematic reviews, regardless of whether they include a meta-analysis. Consistent with recent recommendations,40 we will consider a paper to be a ‘systematic review’ if all of the following are reported: (1) specific research question(s); (2) at least two sources that were searched with a reproducible search strategy (ie, databases and search engines named, search date, complete search terms); (3) inclusion and exclusion criteria; (4) study selection methods and (5) list of studies included in the review (and, optionally, a list of excluded studies, with reasons for exclusion).

We will retain systematic reviews or meta-analyses (including network meta-analyses) that include RCTs of pharmacological and non-pharmacological biological treatments (eg, medication or neuromodulation; not psychotherapy) compared with a placebo or sham treatment in patients with mental health disorders. For inclusion, the systematic review or meta-analysis must report the effect size of placebo (either within-group or compared with no treatment) and the studies retained in the individual systematic review or meta-analysis must define mental disorder according to standardised criteria (ie, Diagnostic and Statistical Manual of Mental Disorders (DSM) or International Classification of Diseases (ICD) criteria). A meta-analysis must offer a quantitative synthesis based on a systematic review, with the methods described in sufficient detail to allow for replication of the study to be included.

Many meta-analyses in the literature contain overlapping studies.41 If duplicate data are included in an umbrella review then this could result in erroneous interpretations of the data. Therefore, consistent with recommendations42 and previously published umbrella reviews,43 if there are multiple systematic reviews or meta-analyses for a given mental disorder, we will use the following algorithm to choose which record to include in our umbrella review:

  1. We will preferably include meta-analyses. If no meta-analysis is available on a specific disorder/treatment, we will include systematic reviews that report the placebo effect size (with 95% CI or other metrics of data dispersion), sample size and design for the majority of individual studies retained in the review.

  2. If there are multiple meta-analyses or systematic reviews with effect size data for the same mental disorder, then we will include the one containing the largest number of studies.

Population

Data for patients aged >18 years and children or adolescents (aged ≤18 years) will be synthesised separately. Where meta-analyses include studies in both age groups, we will use data to meta-analyse the results in adults and in children/adolescents separately.

Interventions

Placebo and biological treatments (not psychotherapy).

Comparisons

We will consider meta-analyses or systematic reviews of RCTs comparing active treatment with placebo/sham.

Primary outcome

The primary outcome will be the effect size of the placebo/sham in terms of disorder-specific primary symptom reduction for each mental disorder.

Secondary outcomes

Secondary outcomes will be any other clinical outcome reported in eligible reviews. In addition, where variance meta-analyses have been carried out and are considered eligible for the current review, we will report the variability of placebo response rate.

Study screening, selection

We will screen records for eligibility in three stages:

  1. Two authors will independently screen non-duplicate titles and abstracts. Discrepancies will be resolved by consensus. If consensus is not reached, a third, senior author will act as arbitrator. If any doubt about inclusion persists, then the record will proceed to the next stage.

  2. Full-text versions of records passing stage 1 will be obtained. Two authors will independently screen these for eligibility. Discrepancies will be resolved by consensus between the two authors and, if needed, a third senior author will act as arbitrator. We will report which articles are excluded at this stage and the reasons for exclusion.

  3. A matrix containing all eligible studies for each category of mental disorder will be created. Two authors will independently assess these for final inclusion based on the criteria detailed above (eg, where component studies overlap, the meta-analysis with more information is included). Discrepancies will be resolved by consensus between the two authors and, if needed, a third senior author will act as arbitrator.

Data extraction

Data will be extracted by two independent reviewers. Discrepancies will be resolved by consensus between the two authors or, if not possible, a third senior author will act as an arbitrator and make a judgement about the data to be entered.

For each included meta-analysis, the following variables will be extracted:

  1. First author surname.

  2. Year of publication.

  3. Patient population, including: diagnosis and diagnostic criteria, demographics, disease stage or severity indicators (ie, first episode, treatment-resistant), presence of comorbidity.

  4. Electronic databases searched by the authors.

  5. Inclusion of unpublished data.

  6. Number of studies included.

  7. Presence of sensitivity or subgroup analyses.

  8. If reported, the nature of the sham or placebo intervention in the included studies.

  9. Predictors of placebo response, either identified via meta-regression or qualitatively.

  10. Type and numerical values for available effect sizes with 95% CI or other appropriate indicator of dispersion.

  11. Measures of heterogeneity (eg, I2, Q, tau).

  12. Test for publication bias or small study effects.

  13. Quality appraisal of the included studies and, if present, which tool was used and the rating for each included study.

  14. Time point for each study outcome.

  15. Modality to assess outcome (eg, symptom rating questionnaire).

Risk of bias (quality) assessment

We will use the ‘A MeaSurement Tool to Assess systematic Reviews’ tool (AMSTAR-2)44 to assess the overall confidence in the results of each meta-analysis as: high, moderate, low or critically low.

Strategy for data synthesis

Overall effect sizes for placebo and, where reported, active treatment will be extracted, as well as the difference between placebo and active interventions where this is reported.

We plan to qualitatively examine how placebo effect size varies across mental health disorders. We also plan to qualitatively summarise predictors of increased placebo response reported in individual systematic reviews or meta-analyses, identified either quantitatively (eg, through meta-regression) or qualitatively.

Additional/sensitivity analyses

We will explore the feasibility of conducting subgroup analyses based on different active treatment modalities.

Patient and public involvement

None.

Ethics and dissemination

We do not believe there are any specific ethical considerations relevant to this study. We expect the results of this review to be of interest to a wide readership, and therefore, plan to publish the results in a high-impact journal.

Ethics statements

Patient consent for publication

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • MS and SC are joint senior authors.

  • Twitter @drjmas

  • Contributors NTMH: conceptualisation, writing–original draft, writing–review and editing. JA: conceptualisation, writing–review and editing. DSB: conceptualisation, writing–review and editing. SRC: conceptualisation, writing–review and editing. CUC: conceptualisation, writing–review and editing. MG: conceptualisation, writing–review and editing. CMH: conceptualisation, writing–review and editing. RH: conceptualisation, writing–review and editing. ODH: conceptualisation, writing–review and editing. JMAS: conceptualisation, writing–review and editing. MS: conceptualisation, writing–original draft, writing–review and editing. SCo: conceptualisation, writing–original draft, writing–review and editing.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests DSB is President of the British Association for Psychopharmacology (BAP) and Editor of Human Psychopharmacology journal (for which he receives an Editor’s honorarium). CMH has acted on an expert advisory board for Neurim pharmaceuticals. ODH is a part-time employee and stock holder of Lundbeck A/s. He has received investigator-initiated research funding from and/or participated in advisory/speaker meetings organised by Angellini, Autifony, Biogen, Boehringer-Ingelheim, Eli Lilly, Heptares, Global Medical Education, Invicro, Jansenn, Lundbeck, Neurocrine, Otsuka, Sunovion, Recordati, Roche and Viatris/Mylan. ODH has a patent for the use of dopaminergic imaging. All other authors declare no competing interests. MS has received honoraria/has been a consultant for Angelini, Lundbeck, Otsuka. SRC receives an honorarium from Elsevier for associate editor work at two academic journals. SCo has received honoraria from non-profit associations (BAP, ACAMH, CADDRA) for educational activities. CUC has been a consultant and/or advisor to or has received honoraria from: AbbVie, Acadia, Adock Ingram, Alkermes, Allergan, Angelini, Aristo, Biogen, Boehringer-Ingelheim, Bristol-Meyers Squibb, Cardio Diagnostics, Cerevel, CNX Therapeutics, Compass Pathways, Darnitsa, Denovo, Gedeon Richter, Hikma, Holmusk, IntraCellular Therapies, Jamjoom Pharma, Janssen/J&J, Karuna, LB Pharma, Lundbeck, MedAvante-ProPhase, MedInCell, Merck, Mindpax, Mitsubishi Tanabe Pharma, Mylan, Neurocrine, Neurelis, Newron, Noven, Novo Nordisk, Otsuka, Pharmabrain, PPD Biotech, Recordati, Relmada, Reviva, Rovi, Sage, Seqirus, SK Life Science, Sumitomo Pharma America, Sunovion, Sun Pharma, Supernus, Takeda, Teva, Tolmar, Vertex and Viatris. He provided expert testimony for Janssen and Otsuka. He served on a Data Safety Monitoring Board for Compass Pathways, Denovo, Lundbeck, Relmada, Reviva, Rovi, Supernus and Teva. He has received grant support from Janssen and Takeda. He received royalties from UpToDate and is also a stock option holder of Cardio Diagnostics, Kuleon Biosciences, LB Pharma, Mindpax and Quantic.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.