Introduction Smoking among persons with a mental health disorder is associated with inequitable health, social and economic burden. Randomised controlled trials (RCTs) are considered the gold standard design for the assessment of healthcare intervention efficacy/effectiveness. However, many RCTs of smoking interventions for persons with a mental health disorder lack rigour due to low participant retention. No systematic review has pooled retention rates in randomised trials of smoking interventions for persons with a mental health disorder or explored associated factors. The aims of the systematic review will therefore be to: (1) summarise overall rates of participant retention in smoking cessation and reduction trials involving persons with a mental health disorder (including for experimental and control groups separately) and (2) determine if retention rates vary according to participant, environmental, researcher and study factors.
Methods and analysis PsycINFO, EMBASE, MEDLINE, CENTRAL and The Cochrane Tobacco Addiction Review Group Specialised Register will be searched for reports of RCTs of outpatient smoking cessation or reduction interventions for adults with a mental health disorder. The search terms will include MeSH terms and free text words, and there will be no language or date restrictions. All databases will be searched from inception to present. Data will be analysed using the Mantel-Haenszel fixed-effect model, and where substantial heterogeneity (I2 >50%) is detected, DerSimonian & Laird inverse-variance random effects model. Pooled estimates and 95% CIs will be calculated for overall participant retention rates and for intervention and control trial arms separately. Associations between participant retention and participant, environmental, researcher and study factors will be assessed via subgroup analyses and, where sufficient data are obtained, meta-regression.
Ethics and dissemination This study does not require ethical approval. The findings of this review will be disseminated via publication in a peer-reviewed open access medical journal and presentations at international scientific meetings.
- mental illness
- mental disorder
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This will be the first systematic review to pool retention rates in randomised controlled trials of outpatient smoking interventions for persons with a mental health disorder and explore associated factors.
Identification of participant, environmental, researcher and study factors associated with retention will be informative when designing future smoking intervention trials for persons with a mental health disorder and may lead to increased retention and, in turn, rigour of research in the field.
It will employ Cochrane’s methods for the conduct of systematic reviews.
Trials of smoking interventions delivered exclusively in psychiatric inpatient setting will not be included.
Tobacco smoking is a leading cause of preventable morbidity and mortality worldwide.1 Smoking prevalence has steadily declined to between 13% and 20% over the past 40 years in the general population of high-income countries;2–4 however, it has remained unchanged among persons with a mental health disorder.5 6 Population surveys suggest that 40%, 44% and 36% of community residing persons with a mental health disorder in the USA,7 UK8 and Australia7 smoke, respectively, with prevalence estimates shown to increase as mental illness severity and number of life time disorders increase.9 10 These groups are also reported to smoke more heavily and be more nicotine dependent than smokers without a mental health disorder9–11 and have been estimated to consume up to 45% of all cigarettes sold in high-income countries.9 12 13 Smoking among persons with a mental health disorder is consequently associated with inequitable health, social and economic burden.14 Despite this, many high-quality randomised trials of smoking interventions exclude persons with a mental health disorder15 16 and those focused on this population group often lack rigour due to, for example, small sample size and low participant retention.17–19 More rigorous intervention research is needed to address this international public health issue.20 21
Randomised controlled trials (RCTs) are considered the gold standard design for assessment of healthcare intervention efficacy and effectiveness.22 In any trial, low rates of participant retention for follow-up data collection can compromise the internal and external validity and reduce statistical power.23–25 Threats to internal validity are particularly salient as retention decreases (<80%25) or is differential between allocation groups,26 increasing the risk of bias and confounding.26 Additionally, statistical methods (eg, multiple imputation) cannot account entirely for the impact of low/differential retention and have been noted to occasionally be used inappropriately.27 Illustrating a consensus regarding the need to consider and address the issue of low retention in clinical and health behaviour trials: the inclusion of items on participant retention in the Consolidated Standards of Reporting Trials (CONSORT) statement,28 29 and prioritisation of trial methodology research to develop novel approaches to increase retention in randomised trials.30 31 In addition, the emergence of recent systematic reviews examining retention rates in health behaviour studies32–36 and publication of retention data from high quality randomised trials37–39 indicates increased interest in the topic of retention and a recognition of its importance.
Participant retention is problematic in trials of smoking cessation interventions,40 41 including those involving persons with a mental health disorder.17–19 For example, review of risk of bias assessments in systematic reviews of smoking cessation intervention trials involving persons with depression17 and substance use disorders19 revealed retention rates as low as 27%–33%, at trial end points. In addition, 65 of the trials (34/49;17 31/3519) included in the reviews transparently reported on participant retention, of which 40 (22/34;17 18/3119) achieved retention rates less than 80%. In addition, there was evidence of differential retention (≥10% difference in retention rates between allocation groups) in 13 (5/34;17 8/3119) trials, with this figure likely an underestimate due to non-systematic reporting of retention data according to allocation group. No study has systematically and quantitatively summarised retention rates of persons with a mental health disorder in smoking trials or considered factors that may impact on such rates.
Factors likely to impact participant retention in any trial could be categorised into four types: (1) ‘participant’ (eg, demographic information) (2) ‘environmental’ (eg, recruitment method/setting) (3) ‘researcher’ (eg, staff qualifications) and (4) ‘study’ (eg, trial design).42 In terms of participant factors, evidence from single studies in the field of smoking and mental illness suggests younger age, higher income, less severe mental health symptomology and readiness to quit at baseline may be associated with higher retention in trials.37 39 43 44 While no research has reported on environmental and researcher factors associated with retention in smoking trials involving persons with a mental health disorder, findings from the broader field of smoking research suggest that higher retention rates are observed when proactive (compared with reactive) recruitment methods are adopted,40 and staff delivering the intervention receive more comprehensive training.29 With regard to study factors, Cochrane systematic review evidence suggests open-label designs increase the relative risk of higher retention rates (risk ratio (RR) 1.37; 95% CI 1.16 to 1.63) in randomised trials of health interventions outside the field of smoking.33 Further, adoption of strategies with demonstrated effectiveness in improving retention in health research could also be considered a ‘study’ factor. Such strategies were developed based on identified barriers to participant retention32 and include monetary reimbursement for time,45 provision of reminders for follow-up assessments46 and participant tracking systems.46 Identification of participant, environmental, researcher and study factors associated with higher participant retention in smoking trials involving persons with a mental health disorder will be informative for researchers when designing definitive trials and in turn potentially improve the rigour of available intervention research in the field.
No review has systematically examined rates of participant retention or explored associated factors in smoking cessation and/or reduction trials involving persons with a mental health disorder. Therefore, the aims of the proposed systematic review will be to: (1) summarise overall rates of participant retention in such trials (including for experimental and control groups separately) and (2) determine if retention rates vary according to participant, environmental, researcher and study factors.
This protocol adheres to the preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) statement.47
Studies identified by a Cochrane review of outpatient smoking cessation and reduction interventions for persons with a mental health disorder48 will be included. Criteria for inclusion:
Study design: RCTs and cluster RCTs.
Participants: adult (aged 18 years or above) smokers, who have either been diagnosed with a mental health disorder (determined via medical record or self-report) in the past 12 months or are currently receiving treatment for the same. Participants will not be required to express an intention to quit smoking to be eligible. No limits will be placed on recruitment setting; however, the majority of the intervention component must be delivered in a community setting. Interventions delivered exclusively during an inpatient psychiatric admission will be excluded as they are the focus of another Cochrane review.49
Intervention type: any intervention to aid smoking cessation or reduction, for example, pharmacotherapy, psychoeducation, cognitive and behavioural therapies. Interventions aiming to increase the uptake/utilisation of smoking cessation supports will be considered. However, whole-of-setting interventions (eg, smoke-free policy) will be excluded.
Intervention delivery: face-to-face, telephone, online, mail, individual and/or group. Interventions can be facilitated by any person (eg, research officer or healthcare clinician) or via unmoderated online/phone-based methods.
Control: all control conditions will be considered, for example, placebo, no treatment, usual care, other smoking cessation/reduction interventions.
Follow-up: undertaken at least 6 months postbaseline assessment.50
The following databases will be searched for reports of trials of smoking cessation and reduction interventions among smokers with a mental health disorder:
The Cochrane Tobacco Addiction Review Group Specialised Register.
Cochrane Central Register of Controlled Trials (CENTRAL).
MEDLINE (OVID SP 1946 to present) & MEDLINE in-process & other non-indexed citations (OVID SP).
EMBASE (OVID SP, 1947 to present).
PsycINFO (OVID SP,1806 to present).
Search terms will include MeSH terms and free text words, and there will be no language or date restrictions. All databases will be searched from inception to present. Reference lists of included studies will be checked for other relevant research. Online supplementary appendix 1 comprises the proposed Medline search strategy. Searches will likely be undertaken in December 2019.
In addition to the above, we will search international clinical trials registers for recently completed trials, including: the Australian and New Zealand Clinical Trials Registry (ANZCTR), UK Clinical Trials Gateway; US Clinical Trials Register and the WHO Portal.
Study screening and data extraction
Two review authors will independently screen the titles and abstracts of search results for relevance, acquire and screen the full texts of potentially eligible articles and extract data from included studies. Disagreements during the screening and data extraction processes will be resolved by referral to a third review author. Covidence (https://www.covidence.org/home) and Microsoft Excel will be used to manage data during screening and extraction phases.
Data to be extracted from included trials are outlined in table 1, with independent variables for consideration identified in the respective column and summarised under relevant headings below. A large number of potential independent variables could be considered, selection will be guided by extant literature, experts in the field and, with exception of ‘participant factors’, focus on factors modifiable by researchers.
Participant characteristics for consideration, collected at the baseline assessment of individual studies, may include: age,39 40 gender,40 43 race/ethnicity,40 socioeconomic status,40 mental health diagnosis,37 severity of distress,13 motivation/readiness to quit smoking,40 44 nicotine dependence,30 40 daily cigarette consumption40 49 and requirement to set a quit date on recruitment.40
Researcher factors include characteristics, skills and roles of the staff that design and implement the study and may include: recruiter qualifications, training and professional role51 52 and relationship of recruiter to participants.51
Study factors comprise features of the study design and implications of such for participants. A number of study factors may be considered: blinding/masking of research personnel,33 adoption of retention strategies,32 37 timing of follow-up assessments,39 40 biochemical validation of self-reported abstinence and potential cost/s to participants of follow-up assessment completion.32 51
The primary outcome of interest for the review is participant retention rate at the longest follow-up. Within each study, retention will be defined as participation in the final follow-up assessment of a trial (eg, completion of telephone or face-to-face interview or response to postal and electronic questionnaires). Retention rate will be calculated using the formula: number retained/number recruited. Study authors will be approached to obtain data required to calculate outcome measures, if not reported in the published manuscript. Secondary analyses will explore follow-up rates separately at other time points, for example, at 6 months, 12 months and 18 months.
Data will be analysed using Review Manager 5.3 and Stata 15.0. Outcome data will be collected in accordance with intention-to-treat principles, where all recruited participants will be included in denominator figure of the retention algorithm. Data that are missing due to participant death will be deducted from the total sample size for all analyses.
According to study aim
Aim 1: the proportion of participants retained will be pooled using the Stata 15.0 prevalence command, ‘metaprop’ using the Freeman-Tukey double arcsine transformation to stabilise variances and prevent exclusion of studies where proportions approached 0 or 1.53 54 We will calculate pooled estimates and 95% CIs for overall participant retention rates as well as for intervention and control trial arms separately. Differences between intervention and control arms will be assessed via risk ratios. Data will be analysed using the Mantel-Haenszel fixed-effect model. If substantial heterogeneity (I2>50%)50 is detected in the pooled estimates, we will run sensitivity analyses using DerSimonian & Laird inverse-variance random effects meta-analysis.55
Aim 2: the association between participant, environmental, researcher and study factors and overall participant retention will be assessed via subgroup analyses and, where sufficient data are obtained (>10 observations),50 meta-regression. Subgroup analyses will be undertaken in line with the meta-analysis approach described for Aim 1. Independent variables (table 1) will be categorised and reduced to two or three levels for the purpose of this analysis. In terms of the meta-regression, we will build models using the ‘regress’ and, where relevant, ‘metareg’ commands. Study level weights will be obtained from the fixed effects meta-analysis and will be included in the regression models using the analytical weights option. Categorical and numerical independent variables to be considered in these models. Ratio data will be natural log transformed prior to analyses. Where data are not amenable to meta-analysis, factors reported in individual studies to be associated with retention will be summarised via narrative synthesis.
Statistical heterogeneity and sensitivity analyses
The I2 statistic, which describes the percentage of total variation across studies that is due to heterogeneity rather than chance, will be used to assess statistical heterogeneity of pooled data. Heterogeneity in pooled estimates will be described as low (I 2~25%), moderate (I 2 ~50%) or high (I 2 ~75%).56 Sensitivity analyses will be conducted to explore the impact of including studies with an overall high risk of bias.
Patient and public involvement
Patients/public were not involved at any stage in the development of this protocol.
This rigorous systematic review will be the first to pool retention rates in RCTs of outpatient smoking interventions for persons with a mental health disorder. Identification of participant, environmental, researcher and study factors associated with retention will be informative when designing future smoking intervention trials for persons with a mental health disorder, and in turn, may lead to increased rigour of research in the field due to higher participant retention. Findings may not generalise to trials of smoking interventions delivered exclusively in psychiatric inpatient settings.
The authors would like to acknowledge all contributors to the overarching Cochrane systematic review.
Contributors APM, ES, JB and JW conceived the review. ES, JB and JW designed the review, and AM is the guarantor and led the writing of the protocol. TR, JB, KB, GT and LW provided critical input across all stages. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding ES and KB are supported by Australian National Health and Medical Research Council Early Career Fellowships (ES #APP1104600; KB #APP1142272). GT is funded by Cancer Research UK Population Researcher Postdoctoral Fellowship award (reference: C56067/A21330).
Disclaimer Funders did not contribute, at any stage, to the development of this protocol or decision to publish.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.