Article Text
PDF
Abstract
Objective A systematic review and meta-analysis to investigate the impact of electronic nicotine delivery systems (ENDS) and/or electronic non-nicotine delivery systems (ENNDS) versus no smoking cessation aid, or alternative smoking cessation aids, in cigarette smokers on long-term tobacco use.
Data sources Searches of MEDLINE, EMBASE, PsycInfo, CINAHL, CENTRAL and Web of Science up to December 2015.
Study selection Randomised controlled trials (RCTs) and prospective cohort studies.
Data extraction Three pairs of reviewers independently screened potentially eligible articles, extracted data from included studies on populations, interventions and outcomes and assessed their risk of bias. We used the Grading of Recommendations Assessment, Development and Evaluation approach to rate overall certainty of the evidence by outcome.
Data synthesis Three randomised trials including 1007 participants and nine cohorts including 13 115 participants proved eligible. Results provided by only two RCTs suggest a possible increase in tobacco smoking cessation with ENDS in comparison with ENNDS (RR 2.03, 95% CI 0.94 to 4.38; p=0.07; I2=0%, risk difference (RD) 64/1000 over 6 to 12 months, low-certainty evidence). Results from cohort studies suggested a possible reduction in quit rates with use of ENDS compared with no use of ENDS (OR 0.74, 95% CI 0.55 to 1.00; p=0.051; I2=56%, very low certainty).
Conclusions There is very limited evidence regarding the impact of ENDS or ENNDS on tobacco smoking cessation, reduction or adverse effects: data from RCTs are of low certainty and observational studies of very low certainty. The limitations of the cohort studies led us to a rating of very low-certainty evidence from which no credible inferences can be drawn. Lack of usefulness with regard to address the question of e-cigarettes' efficacy on smoking reduction and cessation was largely due to poor reporting. This review underlines the need to conduct well-designed trials measuring biochemically validated outcomes and adverse effects.
- electronic cigarettes
- ENDS
- smoking cessation
- GRADE
- systematic review
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
Strengths of our review include a comprehensive search; assessment of eligibility, risk of bias and data abstraction independently and in duplicate; assessment of risk of bias that included a sensitivity analysis addressing loss to follow-up; and use of the Grading of Recommendations Assessment, Development and Evaluation approach in rating the certainty of evidence for each outcome.
The primary limitation of our review is the low certainty consequent on study limitations. Moreover, loss to follow-up was substantial, and our sensitivity analysis demonstrated the vulnerability of borderline effects to missing data. The limitations of the cohort studies led us to a rating of very low-certainty evidence from which no credible inferences can be drawn.
The small number of studies made it impossible to address our subgroup hypotheses related dose–response of nicotine, more versus less frequent use of e-cigarettes or the relative impact of newer versus older e-cigarette models.
Introduction
Tobacco smokers who quit their habit reduce their risk of developing and dying from tobacco-related diseases.1–4 Psychosocial5–7 and pharmacological interventions (eg, nicotine replacement therapy (NRT))5–7 increase the likelihood of quitting cigarettes. Even with these aids, however, most smokers fail to quit.
Electronic nicotine delivery systems (ENDS) and electronic non-nicotine delivery systems (ENNDS) represent a potential third option for those seeking to stop smoking. ENDS are devices that deliver nicotine in an aerosolised form, while ENNDS devices are labelled as not containing nicotine (though labelling may not always be accurate). In theory, these devices as well as the nicotine inhalers may facilitate quitting smoking to a greater degree than other nicotine-based products or no intervention because they deal, at least partly, with the behavioural and sensory aspects of smoking addiction (eg, hand mouth movement).8 The debate about the role of ENDS in smoking cessation, however, is compounded by the lack of clear evidence about their value as a smoking cessation tool, their potential to hook tobacco-naïve youth on nicotine as well as act as a bridge to combustible tobacco use.9 While evidence about all these aspects of ENDS is accumulating, establishing their real place in smoking cessation is essential to outline the public health context of considering them as potential harm-reduced products. There are, however, other reasons for ENDS use such as for relaxation or recreation (ie, the same reason people smoke), with the possibility that adverse health effects may be less than conventional smoking.
There are many types of ENDS. The cigalikes are the first generation of ENDS that provide an appearance of tobacco cigarettes; they are not rechargeable. The second generation of ENDS looks like a pen, allows the user to mix flavours and may contain a prefilled or a refillable cartridge. The third generation of ENDS includes variable wattage devices and are used only with refillable tank systems. The fourth generation contains a large, refillable cartridge and has a tank-style design.
A previous Cochrane systematic review8 summarised results from randomised controlled trials (RCTs) and cohort studies. The authors included two RCTs and 11 cohort studies, and concluded that there was evidence to support the potential benefit of ENDS in increasing tobacco smoking cessation.8 The certainty of evidence supporting this conclusion was, however, deemed low, primarily due to the small number of trials resulting in wide CIs around effect estimates.8 Another systematic review10 including a total of six studies (RCTs, cohort and cross-sectional studies) involving 7551 participants concluded that ENDS is associated with smoking cessation and reduction; however, the included studies were heterogeneous, due to different study designs and gender variation. One other review11 comparing e-cigarettes with other nicotine replacement therapies or placebo included five studies (RCTs and controlled before–after studies) and concluded that participants using nicotine e-cigarettes were more likely to stop smoking, but noted no statistically significant differences.11 In a more recent systematic review, Kalkhoran and Glantz9 included 20 studies (15 cohort studies, 3 cross-sectional studies and 2 clinical trials), and found 28% lower odds rates of quitting cigarettes in those who used e-cigarettes compared with those who did not use e-cigarettes; however, the methodological aspects of the observational studies were rated as unclear or high on outcome assessors, and a RCT was rated as high risk of performance and attrition bias.
Previous reviews were, however, limited in that they did not include all studies in this rapidly evolving field, and all but one did not use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to rating quality of evidence. We therefore conducted an updated systematic review of RCTs and cohort studies that assessed the impact of ENDS and/or ENNDS versus no smoking cessation aid or alternative smoking cessation aids on long-term tobacco use, among cigarette smokers, regardless of whether the users were using them as part of a quit attempt.
Methods
We adhered to methods described in the Cochrane Handbook for Intervention Reviews.12 Our reporting adheres to the preferred reporting items for systematic reviews and meta-analyses (PRISMA)13 and meta-analysis of observational studies in epidemiology (MOOSE) statements.14 This work was commissioned by the WHO.
Eligibility criteria
Study designs: RCTs and prospective cohort studies.
Participants: cigarette smokers, regardless of whether the users were using them as part of a quit attempt.
Interventions: ENDS or ENNDS.
Comparators:
no smoking cessation aid;
alternative non-electronic smoking cessation aid, including NRT, behavioural and/or pharmacological cessation aids (eg, bupropion and varenicline) and
alternative electronic smoking cessation aid (ENDS or ENNDS).
Outcomes:
tobacco smoking cessation, with preference to biochemically validated outcomes (eg, carbon monoxide (CO)) measured at 6 months or longer follow-up;
reduction in cigarette use of at least 50% and
serious (eg, pneumonia, myocardial infarction) and non-serious (eg, nausea, vomiting) adverse events measured at 1 week or longer follow-up.
Data source and searches
A previous Cochrane review with similar eligibility criteria ran a comprehensive search strategy up to July 2014.8 Using medical subject headings (MeSH) based on the terms ‘electronic nicotine,’ ‘smoking-cessation,’ ‘tobacco-use-disorder,’ ‘tobacco-smoking’ and ‘quit’, we replicated the search strategy of that review8 in Medline, EMBASE, PsycInfo, CINAHL, Cochrane Central Register of Controlled Trials (CENTRAL), ISI Web of Science and the trial registry (clinicaltrials.gov). The appendix table 1 shows the search strategy for Ovid MEDLINE. This strategy was adapted for the other databases and run from 1 April 2014 to 29 December 2015. We did not impose any language restrictions.
Supplementary table 1
Study characteristics related to design of study, setting, number of participants, mean age, gender, inclusion and exclusion criteria, and follow-up.
In addition, we established a literature surveillance strategy based on the weekly search alerts by Centers for Disease Control and prevention (CDC)'s Smoking and Health Resource Library of published articles (http://nccd.cdc.gov/shrl/NewCitationsSearch.aspx) as well as the Gene Borio's daily news items (http://www.tobacco.org). The surveillance strategy started from the time of running the comprehensive literature search up to the time of the submission of this manuscript.
Selection of studies
Three pairs of reviewers underwent calibration exercises and used standardised pilot tested screening forms. They worked in teams of two and independently screened all titles and abstracts identified by the literature search, obtained full-text articles of all potentially eligible studies and evaluated them for eligibility. Reviewers resolved disagreement by discussion or, if necessary, with third party adjudication. We also considered studies reported only as conference abstracts. For each study, we cite all articles that used data from that study.
Data extraction
Reviewers underwent calibration exercises, and worked in pairs to independently extract data from included studies. They resolved disagreement by discussion or, if necessary, with third party adjudication. They abstracted the following data using a pretested data extraction form: study design; participants; interventions; comparators; outcome assessed and relevant statistical data. When available, we prioritised CO measurements as evidence of quitting. When CO measurement was unavailable, we used self-report measures of quitting.
Risk of bias assessment
Reviewers, working in pairs, independently assessed the risk of bias of included RCTs using a modified version of the Cochrane Collaboration's instrument15 (http:/distillercer.com/resources/).16 That version includes nine domains: adequacy of sequence generation, allocation sequence concealment, blinding of participants and caregivers, blinding of data collectors, blinding for outcome assessment, blinding of data analysts, incomplete outcome data, selective outcome reporting and the presence of other potential sources of bias not accounted for in the previously cited domains.16
For cohort studies, reviewers independently assessed risk of bias with a modified version of the Ottawa–Newcastle instrument17 that includes confidence in assessment of exposure and outcome, adjusted analysis for differences between groups in prognostic characteristics and missing data.17 For incomplete outcome data in individual studies (RCTs and prospective cohort studies), we stipulated as low risk of bias for loss to follow-up of <10% and a difference of <5% in missing data between intervention/exposure and control groups.
When information regarding risk of bias or other aspects of methods or results was unavailable, we attempted to contact study authors for additional information.
Certainty of evidence
We summarised the evidence and assessed its certainty separately for bodies of evidence from RCTs and cohort studies. We used the GRADE methodology to rate certainty of the evidence for each outcome as high, moderate, low or very low.18 In the GRADE approach, RCTs begin as high certainty and cohort studies as low certainty. Detailed GRADE guidance was used to assess overall risk of bias,19 imprecision,20 inconsistency,21 indirectness22 and publication bias23 and to summarise results in an evidence profile. We planned to assess publication bias through visual inspection of funnel plots for each outcome in which we identified 10 or more eligible studies; however, we were not able to because there were an insufficient number of studies to allow for this assessment. Cohort studies can be rated up for a large effect size, evidence of dose–response gradient or if all plausible confounding would reduce an apparent effect.24
Data synthesis and statistical analysis
We analysed all outcomes as dichotomous variables. In three-arm studies, we combined results from arms judged to be sufficiently similar (eg, Caponnetto 2013,25 two arms with similar ENDS regimens: 7.2 mg ENDS and 7.2 mg ENDS plus 5.4 mg ENDS). When studies reported results for daily or intensive use of ENDS separately from non-daily or less intensive use, we included only the daily/intensive use in the primary pooled analysis (eg, Brose 2015,26–28 we excluded patients with non-daily users; and Biener and Hargraves,29 we excluded patients with intermittent defined use). We conducted a sensitivity analysis in which we included all ENDS users, daily/intensive and intermittent/less intensive use. For this analysis, when necessary, we assumed a correlation of 0.5 between the effects in the daily/intensive and intermittent/less intensive groups.
We synthesised the evidence separately for bodies of evidence from RCTs and cohort studies. For RCTs, we calculated pooled Mantel-Haenszel risk ratios (RRs) and associated 95% CIs using random effects models. For observational studies, we pooled adjusted ORs using random effects models.
After calculating pooled relative effects, we also calculated absolute effects and 95% CI. For each outcome, we multiplied the pooled RR and its 95% CI by the median probability of that outcome. We obtained the median probability from the control groups of the available randomised trials. We planned to perform separate analyses for comparisons with interventions consisting of ENDS and/or ENNDS and each type of control interventions with known different effects (no smoking cessation aid; alternative non-electronic smoking cessation aid including NRT and alternative electronic smoking cessation aid (ENDS or ENNDS)). For meta-analyses, we used 6 months data or the nearest follow-up to 6 months available.
For dealing with missing data, we used complete case as our primary analysis; that is, we excluded participants with missing data. If results of the primary analysis achieved or approached statistical significance, we conducted sensitivity analyses to test the robustness of those results. Specifically, we conducted a plausible worst-case sensitivity analysis in which all participants with missing data from the arm of the study with the lower quit rates were assumed to have three times the quit rate as those with complete data, and those with missing data from the other arm were assumed to have the same quit rate as participants with complete data.30 ,31
We assessed variability in results across studies by using the I2 statistic and the p-value for the χ2 test of heterogeneity provided by Review Manager. We used Review Manager (RevMan) (V.5.3; Nordic Cochrane Centre, Cochrane) for all analyses.32
Results
Study selection
Figure 1 presents the process of identifying eligible studies, including publications in the last systematic review,8 citations identified through search in electronic databases and studies identified through contact with experts in the field. Based on title and abstract screening, we assessed 69 full texts of which we included 19 publications describing three RCTs involving 1007 participants25 ,33–39 and nine cohort studies with a total of 13 115 participants.26–29 ,40–46 The interobserver agreement for the full-text screening was substantial (kappa 0.73).
PRISMA diagram of included studies. *McRobbie, 2014.8 **Further two publications from one RCT included by the Cochrane review were identified only in our search strategy. ***Further one publication from one cohort study identified by our search strategy was identified throughout the expert search.
We contacted the authors of the 12 included studies, 9 of whom26–29 ,33 ,41 ,43 ,44 ,46 supplied us with all requested data; authors of further 3 studies25 ,42 ,46 did not supply the requested information (see online supplementary appendix table S2).
Supplementary table 2
Study characteristics
Table 1 describes study characteristics related to design of study, setting, number of participants, mean age, gender, inclusion and exclusion criteria and follow-up.
Five studies25–28 ,33 ,42 ,46 were conducted largely in Europe, six in the USA,29 ,40 ,41 ,43–45 and one in New Zealand.34–39 Randomised trials sample size ranged from 5033 to 657,34–39 and observational studies from 10046 to 3891.26–28 Typical participants were women in their 40 s and 50 s. Studies followed participants from 4 weeks46 to 36 months.29
Table 2 describes study characteristics related to population, intervention or exposure groups, comparator and assessed outcomes.
Study characteristics related to population, intervention or exposure groups, comparator, and assessed outcomes.
Of the three RCTs, one compared ENDS with NRT and ENNDS,34–39 another compared different concentrations of ENDS with ENNDS25 and the third compared different types of ENDS.33 Only the Borderud study41 included participants who were also currently receiving other behavioural and pharmacologic treatment. The participants from Vickerman 201344 study were all enrolled in a state quitline programme that provided behavioural treatment and in some cases NRT. All nine cohort studies26–29 ,34–46 compared ENDS with no use of ENDS26–29 ,40 ,41 or tobacco cigarettes only;42 in one,41 exposure and non-exposure groups received behavioural and other pharmacologic treatment.
Table 3 describes the mean number of conventional cigarettes used per day at baseline and the end of study.
Mean number of conventional cigarettes used per day at baseline and the end of study*
The mean number at baseline ranged from 11.9 in the no ENDS group45 to 20.6 in the ENDS group.33 In only two studies26–28 ,45 the mean number of conventional cigarettes used per day presented a reduction from the baseline to the end of study in the ENDS group compared with the no ENDS groups, mainly in the daily users.26–28 No included study addressed users of combustible tobacco products other than cigarettes.
Online supplementary appendix table S3 presents the types of e-cigarettes used in the included studies. The three RCTs25 ,33 ,36–39 evaluated only ENDS-type cigalikes. A total of 23.7% of the participants from Brose 201526–28 study used tank and in the Hajek 201546 study participants used either cigalike or tank. The remaining studies did not report the type of ENDS used.
Supplementary table 3
Risk of bias
Figures 2 and 3, and table 4, describe the risk of bias assessment for the RCTs.
Risk of bias assessment for the randomised controlled trials
Risk of bias for RCTs comparing ENDS versus ENNDS.
Risk of bias for RCTs comparing ENDS versus other strategies.
The major issue regarding risk of bias in the RCTs of ENDS versus ENNDS was the extent of missing outcome data.25 ,34–39 RCTs comparing ENDS with other nicotine replacement therapies had additional problems of concealment of randomisation33 and blinding.33–39
Figure 4 and table 5 describe the risk of bias assessment of the cohort studies.
Risk of bias assessment of the cohort studies
Risk of bias for cohort studies.
Seven26–29 ,40–42 ,44 ,45 of nine cohort studies were rated as high risk of bias for limitations in matching exposed and unexposed groups or adjusting analysis for prognosis variables; confidence in the assessment of the presence or absence of prognostic factors; confidence in the assessment of outcome and similarity of cointerventions between groups; all studies suffered from high risk of bias for missing outcome data.
Outcomes
The mean number of conventional cigarettes/tobacco products used per day at the end of the studies ranged from 0.734–39 in ENDS and ENNDS groups to 13.926–28 among non-daily users of ENDS (table 3). The three RCTs25 ,33–39 and one cohort study42 biochemically confirmed nicotine abstinence while the others presented only self-reported data26–29 ,40 ,41 ,43–45 (table 3).
Tobacco cessation smoking
Synthesised results from RCTs
Results from two RCTs25 ,34–39 suggest a possible increase in smoking cessation with ENDS in comparison with ENNDS (RR 2.03, 95% CI 0.94 to 4.38; p=0.07; I2=0%, risk difference (RD) 64/1000 over 6 to 12 months, low-quality evidence) (figure 5, table 6).
GRADE evidence profile for RCTs: Electronic nicotine delivery systems (ENDS) and electronic non-nicotine delivery systems (ENNDS) for reducing cigarette smoking.
Meta-analysis of RCTs on cessation smoking comparing ENDS versus ENND.
A plausible worse case sensitivity analysis yielded results that were inconsistent with the primary complete case analysis and fail to show a difference in the effects of ENDS in comparison with ENNDS (RR 1.16, 95% CI 0.72 to 1.87; p=0.54; I2=0%) (see online supplementary appendix figure 1). Certainty in evidence was rated down to low because of imprecision and risk of bias, due to missing outcome data in all studies and lack of blinding of participants,34–39 caregivers, data collectors, statistician and outcome assessors in the ENDS versus other NRT studies (figure 2, tables 4 and 6).
Supplementary figures
Adriaens 201433 also compared two types of ENDS and ENDS and e-liquid; results failed to show a difference between the ENDS groups with a very wide CI (RR 1.15, 95% CI 0.28 to 4.76, p=0.84).
Bullen 201334–39 also compared ENDS and ENNDS with NRT; results failed to show a difference between these groups with a very wide CI (RR 1.10, 95% CI 0.60 to 2.03, p=0.76) and (RR 0.67, 95% CI 0.20 to 2.19, p=0.50), respectively.
Synthesised results from cohort studies
The adjusted OR from primary meta-analysis of eight cohort studies26–29 ,40–45 comparing ENDS with no ENDS without reported concomitant interventions failed to show a benefit in cessation smoking (OR 0.74, 95% CI 0.55 to 1.00; p=0.051; I2=56%) (figure 6). A sensitivity analysis from the eight cohort studies26–29 ,40–45 using any rather than daily use of ENDS for Brose study,26–28 intensive (used e-cigarettes daily for at least 1 month) and intermittent use (used regularly, but not daily for more than 1 month) of ENDS for Biener study29 and any use versus never used for Vickerman study44 suggested a reduction in cessation smoking rates with ENDS (adjusted OR 0.69, 95% CI 0.53 to 0.91; p=0.01; I2=59%) (see online supplementary appendix figure 2).
Meta-analysis of cohort studies on cessation smoking with adjusted ORs.
Another sensitivity analysis from the same eight cohort studies26–29 ,40–45 examined whether low and high risk of bias limited to the one characteristic in which the studies differed substantially: confidence in whether the outcome was present at the beginning of the study. Although there were substantial differences in the point estimates in the low risk of bias group (adjusted OR 1.00, 95% CI 0.51 to 1.94; p=1.00; I2=67%) and the high risk of bias group (adjusted OR 0.62, 95% CI 0.50 to 0.77; p<0.001; I2=0%), the difference is easily explained by chance (interaction p-value was 0.19) (see online supplementary appendix figure 3).
A second sensitivity analysis from the same eight cohort studies26–29 ,40–45 examined whether low and high risk of bias limited to ‘two or fewer domains rated as low risk of bias’ versus ‘three or more domains rated as low risk of bias’ differed substantially. There were substantial differences in the point estimates between the ‘two or fewer domains rated as low risk of bias’ group (adjusted OR 0.61, 95% CI 0.49 to 0.75; p<0.001; I2=0%) and the ‘three or more domains rated as low risk of bias’ (adjusted OR 1.26, 95% CI 0.68 to 2.33; p=0.46; I2=51%), with an interaction p-value of 0.03 (see online supplementary appendix figure 4).
Certainty in evidence from the observational studies was rated down from low to very low because of risk of bias due to missing outcome data, imprecision in the assessment of prognostic factors and outcomes (figure 4, tables 5 and 7), as well as inconsistency in the results.
GRADE evidence profile for cohort studies: Electronic nicotine delivery systems (ENDS) and no ENDS for reducing cigarette smoking
Borderud 201441 reported cessation smoking in 25 out of 58 patients with cancer using ENDS plus behavioural and pharmacologic treatment versus in 158 out of 356 patients with cancer who received only behavioural and pharmacologic treatment (adjusted OR 0.97, 95% CI 0.71 to 1.33).
Reduction in cigarette use of at least 50%
Synthesised results from RCTs
Two RCTs25 ,34–39 results failed to show a difference between ENDS-type cigalikes versus ENNDS group with regards to reduction in cigarettes but with a very wide CI (RR 0.97, 95% CI 0.57 to 1.66; p=0.92; I2=61%) (see online supplementary appendix figure 5). Certainties in evidence were rated low because of imprecision and risk of bias25 ,34–36 ,38 ,39 (figure 2, tables 4 and 6).
Synthesised results from cohort studies
Two studies26–29 suggested increased reduction rates in those with greater versus lesser use of ENDS. Biener29 reported an adjusted OR for quitting of 6.07 (95% CI 1.11 to 33.2) in those with intensive use versus an OR of 0.31 (0.04, 2.80) in those with intermittent use. Brose26–28 reported a greater likelihood of substantial reduction (but not quitting) in those with daily use of ENDS (OR 2.49, 95% CI 1.14 to 5.45) but not those with intermittent use (OR 0.85 0.43 to 1.71).
Adverse effects
Synthesised results from RCTs
The Bullen 201334–37 ,39 study reported serious side effects in 27 out of 241 participants in the 16 mg ENDS group and 5 out of 57 for the ENNDS group followed at 6 months; results failed to show a difference between these groups with a very wide CI (OR 1.31, 95% CI 0.48 to 3.57; p=0.59). Results suggested possible increase in side effects in the 21 mg nicotine patches group (14 of 215) in comparison with ENDS (OR 1.81, 95% CI 0.92 to 3.55; p=0.08). Serious side effects include death (n=1, in nicotine e-cigarettes group), life threatening illness (n=1, in nicotine e-cigarettes group), admission to hospital or prolongation of hospital stay (12% of all events in nicotine e-cigarettes group, 8% in patches group and 11% in placebo e-cigarettes group), persistent or significant disability or incapacity and other medically important events (6% of all events in nicotine e-cigarettes group, 4% in patches group and 3% placebo e-cigarettes group).
Adriaens 201433 study reported no serious adverse events in ENDS groups as well as in the e-liquid group at 8 months of follow-up; however at 1 week from start of intervention there were three cases of non-serious adverse events in the ENDS groups.
Caponnetto 201325 mentioned that no serious adverse events occurred during the study and authors found a significant reduction in frequency of reported symptoms compared with baseline.
Synthesised results from cohort studies
Manzoli42 reported no significant differences in self-reported serious side effects, but observed four cases of pneumonia, four COPD exacerbations, three myocardial infarctions and one angina as possibly related serious side effects: two among the ENDS users (both switched to tobacco smoking during follow-up); six among tobacco smokers (three quit all smoking) and four among tobacco and ENDS smokers.
Hajek 201546 reported one leak irritating a participant's mouth and some reports of irritation at the back of the throat and minor coughing. The remaining studies did not report adverse effects.
Discussion
Main findings
Based on pooled data from two randomised trials with 481 participants, we found evidence for a possible increase in tobacco smoking cessation with ENDS in comparison with ENNDS (figure 5). The evidence is, however, of low certainty: the 95% CI of the relative risk crossed 1.0 and a plausible worse case sensitivity analysis to assess the risks of bias associated with missing participant data yielded results that were inconsistent with the primary complete case analysis (see online supplementary appendix figure 1). Furthermore, in all these RCTs, the ENDS tested were of earlier generation; it is unknown whether providing later generation of e-cigarettes or a realistic scenario of allowing users to choose e-cigarettes based on self-preference would have greater benefit. There was no robust evidence of side effects associated with ENDS in the RCTs.
Cohort studies provide very low-certainty evidence suggesting a possible reduction in quit rates with use of ENDS compared with no use of ENDS (figure 6). These studies had a number of limitations: an unknown number of these participants were not using ENDS as a cessation device; some were not using ENDS during a quit attempt; many did not have immediate plans to quit smoking and some may have already failed attempts to stop smoking. In our risk of bias assessment, we judged that seven of nine studies did not have optimal adjustment for prognostic variables. Further, as any cohort study, the results are vulnerable to residual confounding. In particular, use of ENDS may reflect the degree of commitment to smoking cessation, and it may be the degree of commitment, rather than use of ENDS, that is responsible for the change in quit rates. For instance, the finding in two studies that daily use of ENDS, but not intermittent use, increased quit/reduction rates could be interpreted as evidence of the effectiveness of daily use. An alternative interpretation, however, is that those that used ENDS daily were more motivated to stop smoking, and the increased motivation, rather than daily use of ENDS, was responsible for their degree of success. It is worth to mention that motivation to quit smoking is a major determinant of success regardless of the aid used.
In terms of bias against ENDS, cohort studies sometimes enrol smokers already using ENDS and still smoking. Such individuals may already be failing in their attempts to stop smoking. If so, enrolling these participants will underestimate ENDS beneficial effects. Additional concerns with cohort studies include their failure to provide optimal adjustment for prognostic variables or provide data regarding use of alternative smoking reduction aids.
Strengths and limitations
Strengths of our review include a comprehensive search; assessment of eligibility, risk of bias and data abstraction independently and in duplicate; assessment of risk of bias that included a sensitivity analysis addressing loss to follow-up and use of the GRADE approach in rating the certainty of evidence for each outcome.
The primary limitation of our review is the low certainty consequent on study limitations. We identified only a small number of RCTs with a modest number of participants resulting wide CIs. Moreover, loss to follow-up was substantial, and our sensitivity analysis demonstrated the vulnerability of borderline effects to missing data. The limitations of the cohort studies led us to a rating of very low-certainty evidence from which no credible inferences can be drawn.
Another limitation of this review is the fact that we could not address our hypothesis about increased rates in smoking cessation in those who used e-cigarettes with higher concentrations of nicotine compared with those using less nicotine, or daily e-cigarette users compared with non-daily e-cigarette users, or those who use newer forms of ENDS compared with users of first generation devices due to lack of evidence. However, although these assumptions seems logical, nicotine delivery from ENDS depends on other factors such as the efficiency of the device in aerosolising the liquid and user experience, apart from the concentration of nicotine in the ENDS liquid.
Furthermore, whether or not ENDS are an effective aid in the cessation of smoking may depend on whether the users were using ENDS as part of a quit attempt or not, and this may play an important role also as a possible confounder. Data is not yet available to conduct a subgroup analysis addressing this hypothesis. Subsequent trials should help provide information regarding whether their impact on cessation of smoking depends on whether users were intended to quit smoking, as well as the other unresolved issues.
Other limitations of this review were the fact of having insufficient number of included studies to allow the complete statistical analysis that we had planned. We were not able to assess publication bias because there were <10 eligible studies addressing the same outcome in a meta-analysis. We also planned to perform subgroup analyses according to the characteristics of:
participants (commitment to stopping smoking, use of e-cigarettes at baseline);
interventions (dose of nicotine delivered by the e-cigarette, frequency of use of the e-cigarette and type of e-cigarettes) and
concomitant interventions in e-cigarettes and control groups.
However, we also were not able to conduct these analyses because they did not meet our minimal criteria, which were at least five studies available, with at least two in each subgroup. A final statistical limitation is that we calculated differences from 6 to 12 months of follow-up. Absolute differences may differ across this time frame and constitute a source of variability. Moreover, there are three schools of thought with respect to use of fixed and random effect models: those who prefer always to use fixed effects, those who prefer (almost) always random effects and those who would choose fixed and random depending on the degree of heterogeneity. Each argument has its proponents within the statistical community. The argument in favour of the second rather than the third is 1) there is always some heterogeneity, so any threshold of switching models is arbitrary and 2) when there is little heterogeneity, fixed and random yield similar or identical results, so one might as well commit oneself to random from the start. We find these two arguments compelling; thus, our choice.
Finally, another limitation of the observational studies in this review is the potential for selection bias as the populations compared differ in terms of intention to quit. Furthermore, in all these RCTs, the ENDS tested were earlier generation; it is possible that later generation of e-cigarettes would have greater benefit.
Although this review presents several limitations, the issue is whether one should dismiss these results entirely or consider them bearing in mind the limitations. The latter represent our view of the matter.
Relation to prior work
The previous Cochrane review8 concluded that due to low event rates and wide CIs, only low-certainty evidence was available from studies comparing ENDS with ENNDS. We excluded some studies included in that Cochrane review as they were either case series, cross-sectional or did not include one arm with ENDS/ENNDS compared with alternative strategies. We also included one additional RCT,33 and nine new cohort studies,26–29 ,40–44 ,46 not included in the Cochrane review. The rationale for including the prospective cohort studies in our review was that it was anticipated that the search would return few RCTs. The authors of the Cochrane review found that ENDS is a useful aid to stop smoking long term compared with ENNDS.
Another review10 including two of our three RCTs,25 ,34–39 and further two case series and two cross-sectional studies, assessed the impact of e-cigarettes in achieving smoking abstinence or reduction in cigarette consumption among current smokers who had used the devices for 6 months or more. The authors concluded that e-cigarette use is associated with smoking cessation; these results are similar to our meta-analysis comparing ENDS with ENNDS (figure 5). Khoudigian's 2016 review11 reported a non-statistically significant trend toward smoking cessation in adults using nicotine e-cigarettes compared with other therapies or placebo. However, the review by Kalkhoran and Glatz 20169 concluded that e-cigarettes are associated with significantly less quitting among smokers.
Implications
Existing smoking reduction aids such as NRT are effective, but their impact is limited: the proportion of those who quit when using these aids remains small. The available evidence, of low or very low quality, can neither verify nor exclude the hypothesis that, because they address nicotine addiction and potentially deal with behavioural and sensory aspects of cigarette use, ENDS may be more effective than other nicotine replacement strategies. This is an important finding, and raises questions regarding how effective it may be addressing the behavioural and sensory aspects of cigarette use in their addictive potential. Thus, the focus of subsequent work should perhaps be on the dose and delivery of nicotine, though teasing out the nicotine effects from sensory aspects is likely to be challenging. It is possible that type of ENDS or dose of exposure may influence quit rates, and that newer models may be more effective, but there is insufficient data to provide insight into these issues. Lack of usefulness with regard to address the question of e- cigarettes' efficacy on smoking reduction and cessation was largely due to poor reporting.
Therefore, owing to the limitations of the studies included in this analysis it is impossible to make strong inferences regarding whether e-cigarette use promotes, has no effect or hinders smoking cessation. This review underlines the need to conduct well-designed trials in this field measuring biochemically validated outcomes and adverse effects.
Acknowledgments
We would like to thank Dr Karolien Adriaens, Dr Frank Baeyen, Dr Wael Al-Delaimy, Dr Lois Biener, Dr Sarah Borderud, Dr Jamie Ostroff, Dr Chris Bullen and Dr Katrina Vickerman to provide us with additional data; and Dr Norbert Hirschhorn to provide us with the results of weekly search alerts. We also would like to thank Rachel Couban for the search strategy, and Dr Konstantinos Farsalinos, Dr Hayden McRobbie and Dr Kristian Filion for the great inputs during revisions.
References
Footnotes
Contributors All authors contributed to all aspects of this study, including conducting the literature search, study design, data collection, data analysis, data interpretation and writing of the paper. AUTHORS’ CONTRIBUTIONS GG, RED, WM and EAA conceived the review. DH-A undertook the searches. RED, EAS, HG, AA, YC, MP and VA screened search results. EAS organised the retrieval of papers. RED, EAS, HG, AA, YC, MP and VA screened the retrieved papers against inclusion criteria. RED, EAS, HG, AA, YC, MP and VA appraised quality of papers. RED, EAS, HG, AA, YC, MP and VA extracted data from papers. RED wrote to authors of papers for additional information. RED provided additional data about papers. RED and EAS obtained and screened data on unpublished studies. RED managed the data for the review. RED and EAS contributed in entering data into Review Manager (RevMan). RED, EAS, GHG, WM and EAA contributed in analysing RevMan statistical data. RED, EAS, GHG, WM and EAA interpreted the data. RED, EAS, GG, WM and EAA contributed in making statistical inferences. RED, GHG, WM and EAA wrote the review. RED, EAS, HG, AA, YC, MP, VA, EAA, WM and GH took responsibility for reading and checking the review before submission.
Funding The study is funded by a WHO grant.
Disclaimer The funding agencies played no role in the conduct of the research or preparation of the manuscript.
Competing interests Regina El Dib received a Brazilian Research Council (CNPq) scholarship (CNPq 310953/2015-4).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data available.