You are here

  • Home
  • Archive
  • Volume 13, Issue 10
  • Evaluating the impact of varying expired carbon monoxide thresholds on smoking relapse identification: insights from the E3 trial on e-cigarette efficacy for smoking cessation

Article Text

Article menu
Smoking and tobacco
Original research
Evaluating the impact of varying expired carbon monoxide thresholds on smoking relapse identification: insights from the E3 trial on e-cigarette efficacy for smoking cessation
  1. Celine Prell1,2,
  2. Andréa Hébert-Losier1,
  3. Kristian B. Filion1,2,3,4,
  4. Pauline Reynier1,
  5. Mark J. Eisenberg1,2,3,4,5
  1. 1Centre for Clinical Epidemiology, Lady Davis Institute, Jewish General Hospital/McGill University, Montreal, QC, Canada
  2. 2Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
  3. 3Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, QC, Canada
  4. 4Department of Medicine, McGill University, Montreal, QC, Canada
  5. 5Division of Cardiology, Jewish General Hospital/McGill University, Montreal, QC, Canada
  1. Correspondence to Dr Mark J. Eisenberg; mark.eisenberg{at}ladydavis.ca

Abstract

Objectives Expired carbon monoxide (ECO) is often used in smoking cessation trials to biochemically validate self-reported smoking status. The optimal ECO threshold to distinguish individuals who smoke from those who do not is debated.

Design The data from the ‘Evaluating the Efficacy of E-Cigarette use for Smoking Cessation (E3) Trial’ were used; the E3 trial was a randomised controlled trial that examined e-cigarettes efficacy for smoking cessation.

Settings Participants were recruited from 17 Canadian sites across 4 provinces.

Participants This substudy included data from participants who returned for at least one of the clinical visits at week 4 (291), 12 (257) or 24 (218) and provided both self-reported smoking status and ECO measures. Analyses were based on 766 paired measures (ie, self-reported smoking status with corresponding ECO).

Results The ability of ECO measurements to discriminate between adults who reported smoking and those who reported abstinence varied with the threshold used. ECO thresholds of 6, 7, 8 and 9 parts per million (ppm) yielded the greatest area under the receiver operating characteristic curve (0.84). These thresholds produced sensitivities of 84%, 82%, 78% and 76% and specificities of 84%, 87%, 90% and 91%, respectively. However, at a threshold of 6 ppm, intersecting sensitivity (84%) and specificity (84%) were maximised with respect to each other. Biochemical validation had the highest agreement with self-report at an ECO threshold of 6 ppm (κ=0.57; 95% CI, 0.51 to 0.64).

Conclusion The classification of participants’ smoking status depends on the ECO threshold used for biochemical validation. We recommend that future smoking cessation trial investigators analyse and report the impact that varying ECO thresholds has on trial results.

Trial registration number NCT02417467.

  • epidemiologic studies
  • public health
  • clinical trials

Data availability statement

Data are available upon reasonable request. Data will be made available upon request with the corresponding author Dr Mark Eisenberg by email (Mark.Eisenberg@ladydavis.ca) to researchers whose proposed use of the data has been approved.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2022-071099

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Individuals enrolled in the E3 trial had to be motivated to quit smoking. Given that participants in a clinical trial are more likely to under-report relapses, the generalisability of these findings to the larger population is unclear.

    • Since the unit of analysis was the participant-visit and participants contributed multiple observations to the analysis, we performed a weighted analysis data to account for potential clustering.

    • We did not adjust for factors that could affect biological levels of expired carbon monoxide (eg, sleep, level of activity, time since last cigarette, environmental exposure) as this information was not available.

    • We repeated our primary analyses stratified by treatment group to determine if the allocated treatment affected study results.

    • Our study involved the use of two different Bedford monitor models, which may have contributed to differences in outcome classification.

    Introduction

    The accurate identification of participants who have quit smoking is a crucial aspect of clinical trials focused on smoking cessation.1–3 Existing data indicate that individuals who have participated in rigorous smoking cessation studies tend to demonstrate high misclassification rates between self-reported and biochemically confirmed abstinence.4 The main reasons are social desirability bias and recall bias associated with relapse.5–7 The revised guidelines published in 2020 emphasise the growing importance of implementing biochemical verification methods to confirm smoking abstinence, particularly in the context of clinical trials.1

    Cotinine and expired carbon monoxide (ECO) testing are the two most common methods of biochemical confirmation of smoking status.1 8–10 Cotinine, a major nicotine metabolite detected in serum, urine and saliva, has a prolonged half-life, providing a 5–7 day detection window for assessing nicotine exposure.1 This makes cotinine a highly sensitive biomarker in evaluating smoking status. However, cotinine activity is influenced by various genetic, hormonal, environmental factors and is relatively expensive compared with ECO measurement.1 3 11 Despite its limitation in capturing only recent exposure, ECO measurement is a widely used, non-invasive, cost-effective, sensitive and real-time method for assessing smoking abstinence in large-scale clinical studies.1 2 8–10

    However, the optimum ECO threshold to distinguish participants who smoked from those who did not is still debated.1 12–14 The primary objective of this study was to determine the effect that varying ECO thresholds had on identifying adults who relapsed to smoking from those who did not in the ‘Evaluating the Efficacy of E-Cigarette use for Smoking Cessation (E3) Trial’, a trial evaluating the efficacy of electronic cigarettes (e-cigarette) for smoking cessation in the general population.15

    Methods

    E3 trial design

    The E3 trial has been described in detail previously.16 Briefly, the E3 trial is a multicentre, randomised controlled trial examining the efficacy of nicotine and non-nicotine e-cigarettes in adults motivated to quit smoking. Adults who smoke were recruited from the general population at 17 Canadian centres from November 2016 to September 2019. To be eligible, they must have smoked, on average, 10 or more conventional (tobacco) cigarettes per day over the past year, be aged 18 years or more and be motivated to quit smoking according to the Motivation To Stop Scale.17 Adults were excluded if they had currently or recently (past 30 days) used a smoking cessation therapy, used an e-cigarette in the past 60 days or had ever used an e-cigarette for more than 7 consecutive days or planned to use tobacco products other than conventional cigarettes during the study period.16

    A total of 376 participants were randomised (1:1:1) to receive (1) nicotine e-cigarettes plus individual counselling; (2) non-nicotine e-cigarettes plus individual counselling; or (3) individual counselling alone.16 The e-cigarette treatment duration was 12 weeks, and the follow-up duration was 52 weeks (inclusive of the 12-week treatment period). At the end of the 12-week treatment period, participants randomised to one of the e-cigarette groups returned their used and unused e-cigarette cartridges to assess e-cigarette consumption. All groups received counselling at baseline (week 0; 30 min), by telephone (weeks 1, 2, 8 and 18; 10 min each) and during in-person follow-up visits (weeks 4, 12, 24, and 52; 15–20 min each).

    At baseline, participants provided information on demographic characteristics, smoking history and clinical history.16 At each in-person visit, smoking status was determined through self-reported 7-day recall and biochemical validation by ECO. Abstinence was defined by a self-report of no cigarettes smoked in the past week and an ECO reading ≤10 parts per million (ppm). If participants self-reported smoking cigarettes in the past week or recorded an ECO>10 ppm, they were classified as smokers.15

    Patient and public involvement

    Patient and public representatives were not involved in the design, recruitment or dissemination of the study.

    ECO measurement

    ECO levels were measured using the Micro 3 Smokerlyzer and the Micro 4 Smokerlyzer (Bedfont Scientific, USA) and are expressed in ppm. Most measurements were obtained using the Micro 4 Smokerlyzer. The instructions for both monitors were identical. Participants were asked to hold their breaths at the initiation of a 15-s countdown. When the countdown had finished, participants were instructed to place their mouth over the mouthpiece and to exhale slowly and gently through the mouthpiece as much as possible until their lungs were emptied. After each use, the mouthpiece was replaced and the air around the sample vent was stirred to prevent contamination between participants. The standard thresholds provided by the manufacturer for both devices were 0–10 ppm for a non-smoker, 11–20 ppm for a light smoker and >20 ppm for a heavy smoker. The monitors have a range of measurement of 0–500 ppm and display CO values in 1 ppm increments with an accuracy of ±2%.18

    Statistical analyses

    This substudy included data from participants who returned for at least one of the clinical visits at week 4, 12 or 24 and provided both self-reported smoking status and ECO measures. Data for the 52-week follow-up visit were not available at the time of this substudy and thus was not included in the analyses. Data analyses were based on a total of 766 paired measures (ie, self-reported smoking status with corresponding ECO level) from the week 4 (291), 12 (257) and 24 (218) clinical visits. The unit of analysis was the participant visit, with participants able to contribute up to three observations to the analyses. Descriptive analyses were used to examine baseline characteristics, with discrete data described using counts and proportions and continuous data described using means and SD or, in the presence of skewed distributions, medians and IQRs.

    For each pair-wise comparison, sensitivity and specificity and corresponding 95% CIs were calculated based on the binomial distribution, with self-reported smoking status used as the reference standard. Positive and negative predictive values (PPV and NPV, respectively) and likelihood ratios for positive and negative test results (LR+ and LR−, respectively) were also estimated. Receiver operating characteristic (ROC) curves were used to assess the ECO cut-offs that discriminated between adults who smoked and those who did not, and the area-under-the-curve (AUC) was calculated to determine the ability of the Micro 3/4 Smokerlyzer to discriminate between these two groups. The ECO threshold that provided the greatest combined sensitivity and specificity was determined. Using this cut-off point, we estimated Cohen’s kappa statistic to assess the agreement between self-reported smoking status and ECO-defined smoking status. Pearson’s correlations were used to determine the relationship between ECO levels and the self-reported mean number of cigarettes smoked per day over the past 7 days.

    We conducted several sensitivity analyses to assess the robustness of our results. First, we repeated our primary analyses stratified by treatment group to determine if the allocated treatment affected study results. These analyses were conducted following an intention-to-treat approach in which participants were included in the treatment group to which they were randomised, regardless of treatment received. Second, we evaluated differences between participants living with smokers and those living in smoke-free households. In order to conclude whether there was a significant difference between AUC curves when comparing these subgroups, we calculated the probability of a significant difference with CIs at the 2.5 and 97.5 percentile by bootstrap with 1000 simulations. Third, since the unit of analysis was the participant-visit and participants contributed multiple observations to the analysis, we performed a weighted analysis data to account for potential clustering. All analyses were performed using SAS V.9.4.19

    Results

    Participant characteristics

    Of 376 participants enrolled in the E3 trial, 298 (79%) participants provided an ECO measurement and were thus included in the analyses (online supplemental appendix 1, efigure 1). Participants’ baseline characteristics are presented in table 1. Participants were, on average, middle-aged (52±12 years) white men (53% male). They had been smoking for 34±14 years, and they smoked a mean of 21±10.3 cigarettes per day over the past year. Twenty-nine per cent of participants reported having other smoker(s) at home.

    View this table:
    Table 1

    Baseline characteristics of included ‘Evaluating the Efficacy of E-Cigarette use for Smoking Cessation’ (E3) trial participants*

    Self-reported smoking versus ECO concentration

    The distribution of participants by expired CO concentration and self-reported smoking status is reported in figure 1. Self-reported non-smokers were more likely to have lower ECO concentrations (0–28 ppm); most self-reported non-smokers had non-negligible ECO concentrations. Specifically, only 9 of 157 (6%) individuals had an ECO≥10 ppm (figure 1, blue). Self-reported smokers had a greater distribution of ECO concentrations (0–102 ppm); however, a substantial number of self-reported smokers (41 out of 609 observations, 7%) had ECO concentrations of 0 ppm (figure 1, orange). In addition, participants who reported smoking the same mean number of cigarettes per week frequently expired at different ECO concentrations (figure 2).

    Figure 1
    Figure 1

    Prevalence of smoking according to expired carbon monoxide (ECO) level. The distribution of ECO concentrations for self-reporting smokers and self-reported non-smokers are visually depicted by the orange and blue curves, respectively. Self-reported smokers ECO levels ranged from 0 to 102 parts per million (ppm) (orange) whereas self-reported non-smokers ECO levels ranged from 0 to 28 ppm (blue).

    Figure 2
    Figure 2

    Expired carbon monoxide (ECO) by self-reported mean number of cigarettes smoked in the past 7 days. Due to several readings for the same average cigarette intake, multiple values are superimposed. The prediction ellipse represents an iso-contour of the Gaussian distribution, defining a region of paired data containing 95% of reported measures. ppm, parts per million.

    Sensitivity and specificity trade-off

    Table 2 shows the impact of varying ECO measurements on test characteristics of ECO and self-reported smoking status (prevalence 80%). The sensitivity ranged from 60% to 91%, and the specificity ranged from 58% to 94%. PPV and NPV ranged from 89 to 98% and from 38% to 62%, respectively. The AUCs ranged from 0.74 to 0.84. Cutoffs of 6, 8 and 10 ppm had AUCs of 0.84, 0.84 and 0.83, respectively (online supplemental appendix 1, efigure 2). Values for specificity and sensitivity intersected at 6 ppm, such that both sensitivity (84%) and specificity (84%) were maximised with respect to each other (figure 3).

    View this table:
    Table 2

    Receiver operator characteristic curve values for expired carbon monoxide level and self-reported smoking status for all participants

    Figure 3
    Figure 3

    Sensitivity and specificity of expired carbon monoxide (ECO) for validating self-reported smoking status. At lower thresholds, ECO measurements are more sensitive and thus better for detecting greater numbers of smokers, whereas at higher thresholds, ECO measurements are more specific and thus better for discriminating smokers from non-smokers. Specificity (orange) and sensitivity (blue) intersect at an ECO value of 6 parts per million (ppm), such that both sensitivity (84%) and specificity (84%) are maximised with respect to each other.

    Threshold effect on smoking status

    To illustrate how varying the ECO cut-off changes the classification of participants as smokers and as non-smokers, we compared participant smoking status using thresholds 10 ppm and 5 ppm (the value that maximises specificity without significant detriment to sensitivity and the value that maximises sensitivity without significant detriment to specificity) (online supplemental appendix 1, etable 1). Three times the number of self-reported non-smokers expired an ECO above a threshold of 5 ppm than 10 ppm (n=34 vs 10), increasing the number of participants classified as smokers by 16%. However, fewer self-reported smokers had an ECO below the 5 ppm cut-off (n=90 vs 166), which decreased the number of participants classified as non-smokers by 12%. Decreasing the ECO threshold from 10 ppm to 5 ppm increased sensitivity from 73% to 85% but decreased specificity from 94% to 78%.

    Biochemical validation had the greatest agreement with self-reported smoking status when the 6 ppm cut-off was used (κ=0.57; 95% CI, 0.51 to 0.64).

    Sensitivity analyses

    Prespecified sensitivity analyses were conducted to examine the robustness of our results. When the primary analyses were stratified by treatment group (online supplemental appendix 1, etable 1 and efigure 3), ECO thresholds that maximised AUC were similar to those obtained in the primary analyses (nicotine e-cigarettes plus counselling ECO=8 ppm, AUC=0.86; non-nicotine e-cigarettes plus counselling ECO=10 ppm, AUC=0.91; counselling alone ECO=6–7 ppm, AUC=0.88). ECO threshold values were also similar when comparing participants living with smokers (ECO=8 ppm; AUC=0.86) to those living in smoke-free households (ECO=7 ppm; AUC=0.85) (online supplemental appendix 1, etable 2) and when weighted by number of visits per participant (online supplemental appendix 1, etable 3file 1).

    We compared AUC for each ECO value in subgroups defined by treatment group (online supplemental etable 4) and the presence or absence of household smokers and those without household smokers (online supplemental etable 5), estimating differences in AUC and corresponding 95% CIs by subgroup. No differences in AUC were observed across all ECO values in either subgroup analysis. However, differences were observed in AUC values among participants randomised to nicotine versus no nicotine e-cigarette for ECO values ≥10 ppm; no difference was observed below the 10 ppm threshold.

    Discussion

    Our study was designed to evaluate the effect that varying ECO thresholds have on diagnostic test characteristics for ECO measurement in clinical trials. We found that ECO monitors can be used to validate self-reported smoking status. Our results suggest that the classification of a participant’s smoking or non-smoking status is dependent on the selected ECO threshold. Therefore, we recommend that future smoking cessation trials analyse the impact that ECO cut-offs have on their results.

    Our results have important implications for the interpretation of smoking cessation trials in which participants transition from combustible to electronic forms of nicotine consumption. Of currently published e-cigarette clinical trials, cut-offs of 10 ppm,15 20 21 9 ppm,22 8 ppm23 and 5 ppm24 have been used to validate smoking status. We found that a cut-off of 6 ppm minimised outcome misclassification by maximising aggregate measures of sensitivity and specificity. While AUC provides the best aggregate measure of sensitivity and specificity, it may be more appropriate to weigh sensitivity and specificity unevenly, given the disproportionate trade-offs between sensitivity and specificity at different thresholds. At lower thresholds, ECO measurements are better for detecting greater numbers of smokers, whereas at higher thresholds, they are better for discriminating smokers from non-smokers. For example, the ECO cut-off of 2 ppm yields the greatest sensitivity (91%), but the specificity is impractically low. By raising the threshold to 5 ppm, the sensitivity (85%) is only slightly reduced while the specificity (78%) is greatly enhanced. Likewise, while the ECO cut-off of 12 ppm yields the greatest specificity (94%), the sensitivity is impractically low (60%), but by lowering the threshold to 10 ppm, the specificity (94%) is unaltered while the sensitivity (73%) is greatly enhanced. The likelihood ratios follow similar trends, where the LR+ reaches its peak at higher ECO values, indicating stronger diagnostic accuracy for positive cases. Conversely, the LR− is minimised at lower ECO values, signifying improved diagnostic accuracy for negative cases. The ROC curve can be used in this manner to evaluate the benefits and drawbacks of choosing different ECO cutoffs to discriminate smokers from non-smokers. Using the E3 trial data in this manner, we conclude that lower ECO thresholds are better suited to minimise outcome misclassification in the context of smoking cessation trials. This is because thresholds that maximise sensitivity will counteract the higher proportion of false-positive reports encouraged by the social desirability bias. This type of reporting bias occurs when reporting socially undesirable behaviours, such as smoking, in which respondents will underestimate the prevalence of this behaviour to fulfil their need for social approval.25 Because it is difficult to quantify the degree to which social desirability impacts participant’s self-reported behaviours, researchers must ensure to design studies that use objective measure to minimise socially motivated misreporting. Our conclusion aligns with prior studies, which found that trials using lower ECO thresholds were less likely to classify participants as having quit than those using higher ECO thresholds, leading to lower false-negative rates.14 26 27

    In smoking cessation trials, practically, biochemical validation is only consequential when participants report abstinence. Participants who self-report smoking with ECO concentrations below the chosen threshold are generally still considered ‘smokers’. This is done for two practical reasons: (1) self-reports generally underestimate rather than overestimate smoking due to the social desirability bias, and (2) participants who self-report smoking might record low ECO levels (ie, false-negative) because too much time has passed between their last cigarette and their ECO measurement. ECO levels fall quickly after smoke inhalation (eg, to as low as 3 ppm in a full day of smoking abstinence).28 Therefore, false-negative readings represent a much smaller threat to the validity of smoking cessation trials than false-positive readings. However, ECO measurements are necessarily reflective only of cigarette consumption within the past few days, given that the half-life of CO is 5–6 hours in the body.29 This limits the accuracy of ECO as a tool to access continuous measures of smoking cessation, although it may still encourage accurate reporting by participants if they know an ECO measurement will be taken.Embedded ImageWhen choosing an ECO threshold to monitor chronic abstinence, it is crucial to note that ECO concentrations vary between individuals depending on activity level, sleep, respiratory muscle use, depth of inhalation, pre-existing conditions (eg, chronic obstructive pulmonary disorder (COPD), lactose intolerance), pregnancy and environmental exposure, to name a few.12 29–32 Higher cut-offs may be necessary to identify smoking habits in participants with asthma or COPD,33–35 in whom ECO concentrations are affected by airway inflammation. Higher cut-offs may also be necessary to assess smoking habits in individuals living in high-pollution areas,36 since ambient CO levels have an important impact on ECO readings. For example, ECO levels of 4 ppm or lower may signify urban living status rather than smoking status.36 37 Therefore, while researchers may be tempted to substantially reduce the ECO cut-off to maximise study sensitivity, doing so might be imprudent for certain study populations.

    Given that there are many factors that can influence ECO other than recent tobacco exposure, researchers may wish to recommend a range of reasonable cut-off points in favour of one ECO concentration, as suggested by a prior meta-analysis comparing ECO-verified cessation trials26 and by the most recent Society for Research on Nicotine and Tobacco (SRNT) publication on biomarker verification for smoking cessation.1 However, absence of a single congruent cut-off point is problematic for smoking cessation trials because intervention efficacy relies on valid measures of smoking status that are consistent across trials. The aforementioned meta-analysis found that an increase in just one ppm unit increased the odds of being classified as abstinent by 18% for thresholds over 3 ppm, highlighting the importance of maintaining threshold consistency across trials.26 For example, researchers may choose the SRNT-recommended 5–6 ppm cut-off1 to compare their results to other trails using ECO thresholds to validate abstinence. Without a standardised and well-defined cut-off, the results of different smoking-cessation trials may be influenced by their chosen cut-off, resulting in important discrepancies between trials. Trials using the higher cut-offs of 9–10 ppm are much more likely to classify participants as abstinent than those using lower cut-offs of 3–4 ppm.26 However, there remains some debate whether this also applies to cut-off thresholds in the more moderate range (ie, 5–6 ppm vs 7–8 ppm). Our study supports the hypothesis26 that different cut-off choices in this range influence the sensitivity and specificity of ECO monitors and therefore the outcome classification for abstinence, while others indicate that such choices are clinically insignificant unless very low thresholds are used.14

    While it remains important to prespecify the primary ECO cut-off to ensure transparency and reproducibility, smoking cessation trials should include prespecified sensitivity analyses that vary the ECO thresholds to provide study transparency, facilitate comparisons across trials and better inform future appraisals of smoking cessation strategies. It may likewise be appropriate to prespecify the monitor brand being used to measure ECO, as several groups report differences in ECO readings between different monitors.33 38 39 Consequently, the extent to which different monitor brands or models impacts outcome classification remains unclear.

    Despite the variability in smoking classification when using different ECO thresholds, and despite the aforementioned factors that confound the results of ECO monitors, such as time since last cigarette, ECO monitors remain the only way to validate self-reported smoking status in the context of e-cigarette use. Other biochemical verification methods for tobacco cessation measure concentrations of nicotine, nicotine metabolites or tobacco-derived alkaloids, which may be present in some e-liquids,40 41 making them impractical methods for e-cigarette users. While the biomarker 4-(methylnitrosamino)−1-(3-pyridyl)−1-butanol has recently been proposed to distinguish users of e-cigarettes from cigarettes, levels remain elevated in the urine for months after quitting,1 limiting its utility in the short-term. Since ECO is an indicator of smoke inhalation rather than nicotine or tobacco ingestion, it is one of the most reliable ways to validate smoking in clinical trials involving e-cigarettes and other nicotine-replacement therapies, despite its drawbacks. Given the limitations of other biomarkers when assessing smoking status of participants using e-cigarettes, researchers must rely on ECO as the reference-standard for biochemical validation. Likewise, comparisons between ECO monitors must rely on self-report as the reference-standard for smoking cessation. Given the choice to select self-report as the reference-standard measurement, our study best assesses the ECO thresholds that predict whether or not participants self-disclose smoking abstinence rather than the ECO thresholds that indicate smoking abstinence.

    Our study had several potential limitations. First, individuals enrolled in the E3 trial had to be motivated to quit smoking. Given that participants in a clinical trial are more likely to under-report relapses, the generalisability of these findings to the larger population is unclear. Second, knowledge that their reports would be biochemically validated may have discouraged participants from falsely reporting that they were abstinent, leading to an inflated correlation between self-report and ECO validation. Third, we did not adjust for factors that could affect biological levels of ECO (eg, sleep, level of activity, time since last cigarette, environmental exposure) as this information was not available. Fourth, our study involved the use of two Bedford monitor brands, which, as mentioned briefly above, may have contributed to differences in outcome classification. Fifth, there was a differential drop-off rate between groups, where more individuals from the counselling group dropped out compared with other groups. Consequently, there may have been an over-representation of participants from the e-cigarette arms included in our analysis. However, sensitivity analyses that stratified by treatment revealed no difference between groups. Finally, the generalisability of our results to trial populations generated using different inclusion and exclusion criteria is unclear. We acknowledge that the long-term effects of e-cigarette use beyond the follow-up duration of our trial (eg, 52 weeks) is not available, therefore we cannot definitively say whether our findings apply to trials that monitor long-term smoking abstinence using ECO measurements beyond this period.

    Conclusion

    ECO levels are useful to validate self-reported smoking status in smoking cessation trials. Smoking cessation trial results may depend on ECO thresholds used for validating self-reported smoking status. To better understand the impact of the ECO on trial results and to facilitate comparisons across trials, investigators should report trial results across a range of ECO thresholds.

    Data availability statement

    Data are available upon reasonable request. Data will be made available upon request with the corresponding author Dr Mark Eisenberg by email (Mark.Eisenberg@ladydavis.ca) to researchers whose proposed use of the data has been approved.

    Ethics statements

    Patient consent for publication

    Ethics approval

    The E3 trial was conducted according to all applicable institutional, provincial and federal regulations concerning clinical trials. The protocol was reviewed and approved by the Medical/Biomedical Research Ethics Committee of the CIUSSS West-Central Montreal Research Ethics Board (Project MP-05-2015-322, 15-012), the research ethic board of each participating institution and No Objection Letters were issued by Health Canada for the protocol and amendments. Written informed consent was obtained from all participants prior to randomisation. Participants gave informed consent to participate in the study before taking part.

    References

    1. ↵
      1. Benowitz NL,
      2. Bernert JT,
      3. Foulds J, et al
      . Biochemical verification of tobacco use and abstinence: 2019 update. Nicotine Tob Res 2020;22:1086–97. doi:10.1093/ntr/ntz132
      OpenUrlCrossRefPubMed
    2. ↵
      1. Grandi SM,
      2. Gervais A,
      3. Joseph L, et al
      . Assessment of the validity of self-report as a measure of smoking status in patients post-myocardial infarction. CJGIM 2014;9:137–9. doi:10.19144/1911-1606.9.4.3 Available: http://csim.ca/csim-journal/index.php/csim/issue/view/2
      OpenUrl
    3. ↵
      1. Murray RP,
      2. Connett JE,
      3. Lauger GG, et al
      . Error in smoking measures: effects of intervention on relations of Cotinine and carbon Monoxide to self-reported smoking. Am J Public Health 1993;83:1251–7. doi:10.2105/ajph.83.9.1251
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Connor Gorber S,
      2. Schofield-Hurwitz S,
      3. Hardt J, et al
      . The accuracy of self-reported smoking: a systematic review of the relationship between self-reported and Cotinine-assessed smoking status. Nicotine Tob Res 2009;11:12–24. doi:10.1093/ntr/ntn010
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Al-Sheyab N,
      2. Kheirallah KA,
      3. Mangnall LJT, et al
      . Agreement between exhaled breath carbon Monoxide threshold levels and self-reported cigarette smoking in a sample of male adolescents in Jordan. Int J Environ Res Public Health 2015;12:841–54. doi:10.3390/ijerph120100841
      OpenUrl
    6. ↵
      1. Stelmach R,
      2. Fernandes FLA,
      3. Carvalho-Pinto RM, et al
      . Comparison between objective measures of smoking and self-reported smoking status in patients with asthma or COPD: are our patients telling us the truth J Bras Pneumol 2015;41:124–32. doi:10.1590/S1806-37132015000004526
      OpenUrlCrossRefPubMed
    7. ↵
      1. Kim Y,
      2. Choi Y-J,
      3. Oh S-W, et al
      . Discrepancy between self-reported and urine-Cotinine verified smoking status among Korean male adults: analysis of health check-up data from a single private hospital. Korean J Fam Med 2016;37:171–6. doi:10.4082/kjfm.2016.37.3.171
      OpenUrlPubMed
    8. ↵
      1. Masaki K,
      2. Tateno H,
      3. Nomura A, et al
      . A randomized controlled trial of a smoking cessation Smartphone application with a carbon Monoxide checker. NPJ Digit Med 2020;3:35. doi:10.1038/s41746-020-0243-5
    9. ↵
      1. Hrabovsky S,
      2. Yingst JM,
      3. Veldheer S, et al
      . Measurement of exhaled breath carbon Monoxide in clinical practice: A study of levels in central Pennsylvania community members. J Am Assoc Nurse Pract 2017;29:310–5. doi:10.1002/2327-6924.12460
      OpenUrl
    10. ↵
      1. Flacco ME,
      2. Ferrante M,
      3. Fiore M, et al
      . Cohort study of electronic cigarette use: safety and effectiveness after 4 years of follow-up. Eur Rev Med Pharmacol Sci 2019;23:402–12. doi:10.26355/eurrev_201901_16789
      OpenUrl
    11. ↵
      1. Murphy SE,
      2. Park S-SL,
      3. Thompson EF, et al
      . Nicotine N-Glucuronidation relative to N-oxidation and C-oxidation and Ugt2B10 genotype in five ethnic/racial groups. Carcinogenesis 2014;35:2526–33. doi:10.1093/carcin/bgu191
      OpenUrlCrossRefPubMed
    12. ↵
      1. Vasthare R,
      2. Kumar S,
      3. Arron LYR
      . Carbon Monoxide breath Analyzers and its role in tobacco cessation: A narrative review of literature. J Int Oral Health 2018;10:71. doi:10.4103/jioh.jioh_273_17
      OpenUrl
    13. ↵
      1. Maclaren DJ,
      2. Conigrave KM,
      3. Robertson JA, et al
      . Using breath carbon Monoxide to validate self-reported tobacco smoking in remote Australian indigenous communities. Popul Health Metr 2010;8:2. doi:10.1186/1478-7954-8-2
    14. ↵
      1. Brose LS,
      2. Tombor I,
      3. Shahab L, et al
      . The effect of reducing the threshold for carbon Monoxide validation of smoking abstinence--evidence from the English stop smoking services. Addict Behav 2013;38:2529–31. doi:10.1016/j.addbeh.2013.04.006
      OpenUrlCrossRefPubMed
    15. ↵
      1. Eisenberg MJ,
      2. Hébert-Losier A,
      3. Windle SB, et al
      . Effect of E-cigarettes plus counseling vs counseling alone on smoking cessation: A randomized clinical trial. JAMA 2020;324:1844–54. doi:10.1001/jama.2020.18889
      OpenUrlCrossRefPubMed
    16. ↵
      1. Hébert-Losier A,
      2. Filion KB,
      3. Windle SB, et al
      . A randomized controlled trial evaluating the efficacy of E-cigarette use for smoking cessation in the general population: E3 trial design. CJC Open 2020;2:168–75. doi:10.1016/j.cjco.2020.03.006
      OpenUrl
    17. ↵
      1. Kotz D,
      2. Brown J,
      3. West R
      . Predictive validity of the motivation to stop scale (MTSS): a single-item measure of motivation to stop smoking. Drug Alcohol Depend 2013;128:15–9. doi:10.1016/j.drugalcdep.2012.07.012
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Bedfont
      . Micro 4 Smokerlyzer Operating Manual. Haddonfield, NJ: Bedfont USA,
    19. ↵
      1. SAS Institute Inc
      . SAS® 9.4 Statements: Reference. Cary, NC: SAS Institute Inc, 2013.
    20. ↵
      1. Bullen C,
      2. Howe C,
      3. Laugesen M, et al
      . Electronic cigarettes for smoking cessation: a randomised controlled trial. The Lancet 2013;382:1629–37. doi:10.1016/S0140-6736(13)61842-5
      OpenUrl
    21. ↵
      1. Lee S-H,
      2. Ahn S-H,
      3. Cheong Y-S
      . Effect of electronic cigarettes on smoking reduction and cessation in Korean male Smokers: a randomized controlled study. J Am Board Fam Med 2019;32:567–74. doi:10.3122/jabfm.2019.04.180384
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Walker N,
      2. Parag V,
      3. Verbiest M, et al
      . Nicotine patches used in combination with E-cigarettes (with and without nicotine) for smoking cessation: a pragmatic, randomised trial. Lancet Respir Med 2020;8:54–64. doi:10.1016/S2213-2600(19)30269-3
      OpenUrl
    23. ↵
      1. Hajek P,
      2. Phillips-Waller A,
      3. Przulj D, et al
      . A randomized trial of E-cigarettes versus nicotine-replacement therapy. N Engl J Med 2019;380:629–37. doi:10.1056/NEJMoa1808779
      OpenUrlCrossRefPubMed
    24. ↵
      1. Lucchiari C,
      2. Masiero M,
      3. Mazzocco K, et al
      . Benefits of E-cigarettes in smoking reduction and in pulmonary health among chronic Smokers undergoing a lung cancer screening program at 6 months. Addict Behav 2020;103:S0306-4603(19)30183-2. doi:10.1016/j.addbeh.2019.106222
    25. ↵
      1. Krumpal I
      . Determinants of social desirability bias in sensitive surveys: a literature review. Qual Quant 2013;47:2025–47. doi:10.1007/s11135-011-9640-9
      OpenUrlCrossRef
    26. ↵
      1. Karelitz JL,
      2. McClure EA,
      3. Wolford-Clevenger C, et al
      . Cessation classification likelihood increases with higher expired-air carbon Monoxide cutoffs: a meta-analysis. Drug Alcohol Depend 2021;221:S0376-8716(21)00065-X. doi:10.1016/j.drugalcdep.2021.108570
    27. ↵
      1. Cropsey KL,
      2. Trent LR,
      3. Clark CB, et al
      . How low should you go? determining the optimal cutoff for exhaled carbon Monoxide to confirm smoking abstinence when using Cotinine as reference. Nicotine Tob Res 2014;16:1348–55. doi:10.1093/ntr/ntu085
      OpenUrlCrossRefPubMed
    28. ↵
      1. Javors MA,
      2. Hatch JP,
      3. Lamb RJ
      . Sequential combination of self-report, breath carbon Monoxide, and saliva Cotinine to assess smoking status. Drug Alcohol Depend 2011;113:242–4. doi:10.1016/j.drugalcdep.2010.07.020
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Goldstein AO,
      2. Gans SP,
      3. Ripley-Moffitt C, et al
      . Use of expired air carbon Monoxide testing in clinical tobacco treatment settings. Chest 2018;153:554–62. doi:10.1016/j.chest.2017.11.002
      OpenUrlPubMed
    30. ↵
      1. Koehler K,
      2. Good N,
      3. Wilson A, et al
      . The Fort Collins commuter study: variability in personal exposure to air Pollutants by Microenvironment. Indoor Air 2019;29:231–41. doi:10.1111/ina.12533
      OpenUrl
    31. ↵
      1. Kotz D
      . Possible reasons for elevated carbon Monoxide levels in self-reported ex-Smokers. Nicotine Tob Res 2012;14:900–1. doi:10.1093/ntr/ntr305
      OpenUrlCrossRefPubMed
    32. ↵
      1. Bailey BA
      . Using expired air carbon Monoxide to determine smoking status during pregnancy: preliminary identification of an appropriately sensitive and specific cut-point. Addict Behav 2013;38:2547–50. doi:10.1016/j.addbeh.2013.05.011
      OpenUrlCrossRefPubMed
    33. ↵
      1. Christenhusz L,
      2. de Jongh F,
      3. van der Valk P, et al
      . Comparison of three carbon Monoxide monitors for determination of smoking status in Smokers and Nonsmokers with and without COPD. J Aerosol Med 2007;20:475–83. doi:10.1089/jam.2007.0606
      OpenUrlCrossRefPubMed
    34. ↵
      1. Ryter SW,
      2. Choi AMK
      . Carbon Monoxide in exhaled breath testing and Therapeutics. J Breath Res 2013;7:017111. doi:10.1088/1752-7155/7/1/017111
    35. ↵
      1. Shaoqing Y,
      2. Ruxin Z,
      3. Yingjian C, et al
      . A meta-analysis of the Association of exhaled carbon Monoxide on asthma and allergic rhinitis. Clinic Rev Allerg Immunol 2011;41:67–75. doi:10.1007/s12016-009-8195-1
      OpenUrl
    36. ↵
      1. Maga M,
      2. Janik MK,
      3. Wachsmann A, et al
      . Influence of air pollution on exhaled carbon Monoxide levels in Smokers and non-Smokers. A prospective cross-sectional study. Environ Res 2017;152:496–502. doi:10.1016/j.envres.2016.09.004
      OpenUrl
    37. ↵
      1. Gregorczyk-Maga I,
      2. Maga M,
      3. Wachsmann A, et al
      . Air pollution may affect the assessment of smoking habits by exhaled carbon Monoxide measurements. Environ Res 2019;172:258–65. doi:10.1016/j.envres.2019.01.063
      OpenUrl
    38. ↵
      1. Karelitz JL,
      2. Michael VC,
      3. Perkins KA
      . Analysis of agreement between expired-air carbon Monoxide monitors. J Smok Cessat 2017;12:105–12. doi:10.1017/jsc.2015.18
      OpenUrl
    39. ↵
      1. Tuck BM,
      2. Karelitz JL,
      3. Tomko RL, et al
      . Mobile, remote, and individual focused: comparing breath carbon Monoxide readings and abstinence between Smartphone-enabled and stand-alone monitors. Nicotine Tob Res 2021;23:741–7. doi:10.1093/ntr/ntaa203
      OpenUrl
    40. ↵
      1. Etter JF,
      2. Zäther E,
      3. Svensson S
      . Analysis of refill liquids for electronic cigarettes. Addiction 2013;108:1671–9. doi:10.1111/add.12235
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      1. Conklin DJ,
      2. Ogunwale MA,
      3. Chen Y, et al
      . Electronic cigarette-generated Aldehydes: the contribution of E-liquid components to their formation and the use of urinary Aldehyde metabolites as biomarkers of exposure. Aerosol Sci Technol 2018;52:1219–32. doi:10.1080/02786826.2018.1500013
      OpenUrlPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors ME and KF had full access to the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. CP, AH-L, KF and ME contributed substantially to the study design and interpretation for the manuscript. CP, KF and PR contributed to the statistical analyses. CP and AH-L drafted the manuscript. KF, PR and ME provided critical revision of the manuscript, and all authors approved the final version to be published. ME is guarantor for the paper.

    • Funding The E3 trial was funded by the Canadian Institutes of Health Research (#7133727 and #155969). CP was supported by Dr Clarke K McLeod Memorial Scholarship award. KF is supported by a senior salary support award from the Fonds de recherche du Québec – santé (Quebec Foundation for Research - Health) and a William Dawson Scholar award from McGill University.

    • Competing interests ME received educational grants from Pfizer for providing continuing medical education in cardiology. The other authors have no relationships to disclose.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.