Objectives We aimed to evaluate the validity of the self-reported smoking indicator used in the Global Youth Tobacco Survey (GYTS).
Setting 43 middle and high-school classrooms from 26 schools were selected from Mexico City and Cuernavaca, Morelos.
Participants A total of 1257 students provided both a questionnaire and a urine sample.
Primary and secondary outcome Sensitivity and specificity of self-reported smoking compared to urinary cotinine. Validity indices were evaluated by subgroups of gender, social acceptability of smoking (ie, smoking parents or friends) and smoking frequency.
Results Sensitivity and specificity for current smoking were 93.2% and 81.7%, respectively. Validity indices remained stable across gender. Parental smoking status moderated the validity of self-report, which had lower sensitivity in adolescents with non-smoking parents (86.7%) than in adolescents with smoking parents (96.6%). Sensitivity and specificity increased with smoking frequency.
Conclusions This first validation study of self-reported current smoking used in the GYTS among Mexican adolescents suggests that self-reported smoking in the past 30 days is a valid and stable indicator of current smoking behaviour. This measure appears suitable for public health research and surveillance.
- tobacco smoking
- sensitivity and specificity
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The WHO-Global Youth Tobacco Survey (GYTS) is an international effort to monitor tobacco consumption in adolescents. A critical indicator in the survey is self-reported current smoking, yet its validity in Latin America is unknown.
We evaluated the validity of self-reported smoking in the GYTS comparing it to urinary cotinine concentrations in 1257 Mexican adolescents. Self-reported smoking was found to be a valid tool for epidemiological surveillance, obtaining similar sensitivity and specificity between genders.
Small variations of validity were observed when comparing levels of social acceptability of smoking, finding higher sensitivity among adolescents with smoking parents; still, the sensitivity among adolescents with non-smoking parents was satisfactory.
Smoking self-report is valid and reasonably stable across gender and contexts of social acceptability of smoking. The increased certainty about the validity of self-report strengthens our confidence on GYTS-derived data for decision-making and the design of public policy aimed at tobacco control in Mexico.
Tobacco control interventions proposed by the WHO Framework Convention on Tobacco Control (WHO-FCTC) have reduced environmental tobacco smoke (ETS) exposure and, to a lesser extent, the prevalence of smoking.1 However, prevalence deceleration has been slowing down since 2006 and policy impact has been heterogeneous across countries.2 These observations call for a reinforcement of smoking monitoring systems, particularly in countries and population segments where smoking behaviour has proven to be refractory to policy interventions. In Mexico, the prevalence of adolescent smoking in 2000 was 14.5% among youth aged 10–19 years, decreased to 10.8% in 2006 and then increased to 12.3% in 2012.3 Surveillance of adolescent smoking is a top priority for tobacco control,1 ,4 requiring affordable and valid approaches to monitor the tobacco epidemic in this susceptible age group.
Self-report is the most common method to assess tobacco use.5–9 However, its validity can vary greatly across populations.7 ,10–12 In adults and adolescents, the validity of self-report has been observed to vary by gender and perceived social acceptability of smoking.5 ,7 ,13 ,14 Considering the critical importance of self-report for tobacco surveillance and intervention research, its validity should be assessed for specific subpopulations to ensure that measurement is unbiased.
The Global Youth Tobacco Survey (GYTS) is the main international data collection effort for the epidemiological surveillance of adolescent tobacco use. The GYTS gathers information about tobacco-related attitudes, knowledge and behaviours, including self-reported tobacco use, cessation and exposure to ETS.15 In Latin America, only Brazil has assessed the validity of self-reported smoking in adolescents, finding poor performance (sensitivity of 16.3% and 22.6%).16 The validity of self-reported current smoking in the GYTS for Spanish-speaking Latin American countries, including Mexico, has not yet been studied.
The present study aims to validate self-reported smoking in Mexican adolescents using a key biomarker of nicotine metabolism (ie, urinary cotinine), and exploring potential differences in validity by gender, social acceptability of smoking and frequency of smoking.
This validation study was undertaken simultaneously and in parallel to the 2011 administration of the GYTS in capital cities from 13 states of Mexico.17
GYTS sample design
The GYTS is an international survey coordinated by the WHO aimed at the 13–15-year-old adolescent population attending middle and high school. The GYTS has been administered in 156 countries using standardised questions and a common methodology that facilitates comparisons across countries and over time.15 In 2011, the GYTS was administered in Mexico selecting students in two stages: in the first stage, high schools were selected with a probability proportional to the number of registered students according to the official active school lists from the Public Education Ministry. In the second stage, classrooms were randomly selected, considering the three grades of middle school and the first grade of high school. All students from selected classrooms present at the time of the survey were invited to participate. Students received full information about the study and gave verbal assent in addition to parental informed consent.
Participant selection for the validation study
The validation study was conducted in schools from Mexico City and Cuernavaca. For the validation, classrooms from the same school and grade were randomly selected among those who were not selected for the GYTS survey, to avoid introducing ‘bogus pipeline’ bias (ie, tendency to provide a more accurate report when subjects are aware that an objective assessment will be included along with self-report, as it was in the case of our study), which would have affected the comparability of GYTS data over time and across sites.18 ,19
Sample size was estimated to obtain a self-report with a ±10% width in the 95% CI of the sensitivity and specificity. However, some assumptions were required as the cotinine-based prevalence of smoking (‘true’ prevalence, proportion >50 ng/mL of urinary cotinine) was unknown.20 Using data from the GYTS 2006 from Mexico City, we assumed 25% of the self-reported smokers (27.8%) would be classified as cotinine-based smokers, given that the proportion of regular smokers was low (2.7% smoked on 20 days or more of the past month). Thus, the estimated sample size was 1136 students (568 for each gender), calculated assuming a 6.75% cotinine-based prevalence of smoking in Mexico City (same for men and women),21 with an expected sensitivity of 90% and 15% replacement. An estimated 60 classrooms with a conservative expected average of 20 students were required to fulfil the sample size.
Following the GYTS protocol, each participant answered an 81-question survey to evaluate tobacco-related attitudes, knowledge and behaviours, both for cigarettes and other tobacco products. The anonymous questionnaire was answered with no teacher intervention. Students received full information about the study and gave verbal assent in addition to parental informed consent.
The question used to establish current smoking was “During the past 30 days (1 month), on how many days did you smoke cigarettes?” with the GYTS predefined possible answers being: 0 days, 1–2 days, 3–5 days, 6–9 days, 10–19 days, 20–29 days and each day for the past 30 days. Participants who answered 0 days were classified as non-smokers, while the rest were considered as smokers.
After answering the survey, participants provided a 50 mL urine sample in a vial, following protocols to avoid dilution, contamination or sample borrowing between students. Vials were kept at 1 to 7°C temperature and transported to the Tobacco Compounds Analytic Laboratory from the National Institute of Public Health in Mexico City, where they were frozen at −30°C until analysed. Concentrations of urinary cotinine were quantified using gas chromatography following a standard protocol.22 Two laboratory technicians were involved in the quantitation. Urine samples were not linked to any specific individual data, so laboratory technicians were blind to the smoking status of the participants. The detection limit was 16.09 ng/mL; internal and external quality controls (blanks, duplicates, spikes) were set and evaluated according to laboratory procedures. Urinary cotinine constituted the gold standard to establish tobacco use; a 50 ng/mL cut-off was used as recommended by the Nicotine and Tobacco Research Society.23–25
On the basis of a literature review, three conditions were considered relevant to evaluate the stability of the sensitivity and specificity of self-report in this population:
Social acceptability of smoking. One of the known factors that influence the accuracy of self-report is social restrictiveness towards smoking.5 ,26 We hypothesised that adolescents closely interacting with smokers were more likely to experience an environment acceptant of smoking and would have less pressure to conceal their smoking habits, increasing the sensitivity of their self-report relative to those for whom smoking is less socially acceptable. To test this hypothesis, we obtained validity indices stratifying adolescents by the smoking status of their parents using the question “Do your parents, step-father, step-mother or legal tutor smoke?” (smoking/non-smoking, at least one parent) or by the smoking status of friends using the question “Do any of your best friends smoke?” (smoking/non-smoking, at least one friend).
Influence of smoking frequency. The half-life of cotinine is 19 h,27 which offsets the cotinine detection window in relation to self-reported use (past 30 days). Thus, the sensitivity and specificity of self-report would increase with smoking frequency. Assuming that the probability of smoking detection by cotinine increases as the adolescent smokes more frequently, we evaluated whether the self-report sensitivity and specificity increased along with the frequency of smoking. Smoking adolescents were classified in groups of frequency of smoking (1–2, 3–5, 6–9, 10–19, 20–29 days and daily) and sensitivity and specificity were estimated comparing each group to those reporting 0 days of use in the past 30 days.
Influence of the cut-off point. A receiver operational characteristics curve (ROC) was used to confirm the discrimination capability of the 50 ng/mL cut-off point in Mexican adolescents compared to the 30 ng/mL cut-off point proposed for Brazilian adolescents.28
The characteristics of the sample were described using frequencies and percentages for categorical variables, and median and percentiles for cotinine concentrations. Differences in cotinine concentrations across subgroups were explored using the non-parametric Mann-Whitney and Kruskal-Wallis tests. Respondents were excluded from the analysis if they had missing data or refused to give a urine sample (n=341) (figure 1).
The overall validity of self-report was first established for the whole population and stratified by gender. The validity indicators used were sensitivity (proportion of positive urinary cotinine respondents who reported smoking), specificity (proportion of negative urine cotinine respondents who reported not smoking), positive predictive value (probability of being a smoker if you have a positive test) and negative predictive value (probability of being a non-smoker if you have a negative test). Additionally, we calculated the positive likelihood ratio defined as the probability of a positive outcome in adolescents who smoke over the likelihood of a positive outcome in adolescents who do not smoke. An ROC curve was built evaluating the sensitivity and 1-specificity for self-reported smoking at different cut-off points of the continuous urinary cotinine concentration; sensitivity, specificity and area under the curve were estimated. CIs were calculated at 95% for all indicators. All analyses were conducted in STATA 12.0 (College Station, Texas, USA).
From a total of 2290 students in 61 selected classrooms from 26 schools (figure 1), 1657 (72.3%) responded to the survey; of them, 75.8% provided urine sample and had complete data to be included in the analyses. There were no statistical differences between providers and non-providers of the urine sample in terms of age or smoking (see online supplementary table S1); a smaller proportion of females were observed among students who provided the urine sample (among urine providers, 51.4% were women, compared to 57.7% of non-providers).
Table 1 describes the characteristics of students who both responded to the questionnaire and provided a urine sample. Of them, 51.4% were women and 72.4% were aged 13–15 years. The majority of adolescents reported not having smoked in the past 30 days (79.1%), being similar for men (78.2%) and women (79.9%). Among smokers, most smoked one cigarette per day (73.4%), followed by those who smoked two to five cigarettes per day (21.7%) and more than six cigarettes per day (4.9%). As for the frequency of smoking, most smokers reported smoking 1–2 days per month (51%) or 3–5 days per month (19.8%), and 6.5% smoked daily. There was no statistically significant difference in the frequency of use of smoking between men and women.
Table 2 shows the distribution of cotinine concentrations. Adolescents reporting smoking in the past 30 days were more likely to have detectable concentrations of cotinine (48.7%) and higher median cotinine (25.4 ng/mL) than non-smoking adolescents (29.7% detectable, median 20.5 ng/mL). Among adolescents with detectable concentrations, median cotinine increased linearly with smoking frequency (number of days smoking per month) and age, but not with smoking intensity (number of cigarettes smoked per day).
Table 3 presents the results for the “During the past 30 days (one month), on how many days did you smoke cigarettes?” question, finding 93.2% sensitivity and 80.3% specificity, with an area below the ROC curve of 0.874 (raw data for estimation available in online supplementary table S2). Stratifying by gender the self-reported use in the past 30 days had 95.5% sensitivity for men and 90% for women, while specificities were 81% for men and 82.4% for women. The negative predictive value was 99.7% for the whole sample, 99.6% for women and 99.8% for men. The maximum positive likelihood was 1.8, with a 0.80 r for the whole population.
The influence of social acceptability of smoking over self-report sensitivity and specificity is shown in figure 2. Adolescents with smoking parents exhibited 96.6% sensitivity compared to 86.7% among adolescents with non-smoking parents; specificity was 78.8% for adolescents with smoking parents and 84.3% for those with non-smoking parents. A sensitivity of 92.7% was observed among adolescents with smoking friends, while adolescents with non-smoking friends had 100% sensitivity; specificity was 75.3% for adolescents with smoking friends and 96.2% for the group with non-smoking friends.
The effect of the window of detection of cotinine over sensitivity and specificity of self-report is presented in online supplementary table S2. The sensitivity and specificity of those smoking 1 or 2 days per month was 62.5% and 99.6%, respectively, increasing to 66.7% and 99.4% among those who smoked 20 to 29 days per month, and to 81% and 99.6% among daily smokers.
The evaluation of the cut-off point showed that the 50 ng/mL cut-off point produced the largest area under the curve (87.4%, see online supplementary table S3). The 30 ng/mL cut-off point produced 76% sensitivity and 82.6% specificity, with a positive credibility of 5.0.
This study aimed to assess the validity of self-reported smoking in the Mexican school-enrolled adolescent population, considering differences by gender and social restrictiveness towards smoking. The standard question used worldwide to classify youth as current smokers (ie, “During the past 30 days (one month), on how many days did you smoke cigarettes?”) had 93.2% sensitivity and 81.7% specificity, which were similar for males and females.
These results are consistent with other studies that have used urinary cotinine with the same cut-off point.6 ,29 Using the 50 ng/mL cut-off in Canadian adolescents, Wong estimated an 81.6% sensitivity and a 96.9% specificity,29 while Park and Kim6 observed an 85.7% sensitivity and a 100% specificity in Korea. In 13 and 14-year old Brazilian teenagers, Malcon et al28 found a 22.6% sensitivity and a 93.7% specificity; however, even though the Brazilian study assessed urinary cotinine, investigators used the 30 ng/mL cut-off commonly used for saliva samples, which could explain the poor performance of self-report. Although the selected cut-off point of urine cotinine for discriminating use was obtained under near experimental conditions,30 diverse studies have suggested that the selection of the cut-off point of 50 ng/mL may not be adequate for all populations.16 In our population, the best cut-off point was found at 50 ng/mL.
Self-reported smoking behaviour can underestimate true smoking prevalence and quantity due to socially desirable responding.26 ,31 ,32 The main validity threat to self-reported smoking in adolescents is the higher probability of under-reporting smoking behaviour when survey administration occurs in a context of social disapproval of smoking.31 ,33 ,34 To test this, our adolescent population was stratified by parental smoking status and, as expected, we found a lower sensitivity among students from families where parents did not smoke compared to those where parents smoked (86.7% and 96.6%, respectively), although specificity was more comparable across both groups (78.6% and 84.6%, respectively). When assessing the potential influence of friend smoking on validity, the effects were less notable, with similar sensitivities and specificities between teenagers with and without smoking friends.
Cotinine is considered the most precise biomarker of nicotine exposure and, by extension, tobacco use in the prior 2 days,7 being more specific and bioavailable than other biomarkers.13 ,35–37 However, cotinine half-life is 19 h, providing a short window of detection to evaluate use that occurs over longer periods of time.5 The narrow detection window seems to not affect the sensitivity, as those reaching the threshold for the cotinine test declared themselves as smokers; however, this limitation does imply a decrease in specificity (as 18.3% of cotinine-negative adolescents self-report as smokers) and, particularly, a decrease in the positive predictive value (only 15.6% of smokers were positive to the cotinine test). Consistent with this observation, the positive predictive value increased as with the frequency of smoking (see online supplementary table S2). Urinary cotinine is a valuable gold standard, being particularly suitable to evaluate recent use; yet, it has some limitations to classify occasional smokers (eg, weekly or monthly smokers). Other biological substrates, such as hair, could provide longer windows of detection, although this benefit must be weighed against the difficulties to obtain a sample, as the procedure is invasive and prone to induce selection bias.5 ,13 ,38 Cigarette smoking is not the only source of urinary cotinine,24 and exposure to ETS could influence the sensitivity and specificity of self-report. Stratifying our sample by ETS exposure at home over the past 7 days showed a higher sensitivity (96.7%) but lower specificity of self-report (77.4%) among those exposed at least one day in the previous week compared to those unexposed (sensitivity 85.7%, specificity 84.8%; data available in online supplementary table S4). This difference is explained by the differential effect of ETS exposure over urinary cotinine in smokers and non-smokers. Smokers exposed to ETS have higher concentrations of cotinine than unexposed smokers, but ETS exposure has a smaller influence over the urinary cotinine concentration of non-smokers (see online supplementary table S5); thus, ETS exposure should increase the proportion of participants over 50 ng/mL of urinary cotinine, but most of them will be self-reported smokers increasing the sensitivity at the expense of lower specificity. While this difference exists, the self-reported smoking is still acceptable for both ETS exposed and unexposed; caution should be exerted when considering this sensitivity analysis as the size of the cells becomes small due to the association between smoking and ETS exposure.
Some limitations must be mentioned regarding our findings. The validation took place in two cities with a relatively high smoking prevalence, Mexico City (21.8%) and Cuernavaca (24.4%), compared to a 14.6% nationwide estimate.17 A higher prevalence suggests higher social acceptability of smoking in the analytic sample for this study, which could potentially limit the generalisability of our findings. However, self-reported current smoking maintained adequate sensitivity and specificity across contexts with varying degrees of social acceptability, as seen in adolescents with smoking and non-smoking parents or friends. Thus, self-report should perform adequately in Mexican states with higher social restrictiveness towards smoking. The GYTS was applied in school environments, eliminating the influence of parents observed in home surveys but excluding adolescents who do not attend school.9 In Mexico, education is mandatory until high school is completed. School attendance in the age group targeted by the GYTS is high (92%) up to age 14, but then decreases (63%) for those aged 15–17 years.39 Therefore, although this validation study applies to the vast majority of Mexican teenagers, further confirmation is needed for adolescents aged 15 years and older. Finally, the prevalence of cotinine-based smoking was lower than expected (3.5% of the sample was above the 50 ng/mL cut-off point), as a reflection of the large proportion of occasional smokers among Mexican adolescents and of the self-reported smoking prevalence reduction observed between 2006 and 2011 in Mexico City and Cuernavaca; therefore, our CIs were larger than the expected ±10% defined in our sample size calculation.
Self-report of tobacco use constitutes a fundamental tool for tobacco monitoring systems and research. This study is the first among Spanish-speaking Latin American countries to assess the validity of the most commonly used method for assessing current smoking behaviour among adolescents, suggesting that this approach is valid and reasonably stable across sex and contexts with different levels of the social acceptability of smoking. Therefore, adolescent smoking self-report constitutes a valid approximation to assess smoking behaviour. The increased certainty about the validity of self-report strengthens our confidence on the GYTS-derived data for decision-making and the design of public policy aimed at tobacco control in Mexico.
Technical support for this study was provided by the Health Ministry of Mexico (Mexico City, Mexico). The authors would like to thank the Pan-American Health Organization (Mexico City, Mexico), the Centers for Disease Control (Atlanta, MA), the Health Ministry of the State of Hidalgo (Pachuca, Mexico), the Education Ministry of Mexico, the National Commission Against Addiction, the National Center for the Prevention and Control of Addictions, the National Office for Tobacco Control and the State Councils Against Addictions for their continued support in the development of this project. The authors are grateful to all research assistants, lab technicians and participants involved in this project.
Review history and Supplementary material
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Contributors MdCV-L analysed and interpreted the data, prepared a first draft of the manuscript and provided critical comments during the manuscript preparation process, approving the final manuscript. TB-G conceived and designed the study, acquired, analysed and interpreted the data and provided guidance and comments for manuscript preparation, approving the final manuscript. TB-G is responsible for the overall content. LMR-S conceived and designed the study, acquired the data and critically reviewed and commented during the manuscript preparation, approving the final manuscript. JFT conceived and designed the study, edited the early version of the manuscript and critically reviewed and commented during the manuscript preparation, approving the final manuscript. IP-B edited the early version of the manuscript, analysed the data and provided methodological and analytical guidance, critically reviewed and commented during the manuscript preparation, approving the final manuscript. EL-P and MH-A provided guidance during the design stage of the study, edited the early version of the manuscript and critically reviewed and commented during the manuscript preparation, approving the final manuscript.
Funding This work was supported by the Health Ministry of Mexico (Mexico City, Mexico) grant number 448–6617.
Competing interests None declared.
Ethics approval The Research, Ethics and Biosecurity Committee from the National Institute of Public Health (920–955).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.