Article Text
PDF
Abstract
Objective To determine whether there is evidence that mass-media campaigns can be effective in reducing illicit drug consumption and the intent to consume.
Design Systematic review of randomised and non-randomised studies.
Methods We searched four electronic databases (MEDLINE, EMBASE, ProQuest Dissertations & Theses A&I and CENTRAL) and further explored seven additional resources to obtain both published and unpublished materials. We appraised the quality of included studies using standardised tools. We carried out meta-analyses of randomised controlled trials and a pooled analysis of interrupted time-series and controlled before-and-after studies.
Results We identified 19 studies comprising 184 811 participants. Pooled analyses and narrative synthesis provided mixed evidence of effectiveness. Eight interventions evaluated with randomised controlled trials leaned towards no evidence of an effect, both on drug use (standardised mean difference (SMD) −0.02; 95% CI −0.15 to 0.12) and the intention to use drugs (SMD −0.07; 95% CI −0.19 to 0.04). Four campaigns provided some evidence of beneficial effects in preventing drug use and two interventions provided evidence of iatrogenic effects.
Conclusions Studies were considerably heterogeneous in type of mass-media intervention, outcome measures, underlying theory, comparison groups and design. Such factors can contribute to explaining the observed variability in results. Owing to the risk of adverse effects, caution is needed in disseminating mass-media campaigns tackling drug use. Large studies conducted with appropriate methodology are warranted to consolidate the evidence base.
- PUBLIC HEALTH
- Substance misuse < PSYCHIATRY
- Health policy < HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- MENTAL HEALTH
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- PUBLIC HEALTH
- Substance misuse < PSYCHIATRY
- Health policy < HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- MENTAL HEALTH
Strengths and limitations of this study
This systematic review is based on an expanded evidence base of both published and unpublished findings and aims to determine whether mass-media campaigns can be effective in preventing the use of or intention to use illicit drugs.
Pooled analyses of eight mass-media interventions provide no evidence of an effect on drug use or intention to use illicit drugs. Four interventions provide evidence of beneficial effects. Two interventions provide evidence of iatrogenic effects.
Owing to the paucity and inconsistency of available evidence, we cannot draw general conclusions as to whether mass-media interventions are effective in preventing the use of or intention to use illicit drugs.
This review provides an insight into research gaps around the impact of mass-media drug prevention interventions and can serve to highlight that new campaigns should be implemented in the framework of rigorous evaluation studies, in order to avoid dissemination of interventions that are ineffective or have unintended effects.
Introduction
Mass-media campaigns are a powerful means for disseminating health promotion messages. A wide and diverse audience can be reached through television commercials, the Internet, mobile phones, newspapers and roadside advertising hoardings. In the field of drug addiction and dependence, advertisements may contribute to shaping patterns of drug use and the intention to use drugs, as well as modifying mediators such as awareness, knowledge and attitudes about drugs.
However, ethical and economic considerations are often raised. Mass-media campaigns—unlike other health interventions—are imposed on populations that have not consented to their implementation.1 This is a considerable ethical issue in modern, person-centred public health, where taking decisions shared with the public is essential for promoting behaviour change. Second, mass-media campaigns can be very expensive, especially when implemented at the national or state level. Large-scale purchasing of public service announcement time during popular shows and broad dissemination via printed media are often accessible only to governmental institutions. For example, the first and second versions of the US Office of National Drug Control Policy's National Youth Anti-Drug Media Campaign cost 2.7 billion dollars over more than 10 years.2 Although such campaigns underwent careful evaluation, most mass-media interventions are not developed in compliance with the classical circle of public health, which consists in designing interventions based on evidence and in evaluating their impact.
A systematic review of the studies assessing media campaigns aiming to prevent use of illicit drugs can inform future strategies and help design effective campaigns. The objective of this review is to assess the effectiveness of mass-media campaigns in preventing or reducing drug use or the intention to use illicit drugs among young people.
Methods
We conducted this systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)3 statement and with the procedures specified in a previously published protocol.4 As described in detail previously,4 we systematically searched four electronic databases: MEDLINE (1966 to 29 January 2013), EMBASE (1974 to 30 January 2013), ProQuest Dissertations and Theses A&I (1861 to 3 February 2013) and CENTRAL (2013, Issue 1). Search strategies are available as supplementary files (see online supplementary appendix 1). We further explored seven additional resources to obtain both published and unpublished materials: four websites of registered studies (ie, http://www.controlled-trials.com, http://apps.who.int/trialsearch/, http://clinicaltrials.gov/, https://eudract.emea.europa.eu/), references embedded in book chapters, references included in the annual national reports written by EMCDDA national focal points and any publications recommended by prominent researchers in the field. We did not set any constraints, such as language or time, to our search.
Selection criteria
As described in detail previously,4 we considered studies involving participants under the age of 26 and evaluating mass-media campaigns explicitly aimed at influencing the use or intention to use illicit drugs.5 The following were deemed acceptable comparison groups: (1) no intervention; (2) community-based or school-based drug prevention programmes; (3) lower exposure to intervention; (4) time before exposure to intervention. We included randomised controlled trials (RCTs), cohort studies, interrupted time-series (ITS) studies and controlled before and after (CBA) studies providing evidence on drug use or intention not to use, to reduce use or to stop use of illicit drugs.
Two authors independently inspected search hits by reading titles and abstracts and assessed studies for inclusion. Any disagreement was solved by consensus. Multiple publications pertaining to the same study were collated as one single study.
Quality Appraisal
Four authors independently performed quality assessments, and any disagreement was solved by consensus. We contacted study authors whenever information was missing or unclear.
We used standardised assessment tools for each study design—details are available as supplementary materials (see online supplementary appendix 2). For RCTs, we used the Cochrane Collaboration Risk of Bias assessment tool.6 For cohort studies, we followed the Scottish Intercollegiate Guidelines Network (SIGN) Quality Criteria.7 For ITS and CBA studies, we used the tool recommended by the Cochrane Effective Practice and Organisation of Care Group.8
Statistical analysis
As for RCTs, we performed a random effects meta-analysis to estimate the pooled effect of mass-media interventions on drug use while accounting for between-study heterogeneity, as described in detail previously.4 We carried out a fixed effects meta-analysis to estimate the pooled intervention effect on intention to use drugs. We tested between-study heterogeneity using the χ2 test and the I2 statistic. A p value lower than 0.10 in the χ2 test and an I2 statistic higher than 50% suggested evidence of heterogeneity. Since most studies assessed their outcome variables with different scales, we used standardised mean difference (SMD) as the summary measure of choice. SMDs were used for both drug use and intention to use drugs and were calculated by dividing the difference in mean outcome between groups by the SD of outcome between participants.6 SMDs and their SEs were then pooled in a meta-analysis performed with RevMan.9 For two clustered RCTs,10 ,11 we inflated SEs to account for within-cluster correlations.6
We pooled the effect estimates of the Meth Project studies using mixed effects logistic regression.12–16 An ITS design was applied for estimating the differences in prevalence of methamphetamine use before and after the Meth Project intervention, adjusting for any underlying temporal trend. We fitted the following model: logit(useij)=β0+u0j+β1timei+β2agei+β3intervi+β4age×intervi, in which use was prevalence of methamphetamine use, time was a continuous variable, age and intervention were two-level categorical variables, u0j was a random intercept and we allowed log odds of methamphetamine use to vary randomly by each jth state.17 The relatively few data points did not allow exploration of more complex models, for example, the temporal trend could not be assumed to vary randomly across states.
Results
Out of 18 343 titles and abstracts, we selected 24 papers corresponding to 19 individual studies (figure 1).
PRISMA flow diagram. Adapted from a previous publication.4
Study Characteristics
Overall, 184 811 participants were included, with most studies comprising participants who were aged between 10 and 19 years (table 1). Although most studies included both boys and girls, two studies focused on girls.18 ,19 One study considered Asian-Americans as the only ethnic group eligible for inclusion,18 while the other study did not focus on any specific ethnic groups. Seventeen studies were conducted in the USA, one in the USA and Canada,19 and one in Australia.11
Characteristics of included studies
Eleven studies (58%) evaluated multicomponent interventions, 3 regarding radio/television and printed advertising,10 ,20 ,21 and 8 regarding radio and television commercials, printed advertisements and Internet advertising.2 ,12–16 ,22 ,23 Eight studies evaluated standalone interventions, four consisting in radio and television commercials,24–27 and four in Internet-based interventions.11 ,18 ,19 ,28 The included studies in this review were grounded in a wide range of underlying theories (table 1).
Comparison groups varied considerably across studies. For thirteen studies (68%), the comparison group consisted in no exposure to any intervention. Four studies compared high exposure versus low exposure to the same mass media intervention.2 ,21–23 For one study, the comparison group consisted in the standard drug education curriculum.11 One study had four study arms consisting either in another intervention or no intervention.26
Eight studies were conducted in an experimental setting by explicitly inviting participants and these studies were randomised controlled trials (RCTs).10 ,11 ,18 ,19 ,25–28 Ten studies were conducted in a field setting without explicitly inviting participants, as would usually happen with most mass-media campaigns. Of them, 2 were cohort studies,22 ,23 6 were ITS2 ,12 ,14–16 ,24 and 2 were CBA studies.13 ,20 One study had a double design as it was conducted in an experimental setting with an RCT design, and in a field setting with a cohort design.21 When specified, follow-up varied from 6 months19 ,28 to 4.7 years.22
Study quality
On the whole, the quality of the RCTs is acceptable (table 2). As described in detail previously,4 the strongest domain appears to be the risk of attrition bias and the weakest domain the risk of selection bias (unclear description of the randomisation procedure). In one paper, findings of secondary outcomes were reported only as a predictor of the primary outcome, and the paper concerned was deemed at high risk for reporting bias.19
Risk of bias of included studies
All cohort studies focused on a clear and appropriate question. Subgroup comparisons between participants and dropouts were carried out in only one study.23 The same study, however, failed to control for potential confounders.
The proportion of participants with no missing data was reported in only one controlled CBA study.20 Potential confounders were accounted for in only one ITS study2 and one CBA study.20 A formal test of trend was not performed in the five Meth Project studies.12–16 One ITS study2 and the two CBA studies13 ,20 had three or less data points, which are generally considered insufficient for drawing reliable conclusions with regard to intervention effectiveness.
Effects of mass-media campaigns
Use of illicit drugs
Experimental studies
Pooled analyses of five RCTs10 ,11 ,18 ,19 ,28 comprising n=5470 subjects showed no evidence (p=0.79) of an effect of mass-media campaigns in modifying use of illicit drugs (standardised mean difference (SMD) −0.02; 95% CI −0.15 to 0.12; figure 2 and table 3). There was some evidence (p=0.020) of heterogeneity between studies.
Main findings for use or intention to use illicit drugs
Pooling of randomised controlled trials. Adapted from a previous publication.4
The RCT part of a mixed RCT-cohort study (n=3236) found evidence of effectiveness (p=0.026) for a media-community intervention (OR 0.60; 95% CI 0.38 to 0.94; table 3).21
Field studies
Two studies found that the Office of National Drug Control Policy (ONDCP) National Youth Anti-Drug Media Campaign (first version) increased use of illicit drugs among adolescents (table 3). One study (n=3529) reported a significant increase in past year-use of marijuana (OR 1.21; 95% CI 1.19 to 1.65).22 The other study (n=2515) found some evidence (‘p<0.05’) of an iatrogenic effect among those aged 15–18 (mean change=0.144), while there was no evidence (‘p>0.05’) of an effect among those aged 13–14 (mean change=−0.022).23
The revamped version of the same ONDCP campaign, Above the Influence, was found effective in a mixed RCT-cohort study (n=3236), whose cohort part found strong evidence (p<0.001) of effectiveness (OR 0.26; 95% CI 0.19 to 0.35).21 On a similar note were the findings of an ITS study (n=130 245) which evaluated Above the Influence and found evidence of reductions in marijuana use in the past month (OR 0.67; 95% CI 0.52 to 0.87) among eighth-grade girls.2
The pooled findings of the five Meth Project studies (n=26 273) suggested no evidence of a change in past-month use of methamphetamine among subjects aged 12–17 (OR 1.16; 95% CI 0.83 to 1.61), nor among those aged 18–24 (OR 1.63; 95% CI 0.70 to 3.79) (figure 3A and table 3). There was, however, evidence (p=0.001) of a reduction in past-year use of methamphetamine among those aged 12–17 (OR 0.59; 95% CI 0.43 to 0.81), while there was no evidence of a similar effect among those aged 18–24 (OR 0.70; 95% CI 0.34 to 1.45; figure 3B and table 3).
Pooling of the meth project interrupted time-series studies: predicted and observed probabilities. Adapted from a previous publication.4
One ITS (n=6371) showed evidence of effectiveness for past 30-day use of marijuana among high sensation seekers (p=0.001 for the Fayette sample, p=0.001 for the first campaign in the Knox sample, p=0.002 for the second campaign in the Knox sample).24
One CBA study found an increase in use of LSD (‘p<0.001’; table 3) while no evidence (‘p>0.05’) of differences was found for marijuana, cocaine, amphetamine and heroin.20
Intention to use drugs
Experimental studies
In one meta-analysis of four randomised controlled studies involving 1270 participants, there was no evidence of an effect (p=0.21) of media campaigns in changing intention to use drugs (SMD −0.07; 95% CI −0.19 to 0.04; figure 2 and table 3).18 ,25–27 There was no evidence (p=0.840) of heterogeneity across studies.
Discussion
Mass-media campaigns are commonly used throughout the world to tackle a broad array of preventable risk factors or injuries. Such campaigns are seldom evaluated, thus making it difficult to inform policymakers regarding their effectiveness and sustainability. In this panorama of overall uncertainty, mass-media campaigns tackling tobacco and traffic accidents are noteworthy exceptions as they have been evaluated more frequently and have shown some evidence for benefit.29 In our attempt to summarise evidence on the effectiveness of mass-media campaigns targeting illicit drugs, we included 19 studies evaluating a number of heterogeneous interventions. We grouped interventions according to whether they were evaluated with studies conducted in experimental settings in which participants were aware of being exposed to media interventions, or were assessed with studies carried out in a field environment which are more likely to show the real-life effects of large national media campaigns, but are also more prone to risk of bias.
Findings appear to vary considerably according to the type of intervention and study design. Pooled analyses of eight interventions evaluated in an experimental setting provided no evidence of beneficial effects for use or intention to use illicit drugs, an indicator of possible future behaviour.30 ,31 Four interventions evaluated with eight field studies revealed some evidence of beneficial effects: (1) the revamped campaign by the Office of National Drug Control Policy (ONDCP) called Above the Influence, which was found effective in one study and effective among eighth-grade girls in another study; (2) the Be Under Your Own Influence media-community intervention; (3) the Meth Project campaign, which was found effective on past-year methamphetamine use, although only among adolescents aged 12–17 years; and (4) the US televised antimarijuana campaigns broadcast in Fayette County (Lexington), Kentucky and in Knox County (Knoxville), Tennessee, which were found to be effective on high-sensation seekers. Two mass media campaigns showed clear iatrogenic effects, most notably, the first version of the ONDCP's media campaign My Anti Drug, which was evaluated by two studies and was found to increase use of marijuana. An adverse effect was also found for a media-community intervention evaluated by a CBA study which provided evidence of increased frequency of LSD use.
No characteristic emerged clearly as a core feature of successful or unsuccessful campaigns, either regarding their explicit or implicit theoretical background or their communication strategies. However, it is worth noting that two out of the four interventions providing evidence of effectiveness, the ONDCP's Above the Influence national campaign and the Be Under Your Own Influence media-community intervention promoted non-use of drugs as a way to support the goals of autonomy and achievement of competence, both of which have been conceptualised as innate psychological needs that persist over the lifespan.21 Among the interventions which provided evidence of harmful effects, the first version of the ONDCP's media campaign My Anti Drug was based on a social marketing approach which emphasised resistance skills, self-efficacy, normative education and negative consequences of drug use.32 These mediators are suspected to have increased the perception of prevalence of drug use in the target population.33
An important reason for the weak evidence obtained by this review is the large variation in mass-media intervention type and study design. Similar interventions were often evaluated with different study designs while different interventions were sometimes evaluated with the same study design. Pooled analyses could thus be undertaken only for a few similar interventions evaluated with the same study design, and such small sets of pooled studies did not allow sensitivity analyses to be carried out. We did not set any time or language constraints to our search, accepted all types of controlled study designs and obtained unpublished data by establishing direct contact with the authors of the original papers. Unfortunately, owing to the paucity and inconsistency of available evidence, we cannot draw general conclusions as to whether media campaigns are effective in preventing the use or the intention to use illicit drugs. This observation is in line with the findings of similar reviews that used more restrictive inclusion criteria.29 ,34
The evidence base accrued so far on media campaigns targeting illicit drugs allows us to make at least two remarks. First, such campaigns can be evaluated—a fact that is often questioned in several parts of the world—and properly conducted evaluation studies can provide benefits to both research and practice. Second, in the worst-case scenario, media campaigns can be both ineffective and harmful. Contrary to common belief, antidrug media campaigns may be damaging and their dissemination is ethically unacceptable without a prior assessment of their effects.35 ,36 New campaigns should be implemented in the framework of rigorous evaluation studies, ideally in field settings with cohort or ITS study designs. A better understanding of which media interventions work best is likely to result in a more effective prevention of drug use and increased efficiency in the management of public resources.
Acknowledgments
The authors are grateful to the Cochrane Drug and Alcohol Group for their support in publishing the Cochrane review from which this paper has been extracted.4 They thank Gregor Burkhart and V Anna Gyarmathy for their input to the drafting of the protocol for this review, and Marie-Christine Ashby and Fozia Hamid for carefully proofreading the manuscript. The authors are grateful to Ling Fang, John Horan, Nicola Newton, Philip Palmgreen, Joan Polansky, Michael Slater, Jennifer Stagnaro, Violeta Taneva and Marco Yzer for their useful advice and for providing their studies’ unpublished data whenever possible.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
- Data supplement 2 - Online supplement
Footnotes
Note: This article is based on a Cochrane Review published in the Cochrane Database of Systematic Reviews (CDSR) 2013, Issue 6, DOI: 0.1002/14651858.CD009287.pub2 (see http://www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and the CDSR should be consulted for the most recent version of the review.
Contributors EA structured and drafted the paper. MF and FF conceived the systematic review from which the paper originates and overviewed the inclusion of studies and their methodological assessment. MF and EA selected the studies for inclusion. EA and AB extracted the data from the studies and contributed to the writing of the review. EA contacted study authors. EA and AB did the meta-analysis of randomised controlled trials. AG and EA conducted the meta-analysis of interrupted time-series studies. All authors regularly discussed each step of the review process and participated equally in each decision regarding the studies and the analysis. They also revised the paper and read and approved the final version of this manuscript.
Funding AG was supported through a Methodology Research fellowship awarded by Medical Research Council-UK (grant ID G1002296). The EMCDDA provided IT equipment and journal access for EA during his traineeship, funding for purchasing of papers not included in standard subscriptions, and funding for the open access publication of this paper.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data and statistical code used for pooled analyses are available and can be requested from the corresponding author (EA, elias.allara@med.unipmn.it).