Article Text
PDF
Abstract
Objectives To examine the volume, relevance and quality of transnational tobacco corporations’ (TTCs) evidence that standardised packaging of tobacco products ‘won't work’, following the UK government's decision to ‘wait and see’ until further evidence is available.
Design Content analysis.
Setting We analysed the evidence cited in submissions by the UK's four largest TTCs to the UK Department of Health consultation on standardised packaging in 2012.
Outcome measures The volume, relevance (subject matter) and quality (as measured by independence from industry and peer-review) of evidence cited by TTCs was compared with evidence from a systematic review of standardised packaging . Fisher's exact test was used to assess differences in the quality of TTC and systematic review evidence. 100% of the data were second-coded to validate the findings: 94.7% intercoder reliability; all differences were resolved.
Results 77/143 pieces of TTC-cited evidence were used to promote their claim that standardised packaging ‘won't work’. Of these, just 17/77 addressed standardised packaging: 14 were industry connected and none were published in peer-reviewed journals. Comparison of TTC and systematic review evidence on standardised packaging showed that the industry evidence was of significantly lower quality in terms of tobacco industry connections and peer-review (p<0.0001). The most relevant TTC evidence (on standardised packaging or packaging generally, n=26) was of significantly lower quality (p<0.0001) than the least relevant (on other topics, n=51). Across the dataset, TTC-connected evidence was significantly less likely to be published in a peer-reviewed journal (p=0.0045).
Conclusions With few exceptions, evidence cited by TTCs to promote their claim that standardised packaging ‘won't work’ lacks either policy relevance or key indicators of quality. Policymakers could use these three criteria—subject matter, independence and peer-review status—to critically assess evidence submitted to them by corporate interests via Better Regulation processes.
- PREVENTIVE MEDICINE
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/
Statistics from Altmetric.com
Strengths and limitations of the study
-
This study builds on the existing literature on corporate influence over public health policy to demonstrate the challenges policymakers face in assessing evidence submitted via public consultations.
-
Further investigation of policymakers’ perceptions of corporate evidence would be beneficial to corroborate the pertinence of our recommendations.
-
Indicators were used to assess quality as these would be suited to the practical demands of the policymaking context.
Introduction
Standardised packaging of tobacco products entails the prohibition of logos, brand imagery, symbols, other images, colours and promotional text from tobacco products and tobacco product packaging. Despite the common use of the term ‘plain packaging’ in media coverage of this issue, graphic and textual health warning labels would still feature prominently on packs and key anticounterfeiting marks would be retained.
Standardised packaging would further restrict the already limited opportunities of transnational tobacco companies (TTCs) to market their products. The policy's objective is to deter smoking initiation, particularly among young people, and promote cessation among existing smokers. Its introduction in Australia in December 20121 sparked a wave of interest: Ireland and New Zealand gave strong indications of their intentions to introduce standardised packaging. In contrast, the UK government announced on 12 July 2013 that it had decided to wait for ‘emerging evidence’ from Australia on the impacts of standardised packaging before taking a policy decision. This announcement followed a lengthy debate which began in 2011,2 included a 4-month public consultation ending in August 2012,3 and was subject to nearly a year's deliberation within the Department of Health before the consultation report was published.4 The consultation aimed to inform policy development and gather additional evidence for an impact assessment. The impact assessment was rated amber (needing more work) by the Regulatory Policy Committee (RPC) in February 2012.5
Public consultations and impact assessments are processes within a global governance innovation termed Better Regulation (also known as Smart regulation or better lawmaking). Drawing heavily on American Administrative Law and the cost-benefit approach to regulatory review in the USA,6 ,7 versions of Better Regulation are in place, for example, in multiple European Union (EU) states (UK, the Netherlands, Czech Republic, Sweden and Germany),8 in the EU itself9 and in Canada10 and Australia.11 A key impetus for Better Regulation has been the pressure put on governments and inter-governmental organisations by TTCs and other transnational corporations to reduce regulatory business costs and prioritise business interests in the policy process.8 ,12
Public consultations effectively frontload problem-resolution in the policy process by offering affected businesses and other interested parties an early opportunity to comment on policy ideas and proposals, and to submit evidence supporting their views.13 ,14 Examples of consultation systems elsewhere include ‘notice and comment’ in the USA15 and the European Commission's ‘Your voice in Europe’.16 Evidence gathered from consultations can then be taken into account in developing impact assessments, which entail quantitative evidence-based assessments of the potential effects of proposed regulations and consideration of alternative policy options.17 Evidence plays a key role in the policy process, and, in practice, Better Regulation underpins a form of evidence-based policymaking which is deliberately open to stakeholder, and particularly business, influence.18 ,19
In the UK, these processes contribute to the attainment of five features of good governance: proportionality, accountability, consistency, transparency and targeting.14 ,20 Under New Labour18 and the Coalition Government,8 Better Regulation has formalised evidence-based policymaking, which is now subject to two stages of scrutiny: first, by the RPC (a body sponsored by the Department of Business Innovation and Skills); and, subsequently, by the Cabinet's Reducing Regulation Committee (RRC). The upfront costs to the government of this process are intended to be offset by an associated reduced impact on businesses post-implementation.
New tobacco control policies developed by the Department of Health are subject to Better Regulation. Thus, TTCs can make submissions to public consultations on tobacco control policies, citing evidence regarding impacts on their businesses, wider impacts and in support of alternative policies. Yet, the government is also required to meet an international commitment made under Article 5.3 of the Framework Convention on Tobacco Control (FCTC) to take steps to ensure that: “…in setting and implementing their public health policies with respect to tobacco control, Parties shall act to protect these policies from commercial and other vested interests of the tobacco industry in accordance with national law.”21 A key rationale for this provision lies in the overwhelming evidence of the tobacco industry's efforts to bias the evidence base of health impacts of tobacco products and public health policies in its favour.22–25 Uniquely in the case of tobacco, the coexistence of these two governance regimes raises the possibility of a regulatory conflict between commitments to include businesses in policy development and commitments to protect public health policies from them.26
The UK's consultation3 and impact assessment5 on standardised packaging provides an opportunity to consider how these two sets of commitments are reconciled by governments. The four largest TTCs in the UK market—Imperial Tobacco (IT), Japan Tobacco International (JTI), Philip Morris Ltd (PM) and British American Tobacco (BAT)—submitted lengthy consultation responses (1521 pages in total, of which 328 comprised their main responses and 1193 provided supplementary materials).27–30 These were just 4 of 668 433 responses the Department of Health received (2444 were ‘detailed responses’).4 Associated time and resource costs raise the question of how governments can effectively make a balanced, informed and transparent assessment of the policy relevance and the quality of all evidence cited in submissions.31
A systematic review of the evidence for standardised packaging, commissioned by the Department of Health, concluded that there is ‘strong evidence’ that standardised packaging would reduce the appeal of tobacco products and increase the prominence of health warnings.32 However, in their submissions, TTCs rejected the findings of the systematic review on the grounds that there is no evidence that standardised packaging would reduce smoking prevalence or initiation.27–30 They cited extensive evidence to support their arguments, claimed that key evidence on smoking behaviour had not been considered in the systematic review, and pointed to the absence of real-world evidence as problematic: the UK consultation preceded implementation of standardised packaging in Australia in December 2012. The TTCs have maintained that advertising and promotional material—including packaging—stimulate only brand switching among current smokers.27 ,29 ,30 ,33 Yet, overwhelming evidence from the tobacco industry's own marketing documents suggests this claim is highly disingenuous.34–37
In this article, we aim to examine the volume, policy relevance and quality of the evidence TTCs cited in their submissions and compare it with that included in the systematic review (further work is underway to investigate TTCs’ interpretation of the evidence itself). We use this analysis to explore the challenges public consultations and impact assessments for tobacco control policies present to governments and begin to unpack the conflict between the Better Regulation agenda and the FCTC. We suggest evidential management strategies for governments developing tobacco control policies in this multilevel governance context.
Methods
Defining ‘evidence’
The comparative analysis methodology employed in this research required that ‘evidence’ was interpreted narrowly as formal written research sources, such as reports or journal articles. This restriction enabled a comparison of similar evidence in the two datasets: TTC citations and systematic review evidence. Other forms of evidence (eg, opinion, political statements, legal rulings and press coverage) cited in the four TTCs’ submissions were excluded.
Selecting and recording TTC evidence
Details (author, title, date and source) of each piece of evidence cited by TTCs in their submissions were extracted into an Excel spreadsheet and categorised under three main arguments made by TTCs: there is no evidence of the beneficial impact of standardised packaging on public health—standardised packaging ‘won't work’; standardised packaging will have negative unintended consequences (including economic impacts on businesses, growth in illicit trade in tobacco products, reduction in the price of cigarettes or contravention of existing trade and intellectual property rules); and the policy process was ‘flawed’. Evidence was also categorised as to whether it was promoted by TTCs as supporting their argument, or contested by them. Only evidence used by TTCs to promote their argument that standardised packaging ‘won't work’ was obtained and examined further.
Evidence from the systematic review
Details of the studies cited in the systematic review were recorded in Excel and were used for further analysis of their relevance and quality.
Criteria for assessing evidence
Three criteria, identified via a review of similar studies of the quality of evidence used to oppose tobacco regulation,24 ,38–42 were used to assess the policy relevance and quality of the evidence: subject matter, independence and peer-review (table 1). These criteria represent an objective and practical means for policymakers to assess the policy relevance and quality of large quantities of evidence cited in submissions to public consultations prior to considering their content.
Coding framework for classifying evidence
The subject matter of the evidence speaks to its relevance to the policy issue.31 Similar work has coded policy position, argument, topic and conclusion.40–42 On independence, research indicates that connection of research with a financially vested interest group can produce results which favour the sponsor, casting doubt on the independence, and therefore quality, of the evidence.40 ,43 ,44 The tobacco industry's efforts to discredit the science on environmental tobacco smoke and bias evidence on the impacts of smoke-free legislation provide historical examples of this.22–25 ,45 On peer-review status, articles which appear in peer-reviewed journals have been shown to be of superior quality to other research outputs in terms of study design, reporting and interpretation.40 ,46 For example, peer-review enables studies to be assessed by experts who are knowledgeable in the subject area, provides strong incentives for authors to heed advice and improve articles and acts as a filter which aims to prevent poorly designed studies from being published. Some alternative publication routes also include external peer-review (eg, government-commissioned research, academic press volumes and conference papers); others rely on internal peer-review (eg, charity and university research reports); research funded and published by the tobacco industry tends not to be subject to an external peer-review process: “[T]he tobacco industry has had a long-standing strategy of funding research and disseminating it through their sponsored, non-peer-reviewed publications.”47 ,48
Evidential coding
Each piece of evidence was obtained via online searches (general search engines and the research database Scopus), with non-digital documents obtained from library sources. Researchers read abstracts, introductions, conclusions, funding statements and cover pages of all evidence documents, and searched documents for key terms (‘plain’, ‘pack*’, ‘standard*’, ‘tobacco’, ‘smok*’). Additional web searches were conducted (eg, the Legacy Library of tobacco industry documents and Scopus) to clarify the independence and peer-review status of evidence.
Analytical process
The researchers used a content analysis methodology to code and analyse the data. Each piece of evidence was accessed and coded by one researcher (JLH) using the criteria outlined in table 1. A second researcher (GJF) blind-coded a random sample of 20% of the data (n=21). This process achieved a 97% level of inter-coder reliability. Once all the data had been analysed, a third researcher (KAE-R) blind-coded 100% of the data. This process achieved a 94.7% level of inter-coder reliability. All disagreements were fully resolved between the coders.
Having quantified and coded the evidence, we compared the policy relevance (subject matter) and quality (independence and peer-review) of the industry evidence with that of the evidence supporting standardised packaging in the systematic review.32 We also examined the relationship between policy relevance and quality by comparing the quality of the industry's evidence on tobacco packaging with its evidence on other topics. Differences were compared using a two-tailed Fisher's exact test. The results were used to develop relevance-quality typologies of the TTC evidence. Evidence was classified as relevant if it focused on standardised packaging/tobacco packaging, and parallel if it focused on other tobacco issues/was unrelated to tobacco. Evidence was classified as featuring quality indicators if it was either independent, published in a peer-reviewed journal or both.
Results
Overview of evidence cited by the TTCs in their submissions
One hundred and forty-three unique pieces of formal written research evidence were referred to or included in the four TTCs’ submissions (22 referenced by more than one company; table 2). Of the 143 documents, TTCs promoted 131 as supporting their arguments and contested the methods, findings or accessibility of the remaining 12, all of which were included in the systematic review. Eighty-eight pieces of evidence were cited to support arguments that standardised packaging would not have beneficial impacts on public health; 36 were cited to argue that standardised packaging will have negative unintended consequences, half of which related to the illicit trade in tobacco; 19 were cited to argue that the policy process—particularly the impact assessment—was ‘flawed’. Seventy-seven pieces of evidence were used to promote the TTCs’ argument that standardised packaging ‘won't work’ and were therefore the subject of further analysis in this article.
Overview of formal written evidence cited by transnational tobacco corporations (TTCs) in their submissions to the UK standardised packaging consultation 2012
Among these 77 documents, TTCs did not cite any research showing that the tobacco industry has extensively studied and holds considerable evidence attesting to the impact of packaging on smoking behaviour.34–37 Instead, they cited industry-funded research which critiqued the systematic review articles, the impact assessment and the consultation document. And they cited a body of independent research into the drivers of youth smoking which, while published in peer-reviewed health and psychology journals with no apparent connection to the tobacco industry, did not explicitly address the role of packaging in youth uptake or prevalence.
Comparison of the TTC and systematic review evidence on the impact of standardised packaging on smoking behaviour
There are marked differences between the relevance and quality of the TTC and systematic review evidence (table 3 and figure 1). Only 17/77 (22%) pieces of evidence promoted by TTCs addressed standardised packaging directly; the majority of these were industry-funded/linked (14/17, 82%) and none were published in a peer-reviewed journal (0/17, 0%). The remaining 60 pieces of evidence (78% of the total, comprising the majority of the evidence the industry cites) did not address standardised packaging. In contrast, 37/37 (100%) pieces of evidence included in the systematic review focused on standardised packaging, none (0/37, 0%) had a connection with the tobacco industry, and 21/37 (57%) pieces of evidence were published in a peer-reviewed journal. A comparison of TTCs’ evidence on standardised packaging (n=17) with the systematic review evidence on standardised packaging (n=37), using Fisher's exact test, showed that the former was of significantly lower quality in terms of independence (p<0.0001) and peer-review status (p<0.0001).
Quality and relevance of transnational tobacco corporation (TTC) and systematic review evidence
Comparison of quality (independence and publication route) of the systematic review and transnational tobacco corporation evidence directly addressing standardised packaging of tobacco products.
Relationships between subject matter, independence and peer-review within the TTC evidence
A low proportion of TTCs’ evidence relating to standardised packaging was independent or peer-reviewed (figure 1 and table 3). When evidence on tobacco packaging was added to the standardised packaging evidence, the same pattern was found: 9/26 (35%) Pieces of evidence were independent (independent/no apparent tobacco industry connection); 1/26 (4%) pieces of evidence was published in a peer-reviewed journal (table 4) However, a greater proportion of the 51 pieces of evidence TTCs cited on parallel topics (including non-packaging drivers of youth and adult smoking behaviour, and drivers of youth behaviour in general) were independent (47/51, 92% independent/no apparent connection) and peer-reviewed (30/51, 59% published in a peer-reviewed journal). These differences are statistically significant (p<0.0001, table 4). We also found a clear relationship between the two indicators of quality—independence and peer-review—in the TTCs’ evidence: industry-funded/linked studies cited by TTCs were significantly less likely to be published in a peer-reviewed journal (3/21, 14%) than independent/no apparent connection studies they cited (28/56, 50%; p=0.0045, table 5).
Relationship between the policy relevance and two indicators of quality in the transnational tobacco corporations' evidence (n=77), number and per cent in parenthesis
Relationship between two indicators of quality in the transnational tobacco corporation evidence, number and per cent in parenthesis
TTCs’ evidence was classified into four typologies (table 6): relevant/quality indicators, relevant/no quality indicators, parallel/quality indicators and parallel/no quality indicators. While 100% of the systematic review evidence was relevant and featured at least one of the two quality indicators, only 12% of evidence cited by TTCs in their submissions qualified for this category.
Distribution of transnational tobacco corporation evidence across typologies
Discussion
Four main findings are apparent. TTCs cited a large volume of evidence in their submissions to the UK standardised packaging consultation. Seventy-seven pieces of evidence were used to support the claim that standardised packaging ‘won't work’ yet just 17 of these actually focused on standardised packaging, 14 of which had a connection with the industry. The quality of the TTCs' evidence on standardised packaging is significantly lower, as judged by independence and peer-review, than that included in the systematic review. Overall, the evidence cited by TTCs is shown, with few exceptions, to fit one of two typologies—either relevant/no quality indicators or parallel/quality indicators (table 6). Furthermore, we show that evidence funded by or otherwise linked to industry is significantly less likely to be peer-reviewed than non-industry-connected evidence.
These findings raise a number of concerns regarding the potential impact of Better Regulation on tobacco control policymaking in jurisdictions around the world. First, our findings highlight how Better Regulation, with its requirement for public consultations and impact assessments, imposes costs on government departments in the earliest stage of policy development. Just as TTCs habitually launch legal challenges in the postdecision phase of policy-making,49 so too can they use their resource advantage to exploit Better Regulation processes by commissioning new research and submitting extensive and complex responses in the predecision phase of the policy process, effectively frontloading their opposition. The combination of a requirement for due diligence and the volume and nature of responses may have contributed to the 11-month delay in the publication of the Department of Health's consultation report.
Second, Better Regulation's requirement that policymakers consider alternative policy options, with its underlying intention of preventing unnecessary regulation, imposes additional upfront costs on governments. In the case of standardised packaging, this requirement encouraged the extensive citation of evidence beyond the focus of the policy proposal. This may partly explain why nearly two-thirds of the evidence TTCs cited to claim that standardised packaging ‘won't work’ addressed non-packaging drivers of youth and adult smoking: studies which do not consider standardised packaging in their methodology or analysis. A second possible explanation is that the level of independence and peer-review of this parallel evidence is significantly higher than that of the evidence they cite on tobacco packaging, and its inclusion may therefore have been intended to add legitimacy to the TTCs' arguments.
Third, the absence of guidelines requiring a declaration of any conflict of interest between corporations and the evidence they cite enable tobacco industry-funded/linked work to be cited by TTCs in such a way that any link is undeclared, implying independence. For example, BAT, IT and PM all cited tobacco industry-funded/linked evidence in their submissions without explicitly acknowledging their connection with it.27 ,29 ,30 This speaks to a lack of transparency in the policy process regarding the provenance of evidence submitted by corporate interests.
Fourth, the lack of clarity regarding whether or how civil servants assess the policy relevance and quality of evidence is reflected by an equivalent lack of clarity regarding how governments handle the absence of evidence. We have identified a clear omission in the TTCs’ submissions of evidence, which TTCs are known to have,34–37 on the importance of packaging in the marketing of their products.
Taken collectively, evidence present in and absent from TTCs’ submissions highlights an important transparency deficit within Better Regulation processes. This deficit obscures the view of policymakers, potentially preventing them from identifying and taking account of the judicious selection and exclusion of evidence by corporate actors with vested interests in policy outcomes. Because Better Regulation requires evidence-based impact assessments and invites evidence-based submissions to public consultations, the potential exists for corporations to exert undue and unnoticed influence on the policy process. Recently leaked industry documents outlining Philip Morris International's strategy for ensuring standardised packaging is not adopted in the UK indicate that the TTCs had identified and planned to use the opportunities presented by Better Regulation in making its arguments.50 ,51
Considering the statutory requirement imposed by Article 5.3 of the FCTC to ‘protect’ tobacco control policies ‘from commercial and other vested interests of the tobacco industry’,21 it would be advisable for the 177 states which are party to the Convention to implement and publish clear guidelines on how TTC submissions to public consultations, and evidence cited within, should be managed by policymakers. Two steps could be taken by governments to achieve this. First, conflict of interest declarations regarding evidence cited could be made a mandatory element of public consultations. Second, policymakers could adopt a similar methodology to that used in this research. Adopting a process of classifying evidence for subject matter, independence and peer-review status may help policymakers to systematically prioritise good quality and policy-focused evidence, and to flag evidence about which they need to be more sceptical, such as that which is not policy-focused, not independent or not peer-reviewed.
These recommendations have relevance across government departments in all states which are signatories to the FCTC. It would also be appropriate to explore applying this critical perspective to the development of non-tobacco public health regulation—for example, of the alcohol and food industries—where corporate interests also seek to influence policy being developed for the public good.52 ,53
The strength of the findings is limited by the use of indicators of quality, rather than a validated quality-assessment framework, to assess the evidence. Peer-review status and independence from the tobacco industry are used as proxy indicators of quality. While we acknowledge that peer-review standards can and do vary in practice,54–57 our rationale for choosing these proxies is based on our interest in addressing the challenges policymakers face in assessing large volumes of evidence. Unlike quality assessment tools, the criteria we have selected do not require scientific expertise or lengthy data extraction processes and can be used systematically in the policy environment to assess the relevance and quality of evidence cited by consultation respondents. Where the need is identified, more in-depth analysis of study design, data and methods38–40 can be undertaken to review key pieces of evidence.
What has been learned from the UK Government's 2013 decision to postpone any decision on standardised packaging until further evidence is available is that Better Regulation ensures that evidence occupies a critical instrumental role in policymaking. Thus, how government departments handle and interpret evidence in the development of public health policy, and what evidential relevance and quality thresholds are set for policy progression in the context of Better Regulation, are of vital importance.
Acknowledgments
The authors would like to thank Dr Michelle Sims for her assistance in reviewing the statistical content of this paper. Funding from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council, Medical Research Council and the National Institute for Health Research, under the auspices of the UK Clinical Research Collaboration, is gratefully acknowledged.
References
Supplementary materials
Press release
Press release
Files in this Data Supplement:
Footnotes
-
Contributors JLH, GJF and ABG conceived the idea for this study; ABG conceived the idea for the statistical analysis. JLH was responsible for data gathering, coding, analysis and write-up. GJF and KAE-R were responsible for secondary coding. JLH, GJF, KAE-R, SU and ABG were involved in discussion of findings and commented on and edited drafts of the article.
-
Funding The work was supported by Cancer Research UK (Grant no. C38058/A15664).
-
Competing interests ABG is a member (unpaid) of the Council of Action on Smoking and Health, and was a member of the WHO Expert Committee convened to develop recommendations on how to address tobacco industry interference with tobacco control policy. JLH, GJF, KAE-R, SU and ABG are part of the UK Centre for Tobacco and Alcohol Studies (Grant no. MR/K023195/1), a UKCRC Public Health Research Centre of Excellence. KAE-R is supported by Cancer Research UK (Grant no. C48078/A16481). GJF, SU and ABG are supported by the National Cancer Institute at the National Institutes of Health (Grant no. RO1CA160695).
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement No additional data are available.