Article Text
Abstract
Objective To assess if different forms of regulation lead to differences in the quality of journal advertisements.
Design Cross-sectional study.
Participants Thirty advertisements from family practice journals published from 2013 to 2015 were extracted for three countries with distinct regulatory pharmaceutical promotion systems: Australia, Canada and the USA.
Primary and secondary outcome measures Advertisements under each regulatory system were compared concerning three domains: information included in the advertisement, references to scientific evidence and pictorial appeals and portrayals. An overall ranking for advertisement quality among countries was determined using the first two domains as the information assessed has been associated with more appropriate prescribing.
Results Advertisements varied significantly for number of claims with quantitative benefit (Australia: 0.0 (0.0–3.0); Canada: 0.0 (0.0–5.0); USA: 1.0 (0.0–6.0); p=0.01); statistical method used in reporting benefit (relative risk reduction, absolute risk reduction and number needed to treat; Australia: 6.7%, n=2; Canada: 10.0%, n=3; USA: 36.6%, n=11; p=0.02); mention of adverse effects, warnings or contraindications (Australia: 13.3%, n=4; Canada: 23.3%, n=7; USA: 53.3%, n=16; p=0.002); equal prominence between safety and benefit information (Australia: 25.0%, n=1; Canada: 28.6%, n=2; USA: 75.0%, n=12; p=0.04); and methodological quality of references score (Australia: 0.4150 (0.25–0.70); Canada: 0.25 (0.00–0.63); USA: 0.25 (0.00–0.75); p<0.001). The USA ranked first, Canada second and Australia third for overall quality of journal advertisements. Significant differences for humour appeals (Australia: 3.3%, n=1; Canada: 13.3%, n=4; USA: 26.7%, n=8; p=0.04), positive emotional appeals (Australia: 26.7%, n=8; Canada: 60.0%, n=18; USA: 50.0%, n=15; p=0.03), social approval portrayals (Australia: 0.0%, n=0; Canada: 0.0%, n=0; USA: 10.0%, n=3; p=0.04) and lifestyle or work portrayals (Australia: 43.3%, n=13; Canada: 50.0%, n=15; USA: 76.7%, n=23; p=0.02) were found among countries.
Conclusions Different regulatory systems influence journal advertisement quality concerning all measured domains. However, differences may also be attributed to other regulatory, legal, cultural or health system factors unique to each country.
- health policy
- quality in health care
- primary care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The information assessed from ads is associated with more appropriate prescribing.
All information was abstracted by two independent authors and disagreements were resolved through consensus or a third author if consensus could not be reached.
The accuracy of information in ads was not assessed.
The effect of ads on prescribing was not assessed.
Other regulatory, legal, cultural or health system factors unique to each country were not controlled for which may also account for differences in the quality of advertisements.
Introduction
Journal advertising in medical journals is a ubiquitous form of drug promotion, although it only represents a small fraction of total promotional spending. Figures for the USA from 2012 show that medical journal advertising cost companies US$90 million out of a total promotional budget of US$27 billion (0.3%).1 The bulk of the budget, US$15 billion, is primarily dedicated to detailing efforts. Canadian data for 2016 are equally skewed in favour of detailing over journal advertising—US$408.9 million for the former compared with US$12.5 million for the latter.2
However, according to a study published in Medical Marketing & Media ‘advertising magnifies the detailing effort at a fraction of detailing expense. In effect, detailing provides the power in the marketing effort and advertising provides the efficiencies’.3 For every dollar spent on medical journal advertisements during the first 4 years, drugs are on the market in the USA, the return on investment (ROI) was US$2.43; after this time, ROI increased to over US$4.00. In addition, advertising magnifies the effects of detailing, increasing the ROI from detailing 75% of the time by 30%–40%.3 Neslin claimed that journal advertising generated the highest ROI of all promotional strategies, ranging from US$2.22 to US$6.86 per advertising dollar spent.4
Journal advertisements are directly influenced by the standards and approaches to regulation in the jurisdiction in which they appear; however, it is unclear how this affects the quality of advertisements. One previous study examined journal advertisements in different countries and concluded that the quality of advertisements, as measured by six characteristics including the relative frequency and size of the generic and trade names and the amount of space allocated to indications and safety information, was affected by the method of regulation; however, it analysed only advertisements published between 1961 and 1977.5 More recent literature has compared drug advertisements in different countries but did not explicitly assess approaches to regulation.6 7 Given that drug promotion has an established effect on physician prescribing practices,8 it is essential to examine how current regulations affect the quality of journal advertisements.
Three methods of regulating medical journal advertising have evolved in developed countries: direct government control (eg, the Food and Drug Administration (FDA) in the USA),9 industry self-regulation (eg, in Australia and New Zealand)10 and regulation by a multistakeholder body (eg, the Pharmaceutical Advertising Advisory Board in Canada, table 1).11 Of note, in Australia, the industry code must be approved by the Australian Competition and Consumer Commission. Despite differences in details in the requirements in the regulations in each country, the overall goals in each country with respect to how advertisements should portray the benefits and harms of the medicines are broadly similar:
Australia: ‘The content of all promotional material provided to healthcare professionals must be current, accurate, balanced and fully supported by the Australian Approved Product Information’.10
Canada: ‘PAAB ensures that any information provided about a product is evidence-based and that there is a balance between claims about benefits and possible risks’.11
USA: ‘Product claim ads must present the benefits and risks of a prescription drug in a balanced fashion’.9
The objective of this study is to examine the quality of advertisements in Australia, Canada and the USA to determine if different forms of regulation lead to differences in the quality of the advertisements. Based on previous literature describing the failure of voluntary industry regulation,12 13 our a priori assumption was that advertisements produced under a self-regulatory system (Australia) will be of inferior quality compared with ads produced under the other two systems (Canada and the USA).
Methods
This was a cross-sectional study of medical journal advertisements from Australia, Canada and the USA.
Selection criteria and method of choosing ads
We applied selection criteria for ads for prescription medicines that controlled for as much variability as possible, aside from the type of regulatory control that they are subject to. Table 2 lists the inclusion criteria. We selected ads with both text and images from the same type of journal, targeted at the same audience and published in the same years. Ads came from family practice journals (American Family Physician, Australian Family Physician and Canadian Family Physician) from 2014 to 2015. Family practice journals generally have a greater number of ads and advertise a wider range of drugs compared with specialty journals. Of the ads that met inclusion criteria, we used a random number generator to select 15 ads from each journal in each of the 2 years. Journals were accessed through the library system at the University of Toronto. Ads were scanned, and the electronic versions were used for evaluation.
Evaluation components of ads
For each ad, we recorded the country where it appeared, year, brand and generic name of the drug, manufacturer and the number of pages in the journal that the ad occupied. We recorded the therapeutic category for each drug by using the WHO Anatomic Therapeutic Chemical (ATC)/Defined Daily Dosage Index at the second level (https://www.whocc.no/atc_ddd_index/) to examine whether the drugs being advertised were for a broad range of conditions.
Ads typically consisted of three components—advertising copy, prescribing information and visual messages. Advertising copy was distinguished from prescribing information based on the following criteria: no colour used in the prescribing information (eg, black print on white background/white print on a black background); the clear visual distinction between the advertising copy and prescribing information; no claims made in prescribing information; and the use of different fonts. Only the advertising copy and the visual messages were evaluated.
Our scoring system assessed three main quality domains: (1) information included in the advertisement, (2) references to scientific evidence and (3) advertising appeals and portrayals. The first domain included criteria that assessed whether generic drug names were given the same prominence (ie, mentioned as frequently) as brand names because the use of generic names is associated with more appropriate prescribing.14–16 If the ad made one or more quantitative claims about benefits then, if possible, based on the information in the ad, we assessed whether the claim was in the form of a relative risk reduction (RRR), absolute risk reduction (ARR) or number needed to treat (NNT). Specific mention of ARR and NNT have been shown to lead to more conservative prescribing.17–20 We examined the main claim(s), that is, the one(s) in the largest font to see if they referred to clinically relevant or non-clinically relevant features of the drug. Mention of clinical benefit was considered to be more important than the mention of a surrogate benefit since the latter is not necessarily predictive of a clinical benefit21 and because surrogate outcomes are likely to exaggerate treatment benefits as compared with patient-relevant clinical outcomes.22 Clinical outcomes were defined as ‘a characteristic or variable that reflects how a patient (or consumer) feels, functions or survives’ whereas surrogate endpoints were expected to predict clinical benefit (or harm or lack of benefit or harm) based on epidemiological, therapeutic, pathophysiological or other scientific evidence.23
Other types of claims (eg, on convenience, listing in a guideline, popularity of the product and mechanism of action) were considered to be less relevant to appropriate prescribing. Finally, mention of harm was assessed as physicians must be able to assess the benefit-to-harm ratio to prescribe appropriately. Specifically, we looked at whether the ad gave the same prominence to benefits and harms in terms of font size and position of the information. If more than one claim or harm was mentioned or more than one statement about safety information was provided, each one was evaluated separately.
The second domain included criteria that assessed the methodological quality of all of the references used to support claims made in the advertisement and the degree to which the reference supported the statement in the ad (assessed by reading only the abstract). Peer-reviewed journals are generally considered to publish higher-quality material than non-peer reviewed journals or other types of publications. The rating scales used for the methodological quality of the references and their support for claims came from the study by Lexchin and Holbrook.24 Reliance on observational data to evaluate drug efficacy is highly problematic,25 and the bias is, on average, larger than the estimated effect.26 Furthermore, there are many recent examples where observational studies that suggested a treatment benefit were overturned by randomised controlled trials (RCTs).27 Although there has not been any research into whether the strength of the link between the reference and the claims leads to more appropriate prescribing, it seems logical to assume that a stronger link would be beneficial in improving the reliability of the information.
The third domain included criteria that assessed different appeals and portrayals used by ads to market the product, and by doing so, provide prescribers different impressions regarding the value of the drug. The criteria used—the type of appeal, lifestyle or work portrayal, condition portrayal, the portrayal of effects of product use, product portrayal—were adapted from a study of direct-to-consumer television ads.28 Scott and colleagues have argued that drug ads ‘use images to construct mythical and potentially misleading associations between diseases and products’.29 In particular, drug advertising for psychiatric conditions can replicate and construct stereotypes about mental disabilities,30 especially in the case of women and the elderly. We counted the per cent of ads in each country using each of the different categories of appeals or portrayals.
Online supplementary file 1 outlines in detail the scoring system used for the quality assessment of advertisements. The overall quality of drug advertisements was measured by summing the ranking of selected criteria. Only criteria from the first two domains which revealed significant differences between countries were chosen. The first two domains were selected because they could be objectively measured, whereas the evaluation of the appeals and portrayals involved a subjective element.
Supplemental material
Scoring of ads
The initial scoring system was developed based on the results of a systematic review of the quality of journal ads.31 The scoring system was then refined through independent pilot testing by two authors (DD and AMS) with a review by the third author (JL) using 10 ads that were not included in the main study. Subsequently, two independent assessors (DD and AMS) used the scoring system to assess all the ads. Disagreements were solved by consensus or a third author (JL), if consensus could not be reached. The third author (JL) also evaluated the first 10 ads and every subsequent third ad to ensure consistency in coding.
Data analysis
Criteria were scored in one of two ways; some on a yes/no basis and in other cases we computed the percent of the total possible maximum score (eg, if the maximum score was 4 and the particular criterion for that ad was scored as 1 then we recorded a score of 0.25 (1/4)). If an ad had two claims, then the score for each claim was computed separately and then the scores were summed and the mean was calculated and reported. Then we performed two different quantitative analyses:
We compared scores for each criterion for the 30 ads for each country. Nominal data (yes or no) were presented as counts and percentages and compared with the χ2 test. Post-hoc analyses using adjusted residuals with Bonferroni corrections were done for all significant tests. For numerical data, Shapiro-Wilk tests were first used to assess normality. Our data were not normally distributed; hence non-parametric Kruskal-Wallis mean rank comparisons were used.32 Results were presented as medians and ranges. Post-hoc pair-wise comparisons with Bonferroni corrections were made for all significant tests.
In the absence of any validated research about whether any of the 10 criteria were more important in terms of influencing prescribing, we weighted all the criteria equally and ranked the countries from 1 (best score) to 3 (worst score) for each criterion. Ranks for each criterion were then summed, where the total rank was obtained to draw comparisons regarding the overall quality of ads per country. Lower total scores represented a better quality of journal drug advertising in the respective country.
Statistical calculations were done using IBM SPSS V.25.0. A two-sided alpha level of 0.05 was set for significance.
Patients and public involvement
No patients were involved in this study. There was no public involvement in this study.
Results
A total of 30 ads were included from each country. Only 14 unique ads were available from the American Family Physician for 2014, and therefore one ad from 2013 was used. AstraZeneca was the most common manufacturer for Australian ads (13.3%, n=4); Novartis Pharmaceuticals (13.3%, n=4) for Canadian ads; and Boehringer Ingelheim (20.0%, n=6) for USA ads. The mean total number of pages for the advertising copy of the ad was 1.15 (SD ±0.30) for Australia, 1.22 (SD ±0.34) for Canada and 2.18 (SD ±0.87) for the USA. For Australia, Canada and the USA, drugs came from 12, 15 and 16 different second level ATC groups, respectively. For Australia and Canada, the most common therapeutic group was Drugs for Obstructive Airway Diseases (7/30 ads in both); for the USA it was Drugs Used in Diabetes (7/30). Online supplementary file 2 lists the included advertisements.
Supplemental material
Information included in the advertisement
There was a statistically significant difference in the number of claims with quantitative benefit among the different countries: Australia 0 (0–3), Canada 0 (0–5), USA 1 (0–6), x2=8.761, p=0.01, with a mean rank of 37.6 for Australia, 43.9 for Canada and 55.0 for the USA. Post-hoc analysis revealed a difference in claims between Australia with a median of 0.0 (0.0–3.0) compared with the USA, with a median of 1.0 (0.0–6.0; p=0.01).
Differences were observed among countries concerning the reporting of RRR, ARR and NNT. RRR was most frequently reported by the USA (33.3%, n=10), followed by Canada (10.0%, n=3) and Australia (6.7%, n=2; p=0.02). Only one US ad provided sufficient information to calculate ARR or NNT.
Information on adverse effects, warnings or contraindications were most frequently reported by the USA (53.3%, n=16), then Canada (23.3%, n=7) and Australia (13.3%, n=4; p=0.002). Similarly, if safety information was given it had the same prominence as benefits information most frequently in the USA (75%, n=12), then Canada (28.6%, n=2) and Australia (25.0%, n=1; p=0.04). There were no statistically significant differences among countries with respect to how often generic names were mentioned compared with brand name mentions, presence of claims of clinical benefit or harm and how close each claim was to a clinically relevant drug characteristic. See table 3 for an overview of the information elements in the advertisements.
References to scientific evidence
Advertisements varied per country regarding the citation of scientific evidence (table 4). There was a statistically significant difference in methodological quality of evidence among the different countries, x2=17.066, p<0.001, with a mean rank of 35.9 for the USA, 39.6 for Canada and 61.0 for Australia. Post-hoc analysis revealed a difference in favour of Australia compared with Canada (p=0.003) and the USA (p<0.001). The median score, that is, the methodological quality score, of this criterion for Australia was 0.42 (0.25–0.70) compared with Canada at 0.25 (0.00–0.63) and the USA at 0.25 (0.00–0.75), where the maximum score was 1. There were no significant differences among countries with respect to supportive score for meta-analyses, systematic reviews, and RCTs.
Overall scoring of advertisements
The overall quality of drug advertisements, as measured by summing the ranking on five criteria that revealed significant differences among countries, was highest in the USA, followed by Canada and then Australia. Table 5 provides a summary of country rank per criterion.
Advertising appeals and portrayals
The distribution of different types of appeals images, portrayals of the effects of product use and product portrayals were equal in all three countries (p=0.55, p=0.34, p=0.15, respectively). However, there were differences in the distribution of lifestyle or work portrayal images and condition portrayals (p=0.04, p=0.02, respectively, online supplementary file 3a-3e). Overall, the most used appeals by all ads were rational (100%), followed by positive emotional appeals (46%). The most used portrayal was that the product enables health, recreational or work activities (48%). Ads were least likely to use product portrayals (36%), the portrayal of effects of product use (23%) and condition portrayals (16%).
Supplemental material
There were various statistically significant differences found between countries and types of appeals and portrayals (table 6). Positive emotional appeals were less common in Australia (26.7%, n=8) compared with Canada (60.0%, n=18) and the USA (50.0%, n=15; p=0.03). Humour appeals were more common in the USA (26.7%, n=8) compared with Canada (13.3%, n=4) and Australia (3.3%, n=1; p=0.04). Lifestyle or work portrayals were more commonly employed by the USA (76.7%, n=23) compared with Canada (50.0%, n=15) and Australia (43.3%, n=13). Portrayals that lifestyle change is an adjunct to product use were infrequently used in all countries: USA (26.7%, n=8), Canada (3.3%, n=1) and Australia (0.0%, n=0; p<0.001). Similarly, portrayals of social approval as a result of product use were also rarely used (USA (10.0%, n=3), Canada (0.0%, n=0) and Australia (0.0%, n=0); p=0.04) as were portrayals of loss of control caused by the condition (Canada (20.0%, n=6), Australia (3.3%, n=1), USA (3.3%, n=1); p=0.03). Post-hoc analyses were done for each χ2 comparison to see if there was a specific country that contributed most to the value of significance, but these analyses did not find any countries that were specific contributors of significance in any comparison.
Discussion
Our study revealed significant differences among countries regarding the following criteria: number of claims with quantitative benefit; RRR, ARR and NNT reported or calculated; mention of adverse effects, warnings or contraindications; equal prominence between safety and benefit information; and methodological quality of references. Taken together, our overall scoring ranked the USA first, Canada second and Australia third for the quality of journal ads, which confirms our original hypothesis in that self-regulatory systems (ie, the one used in Australia) may have the greatest influence in yielding the lowest quality ads compared with other regulatory regimes.
Although the US ads ranked first in quality, this finding should not be taken to imply that using them as a source of information would lead to appropriate prescribing. Only 13% of ads in American Family Physician mentioned the generic name every time the brand name was mentioned; only a single ad either gave an ARR or NNT or the information to calculate one; the maximum score for whether the main claim in the ads was to a clinically relevant issue was 3 but the median score was only 2; and only 30% of the ads referenced a meta-analysis, systematic review or RCT. The limitations seen in US advertisement quality might be due to a lack of resources needed to properly evaluate the volume of advertising. As of 2016, the FDA’s Office of Prescription Drug Promotion with a staff of just over 70 people received nearly 100 000 promotional material submissions related to prescription medications annually.33 Finally, the FDA only evaluates ads before they appear in relatively rare circumstances (table 1).
Countries only differed with respect to humorous, positive emotional and social approval portrayals as well as the presence of lifestyle or work portrayals. Although post-hoc testing was not significant, these portrayals were generally most commonly used by the US ads. Some of the pictorial features of the ads such as the frequent use of emotional appeals in ads, the relative absence of both the portrayals of lifestyle change as an adjunct to product use as well as the portrayal of the product enabling health, recreational or work activities all suggest that some aspects of the ads were not intended to give physicians an accurate view of the value of the medications that they were promoting.
Our findings are consistent with a previous study that concluded ad quality was affected by different regulations.8 Although that study examined ads published between 1961 and 1977, it appears that different regulatory regimes continue to influence ad quality. Another study compared ads between Australia, Malaysia and the USA between 2004 and 2006.9 Our study yielded similar results in that warning information was most likely to be provided in the US ads and least likely to be provided in Australian ads. We also found consistently incomplete product information in the advertising copy (eg, lack of safety information and support for claims made in ads) irrespective of the country. However, there was a large contrast between the two studies when comparing the percentage of ads that mention the generic name. Our study yielded a lower percentage, likely due to our more stringent criteria in that the generic name had to be mentioned every time the brand name was mentioned. Our findings regarding the supportive score for references were also higher compared with a past study that analysed the accuracy of scientific claims in Spanish drug ads.34 All known previous studies comparing ads used criteria focused on product information data but did not include additional comparisons known to influence prescriber behaviour, such as references to scientific evidence as well as advertising appeals and portrayals.8–10
Limitations
Despite examining information in ads that may affect prescribers’ behaviour, our study had some limitations. First, we only examined in-print journal advertisements and not other forms of promotion that affect prescribing practices. Additionally, we did not assess the accuracy of the information in the ads. While this would have been desirable, the lack of information about many important aspects of drug efficacy and safety speaks the poor educational quality of the ads. We also did not directly examine whether the ads all conformed to regulatory requirements in the country in which they were published or whether they had been subject to complaints to the regulator. We suspect that violations of regulations may have confounded our results. For instance, we found that advertisements from the USA were significantly more likely to report on adverse events, despite all regulatory bodies requiring a fair balance between benefits and harms, suggesting that advertising originating in Australia and Canada may not have been complaint with the relevant codes. Advertisements for different drugs and from different manufacturers may also yield differences in the type of product information, references to scientific evidence, as well as appeals and portrayals. We only examined one country per regulatory regime and therefore we could not determine whether the differences were due to the regulatory framework or to other regulatory, legal, cultural or health system factors specific to each country. For instance, our finding that USA ads contain more information on adverse effects, warnings or contraindications may also reflect industry concerns with litigation in addition to FDA regulation. To the extent that our findings do reflect different regulatory regimes, they only apply to ads in family practice journals in three developed countries over the period 2014–2015. Finally, we only examined parts of the ads that could be objectively scored and our scoring system for some elements while used before has not been validated against the effects that ads have on prescribing behaviour.
Conclusion
This is the first study to compare advertising quality under different regulatory frameworks. We found differences in the quality of journal advertisements concerning product information, references to scientific information, as well as appeals and portrayals that were produced under different regulatory regimes. Regulation via direct government control (ie, the USA) yielded the highest-quality ads, followed by regulation by autonomous bodies (ie, Canada) and then by industry self-regulation (ie, Australia). Despite this, all forms of regulation as they are currently practiced have limitations in terms of the quality of the ads. Our results suggest that well-resourced government regulation might be the best way to ensure that journal advertising provides physicians with the accurate, complete and objective information that they need.
Acknowledgments
The authors thank Drs. Richelle Cooper, Barbara Mintzes, Adrienne Shnier, Agnes Vitry and Michael Wilkes for providing helpful comments on an earlier version of the manuscript. They were not compensated for their contribution.
References
Footnotes
Contributors JL was responsible for the study conception and design. DD, AM-S and JL were responsible for data extraction and validation. DD, AM-S and JL analysed and interpreted results. DD, AM-S and JL drafted the manuscript. All authors provided a critical review and approved the final manuscript. JL is the guarantor.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf. In 2016–2019, Joel Lexchin was a paid consultant on two projects: one looking at developing principles for conservative diagnosis (Gordon and Betty Moore Foundation) and a second deciding what drugs should be provided free of charge by general practitioners (Government of Canada, Ontario Supporting Patient Oriented Research Support Unit and the St Michael’s Hospital Foundation). He also received payment for being on a panel at the American Diabetes Association, for a talk at the Toronto Reference Library, for writing a brief in an action for side effects of a drug for Michael F Smith, Lawyer and from the Canadian Institutes of Health Research for presenting at a workshop on conflict of interest in clinical practice guidelines. He is currently a member of research groups that are receiving money from the Canadian Institutes of Health Research and the Australian National Health and Medical Research Council. He is a member of the Foundation Board of Health Action International and the Board of Canadian Doctors for Medicare. He receives royalties from the University of Toronto Press and James Lorimer & Co, Ltd for books he has written. Dion Diep and Abnoos Mosleh-Shirazi have no conflicts to declare.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Patient consent for publication Not required.
Ethics approval All data were publicly available and therefore, ethics consent was not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available in a public, open access repository. Extra data can be accessed via the Dryad data repository at https://datadryad.org/ with the doi: 10.5061/dryad.6t1g1jwtz.