Article Text
Abstract
Objective To assess the accuracy of self-reported financial conflict-of-interest (COI) disclosures in the New England Journal of Medicine (NEJM) and the Journal of the American Medical Association (JAMA) within the requisite disclosure period prior to article submission.
Design Cross-sectional investigation.
Data sources Original clinical-trial research articles published in NEJM (n=206) or JAMA (n=188) from 1 January 2017 to 31 December 2017; self-reported COI disclosure forms submitted to NEJM or JAMA with the authors’ published articles; Open Payments website (from database inception; latest search: August 2019).
Main outcome measures Financial data reported to Open Payments from 2014 to 2016 (a time period that included all subjects’ requisite disclosure windows) were compared with self-reported disclosure forms submitted to the journals. Payments selected for analysis were defined by Open Payments as ‘general payments.’ Payment types were categorised as ‘disclosed,’ ‘undisclosed,’ ‘indeterminate’ or ‘unrelated’.
Results Thirty-one articles from NEJM and 31 articles from JAMA met inclusion criteria. The physician-authors (n=118) received a combined total of US$7.48 million. Of the 106 authors (89.8%) who received payments, 86 (81.1%) received undisclosed payments. The top 23 most highly compensated received US$6.32 million, of which US$3.00 million (47.6%) was undisclosed.
Conclusions High payment amounts, as well as high proportions of undisclosed financial compensation, regardless of amount received, comprised potential COIs for two influential US medical journals. Further research is needed to explain why such high proportions of general payments were undisclosed and whether journals that rely on self-reported COI disclosure need to reconsider their policies.
- MEDICAL ETHICS
- Clinical trials
- CLINICAL PHARMACOLOGY
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Data posted at medRxIv at: https://www.medrxiv.org/content/10.1101/2021.09.12.21263468v1.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first effort to systematically evaluate conflict-of-interest (COI) disclosures of practising US physicians publishing in the Journal of the American Medical Association and the New England Journal of Medicine.
The US Center for Medicare and Medicaid Services’ Open Payments database is unlike that of many other countries in that it provides mandatory reporting of financial COIs.
The sample size for physician-authors that met all inclusion criteria (n=118) was only moderate, and findings were from 2017.
These results may be less generalisable to authors from outside of the US, non-physicians (PhD, NP, PA), or to those publishing in less-influential journals.
Background
Financial conflicts of interest (COIs) are a perennial problem for medical research and practice.1 2 Physician researchers who receive industry payments are more likely to demonstrate results favourable to the companies funding them;3 4 are more likely to prescribe drugs and use of medical devices produced by these companies, from statins5 to opioids6 to endoscopic7 and orthopaedic devices;8 and they may unduly influence other physicians by contributing to research that others use to guide their own clinical practice.9–16 Industry payments to physicians therefore may bias healthcare providers’ delivery of evidence-based medicine and interfere with their responsibilities to their patients.
In order to increase the transparency of the financial relationships between physicians and pharmaceutical and medical device manufacturers, the US government passed the Physician Payment Sunshine Act as part of the Patient Protection and Affordable Care Act in 2010.17 This law required manufacturers reimbursed by Medicare, Medicaid, or the Children’s Health Insurance Program to submit information regarding payments received by physicians to the Centers for Medicare and Medicaid Services (CMS). CMS shares these payment data with the public on an annual basis through the Open Payments website,18 which was introduced in 2014 with data starting in August of 2013. The International Committee of Medical Journal Editors (ICMJE) has produced its own COI form to help medical journals maintain COI disclosure standards for physician-authors seeking to publish articles in peer-reviewed medical journals.19 Many journals have adopted the use of this form, requiring authors submitting manuscripts to them to disclose payments received from manufacturers of products related to the article content in the 36 months prior to submission.19 These author disclosures can be verified by viewing the physician-author’s record in the Open Payments database.18
Despite these attempts to address COI disclosures, COI disclosure opacity has persisted across a diversity of specialties,20 21 forms of compensation,22 and investigational products of clinical trials.23–26 Previous studies of inaccurate, or ‘discordant,’ COI disclosures have emphasised financial COIs,27 COIs differing significantly by specialty,28 and inaccurate COI disclosures appearing in high-impact journals, such as the New England Journal of Medicine (NEJM).25 In 2017, the Journal of the American Medical Association (JAMA) published an issue dedicated to the subject of COI disclosures to highlight the multifaceted nature of the problem.29
NEJM and JAMA are the peer-reviewed general medical journals published in the United States with the highest and second-highest impact factors, respectively. Both journals publish with similar frequency (weekly for the former and 48 times per year for the latter), emphasise publication of original research as well as reviews, are popular for physician-authors, and publish articles that receive wide coverage both within the scientific community and in the popular news media. The impact and reach of these two journals have substantial potential to shape future research and patient care. To date, there has been no comprehensive study of COI disclosures in these two journals. The objective of this study was to examine COI disclosures among physician-authors who published articles in either (or both) of these journals in 2017, the first year for which complete data exist for the earliest possible disclosure period following the inception of Open Payments. Identifying patterns of disclosure transparency at the beginning of the existence of Open Payments revealed the extent to which physician-authors publishing in NEJM and JAMA follow COI disclosure policies.
Methods
Inclusion and exclusion criteria
Original research articles (n=394) detailing the results of randomised controlled trials and published in NEJM (n=206, 52.3%) and JAMA (n=188, 47.7%) from 1 January 2017 to 31 December 2017 were examined. The first and last author of each article were identified and located on Open Payments using their full names, specialties, and department affiliations. Articles were excluded from further examination if either the first or last author did not have an MD or DO degree, if either author did not have a record in Open Payments, and if either author had incomplete Open Payments data from the start of the data collection window (figure 1).
This cross-sectional study explored COI disclosures of physician-authors by comparing their Open Payments records with self-disclosures made by the authors to the journals. Open Payments records consisted of disclosures of payment amounts by the companies making the payments, and self-disclosures consisted of the authors’ identifications of the companies that paid them, not of the amounts.
Data collection
Data reported to Open Payments from 2014 to 2016 were compiled and compared with self-reported disclosure forms that had been submitted to NEJM or JAMA with the authors’ published articles. Data collected about the authors included sex, specialty, journal(s) of publication, and yearly payment information. Open Payments defines ‘payments’ as either general ‘payments that are not associated with a research study’ or research payments that ‘are associated with a research study.’30 As other studies have focused on research payments,4 9 31 32 this cross-sectional study focused solely on general payments, which included compensation for promotional speaking, consulting, travel and lodging, food and beverage, honoraria, and current or prospective ownership or investment interest.
The articles were examined to determine their areas of investigation (eg, cardiovascular disease, diabetes, cancer). This occurred with reference to the title, key words, abstracts, and content of each article. Three coauthors of this study collected the data, resolving by discussion any disagreement in interpretation of article topics for the purpose of disclosure analysis. Each article’s area of investigation was compared against the product portfolios and research pipelines of the companies that paid the physician-authors, according to their Open Payments data. Payments from companies disclosed by the author were labelled as ‘disclosed’ for this study. Payments from companies not listed on the respective authors’ disclosure forms were investigated further and categorised as ‘undisclosed,’ ‘indeterminate,’ or ‘unrelated.’ Occasionally, a company that did not match the disclosures on the author’s form was later determined to have made a disclosed payment. For example, a company with multiple names was determined to have made a disclosed payment under only one name, prompting a review that revealed the fact that the company’s payment had been disclosed, just under a name that was not immediately recognisable on the disclosure form. Every payment from a given company was analysed by researching that company, thus putting all payments from that company under one umbrella for the purpose of COI disclosure. For example, if a company produced a drug related to the content of an author’s research article, then every general payment that that company made to the author, regardless of the nature of the payment, was construed as a COI. Payments were considered for both the parent and subsidiary companies.
The criteria in table 1 were followed to categorise payments as ‘disclosed,’ ‘undisclosed,’ ‘indeterminate,’ or ‘unrelated’. These definitions were adapted from the ICMJE disclosure form used by both journals, which states:
“(Authors) should disclose interactions with any entity that could be considered broadly relevant to the work. For example, if your article is about testing an epidermal growth factor receptor (EGFR) antagonist in lung cancer, you should report all associations with entities pursuing diagnostic or therapeutic strategies in cancer in general, not just in the area of EGFR or lung cancer.”33
ICMJE requires disclosures for 36 months prior to submission. Therefore, this study focused on all payments within 36 months of the submission dates. A copy of the ICMJE form reviewed for this study is available as online supplemental file 1.33
Supplemental material
Payment data in the study window, including company name, amount, and purpose, were extracted from the Open Payments database into a spreadsheet accessible to all authors of this study. Payments were then categorised based on the ICMJE guidelines (table 1) by three coauthors in their first two years of medical school (MD class of 2021 and 2022), and disagreements were resolved by discussion. NEJM provided the disclosure forms as attachments to their articles; JAMA provided a list of disclosures at the end of each article. The NEJM articles investigated in this study stated their submission dates or included this information on the author disclosure forms, which are made available to the public as attachments to each article. For JAMA the submission date was approximated by using the date when the article was published. JAMA’s official position is that, as of 2016, the median time from article submission to acceptance was eighteen days, and the median time from acceptance to first online publication another fourteen days, roughly totalling one month.34 Therefore, unknown submission dates were estimated as 30 days prior to respective publication dates; this caveat is important for data interpretation. It was assumed that a COI encountered within these 30 days would be unlikely to influence the manuscript, presumably already written in its nearly final form.
Data were collected for payments from 2014 to 2016. Data in Open Payments are periodically updated. Our data were last updated in August of 2019. See online supplemental file 2 for the full data, which are also available in a public, open-access repository.35 The sample size depended in part on the labour intensivity of COI evaluations for authors with extensive relationships with industry: 1–3 hours per physician-author were required for this study’s investigators to delve into each company, its portfolio of products, and its research pipeline during the 36-month disclosure window. The payments to the one author who contributed to both NEJM and JAMA were counted once in calculating total general payments but twice (once per journal) for between-journal analysis (online supplemental file 3). Our framework conceptualised a broadly construed COI, rather than impact on research per se.
Supplemental material
Supplemental material
Patient and public involvement
Patients or the public were not involved in the design of this investigation.
Statistics
Analysis focused on payments received within the years 2014, 2015, and 2016, as all of the authors’ respective 36-month disclosure windows overlapped these years. GraphPad Prism (V.9) was used for statistical analysis and for figure generation. Descriptive statistics (median, quartiles, and mean±SD) were calculated. Robust non-linear regression with outlier removal (ROUT) analysis with a maximum desired false discovery rate Q=1% identified outliers.36 The Wilcoxon rank-sum test assessed the extent of parity between distributions. A p<0.05 was considered statistically significant. The flow chart was generated using Lucidchart (Lucid Software, South Jordan, Utah, USA).
Results
A total of 394 original research articles published in NEJM (n=206) and JAMA (n=188) from 1 January 2017 to 31 December 2017 were examined. Articles containing a first or last author without an MD or DO degree, who did not appear in the Open Payments database, or who had incomplete Open Payments profiles at the start of the study were excluded. This left 31 articles from NEJM and 31 articles from JAMA that met all criteria for inclusion, with a total of 118 unique authors (figure 1).
Within their respective 36-month disclosure windows, the 118 authors received US$7 476 049.87 in general payments combined. Payments to authors who published in NEJM totaled US$3 635 791.81 (48.4% of the total) and to JAMA totaled US$3 876 107.75 (51.6%). These journal totals sum to US$7 511 899.56; the discrepancy of US$35 849.69 consists of the payments of the sole author who published in both journals. The median payment amount for NEJM authors was US$11 224.53; at Q1 (25th percentile) the amount was US$755.67 and at Q3 (75th percentile) was US$80 179.56. For JAMA authors, the median payment was US$2400.00, with Q1 at US$65.20 and Q3 at US$30 964.21. Mean payment amounts were US$58 641.80 (±US$102 337.65) for NEJM and US$68 001.89 (±US$215,813.16) for JAMA (figure 2). Total payments by category were similar between the two journals (figure 3 and online supplemental file 3).
Of the 118 authors, twelve (10.2%) received no payments. Of the 106 (89.8%) who did, payment amounts ranged from a minimum of US$6.36 to a maximum of US$1 486 929.34. Eighty-six of these 106 authors (81.1%) received undisclosed payments. Twenty-three outliers were identified, ranging from US$93 165.88 to US$1 486 929.34, reflecting the payment amounts received by the 23 most highly compensated physician-authors; our definition of outliers did not include low or no amounts. All 23 had MD degrees; three additionally had PhDs, and two others additionally had MPHs. Sixteen (69.6%) were internal medicine specialists or subspecialists. Fifteen (65.2%) published in NEJM, and eight (34.8%) published in JAMA. Twelve (52.2%) were first authors, and eleven (47.8%) were last authors. Of the outliers, 21 (91.3%) were males, and two (8.70%) were females. Of the entire sample, 101 (85.6%) were males, and seventeen (14.4%) were females. The top 23 most highly compensated physician-authors received US$6 316 025.03, of which US$3 004 703.54 (47.6%) was undisclosed. The total amount that the top 23 most highly compensated physician-authors received (US$6 316 025.03) comprised 84.5% of all compensation received by all 118 physician-authors (US$7 476 049.87, table 2). The total amount that the NEJM outliers received (US$2 965 974.61) comprised 81.6% of all compensation received by all 62 NEJM authors, and the total amount that the JAMA outliers received (US$3 350 050.42) comprised 86.4% of all compensation received by all 57 JAMA authors. One author published in both journals.
COI disclosure rates
Of the 106 authors who received payments, 55 made disclosures of which the disclosed companies reported dollar amounts that summed to at least half of the authors’ total payment amounts. Twenty had a three-year disclosure rate of 100%; ten of these published in NEJM, and the other ten published in JAMA. The other 35 authors who disclosed at least half of their payments had disclosure rates that ranged from 54.5% to 99.9%. Of the 51 authors who disclosed less than half of their payment amounts, 18 disclosed between 0.007% and 42.3%. Thirty-three authors who received payments disclosed 0%, or no amount, of their payments received. Of the authors who disclosed 0%, 21 of them published in JAMA, and twelve published in NEJM (figure 4).
A Wilcoxon rank-sum test also found no significant difference in COI disclosure rates between NEJM and JAMA authors (p=0.0849).
COI by year
Little variability was observed across the individual years that fell within the 36-month disclosure window. In 2014, 79 authors (67.0%) received payments; in 2015, 84 (71.2%) received payments; and in 2016, 81 (68.6%) received payments. In all three years, the majority of physician-authors received no payments in the disclosed, indeterminate, and unrelated categories, whereas the majority did receive undisclosed payments in each year. Some authors who received no payments in one year within the disclosure window received payments in one or more of the other two years (online supplemental file 2).
COI by specialty
The physician-authors in this study represented 33 distinct medical specialties. Fourteen (42.4%) of these specialties were subspecialties of internal medicine. The two most-represented specialties among the 118 physician-authors were cardiovascular disease (n=16) and general internal medicine (n=11). Ten specialties each were represented by four to nine individuals, seven specialties each were represented by three individuals, five specialties each by two individuals, and eleven specialties each by a single individual.
Discussion
These novel data from highly influential US general medical journals (NEJM and JAMA) extend a sizeable evidence base that has raised doubts about whether self-reported financial disclosure is a trustworthy mechanism for point-of-care databases,13 clinical practice guidelines,37 38 or other authoritative resources.3 Financial COIs are important to identify in order to recognise sources of potential bias in research works published by physicians and other researchers. Such bias can have devastating consequences: it undermines public trust in science,39–42 confounds understanding of treatment efficacy43–45 and clinical practice guidelines,9–13 and even continues to obstruct investigation into the origins of SARS-CoV2.46 47 Such instances provoke popular outrage48–50 and incite corrective action,51 52 often to little avail.29 53 On multiple occasions, both NEJM and JAMA, as well as many other publications, have confronted the resignation or dismissal of their editors-in-chief for COIs, financial and otherwise.54–58 On 2 May 2017, JAMA published a ‘theme’ issue dedicated to the topic of COI disclosures and consisting of a variety of articles exploring this theme.53 Perhaps JAMA’s submitting authors have exercised greater COI disclosure transparency since publication of that 2017 special issue, but the results of this study are not consistent with this supposition.
COI disclosure rules and procedures venture to mitigate the impact of COI bias on the integrity of published manuscripts. This premise means that ascertaining the impact of payments on researchers, or how payments influence those receiving them, may help to delineate the process of this insult to publication integrity. Key to identifying such bias in the first place is examination of COI disclosure accuracy. This was the purpose of this study. Accordingly, the authors of this study take no position on the physician-authors’ intentions in non-disclosure of their COIs; we characterise the issue as a ‘process problem’ rather than a ‘people problem,’ especially in light of the patterns observed in COI disclosure rates regardless of the journal’s disclosure process and regardless of payment amount. The publication process could be improved by requiring US-based physicians to provide links to their Open Payments reports with their manuscript submissions.
It was anticipated that different disclosure processes between the two journals could produce different patterns in payment distributions and disclosure rates. NEJM provided a copy of the original disclosure, while JAMA provided a list of disclosures; this prompted speculation that different disclosure processes between the two journals could produce different patterns in payment distributions and disclosure rates. On the contrary, the data demonstrate no such significant differences between the two journals. The finding that the top 23 most highly compensated physician-authors received 84.5% of all monies analysed demonstrates a Pareto-principle pattern reflected across the two journals. This lack of differences in payment distributions and disclosure rates, despite a difference in the disclosure process, may imply that the journals’ differing disclosure processes had no effect on payment distributions and disclosure rates.
Limitations
A major limitation of this study is that of generalisability, especially of the findings to journals other than NEJM and JAMA. Our study assessed the data within a non-parametric analytical framework because there is no methodological justification for making inferences about payment distribution patterns among the broader community of physician-authors. The 118 physician-authors that met inclusion criteria are not presumed to represent this broader community, despite the more general-interest nature of the content that NEJM and JAMA tend to publish. The authors who publish results of randomised controlled trials in these two journals may be more likely to receive funding, and may receive more funding, than those who publish in other journals generally. Moreover, despite the high-impact and high-profile nature of NEJM and JAMA, results of randomised controlled trials that exert the greatest influence on clinical practice may appear instead in the highest-impact medical specialty journals. A limitation contributing to that of generalisability was a moderate, although carefully selected, sample size across the two journals. This moderate sample size may have obscured possible differences due to low statistical power. Exclusion of middle authors, a factor influencing sample size and, therefore, statistical power, occurred because of the convention in academic publishing of the first and last authors having an undoubted impact on manuscript content with the influence of the middle authors varying greatly. This study’s dataset is based on information provided by CMS Open Payments. Use of this system to disclose information regarding investigator compensation is mandatory for many US pharmaceutical and medical device companies that make products used by Medicare, Medicaid, or Children’s Health Insurance patients.59 However, this database does not capture remuneration from entities for products or services that are not covered by the Food and Drug Administration (eg, medical marijuana, many complementary and alternative medicines, or psychotherapies) or remuneration to non-US-based authors. Finally, this report was limited to financial COIs. Non-financial COIs in the form of strong intellectual, emotional, political, and religious convictions might exert at least as much influence as financial COIs. An example is the allegiance effect, the phenomenon of researchers and clinicians who develop or are otherwise invested in a treatment tending to find larger effect sizes supporting their treatment groups.60–62 Non-financial COIs merit further exploration, particularly in highly influential publications. However, the feasibility of reporting them is fraught with practical, ethical, and even epistemic issues.63 64 Therefore, non-financial COIs were not considered in this study.
Future directions
Future research should examine differences between general and specialty journals in the comprehensiveness of their reporting of financial COIs65 and the effects of the COVID-19 pandemic on interactions between physician-authors and pharmaceutical and medical device manufacturers. Additional investigations may benefit from larger sample sizes, inclusion of middle authors and authors located outside of the USA, and examination of observational reports, reviews, and editorials. This report, as well as prior ones,1–3 5–7 9–16 may form the foundation for additional investigations with clearly specified a priori hypotheses. Inquiries may also seek to explain the role of COI bias in the conduct and outcomes of randomised controlled trials.66 A future study comparing COI disclosure patterns with this earliest period of Open Payments data might show a change in such patterns or the effect of Open Payments on COI disclosure transparency. Future research may be more robust now that the SUPPORT Act has expanded the range of researchers whose data are collected by Open Payments.67 Since January 2021, physician assistants, clinical nurse specialists, certified nurse midwives, certified registered nurse anaesthetists, and anaesthesiologist assistants have had entries on the Open Payments website.68 This new data source may help to assess whether the observations of this study are applicable to mid-level healthcare providers and to make direct comparisons of mandated versus self-reported disclosures14 among different members of the healthcare team.
Conclusion
The fact that the preponderance (81.1%) of physician-authors in this novel study received payments that they did not disclose but that they nonetheless were supposed to disclose as COIs per ICMJE guidelines and journal requirements demonstrates that these disclosure requirements in conjunction with the expectation of COI self-disclosure have been inadequate to ensure full COI transparency in either NEJM or JAMA and regardless of general payment amount received. Making industry payments a matter of public record in the form of Open Payments presumes to mitigate this problem of COI disclosure opacity. Readers are encouraged to compare self-reported with industry-reported disclosures.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Data posted at medRxIv at: https://www.medrxiv.org/content/10.1101/2021.09.12.21263468v1.
Ethics statements
Patient consent for publication
Ethics approval
The procedures of this study were approved by the IRBs of the Wright Center of Scranton and Geisinger.
Acknowledgments
Thanks to Iris Johnston and the Department of Medical Education of GCSOM for technical support.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors JHB was responsible for data analysis, figure preparation, and writing and revising the manuscript. SAP, JCA, and KV were responsible for experimental design, data collection, and writing. MRG contributed to data analysis and writing. BJP was responsible for research design, general project administration, and writing. All authors contributed to revising the manuscript and approved the final version.
Funding This study received no direct support. BJP is supported by the Health Resources Services Administration (D34HP31025) and is part of an osteoarthritis research team supported by Pfizer and Eli Lilly. Software used in this effort was provided by the NIEHS (T32- ES007060-31A1).
Disclaimer Funders had no involvement in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.
Competing interests BJP is part of an osteoarthritis research team supported by Pfizer and Eli Lilly. The authors have no personal or institutional interest with regards to the authorship and/or publication of this manuscript.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.