You are here

  • Home
  • Archive
  • Volume 9, Issue 2
  • Effect of different financial competing interest statements on readers’ perceptions of clinical educational articles: a randomised controlled trial

Article Text

Article menu

Download PDFPDF

XML
Medical publishing and peer review
Research
Effect of different financial competing interest statements on readers’ perceptions of clinical educational articles: a randomised controlled trial
  1. Sara Schroter1,
  2. Julia Pakpoor2,
  3. Julie Morris3,
  4. Mabel Chew1,
  5. Fiona Godlee1
  1. 1 The BMJ, London, UK
  2. 2 Academic Clinical Fellow in Public Health, Oxford University Clinical Academic Graduate School, Oxford, UK
  3. 3 Education & Research Centre, University Hospital of South Manchester, Wythenshawe Hospital, University of Manchester, Manchester, UK
  1. Correspondence to Dr Sara Schroter; sschroter{at}bmj.com

Abstract

Objectives To investigate how different competing interest (COI) statements affect clinical readers’ perceptions of education articles.

Design Randomised controlled trial.

Setting and participants Random sample of UK doctors.

Interventions We created four permutations of each of two clinical reviews (on gout or dyspepsia), which varied only in terms of the COI statement. Volunteers were blinded and randomised to receive one review and asked to complete a questionnaire after reading it. Blinded factorial analyses of variance and analyses of covariance were carried out to assess the influence of each review and type of COI on outcomes.

Primary and secondary outcomes Confidence in the article’s conclusions (primary outcome), its importance, their level of interest in the article and their likelihood to change practice after reading it.

Results Of 10 889 doctors invited to participate, 1065 (10%) volunteered. Of these, 749 (70%) completed the survey. Analysis of covariance (adjusting for age, sex, job type, years since qualification) showed no significant difference between the groups in participants’ confidence in the article (gout: p=0.32, dyspepsia: p=0.78) or their rating of its importance (gout: p=0.09, dyspepsia: p=0.79). For the gout review, participants rated articles with advisory board and consultancies COI as significantly less interesting than those with no COI (p=0.028 with Bonferroni correction). Among participants indicating that they treat the condition and that the article’s recommendations differed from their own practice, there was no significant difference in likelihood to change practice between groups (gout: p=0.59, n=59; dyspepsia: p=0.56, n=80).

Conclusions Doctors’ confidence in educational articles was not influenced by the COI statements. Further work is required to determine if doctors do not perceive these COIs as important in educational articles or if they do not pay attention to these statements. More meaningful COI disclosure practices may be needed, which highlight context-specific potential sources of bias to readers.

Trial registration number NCT02548312; Results.

  • general medicine (see internal medicine)
  • medical journalism

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2018-025029

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • Competing interest (COI) statements have been shown to influence readers’ perceptions of research, but this is the first experimental study to look at the effect of COIs in clinical educational material.

    • A key strength of this study is its randomised study design in a research area where there are few experimental studies.

    • Financial competing interests are varied; we were only able to evaluate the effect of three financial COI statements compared with none due to the large sample size required.

    • Our outcome measures were all self-reported and we did not assess objective changes to practice.

    Introduction  

    Researchers, clinicians and academic institutions often have competing interests, also known as conflicts of interest (COIs), and collaborations with industry are often considered necessary to facilitate progress and innovation in medical research.1 COIs are defined as ‘circumstances that create a risk that professional judgements or actions regarding a primary interest will be unduly influenced by a secondary interest’.1 2 The possibility that COIs may bias the medical literature and potentially affect patient care has been highlighted in many studies. For example, a 2017 Cochrane review found that drug and device studies sponsored by the manufacturer demonstrated more favourable efficacy and conclusions than studies sponsored by other sources.3 Whether bias is conscious or unconscious, COIs may therefore compromise the medical evidence which drives development of recommendations for clinical care.3

    Widespread recognition that COIs may potentially influence decision-making has rendered their open disclosure a common requirement for the publication of research articles in academic medical journals.3 A systematic review reported that the presence of financial COIs and industry collaborations are concerning to academic and clinical researchers, particularly as such interests may potentially influence research project decisions, the conduct of research and subsequent publication.4 Financial ties with industry were considered more acceptable where these were not directly related to the research, disclosure of COIs was upfront and the results of research was freely published.4

    However, there has been little research exploring the effect of competing interests on reader perceptions. In a randomised trial investigating the effect of the funding source of a clinical trial on clinicians’ interpretation of trial results, it was observed that industry sponsorship negatively affected the perceived methodological rigour of a trial and the willingness to change practice based on its findings, independent of trial quality.5 We have previously reported the results of two randomised controlled trials comparing the effect of COI statements related to financial interests against no competing interests and demonstrated a significantly negative influence of COIs on readers’ perceptions of the credibility of medical research.6 7 Surveys have previously reported a similarly low perceived credibility of industry-initiated or funded drug trials among clinicians.8 However, in a trial of US physicians who were provided with a clinical trial abstract presenting positive findings of a new drug, randomised to differ in their COI statement, it was found that doctors did not significantly discount for COIs when reporting their likelihood to prescribe the fictitious drug.9 Nonetheless, when directly asked about the COI, the majority reported that they feel that they should to some degree discount information on the basis of COIs, highlighting that simply publishing COI disclosures may not be sufficient.9 Similarly, a randomised trial of French general practitioners (GPs) found no evidence of a significant impact of reporting of COIs on GP’s confidence in the conclusions of trial abstracts.10

    Clinical education articles are intended to provide guidance on clinical care for clinicians, yet our understanding of the role of COIs on readers’ perceptions of the credibility of such articles, rather than primary research articles, is limited. Educational articles are prone to bias as they typically use non-systematic methods of literature acquisition, and broadly rely on the interpretation of one author, or a small number of authors, on their chosen included literature. Of concern too is evidence from the social sciences suggesting that disclosure of COIs may even enhance bias: conflicted authors may feel a ‘moral release’ from having simply declared they are conflicted, or may even exaggerate to counteract any expected discounting of their opinion.11 12 Such potential biases may therefore be extensive, but potentially less visible to their targeted broader clinician readership.

    Many years ago, the American Family Physician became the first journal to introduce a ‘zero tolerance’ policy for COIs in clinical educational articles.13 In the 1990s, The New England Journal of Medicine implemented a stringent policy, whereby editorialists had to be free from financial ties to drugs or devices discussed in the editorial,14 15 but this policy was relaxed in 2002 to exclude only those with significant (US$10 000) financial interest due to difficulties in recruiting authors.16 In 2015, The BMJ implemented a ‘zero tolerance’ policy on the presence of any relevant financial COI related to industry for authors of its clinical editorials and some education articles.17 However, some have questioned the need for strict restrictions on the presence of COIs, discussing whether such policies may limit trust, effective industry collaborations or the ability for some experts to contribute to clinical education.18–21 Evidence is missing in characterising how COI statements influence reader perceptions of educational articles, or indeed if awareness exists of the potential for COIs to influence the conclusions of such articles. We describe a randomised controlled trial to test the effect of a range of common COI statements in educational articles on a clinician readership’s confidence in the conclusions of an article, their interest in the article, its perceived importance and on the likelihood that they would change their clinical practice based on the article’s findings.

    Methods

    Design

    Parallel-group randomised controlled trial. The study protocol has previously been published.22

    Study sample

    We took two approaches to the sampling for this study as the first approach did not yield adequate numbers.

    Inclusion criteria and exclusion criteria

    We included practising doctors in the UK who were receiving The BMJ through their membership of the British Medical Association (BMA). We excluded members who had opted out of receiving a copy of The BMJ, public health doctors, consultant oral/dental surgeons, retired doctors and student members. We also excluded doctors listed as doing private practice as this was necessary due to the way the data about specialty and grade are stored to ensure compliance with our other exclusion criteria.

    Sample 1

    We generated a random sample of 2040 BMA members (680 GPs, 680 hospital consultants and 680 junior doctors), randomised each to a group (see ’Methods' section below), and sent them a personalised email invitation from The BMJ’s editor-in-chief in September 2015 to take part in a research project along with the relevant study materials. A range of clinical specialties and stages of training were included to facilitate generalisability of study findings to the clinical workforce, and the clinical conditions of the educational articles were accordingly selected to reflect conditions which the vast majority of clinicians would be expected to have had experience in managing. Participants were not told the purpose of the study to avoid biassing responses.

    Sample 2

    We broadened the sampling frame and took a very large random sample of 11 004 BMA members and asked for volunteers to take part in a research project before assigning them to a study group, as per our protocol. Recruitment for volunteers was open between 6 January 2016 and 28 January 2016.

    Intervention

    Participants were sent an email with a link to one of two clinical reviews depending on randomised group allocation, on the management of dyspepsia (online supplementary appendix 1) or gout (online supplementary appendix 2), and a link to a short questionnaire on SurveyMonkey on 2 February 2016. Study participants were asked to read the article and then complete the questionnaire. Data collection closed on 3 May 2016.

    Supplemental material

    Supplemental material

    We selected two clinical reviews previously published by The BMJ in 2013 describing two conditions commonly seen by doctors, requiring treatment by drugs, and familiar to all clinical specialties. We shortened and modified these after obtaining permission from the original authors. We changed the authors on the authorship byline to fictional author names and listed fictional institutions. Each of the two clinical reviews had four permutations differing only in the COI statement (from no competing interests to a range of financial interests) for the last of the three authors (table 1). All COI statements appeared at the end of the article’s main text, just before the references, in line with usual practice. These statements all had the same fictional author names and where there was a financial COI we used the same fictional pharmaceutical company name but did not mention the company name in the main text of the clinical reviews.

    View this table:
    Table 1

    Group allocations and competing interest (COI) statements

    Randomisation and blinding

    A random sample of eligible BMA members was generated from the database of all members by staff at the BMA using computer-generated random numbers. JM then randomised members to one of eight groups to receive one of the eight permutations of the clinical reviews using a computer-generated block randomisation procedure, stratified by type of doctor (GPs/hospital consultants/junior doctors) and gender. The eight permutations of the clinical reviews were then randomly assigned a number from 1 to 8 by SS. SS enrolled participants and managed the survey. JM conducted the statistical analysis blinded to the group allocation; participants were identified only by study group number, which was not revealed to JM until after all analysis was completed. Participants were blinded to their study group and were not told that we were testing the effect of various COIs on their perceptions of the articles.

    Data collection

    We piloted the draft survey with a convenience sample of doctors to ensure the instructions were clear and the questions were not ambiguous. Study participants were asked to read the article and indicate on a 10-point Likert scale (0=not at all, 10=extremely): how confident they were in the conclusions drawn in the article they received, how interesting and important they found the article and how likely they were to change their practice on the basis of the article (see the online supplementary materials of the published protocol for the questionnaire).22 To reduce question order bias, the presentation order of the questions was randomised for the first three items (confidence, interest and importance).

    Contact details and demographic information about BMA members were obtained from the BMA membership database: name, title, email address, specialty, sex, age and date qualified. Survey data were gathered using SurveyMonkey. Non-responding volunteers were sent up to five reminders to complete the survey.

    Outcome measures

    Primary outcome measure

    The primary outcome was the readers' level of confidence in the conclusions drawn in the article, measured on a 10-point Likert scale from 1= ‘not at all confident’ to 10= ‘extremely confident’.

    Secondary outcome measures

    The three secondary outcomes, all measured on similar 10-point Likert scales, were: readers' ratings of the importance of the article, interest in the article and likelihood to change practice on the basis of the article.

    Ethics and trial registration

    We did not submit the study for ethical approval as this is not required for this type of survey with doctors in the UK. However, the study proposal and study materials were reviewed by The BMJ’s Ethics Committee and they did not have substantive ethical concerns. To avoid biassing participants' responses, details of the study objectives and design were not given to participants. The study protocol22 was not published until data collection was complete so as not to potentially influence participants’ responses. Consent to take part was assumed by completion of the study questionnaire. The trial was registered at ClinicalTrials.gov (NCT02548312) just before recruitment commenced.

    Statistical analysis

    Sample size justification

    We calculated that to have 90% power to detect a one-unit difference on the 10-point ‘confidence’ scale between the groups, 121 readers were needed in each of the four COI statement groups, based on a simple Student’s t-test with an estimated SD of 2, with a two-sided 1% significance level to provide some adjustment for multiple testing between the four COI statements. However, as differences between the results for the two clinical reviews were considered important to quantify, a total of 968 readers were required to account for the eight permutations. Assuming a response rate of around 50% based on previous BMJ trials of similar design,6 7 we calculated we needed to invite at least 1936 readers to take part. Accordingly, in sample 1, for each of the eight groups, 255 readers (85 GPs, 85 consultants and 85 junior doctors) were invited to take part. We assumed that a one-unit difference on the 10-point scale was important on the basis that a 0.5-unit difference was important in our previous studies using a 5-point scale.6 7 Similarly, the observed SD for the 5-point scale was ∼1, and hence we assumed that, for a 10-point scale, the SD would be twice as large. As sample 1 only yielded a 9% response rate and we anticipated a similar yield when asking for volunteers, we broadened the sample to 11 004 in sample 2.

    Statistical analysis

    A factorial analysis of covariance (with COI statement and clinical review type as the two factors) was carried out to assess their impact on the primary outcome (level of confidence) and secondary outcomes (importance, interest and likelihood to change practice) adjusting for the effect of doctor type (GP, consultant or junior doctor), gender, age and the number of years since qualification. Separate analyses of covariance were performed for each of the two clinical reviews, and, in addition, for the subgroups who were currently treating the conditions. The impact on the likelihood to change practice was assessed using Χ2 tests. Analyses of variance and Χ2 were used to compare non-responders with responders in terms of age, gender, doctor type (GP, consultant or junior doctor) and number of years since qualification.

    Patient and public involvement

    We did not include patients as study participants. Patients were not involved in setting the research question, designing the study, the conduct of the study or the interpretation of the results. One of The BMJ’s patient editors, recently invited patients and members of the public attending a workshop at the Cochrane Colloquium 2018 on ‘Meeting the challenge of research empowerment through co-production and expert patient review’ how patients and the public could have been involved in this research and they reported that they did not see relevant opportunities to do so. However, a patient and public reviewer for The BMJ did make an interesting suggestion for a further study with the general public as the participants as it is important to know how people value and consider COIs when reading articles.

    Results

    Samples

    Sample 1

    Overall, 182/2040 (9%) responded, but the response rate was lower for the article on dyspepsia (81/1020, 8%) than gout (101/1020, 10%). On reading responses to the survey for those who received the dyspepsia review, we identified a problem with the content of the manuscript. A few respondents queried the appropriateness of including a section on prokinetic drugs given the recent withdrawal of drugs and safety concerns but no mention of this in the article. As such, we removed this section from the manuscript before using it in sample 2 and started the study again.

    Sample 2

    We obtained a random sample of 11 004 BMA members meeting the study eligibility criteria. After removing overlap with sample 1, we invited 10 889 doctors to volunteer to take part in a research project for The BMJ (figure 1). On sending the email invitation, 96 email addresses bounced and 97 had already opted out of SurveyMonkey so had to be excluded. Of the 10 696 eligible email addresses, we recruited 1065 volunteers (10%) and 749 (70% of those who volunteered) completed the survey; n=376 dyspepsia and n=373 gout. A third of respondents were consultants, a third GPs and a third junior doctors; 46% were male and the mean age was 44 years (table 2). All analyses are based on data collected in sample 2.

    Figure 1
    Figure 1

    Participant flow chart.

    View this table:
    Table 2

    Baseline characteristics by group of allocation

    Primary outcome

    There was no significant difference between the groups in the readers' level of confidence in the conclusions drawn in the article for the gout (p=0.32) or dyspepsia (p=0.78) reviews (table 3). The mean confidence rating scores for all of the groups receiving the gout review was at least 7 out of 10, and for the dyspepsia review it was at least 6.

    View this table:
    Table 3

    ANCOVA analysis of the level of confidence, importance and interest in the reviews by intervention group adjusting for age, sex, job type and years since qualification

    Combining results over both reviews showed no differences in confidence between the COI groups (p=0.54), and no evidence of a difference between reviews in the variability between the COI groups (p=0.53), but respondents had a higher level of confidence in the gout review than the dyspepsia review (p<0.001).

    Secondary outcomes

    Importance of the article

    There was no significant difference between the groups in readers' ratings of the level of importance of the article for the gout (p=0.09) or dyspepsia (p=0.79) reviews (table 3).

    Combining results over both reviews showed no overall differences in level of importance between the COI groups (p=0.79) and no evidence of a difference between reviews in the variability between the COI groups (p=0.14), but respondents gave higher ratings of importance for the gout review (p=0.002).

    Interest in the article

    For the gout review, participants rated reviews with advisory board and consultancies COI as significantly less interesting than those with no COI (p=0.018 with Bonferroni correction), but there was no significant difference between the groups for the dyspepsia review (p=0.83) (table 3).

    Combining results over both reviews showed no overall differences in level of interest between the COI groups (p=0.46) and no evidence of a difference between reviews in the variability between the COI groups (p=0.12), but respondents gave higher ratings of interest for the gout review (p<0.001).

    Likelihood to change practice

    Almost half of respondents (178/373, 48%) who received the gout review reported that they were currently treating patients with gout, 28% (103/373) were not currently treating them and 24% (90/373) reported they do not treat patients with this condition. Of those who were currently treating gout, 33% (59/103) indicated that the article recommended practice differing from their current practice.

    Over half of respondents (207/376, 55%) who received the dyspepsia review reported that they were currently treating patients with dyspepsia, 23% (85/376) were not currently treating them and 22% (83/376) reported they do not treat patients with this condition. Of those who were currently treating dyspepsia, 39% (80/207) indicated that the article recommended practice differing from their current practice.

    Among participants indicating that they treat the condition and that the article’s recommendations differed from their own practice, there was no significant difference in likelihood to change practice between groups (gout: p=0.59, n=59; dyspepsia: p=0.56, n=80) (table 4).

    View this table:
    Table 4

    Likelihood to change practice for those currently treating gout/dyspepsia and their own practice differed from the recommendations given in the review

    Subgroup analysis

    Analysis of the subgroups who were currently treating the conditions showed no significant differences between the groups for the level of confidence (primary outcome) in the article (gout: p=0.18; dyspepsia: p=0.64) (table 5).

    View this table:
    Table 5

    Analysis of covariance of the level of confidence in the reviews by intervention group adjusting for age, sex, job type and years since qualification for subgroups who were currently treating patients with gout or dyspepsia

    Analysis of non-responders

    Respondents who completed the survey were significantly older, had been qualified for longer and were more likely to be female than those who did not complete or volunteer, p<0.05 (table 6).

    View this table:
    Table 6

    Characteristics of volunteers, completers and non-responders

    Discussion

    Doctors’ confidence in the conclusions drawn in two educational reviews was not significantly influenced by a range of financial COI statements that are commonly reported to journals and frequently occur in medical practice. When the results for the two reviews were combined, we found no significant difference between the COI statement groups in the importance or interest doctors attached to the article or their self-reported likelihood to change practice based on the article. However, we did find a significant difference between the groups in level of interest for the gout review; doctors rated the gout review with advisory board and consultancies COI as significantly less interesting than when there was no COI. Subgroup analysis of those who were currently treating the conditions found no significant difference in the level of confidence in the article.

    Three previous randomised controlled trials5–7 on the effect of financial COIs on readers’ perceptions of research found strikingly different results. In our first trial,6 readers randomised to receive a drug study written by three authors with financial COIs (employees of a fictitious company who potentially held stock options in the company) indicated these as significantly less interesting, important, relevant, valid and believable than those randomised to receive the same article written by authors with no COIs declared. In our second trial,7 we tested the effect on a non-drug study and also varied the type of COI statement (author potentially held stock options in the company vs author was a recipient of funding for studentships and research grants vs no competing interest declared). Once again, we found that overall, importance, relevance, validity and believability ratings were significantly lower in the group with the financial COI statement than in the no competing interest group. Validity ratings for the financial COI statement group were also significantly lower than for the group receiving the research grants statement. The current study sampled doctors from the same large membership database and applied a similar methodology, but found no significant difference in the confidence in the conclusions drawn (primary outcome) between the groups. A third randomised trial exploring the influence of clinical trial funding on clinician perceptions of trials with a high, medium or low methodological rigour, found that industry funding negatively impacted on perceived methodological quality and willingness to implement trial findings regardless of trial quality.5 However, two further trials of the effects of COIs in trial abstracts, one with US physicians and another with French physicians, found no significant evidence, respectively, that COIs influenced the likelihood to prescribe a fictitious drug or the confidence of physicians in the abstract conclusions.9 10 In contrast to previous trials evaluating the influence of COIs on readers’ perceptions, our study used a clinical review article (where possible biases may be less visible) and subtler financial COIs (although these were still typical of those seen in medical practice).6 7

    A key strength of this study is its randomised study design in a research area where there are few experimental studies. This study had several limitations. First, our initial sampling approach yielded a very low response rate of 9% (not unusual for surveys of doctors and researchers)23–25 and the study was underpowered to show an effect. As such, we broadened the sampling frame by seeking volunteers at the outset and this yielded a response from 70% of those recruited. The initial low response rate was surprising as in both our earlier trials, with postal administration, we achieved response rates of 59%6 7 and sampled readers from the same membership database. However, mode of administration can influence response rates26 and response rates across all methods of survey administration have declined over time.27 The extent to which we can generalise the findings from the small sample of volunteers who had the time, interest and motivation to take part to all readers is unknown; those who volunteered may differ from their peers in ways we did not capture. Second, we excluded doctors in private practice, so we are unable to generalise the results beyond practising National Health Service doctors. However, to get a representative sample, we sampled doctors from a wide range of clinical specialties. We also used two clinical reviews to help make the findings generalisable beyond a single clinical topic. Third, participants were told they were taking part in a research project and this may have influenced the way they read the article and responded to the questions. Fourth, participants were asked to read an article that they might not usually read; approximately half of participants in each group reported that they were not currently treating patients with gout/dyspepsia. While this may have influenced responses, we deliberately selected two general clinical topics commonly presented and the task did not require respondents to have in-depth knowledge of the assigned topic. Further research could study the effects of COI statements in the context of articles that are highly relevant to readers’ own clinical practice. Fifth, respondents were significantly older than non-respondents, but this was in keeping with both our previous trials.6 7 Sixth, we only looked at the effect on self-reported outcome measures not on actual changes to practice. Finally, we were unable to pool the results from our two sampling approaches as we used different recruitment processes and modified one of the reviews.

    In a recent cross-sectional study, the authors reported a higher prevalence of disclosed COIs in commentaries, editorials and narrative reviews.28 This finding, combined with the fact that author bias in educational articles may be less obvious to readers, and our own finding that COI statements do not seem to affect reader perception of such articles, is particularly concerning. Such articles are widely read by clinicians for their summaries of available evidence and clinical care recommendations.29 Furthermore, our trial used articles on common conditions with relatively uncontroversial treatments, but the role of COIs may be particularly pertinent in articles on the clinical use of novel, potentially expensive, therapeutic agents.

    Our findings may be explained by a lower awareness among clinicians that competing interests may influence the conclusions of educational articles, just as they may research articles. Furthermore, educational articles are typically written by highly regarded clinicians who are well known or ‘trusted’ experts in their field, which may mean that COIs are considered by readers to be less influential or less important in this context. It is also possible that readers did not consider the COIs to be directly relevant to the topics of the educational articles, and the perceived role of COIs may be context-dependent. COI statements may therefore be more meaningful if they were to specify the relevance of a COI to the subject topic, rather than, as an example, simply stating the existence of a tie with a pharmaceutical company. Alternatively, readers may have considered the included COI statements too mild and not sufficiently alarming to warrant greater scepticism of the review’s conclusions. For example, a prior randomised trial has demonstrated that COIs incorporating stocks and shares influenced perceptions of research articles more than COIs involving research grants.7 We only included a COI for the last of the three listed authors and this may have influenced the magnitude of any effect, but our earlier trial only reported a COI for one of three authors and did find a significant effect.7 Many readers in this study may also have been familiar with the medical conditions under discussion, and their own clinical practice already in alignment with the review conclusions. We further speculate that levels of trust in the educational reviews used in this study may also have been high due to their dissemination by The BMJ, a widely read and recognised UK-based general medical journal. Accordingly, our findings may not be generalisable to articles in smaller/specialty journals. However, the disseminated articles were not portrayed to participants as accepted or published BMJ articles, but rather formatted to mimic manuscript submissions without indication of whether the submission would be published in the journal.

    Future research should aim to explore why COI statements in educational articles may not affect reader perceptions. For instance, as publishing COI statements has become standard practice, do readers now no longer pay attention to such statements? Or do they perceive these as unimportant and unlikely to bias an article’s conclusions? Furthermore, our research has focused only on financial COIs but it would also be important to evaluate the effect of non-financial or indirect COIs on readers’ perceptions, such as unpaid consultancies which may include reimbursements for travel expenses, meals and drinks.29 Against the backdrop of risk of industry-guided bias in clinical practice, journal editors need to tackle possible reader inattention towards COIs in educational articles. Possible mitigating solutions include policies that exclude authors with relevant COIs from authoring clinical educational articles (as has been adopted by The BMJ),13 17 30 or a requirement for such articles to be based on systematic, rather than narrative, reviews. Tackling readers’ understanding of COIs in educational articles is also crucial. This may involve emphasising the role of COIs in critical appraisal, as part of the medical curriculum. In addition, given that some form of COI among leadership figures in clinical research is now very common, it is possible that the simple presence of a COI is not sufficient to attract attention. Rather, in addition to reporting COIs, authors or journal editors should consider positioning the COI in relation to the topic of the article so that any context-specific risk of bias is clearer to the reader.

    Acknowledgments

    The authors would like to thank the editors in The BMJ education team (Cath Brizzell, Tony Delamothe, Giselle Jones, Navjoyt Ladher, Emma Parish, Alison Tonks, Sophie Cook), and Theo Bloom for their help in designing the study; the authors of the original clinical reviews for giving permission to use their work and Emma Parish for editing these articles for use in the study; Keith Bates, Data Analyst at the British Medical Association, for providing data about BMA members and help with sampling and our patient editor Amy Price, Julie Sprakel (BMJ patient and public reviewer) and participants at the workshop at the Cochrane Colloquium 2018 for their thoughts on how patients might have been involved in this study and ideas for further research.

    References

    1. 1.↵
      2. Lo B ,
      3. Field MJ
      . Institute of Medicine (US) Committee on Conflict of Interest in Medical Research, Education, and Practice. Conflict of interest in medical research, education, and practice: National Academies Press, 2009.
    2. 2.↵
      2. Thompson DF
      . Understanding financial conflicts of interest. N Engl J Med 1993;329:573–6.doi:10.1056/NEJM199308193290812
      OpenUrlCrossRefPubMedWeb of Science
    3. 3.↵
      2. Lundh A ,
      3. Lexchin J ,
      4. Mintzes B , et al
      . Industry sponsorship and research outcome. Cochrane Database Syst Rev 2017;2:MR000033.doi:10.1002/14651858.MR000033.pub3
      OpenUrlCrossRefPubMed
    4. 4.↵
      2. Glaser BE ,
      3. Bero LA
      . Attitudes of academic and clinical researchers toward financial ties in research: a systematic review. Sci Eng Ethics 2005;11:553–73.doi:10.1007/s11948-005-0026-z
      OpenUrlCrossRefPubMedWeb of Science
    5. 5.↵
      2. Kesselheim AS ,
      3. Robertson CT ,
      4. Myers JA , et al
      . A randomized study of how physicians interpret research funding disclosures. N Engl J Med 2012;367:1119–27.doi:10.1056/NEJMsa1202397
      OpenUrlCrossRefPubMedWeb of Science
    6. 6.↵
      2. Chaudhry S ,
      3. Schroter S ,
      4. Smith R , et al
      . Does declaration of competing interests affect readers' perceptions? A randomised trial. BMJ 2002;325:1391–2.doi:10.1136/bmj.325.7377.1391
      OpenUrlFREE Full Text
    7. 7.↵
      2. Schroter S ,
      3. Morris J ,
      4. Chaudhry S , et al
      . Does the type of competing interest statement affect readers' perceptions of the credibility of research? Randomised trial. BMJ 2004;328:742–3.doi:10.1136/bmj.38035.705185.F6
      OpenUrlFREE Full Text
    8. 8.↵
      2. Sackett D
      . Industry-initiated drug trials are far less credible to Canadian internists than investigator-initiated trials. Can J Gen Intern Med 2008;3:29–32.
      OpenUrl
    9. 9.↵
      2. Silverman GK ,
      3. Loewenstein GF ,
      4. Anderson BL , et al
      . Failure to discount for conflict of interest when evaluating medical literature: a randomised trial of physicians. J Med Ethics 2010;36:265–70.doi:10.1136/jme.2009.034496
      OpenUrlAbstract/FREE Full Text
    10. 10.↵
      2. Buffel du Vaure C ,
      3. Boutron I ,
      4. Perrodeau E , et al
      . Reporting funding source or conflict of interest in abstracts of randomized controlled trials, no evidence of a large impact on general practitioners' confidence in conclusions, a three-arm randomized controlled trial. BMC Med 2014;12:69.doi:10.1186/1741-7015-12-69
      OpenUrlCrossRefPubMed
    11. 11.↵
      PLoS Medicine Editors. Does conflict of interest disclosure worsen bias? PLoS Med 2012;9:e100120.doi:10.1371/journal.pmed.1001210
      OpenUrl
    12. 12.↵
      2. Katz D ,
      3. Caplan AL ,
      4. Merz JF
      . All gifts large and small: toward an understanding of the ethics of pharmaceutical industry gift-giving. Am J Bioeth 2003;3:39–46.doi:10.1162/15265160360706552
      OpenUrlCrossRefPubMedWeb of Science
    13. 13.↵
      2. Siwek J
      . AFP’s conflict of interest policy: disclosure is not enough. Am Fam Physician 2014;89:161–7.
      OpenUrlPubMed
    14. 14.↵
      2. Relman AS
      . New "Information for Authors"--and readers. N Engl J Med 1990;323:56.doi:10.1056/NEJM199007053230111
      OpenUrlCrossRefPubMedWeb of Science
    15. 15.↵
      2. Kassirer JP ,
      3. Angell M
      . Financial conflicts of interest in biomedical research. N Engl J Med 1993;329:570–1.doi:10.1056/NEJM199308193290810
      OpenUrlCrossRefPubMedWeb of Science
    16. 16.↵
      2. Drazen JM ,
      3. Curfman GD
      . Financial associations of authors. N Engl J Med 2002;346:1901–2.doi:10.1056/NEJMe020074
      OpenUrlCrossRefPubMedWeb of Science
    17. 17.↵
      2. Chew M ,
      3. Brizzell C ,
      4. Abbasi K , et al
      . Medical journals and industry ties. BMJ 2014;349:g7197.doi:10.1136/bmj.g7197
      OpenUrlFREE Full Text
    18. 18.↵
      2. Rosenbaum L
      . Reconnecting the dots — reinterpreting industry–physician relations. N Engl J Med Overseas Ed 2015;372:1860–4.doi:10.1056/NEJMms1502493
      OpenUrl
    19. 19.↵
      2. Rosenbaum L
      . Understanding bias--the case for careful study. N Engl J Med 2015;372:1959–63.doi:10.1056/NEJMms1502497
      OpenUrlCrossRefPubMed
    20. 20.↵
      2. Rosenbaum L
      . Beyond moral outrage--weighing the trade-offs of COI regulation. N Engl J Med 2015;372:2064–8.doi:10.1056/NEJMms1502498
      OpenUrlCrossRefPubMed
    21. 21.↵
      2. Drazen JM
      . Revisiting the commercial-academic interface. N Engl J Med 2015;372:1853–4.doi:10.1056/NEJMe1503623
      OpenUrlCrossRefPubMed
    22. 22.↵
      2. Schroter S ,
      3. Pakpoor J ,
      4. Morris J , et al
      . Effect of different financial competing interest statements on readers' perceptions of clinical educational articles: study protocol for a randomised controlled trial. BMJ Open 2016;6:e012677.doi:10.1136/bmjopen-2016-012677
    23. 23.↵
      2. Asch DA ,
      3. Jedrziewski MK ,
      4. Christakis NA
      . Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997;50:1129–36.doi:10.1016/S0895-4356(97)00126-1
      OpenUrlCrossRefPubMedWeb of Science
    24. 24.↵
      2. Mulligan A ,
      3. Hall L ,
      4. Raphael E
      . Peer review in a changing world: an international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology 2013;64:132–61.doi:10.1002/asi.22798
      OpenUrlCrossRef
    25. 25.↵
      2. Cunningham CT ,
      3. Quan H ,
      4. Hemmelgarn B , et al
      . Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol 2015;15:32.doi:10.1186/s12874-015-0016-z
      OpenUrlCrossRefPubMed
    26. 26.↵
      2. Bowling A
      . Mode of questionnaire administration can have serious effects on data quality. J Public Health 2005;27:281–91 https://doi.org/.doi:10.1093/pubmed/fdi031
      OpenUrlCrossRefPubMedWeb of Science
    27. 27.↵
      2. Walker A ,
      3. Maher J ,
      4. Coulthard M , et al
      ; Living in Britain. Results from the 2000 General Household Survey. London: The Stationary Office, 2001.
    28. 28.↵
      2. Grundy Q ,
      3. Dunn AG ,
      4. Bourgeois FT , et al
      . Prevalence of disclosed conflicts of interest in biomedical research and associations with journal impact factors and altmetric scores. JAMA 2018;319:408–9.doi:10.1001/jama.2017.20738
      OpenUrlCrossRefPubMed
    29. 29.↵
      2. Mintzes B ,
      3. Grundy Q
      . The rise of ambiguous competing interest declarations. BMJ 2018;361:k1464.doi:10.1136/bmj.k1464
      OpenUrlFREE Full Text
    30. 30.↵
      2. Loder E ,
      3. Brizzell C ,
      4. Godlee F
      . Revisiting the commercial-academic interface in medical journals. BMJ 2015;350:h2957.doi:10.1136/bmj.h2957
      OpenUrlFREE Full Text

    Footnotes

    • Patient consent for publication Not required.

    • Contributors FG initiated the study. SS, JP, JM, MC, FG contributed to the study design, including the wording of the competing interest statements, outcome measures and sampling strategy. JM estimated the required sample size and carried out the statistical analysis. SS was responsible for running the trial and wrote the first draft of the manuscript with help from JP. SS, JP, JM, MC, FG contributed to the interpretation of the results and the writing of the manuscript.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests SS and FG are employed full time by The BMJ; MC was formerly employed by The BMJ; JP was a Clegg Scholar at The BMJ and JM is a statistics editor for The BMJ. None of the authors work directly for BMJ Open or are involved in the decision-making process for articles submitted to BMJ Open.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement The anonymised individual participant data will be shared on reasonable request.