Article Text

Download PDFPDF

Original research
Mixed methods evaluation of workshops for citizen health advocates about financial conflicts of interests in healthcare
  1. Ray Moynihan1,
  2. Alice Fabbri2,
  3. Lisa Parker3,
  4. Lisa Bero2
  1. 1Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
  2. 2Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia
  3. 3Centre for Values, Ethics and the Law in Medicine, University of Sydney, Sydney, New South Wales, Australia
  1. Correspondence to Ray Moynihan; raymoynihan{at}


Objectives To evaluate workshops delivered to citizen health advocates about financial conflicts of interest in healthcare, transparency databases which disclose industry payments in the USA and Australia and the pros and cons of advocacy groups accepting industry sponsorship.

Design Thematic analysis of workshop participant recorded discussions, and pre, post and 3-month follow-up questionnaires on confidence and knowledge about financial conflicts of interest, transparency databases and the merits of advocacy organisations accepting industry sponsorship.

Participants and setting 48 citizen health advocates participated in a half-day workshop, held in four Australian cities, which ended with a 1-hour recorded discussion. Participants were recruited with assistance from leading state-based health advocacy organisations.

Results The thematic analysis of the recorded discussions revealed two major themes, (i) transparency and (ii) relationships with industry; and three minor themes: a lack of awareness about conflicts of interest and transparency, issues relating to trust and next steps in terms of potential reforms. In relation to transparency, participants felt strong support for transparency, strongly favouring the mandatory, extensive and accessible US Open Payments over the self-regulatory Australian model. Participants also noted that transparency had limitations, including the utility of disclosed information. In relation to industry sponsorship of advocacy groups, some participants expressed an openness to and support for accepting sponsorship, while many expressed a caution around potential downsides. Questionnaire results showed increases in both confidence and knowledge after the workshop, though only 23 of 48 participants returned the 3-month follow-up questionnaire.

Conclusions Following a half-day workshop, citizen health advocates recruited by leading health advocacy organisations expressed strong support for tough transparency rules, and mixed feelings about advocacy groups accepting sponsorship from industry. Study limitations include a non-representative sample and a large drop-out at the 3-month post-workshop follow-up.

  • public health
  • ethics
  • health services administration

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Strengths and limitations of this study

  • A key strength of the study is the use of mixed methods to evaluate an innovative series of workshop for citizen health advocates, about conflicts of interest in healthcare.

  • The qualitative analysis of participant discussion offers a rare insight into citizen health advocate perspectives of two major contemporary issues, transparency and the merits of financial relationships with industry.

  • The workshops were planned and delivered with input and participation from leading Australian citizen health advocacy groups.

  • A key limitation is that the workshop participants were not representative, but rather recruited through advertising by advocacy groups.


Extensive financial conflicts of interest are recognised as a major challenge within healthcare, potentially jeopardising the integrity of medical research, education and practice.1 2 Industry-sponsored trials are more likely to find favourable results for sponsors’ products.2 Doctors attending industry-funded educational events are more likely to prescribe sponsors’ products.3 Specialists with financial ties to industry tend to make favourable statements about sponsors’ products.4 And guideline panels which set and expand disease definitions often comprise a majority of members with financial ties to industry.5 At the same time there is growing evidence that many consumer or citizen health advocacy groups are reliant on industry funding.6 These groups play increasingly important roles, including advocating for patients, co-designing research and contributing to guideline development, and there are concerns that financial conflicts may be distorting the patient voice in healthcare.7 8

As a result of growing concerns about the distorting impacts of endemic financial ties to industry, there have been moves in many countries towards more transparency in healthcare. As a result, new databases are becoming available with details about ties between industry, individuals and organisations. In the USA, a series of high-profile congressional investigations into financial conflicts of interest in healthcare ultimately led to the passing of a bipartisan law, The Physician Payments Sunshine Act, commonly known as the Sunshine Act. As a result of this law, since 2013 pharmaceutical companies have had to disclose all payments to doctors in a highly accessible government-run database, called Open Payments. ( In 2018, new legislation was introduced in the USA proposing to extend the existing law, so that all industry payments to advocacy groups would have to be disclosed in the database.9 Australia, like many countries, does not have legally binding mandatory disclosure, although its industry-controlled system does include disclosure of payments to consumer groups.10

Alongside moves to transparency, there are growing calls for more independence between industry and other sectors within healthcare, including in research, education and practice,1 particularly in light of growing evidence of the problems of overdiagnosis and overuse within healthcare.11 For example, the British Medical Journal (BMJ) has undertaken a number of independence initiatives, such as excluding authors with financial ties from publishing educational content or clinical guidelines, and ending advertising revenues from breast milk substitute manufacturers.11 Others have advocated that the leadership of influential professional medical associations, responsible for much research and education, should be entirely free of financial ties to industry.12

There have been limited attempts to share the evidence about financial conflicts of interest in healthcare in non-medical settings. Odierna and colleagues conducted and evaluated workshops for non-medical participants in the USA, which included citizen/consumers, and covered elements related to financial conflicts of interest.13 14 In Australia, we have previously conducted two workshops with journalists, introducing the evidence about financial conflicts of interest and interacting with transparency databases,15 16 though without any evaluation. In light of the importance of public understanding of financial conflicts of interest,1 increasing scrutiny of industry funding of advocacy groups8 and calls for greater independence,11 we aimed to evaluate workshops which introduced citizens to financial conflicts of interest and transparency databases, and engendered discussion about industry sponsorship of advocacy organisations. While the terms ‘consumer’, ‘patient’, ‘citizen’ and ‘advocate’ are all used, for the purposes of consistency, we have used ‘citizen’ in this manuscript.



We worked with leading state-based Australian citizen (or consumer) peak health organisations to recruit people involved in advocacy to attend a half-day workshop, in each of four state capital cities in Australia. We used both quantitative and qualitative methods to evaluate workshops which: (i) introduced participants to the evidence about the extent and impacts of financial conflicts of interest in healthcare; (ii) introduced workshop participants to the transparency databases, from the USA and Australia, which feature details including the financial ties to industry, of individual health professionals, researchers and consumer groups; (iii) facilitated discussion among participants about financial conflicts of interest including their relevance to citizen health advocacy organisations. For the small quantitative component, we evaluated the workshop using very brief questionnaires to measure effects on knowledge and confidence about financial conflicts of interest and transparency databases, pre, post and after 3 months. The questionnaires were based on similar tools used previously by Odierna and colleagues.14 For the qualitative evaluation, we evaluated transcripts of 1-hour facilitated discussions with all participants which were recorded at the end of each workshop, drawing on framework analysis methods previously used in a research project with community focus groups,17 and based on the methods outlined by Ritchie and colleagues.18

Workshop content

The content of the 4-hour workshop was based on similar workshops on conflicts of interests, delivered to Australian journalists, with support from Cochrane Australia,15 16 but modified for citizen participants. The workshop materials consisted of a series of short didactic PowerPoint presentations, (online supplementary file 1) followed by a series of small-group activities and discussions. The structure of the workshop opened with an introduction to definitions of conflicts of interest in healthcare, followed by four main sections (see box 1.) Workshop content was developed and delivered by two academic researchers who conduct research on the impacts of conflicts of interest on research and practice, and have previously conducted qualitative research including focus groups, authors RM (male) and LB (female), who disclosed their conflicts of interests, and interests in the topic, at the beginning of the workshop.

Box 1

Content of workshops for citizen health advocates

The first section introduced evidence about industry sponsorship of research and medical education. The first small group session used the Australian transparency database22 which included company provided PDFs to find examples of industry sponsored educational events.*

The second section introduced evidence about industry sponsorship of individual scientists, or ‘key opinion leaders’, and the small-group exercise involved live searching of the US Open Payments database to find disclosures of individual doctors.

The third section focussed on industry funding of patient/advocacy groups, and the small-group exercise involved live searching of the Australian transparency database to find and discuss examples of advocacy groups receiving funding.

The fourth section involved a recorded 1-hour discussion, to explore participants views and feelings about the workshop content. While a structured guide was available, the aim was for an open and free-flowing discussion.

  • *One of several elements of the Australian transparency system moved from PDFs to a searchable database in August 2019.

Workshop recruitment

Participants were recruited with assistance from each of the state-based peak or umbrella groups which hosted a workshop. Each organisation sent a short description of the workshop to their networks. Participation was open to both organisation staff, and citizens connected to the organisation. Interested participants were sent an Explanatory Statement (online supplementary file 2) about the workshops and evaluation research, and all signed consent forms before participating. We aimed to recruit between 50 and 100 participants, at up to five workshops. Participants were not reimbursed, but received travel assistance when requested, and refreshments and a light lunch, provided by the citizen organisations and in one instance, the University of Sydney. LB and RM paid for their own travel expenses.

Workshop evaluation and analysis

The theoretical framework underpinning the qualitative aspects of the study, was drawn from the phenomenology approach, which seeks to comprehend the ‘constructs, concepts or ideas people use in everyday life to make sense of their world’.18 The methods used to evaluate the themes of the 1-hour recorded discussions with participants were a simplified version of ‘framework analysis’ method as described by Ritchie and colleagues18 and used previously by Moynihan and colleagues.17 Two authors, RM and AF independently assessed the transcripts of all four workshops and developed a draft framework of themes and subthemes, in Excel. These were merged and shared with co-authors LP and LB, who each read one transcript, and offered feedback on the themes and subthemes. RM and AF then each coded all four transcripts to the framework, and preliminary findings were shared with co-authors for comment, and then refined.

The quantitative evaluation of confidence and knowledge was a simple, very brief questionnaire, (online supplementary file 3) based on the short questionnaire used by Odierna and colleagues in evaluating a similar workshop on critical appraisal, which was based on validated and previously used instruments,14 and modified during pilot testing. Six questions about confidence covered: confidence in explaining a financial conflict of interest, and a systematic review of studies; confidence in ability to use the Australian transparency database, and the US database; confidence in ability to discuss evidence about the impacts of industry sponsorship of research, and the pros and cons of accepting industry sponsorship from a pharmaceutical company. Three questions about knowledge covered the most reliable source of evidence, the source of information on industry payments in Australia and the minimum threshold of payment for disclosure in the USA.

The questionnaire was given to participants prior to the start of the workshop, at the conclusion and was emailed 3 months later, with one email reminder. The pre-questionnaire also asked basic demographic questions including: how participants would describe themselves, gender, education and age. For the quantitative analysis, for each confidence question, we compared the proportion of respondents who indicated 4 or 5 on a 1 to 5 scale of confidence, pre workshop, post workshop and at 3 months, using McNemar’s test. For each knowledge question in our questionnaire, we compared the proportion of correct answers, pre and post workshop, and at 3 months, using McNemar’s test.

Research team

In line with the Consolidated criteria for Reporting Qualitative research checklist for the reporting qualitative research,19 we report that we have expertise across overdiagnosis, pharmacy, research integrity, public health ethics, medicine and evidence-informed practice. RM is a postdoctoral researcher with a long-term interest in conflicts of interest, greater independence between industry and other sectors within healthcare and overdiagnosis. AF is a postdoctoral researcher whose work focusses on pharmaceutical policy and commercial influences on health. LP is a bioethicist, medical clinician and a postdoctoral researcher whose work focusses on pharmaceutical policy and commercial influences in health. LB uses mixed methods in her research on bias, conflicts of interest, research integrity and research implementation in biomedicine and public health, which spans decades.

All workshop participants read the Explanatory Statement and gave written informed consent.

Patient and public involvement

Staff from leading state-based citizen/consumer health organisations gave feedback on the plans to deliver and evaluate the workshops, helped recruit participants, hosted three of the workshops and co-hosted the fourth.


Four organisations agreed to host a workshop in a state capital city: Health Consumers Queensland in Brisbane; Health Consumers New South Wales in Sydney; Health Issues Centre in Melbourne and the Health Consumers Alliance of South Australia, Inc, in Adelaide. A total of 48 participants took part in the workshops, with an average age of 60, and basic demographic information is available in table 1.

Table 1

Workshop participant characteristics. (n=48)

Qualitative findings


The structured guide for the recorded 1-hour discussion proved largely redundant, as participants enthusiastically engaged in discussion about the material covered in the workshop. Our framework analysis of the transcripts of the four recorded discussions revealed two major themes, and three minor themes. The first major theme was transparency, with a range of views and feelings, but overall very strong support for public disclosure of financial relationships between industry and other players within healthcare, including professionals and advocacy groups. The second major theme was about financial relationships between industry and advocacy organisations, with discussions featuring an openness to and sometimes support for relationships, as well as common and strong concerns about potential downsides. Minor themes included: a general lack of awareness about the evidence around conflicts of interest and transparency databases; trust, which emerged briefly in discussions in different ways and next steps, in terms of possible reforms.


The dominant theme of the recorded discussions was around the importance of transparency, with almost universal support for the idea of improving transparency systems to maximise disclosure. After using both the USA and Australian databases in exercises during the workshop, participants were able to compare both systems. Two important subthemes were the weakness of the Australian system as compared with the strengths of the US system, and the limits to the impacts of transparency alone.

Strength of US transparency database and weakness of Australian database

The US Open Payments transparency database was discussed very positively by many of the participants, described variously as: absolutely fantastic’, ‘far more extensive’, ‘much more accessible’, ‘far superior’, ‘much more transparent’. The Australian database, managed by the pharmaceutical industry under a self-regulated system, rather than run by the national government, was criticised as much harder to search, much less extensive due to exclusion of hospitality payments and not mandatory. At the time of the workshops Australian data was only available in individual PDFs supplied from each company.

Yeah, it's like we put something there, there you go, we're being transparent, but we don't want you to actually know how to actually find anything and if you do, you're going to spend the next 3 weeks stuck in this rabbit hole…

The limitations of transparency

While transparency was strongly supported, it was not regarded by all participants as a panacea for addressing the problem of financial conflicts of interest. It was viewed by some as a ‘really important first step’ towards more independence, but for others transparency had limited utility.

I'm not sure whether increased transparency will change the culture; and I think it is a cultural problem that exists between doctors and the pharmaceutical and the medical technology companies and it is dangerous, very.

Relationships with industry

The second key theme was discussion of potential positives and potential negatives of financial relationships between citizen/patient/consumer groups and industry sponsors. While there was some support for the idea from some participants, and an openness to it, there was much more discussion of potential downsides. Importantly, while pharmaceuticals is the most identifiable industry in healthcare, there was a common view that similar debates apply to other industries, including device-makers or private insurers.

Positive views of ties with industry

While several participants expressed an openness to the positive impacts of accepting industry sponsorship, one participant repeatedly offered very strong endorsement of the relationship. A key argument in favour of sponsorship from industry was that sometimes it is the only source of funding, that there is not enough non-industry funding, and funding is needed to enable citizen groups to undertake valuable advocacy and produce materials for patients:

…if the pharmaceutical companies weren't funding it…who would? You wouldn't get it from anybody else

The concern expressed was that valuable activities of advocacy groups may stop if industry sponsorship was lost:

…in consumer advocacy groups, the main one that I'm involved with, we don't receive any funding at all, but I know of other groups that have and it's been used in a very appropriate way and it's raised awareness, it's created fundraising, awareness with government and ultimately influencing more funding for particular research…

Other support for industry sponsorship came from participants involved with groups who accepted it, who argued there was benefit, and there was no evidence of unhealthy influence:

As I mentioned before, I'm part of an organisation that relies quite heavily on that and so I see the benefits all the time. I, to this point, haven't seen any negatives from it, so I just see all the work that it's doing and what it can potentially be doing

Another participant reflected that sponsorship was more acceptable, given that advocacy groups did not prescribe:

…in terms of then their real influence, they're not actually the prescribers of the medication.

Negative views of industry sponsorship

The most common concern from participants was that industry sponsorship would bring potentially unhealthy influences or biasses to their activities:

…you're going to think more favourably of that company. Anybody that's going to sponsor you, you're going to be more favourable towards…

One specific potential impact of pharmaceutical company sponsorship on advocacy groups which came up in discussion was the possibility the group may be used as part of the marketing of a product, including lobbying for listing for reimbursement on Australia’s national formulary, the Pharmaceutical Benefits Scheme, or PBS.

…it is my understanding that there is encouragement by pharmaceutical companies for patient - perspective patients, shall we say, to write to the PBS prior to listing in an effort to get products listed.

The threat that sponsorship poses to independence was another issue which emerged in discussion:

…we've been really clear that we want our policies to be fit for the purpose of our organisation and if we are saying we're 100 per cent independent volunteer organisation, then we're not going to accept money from pharmaceutical companies, because that would bias our independence

Responding to the view that industry sponsorship was often all that was available to fund advocacy activity, one participant stated:

my final comment would be something is not always better than nothing.

Lack of awareness of evidence and databases

There was a general view that participants had not heard much about the existing research on conflicts of interest in healthcare, or the publicly accessible transparency databases:

We are all involved and reasonably knowledgeable and yet I think many of us, I have, been taken by surprise at what's happening


The issue of trust came up a number of times in different ways during discussion. Some participants recognised that their groups independence was seen by patients as a reason for their trustworthiness :

People phone our information phone line and say, what doctor should I see, should I take this drug, they're asking—and we say, we can't give medical advice, but they ask us because they trust us.

Other participants noted that ties with industry might be seen by some to threaten both their independence and reputation:

The question really needs to be raised to clinicians and to patient support groups about how their reputation comes across

One participant thought that wider public debate about conflicts of interest might undermine public trust:

I actually worry a little bit about this topic getting out there, because it does destroy trust

And others thought transparency could help to build trust:

I think the whole question is really important and this thing about trust and transparency is really, really important.

Next steps

In discussion about possible future reforms, as mentioned above there was strong support for greatly enhanced transparency rules, in line with the mandatory government-run US system, but extended to cover other industries active in health, and to cover all payments to all organisations, including advocacy groups. In relation to industry sponsorship of advocacy groups, participants felt a need for more scrutiny of what sponsors wanted in return, new policies to help guide relationships and wider awareness-raising and debate about pros and cons of sponsorship, within a context of respect:

…getting back to the patient groups, I really, really feel that however (the topic of financial relationships with industry) is targeted has to be done with a great deal of sensitivity if you're opposing that, because we're talking about people and their narrative and there is so much power in the narrative and so that has to be honoured. So if you're actually going to come up in any way at all, it's not from a point of discrediting, it's a means of opening up further and it can never be, well you're wrong and we're right because it can never be that.

Quantitative findings

Analysis of the brief questionnaire found small improvements in confidence and knowledge. Forty-eight participants completed the pre and post questionnaires, with minimal missing data for each question. Only 23 participants completed the questionnaire at 3 months follow-up. For all six questions about confidence in ability to understand and discuss concepts about conflicts of interest and ability to use databases, comparisons showed increases in confidence between pre and post workshop, and pre workshop and 3-month follow-up (online supplementary file 4).

As for knowledge, there were small but statistically significant increases in the number of correct answers to two of the three questions, between pre and post, and between pre and 3-month follow-up, using McNemar’s test, with the important caveat that just under 50% of participants did not submit a questionnaire at 3 months (online supplementary file 4). Reasons for not submitting were not solicited from participants. There were increases for question 8 (about which form of evidence is most reliable), and question 10 (about the threshold for disclosure in the USA). There was no statistically significant change in the numbr of correct answers to question 9, about the current best source of transparency information in Australia.


Our results suggest an appetite among citizen health advocates and their organisations to understand and engage more with evidence and data about financial conflicts of interest in healthcare, and that accessible workshops on the topic can be successfully designed and delivered. Analysis of brief questionnaires given pre and post workshop and at 3 months follow-up suggest the potential for workshops to modestly improve confidence and knowledge around the major challenge of addressing financial conflicts of interest within healthcare. Qualitative analysis of recorded discussions indicates strong support for enhanced transparency rules, mixed feelings about the merits of advocacy groups accepting industry sponsorship and a need for more scrutiny and debate around these relationships.

Our study had important limitations. The primary limitation is that the sample of 48 participants was not designed to be representative: they were recruited via information from leading organisations, and clearly have an interest in the topic. Second, just over half of all participants did not return a questionnaire at 3 months, despite one email and a friendly reminder. And third the number of participants is relatively small. The chief strength of the study was its novel ability to facilitate a more informed discussion among active citizen health advocates about the evidence on financial conflicts of interest. The study also offered an extremely rare opportunity to hear the voices of citizens reflecting on highly salient contemporary debates within healthcare around both transparency, and independence from industry. A final strength lies in demonstrating a capacity to design and deliver complex materials on these topics in non-medical settings.

To our knowledge this was one of the first projects to design and evaluate workshops for citizen advocates about the issue of financial conflicts of interest. Odierna and colleagues have previously evaluated workshops for non-medical participants in the USA, which included some materials related to financial conflicts of interest.13 14 An analysis of the workshops published in 2012, which involved 102 consumers and 33 journalists, found improvements in confidence and knowledge in relation to critical appraisal of health evidence.14 A later analysis demonstrated how the workshops on critical appraisal had successfully included material on the ‘cycle of bias’ in healthcare and detecting financial and other conflicts within medical research.13

Evidence and concerns about financial conflicts of interest are intersecting with evidence and concern about too much medicine, including the challenge of overdiagnosis, resulting in stronger calls for more independence from commercial influence in healthcare.11 20 21 Notwithstanding important limitations, our study demonstrates it is both feasible and desirable to engage citizen advocacy organisations and their wider networks in a broader evidence-informed conversation about these challenges. Our findings suggest such workshops could feasibly be made available more broadly to both the general population, as well as within healthcare settings. On the question of transparency, participants expressed almost universal support for tough transparency rules, along the lines of the mandatory US system, strengthening the case for reform.20 One very practical outcome has been a new national meeting on these issues taking place in Australia in 2020, where citizen advocates will discuss the merits of industry funding and the challenges of finding funding from other sources.

Our results suggest three possible areas for future research. First, further qualitative research with citizen health advocates in different nations, about both transparency and the merits of independence, may be fruitful. Second, studies comparing processes and experiences with different transparency systems may be warranted. Third, more quantitative work on the ways in which industry sponsorship impacts in both hidden and tangible ways on advocacy groups is urgently needed. As emerged in the workshop discussions, like health professionals, and medical researchers, advocacy groups believe that they currently command a certain trust in the public imagination. As more groups become reliant on industry funding, and evidence about conflicts of interest becomes better known, the impacts on that trust are unclear.


We thank the following for their engagement with and support for this project: Melissa Fox, Chief Executive Officer, Health Consumers Queensland; Pip Brennan, Executive Director, Health Consumers Council (WA); Ellen Kerrins, Manager Advocacy and Policy, Health Consumers Alliance of SA Inc; Danny Vadasz, Chief Executive Officer, Health Issues Centre (Vic) and Anthony Brown, Executive Director, Health Consumers NSW; from Cochrane Australia Shauna Hurley, Communications and Engagement and Steve McDonald, co-director.



  • Contributors RM and LB designed the study and presented at all workshops; RM, LB, AF and LP were involved in data analysis; RM wrote the first draft of the manuscript; RM, LB, AF and LP were all involved in critically revising the manuscript and approving the final version. We thank Mark Jones for assistance with statistical analysis.

  • Funding RM is the recipient of an Australian National Health and Medical Research Council (NHMRC) Early Career Fellowship, 1124207, and co-investigator on an NHMRC CRE, 1104136. AF was supported as a postdoctoral fellow by the NHMRC, project grant 1122332

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval Ethics assessment was undertaken, and approval given, by the Bond University Human Research Ethics Committee in May 2018, (#RM03013).

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement No additional data are available.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.