Article Text
PDF
Abstract
Objectives Although the media can influence public perceptions and utilisation of healthcare, journalists generally receive no routine training in interpreting and reporting on medical research. Given growing evidence about the problems of medical overuse, the need for quality media reporting has become a greater priority. This study aimed to codesign and assess the feasibility of a multicomponent training intervention for journalists in Australia.
Design A small pragmatic feasibility study using a pre- and postdesign.
Setting 90 min online workshop.
Participants Eight journalists currently working in Australia, recruited through the study’s journalist advisor and existing contacts of the researchers.
Intervention The training intervention covered a range of topics, including study designs, conflicts of interest, misleading medical statistics, population screening and overdiagnosis. The intervention also provided tools to help journalists with reporting, including a Tip Sheet and list of expert contacts in health and medicine. Preworkshop and postworkshop questionnaires were administered via Qualtrics.
Measures Acceptability and feasibility of the intervention, and journalists’ knowledge of overdiagnosis and common issues with health stories. Quantitative results were analysed descriptively using SPSS. Qualitative data were thematically analysed.
Results All participants completed preworkshop and postworkshop questionnaires, and 6 completed the 6-week follow-up (75% retention). Feasibility findings suggest the intervention is acceptable and relevant to journalists, with participants indicating the workshop increased confidence with reporting on medical research. We observed increases in knowledge preworkshop to postworkshop for all knowledge measures on overdiagnosis and common issues with media coverage of medicine. Analysis of free-text responses identified several areas for improvement, such as including more examples to aid understanding of the counterintuitive topic of overdiagnosis and more time for discussion.
Conclusions Piloting suggested the multicomponent training intervention is acceptable to journalists and provided important feedback and insights to inform a future trial of the intervention’s impact on media coverage of medicine.
- education & training (see medical education & training)
- public health
- medical education & training
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
First pilot study to test the acceptability and feasibility of a codesigned, multicomponent training intervention for journalists in Australia.
This study was based on qualitative findings from journalist interviews and continuous discussion with the team’s journalist advisor.
While recruitment was challenging and the final sample was small, limiting generalisability, this study provides useful data for planning a larger trial.
Acceptable retention at 6 weeks postintervention, despite coinciding with the Omicron outbreak in Australia.
We did not assess the impact of the training on actual medical reporting, which is an important direction for future research.
Introduction
Journalists who report on health have a very difficult role, often translating complex science under deadlines into news that the public can understand.1 Despite many initiatives in this area,2 3 journalists generally receive no routine training in how to interpret or present medical research,4 5 and studies have found the quality of media coverage of medicine is often poor globally.6 Research has shown that media stories frequently emphasise the potential benefits of health interventions and under-report the harms, ignore important conflicts of interest, report numbers in misleading ways and frequently fail to report important study limitations.6 7As the media has a powerful influence on the public’s perceptions, behaviours and utilisation of healthcare,8 9 quality medical reporting is essential to ensure accurate portrayal of health and science information, which in turn can shape the health of the population.10 Given evidence has grown on the broader problems of medical excess, including overdiagnosis and overuse,11–13 the need for quality media reporting has become a greater priority.
The promotion of early detection tests through the media has been recognised as an important driver of overdiagnosis.14 Overdiagnosis occurs when individuals are labelled with a technically correct diagnosis that does not improve health outcomes or causes more harm than benefit,15 and is now recognised as a key challenge to human health and health system sustainability.16 A recent cross-sectional study of global media coverage of over 1000 media stories about five early detection tests (eg, three-dimensional mammography for breast cancer, liquid biopsy for cancer, blood biomarker tests for dementia) found that the potential benefits of testing were presented far more frequently than any potential harms (97% vs 37%, respectively), and the risk of overdiagnosis was only mentioned in 5% of stories.17 The general public and patients already tend to overestimate the benefits of early detection,14 18 meaning that the media’s often unrealistic and overly optimistic portrayals19 can reinforce these perceptions. Research has also found media coverage can influence patterns of healthcare utilisation, with positive coverage of a test or treatment associated with increases in utilisation.20
Given the powerful role that media can play in perpetuating the lack of awareness of the downsides of healthcare interventions, including early detection tests, strategies to improve media reporting of tests and overdiagnosis are needed.17 A recent qualitative study with 22 Australian health journalists found that lack of knowledge, training and time pressure were perceived to be the main barriers to balanced, critical reporting.21 Journalists felt that access to very short training programmes, ongoing support and information about both benefits and potential harms would enable more high-quality medical reporting.21 Building on these findings, this study aimed to develop and codesign a multicomponent training intervention for journalists in Australia, with a particular focus on the benefits and harms of diagnostic tests, including overdiagnosis. We also sought to explore the feasibility and acceptability of the intervention in a small pragmatic pilot with journalists, to inform the development of a randomised trial to improve journalists’ capacity to report more responsibly on medical tests and treatments.
Methods
Phase 1: codesign and development of the multicomponent intervention
We developed a multicomponent, educational and behavioural intervention for Australian journalists, with the aim of trying to help improve media reporting of medical tests, treatments and diagnoses. This was based on our qualitative findings with journalists and continuous discussion with the team’s journalist advisor and coinvestigator (LM; a national science reporter), which found high interest in very short training workshops as well as access to more resources and ongoing support.21 Suggestions from journalists included checklists, access to expertise for comment, fact-checking and reminders.21 Building on these findings and the key components of the Capability, Opportunity, Motivation and Behaviour (COM-B) model,22 our multidisciplinary team of journalists, researchers and clinicians developed the intervention components and content, resulting in (1) a 1.5-hour training workshop, (2) a tip sheet for guiding medical reporting and (3) an extensive list of expert contacts for independent comment or fact-checking. These intervention components were designed to address key constructs of the COM-B model by increasing knowledge, awareness, education and skills (capability), provide resources (opportunity) and increase confidence in reporting on medical research (motivation).22
Workshop
Based on our previous findings21 and discussions, the workshop was kept as short as possible, delivered over 90 min during a journalists’ lunchtime, by one journalist and three researchers with expertise in evidence-based medicine and overdiagnosis. The workshop covered a range of topics to address issues with media coverage of medicine highlighted in the current literature, with a particular focus on overdiagnosis (table 1). The workshop also included one group activity and several short pauses for questions (see online supplemental file 1 for workshop slides).
Supplemental material
Key topics covered in workshop
Additional materials to support reporting on medicine
A Tip Sheet was created to help guide journalists’ reporting on medical tests, treatments and diagnoses.21 Informed by previous work,17 23 the Tip Sheet included five simple, important questions and cautions to consider when writing stories on medicine, reflecting the content of the workshop (see online supplemental appendix B). A short and long version were designed to ensure the Tip Sheet was as clear and concise as possible, with links to additional further information and references (www.wiserhealthcare.org.au/tips-for-journos/). In addition, a list of 16 health and medicine research contacts with expertise in overdiagnosis and overuse of tests and treatments was created to facilitate access to experts to help with fact-checking stories.
Supplemental material
Phase 2: pilot testing
We conducted a small pilot study using a pre–post design to evaluate the feasibility and acceptability of the training intervention.
Eligible participants were journalists working in Australia, who were recruited by email through the team’s journalist advisor (LM) and existing contacts of the researchers. Interested participants emailed the study coordinator their interest or completed the preworkshop questionnaire after giving informed consent. Although preference was for face-to-face delivery, the workshop was delivered via zoom due to COVID-19 restrictions. Online delivery was limited to a maximum of 10 participants to facilitate discussion. Advertisement and recruitment commenced 2 weeks before the workshop, in the midst of the COVID-19 pandemic, and on the eve of the Omicron outbreak in Australia.
Outcome measures
Data were collected online via Qualtrics survey software immediately before (preworkshop), directly after (postworkshop) and 6 weeks after exposure (6-week follow-up) to the workshop.
Efficacy and acceptability outcomes
Along with collecting demographics (age, gender, years’ experience as a journalist, current role, media outlet), baseline confidence understanding medical statistics24 and awareness of the term ‘overdiagnosis’,25 a number of adapted or purpose-designed measures were administered preworkshop, postworkshop and at 6 weeks follow-up to assess the efficacy of the intervention in improving journalists’ knowledge of overdiagnosis and common issues with media coverage of medicine.
Awareness and knowledge of overdiagnosis.26 27 Six items assessed awareness and knowledge of overdiagnosis (eg, ‘All cancers will cause illness and death if they are not found or treated), measured on a three-point scale (eg, True, False, Don’t know). A newly developed free-text item was added to assess recall understanding of the definition of overdiagnosis (‘Please briefly describe overdiagnosis in your own words’).
Understanding of how diseases are defined. Four items were developed by the investigators to assess participants’ understanding of how diseases are defined (eg, ‘Disease definitions are based on distinct and objective biological structures or processes’), measured on a 5-point Likert scale (1=strongly disagree to 5=strongly agree).
Understanding of cancer screening statistics.28 Three questions asked participants to indicate what proves a cancer screening test saves lives (eg, ‘More cancers are detected in screening populations than in unscreened populations’), measured on a three-point scale (Proves, Does not prove, Don’t know).
Knowledge of the hierarchy of evidence. A single item measure assessed knowledge of study design strength (‘which study type is considered the strongest level of evidence?’), with six options (eg, Expert opinion, Randomised controlled trials, Systematic review and meta-analysis, Cohort).
Interpretation of relative and absolute risks.29 Two items from the validated 18-item test of patients’ medical data interpretations skills29 were utilised to assess journalists’ interpretation of relative and absolute risks.
Acceptability and feasibility. A number of quantitative items measured on five-point Likert scales (Strongly agree to Strongly disagree) as well as free-text questions assessed key feasibility and acceptability outcomes, including attitudes, perceived burden, understanding, perceived effectiveness and self-efficacy.30 Personal thoughts and reflections of the workshop facilitators were also collected and recorded.
Patient and public involvement
This study was based on our qualitative findings with journalists and continuous discussion with the team’s journalist advisor, who was involved from study conception, gave continuous feedback on workshop format, length and materials, and played a key role in recruiting study participants and facilitating the workshop.
Analyses
Quantitative results for each timepoint were analysed descriptively using absolute and relative frequencies using SPSS V.26. Qualitative data were analysed using thematic analysis31 to identify preliminary themes and patterns in the small amount of free-text responses, along with observations from workshop facilitators.
Drawing on our previous work26 32 to assess participants’ written understanding of overdiagnosis (‘please briefly describe overdiagnosis in your own words’), responses were coded as correct if they stated it was a diagnosis that is either (1) unnecessary, (2) does not improve health outcomes or would not cause harm if left undetected or (3) leads to harm or causes more harm than benefit. Responses that did not mention one of these three aspects or described a false positive (an incorrect diagnosis) were marked as incorrect. Responses were double-coded by two researchers independently (TC and BN) to ensure rigour, with any discrepancies discussed with a third researcher (TD).
Results
Pilot demographics
Out of the 10 journalists who registered, 8 journalists participated in the training workshop, held on 3 November 2021. All eight participants completed the preworkshop and postworkshop questionnaire, and 6 completed the 6-week follow-up (December 2022; 75% retention). Journalists worked at a range of media outlets (a major newspaper group, a public broadcaster, a specialty medical website, a major publisher of research-based news) and varied in years of experience working as a journalist (range: 1–25 years; see table 2). Most, however, were female, and had current roles in health and medical reporting.
Demographic characteristics of sample (N=8)
Reported confidence in understanding and interpreting medical statistics was relatively high preworkshop, with five participants (62.5%) indicating it was ‘easy’ or ‘very easy’ to understand medical statistics, six participants ‘somewhat’ or ‘strongly’ agreeing with the statement ‘I am confident that I can make sense of medical statistics’, and six participants ‘somewhat’ or ‘strongly’ disagreeing with the statement ‘I feel like I do not know how to interpret medical statistics’.
Efficacy outcomes
All participants indicated having seen or heard of the term ‘overdiagnosis’ before the workshop, although only 50% of free-text descriptions of overdiagnosis preworkshop were coded as correct. We observed an increase in overall knowledge from preworkshop to postworkshop, with knowledge increases observed for all 15 items, with the largest observed improvement in understanding the misleading nature of 5-year survival rates (see table 3 for all outcome measures at each time point).
Awareness, knowledge and beliefs preworkshop, postworkshop and at 6-week follow-up
Acceptability of the workshop
All participants agreed the workshop was relevant, something they would attend again and would recommend to other journalists (see figure 1). All participants agreed it would improve their reporting on medical tests, treatments and diagnoses and had increased their confidence with reporting on medical tests and treatments. Most agreed that the workshop was interactive, was interesting and kept their attention (see figure 1 for all acceptability outcomes).
Acceptability graph. Acceptability outcomes measured on a 5-point Likert scale (strongly disagree to strongly agree).
In terms of workshop length, half of participants (n=4) indicated the workshop was just the right length, three participants indicated it was too short and one participant indicated it was too long. There was, however, high interest in a longer version, with three participants indicating interest in a longer 2-hour workshop and four participants in a longer 3–4 hours workshop. When asked how much of the information in the workshop was new, 1 indicated ‘none’, 5 indicated ‘some’ and 2 indicated ‘most’.
Analysis of free-text responses indicated that participants thought the workshop was relevant and interesting, with many valuing the opportunity to attend the workshop and appreciative of the interventions’ efforts to provide journalists with more support. A few raised the counterintuitive and confronting nature of the concept of overdiagnosis, and that the information was at times difficult to digest and would have benefited from further explanation and examples. A few described feeling the workshop was not interactive enough and two noted it ‘felt a bit rushed’. Participants’ suggestions for improvement included spending more time describing the concept and consequences of overdiagnosis, including more examples, more interaction and discussion, as well as preference for the workshop to be held face to face (see table 4 for illustrative quotes).
Illustrative quotes from free-text responses
Acceptability of the Tip Sheet and list of expert contacts
All six respondents who responded to the 6-week survey indicated the workshop had informed their writing ‘a little’ (n=1) to ‘somewhat’ (n=5). Only two participants reported having written about new medical tests, treatments or diagnoses since the workshop. Two participants indicated they had used the Tip Sheet when writing, and one participant indicated having used the list of expert contacts. Three participants indicated they have not yet used the resources as they were on leave or had not written about a relevant topic (eg, ‘…I will certainly use it when required.’). Most gave positive feedback regarding both the Tip Sheet (‘Fantastic resource’, ‘Great layout—super easy to go through before/during writing’) and the list of expert contacts (‘Fine as is’). Suggestions for the Tip Sheet included to update it regularly.
Learnings by research team
Recruitment of journalists was challenging. The 2 weeks lead up for advertisement and recruitment seemed too short notice for many journalists, while others reported not knowing until the day of the workshop if they were able to attend. This introduced challenges with the need to complete a preworkshop questionnaire before attending the workshop. Similar to participant feedback, the facilitators reported that the workshop felt too rushed for the amount of content covered, with not enough time to answer questions. Facilitators also perceived some participants’ difficulty grasping the concept of overdiagnosis. More case examples to clearly illustrate the unintended harms of an unnecessary diagnosis would be beneficial for understanding in future iterations.
Discussion
This paper describes the development and feasibility testing of a novel training intervention for journalists in Australia. Piloting suggested that the multicomponent intervention is acceptable to journalists and may improve knowledge across several topics, including disease definitions, overdiagnosis and misleading statistics. Journalists were very interested, engaged and appreciative of receiving research training and support to improve critical reporting of new tests, treatments and diagnoses, and expressed a desire for further training.
Study results pointed to several strategies to improve the intervention to suit journalists with varying degrees of knowledge. Findings indicated journalists wanted more information and examples of overdiagnosis, as well as more time for group discussion. Facilitators also indicated there was not enough time to go through complex issues. Overdiagnosis is counterintuitive and challenging to both understand and communicate, particularly in light of strong beliefs in the benefits of early detection.33–35 Including more specific case studies and examples of overdiagnosis in future iterations may be helpful in conveying the unintended harms of an unnecessary diagnosis. Both participants and facilitators felt the workshop was too short, with most participants expressing interest in a longer version. This is in contrast to previous feedback from journalists about the ideal workshop length, with strong preferences for workshops of short duration (60 min) due to their limited availability and time constraints.21 Importantly, the successful Medicine in the Media workshops, run in the USA, were run over multiple days.36 In terms of examining the impact of the workshop, a few free text responses also indicated that the 6-week follow-up time frame was too short, as it had not been enough time for them to have written about a new test, diagnosis or treatment.
The study has important limitations as well as strengths. Gaining the attention of time poor journalists was a challenge and recruitment was difficult, particularly during COVID-19, when the pandemic remained a key focus for journalists covering healthcare. Employing additional recruitment channels and a longer lead time for recruitment may improve participation rates. In this context, the follow-up rate of 6/8 journalists at the time of the Omicron outbreak in Australia was a strong result reflecting the journalists’ engagement with the training and the study. As a feasibility study with short-term follow-up, we did not intend to detect significant differences in pre and post outcomes, but to provide insights into how the workshop and intervention materials can be optimised ahead of a randomised controlled trial. Strengths of the study include its novelty, because to our knowledge this is one of the first attempts to design and pilot an intervention to improve journalist reporting on the challenge of overdiagnosis. Although the sample is small, another of this study’s strengths is that it has provided useful data for planning a larger trial, and it achieved a diverse sample in terms of years of experience and across a number of media outlets. While participants self-reported how much the workshop had informed their reporting, we did not assess the impact of the intervention on actual medical reporting. This is an important direction for future research.
Improving the quality of medical reporting is a crucial endeavour given the powerful influence of the media on the public’s perceptions and usage of healthcare.17 37 This study is part of a global effort to offer journalists more training and support on these challenging issues.36 38 Such moves recognise the need to improve coverage of medical research but could be augmented by improvements in the quality of information provided by sources journalists rely on, such as press releases.39 40 Feasibility findings suggest that this intervention is acceptable to journalists and may improve knowledge across several important topics. The results of this acceptability pilot and ‘lessons learnt’ will inform the development of a randomised control trial to test the effectiveness of a revised intervention on the quality of journalists’ media coverage of medicine, with the overall aim of translating this programme into routine training for medical journalists.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by The University of Sydney Human Research Ethics Committee (protocol no: 2020/688). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would like to thank Professor Rachelle Buchbinder for her help with facilitating the online workshop and all participating journalists. We would also like to thank Dr Mary O’Keeffe for multiple detailed discussions regarding study conceptualisation and design, for important preliminary work on all aspects of the intervention being piloted, and for leading key studies which informed development of this intervention, including the cross-sectional study of media coverage of new tests, and the qualitative study with journalists.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @TessaCopp
Contributors RM coconceived the study, with input from all coauthors, and MO'K, as per acknowledgements. TC, TD, BN, LA, LM, KM, AB and RM were involved in designing the study and developing the methods. TC, RM and LM facilitated the workshop, as well as RB, as per acknowledgements. TC, BN, TD and RM contributed to the analysis. TC drafted the manuscript. TC, TD, BN, LA, LM, KM, AB and RM critically revised and approved the final manuscript. TC is the guarantor.
Funding This project was supported by a National Health and Medical Research Council (NHMRC) Centre for Research Excellence grant (1104136). TC is supported by an NHMRC Emerging Leader Research Fellowship (2009419). RM is supported by a research fellowship funded by the NHMRC (1124207) and a chief investigator on the Centre for Research Excellence (grant No 1104136). BN is supported by an NHMRC Emerging Leader Research Fellowship (1194108).
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.