Objectives To describe radiologist's attitudes and perspectives on evidence-based medicine (EBM) and their practice.
Design Face-to-face semistructured interviews, thematic analysis.
Setting 24 institutions across six Australian states and New Zealand. Transcripts were imported into HyperRESEARCH software and thematically analysed.
Participants 25 radiologists.
Results Six themes were identified: legitimising decisions (validated justification, prioritising patient preferences, reinforcing protocols), optimising outcomes (ensuring patient safety, maximising efficiency), availability of access (requiring immediacy, inadequacy of evidence, time constraints, proximity of peer networks, grasping information dispersion), over-riding pragmatism (perceptibly applicability, preserving the art of medicine, technical demands), limited confidence (conceptual obscurity, reputation-based trust, demands constant practice, suspicion and cynicism), and competing powers (hierarchical conflict, prevailing commercial interests).
Conclusions Radiologists believe EBM can support clinical decision-making for optimal patient outcomes and service efficiency but feel limited in their capacities to assimilate and apply EBM in practice. Improving access to evidence, providing ongoing education and training supplemented with practical tools for appraising evidence; and developing evidence-based guidelines and protocols may enhance feasibility and promote the confidence and skills among radiologists in applying EBM in radiology practice for better patient care.
- EDUCATION & TRAINING (see Medical Education & Training)
- QUALITATIVE RESEARCH
- PUBLIC HEALTH
- RADIOLOGY & IMAGING
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- EDUCATION & TRAINING (see Medical Education & Training)
- QUALITATIVE RESEARCH
- PUBLIC HEALTH
- RADIOLOGY & IMAGING
Strengths and limitations of this study
Interviews were conducted until saturation and we recruited participants from a range of demographic characteristics, years of practice in radiology, and evidence-based medicine training.
Preliminary findings were sent to the participants for to ensure that the results reflected the full breadth of their perspectives.
Participants were recruited from two countries and therefore transferability to other countries beyond Australia and New Zealand is uncertain.
Evidence-based medicine (EBM) is the “integration of best research evidence with clinical expertise and patient values”1 ,2 and is widely promoted as a tool to improve patient care. EBM was originally designed to address clinical problems within internal medicine, but has been generalised and expanded to enable applicability to other medical specialties. Recognised barriers to the optimal use of EBM in a variety of specialties include competing priorities and lack of time, inability to cope with ‘information explosion’,3 misconceptions about what constitutes quality evidence,4 lack of awareness of EBM resources,3 threat to professional autonomy,5 ,6 concerns about the trustworthiness of the data,5 fear of conflict with colleagues,7 and institutional dynamics and culture.8–10 However, there is little information on how EBM is regarded and practised by radiologists and whether barriers to optimal use may differ.
For example, technological innovation in imaging and interventional technologies has intensified the demand on radiologists to assimilate the proliferation of new information to deliver safe and cost-effective care,11 ,12 and the promotion of new tests may occur prior to published evidence supporting widespread use. EBM use within radiology has not been examined, but a survey of radiation oncologists found widespread support for EBM, although most had not undertaken formal EBM training and were unaware of integral resources such as the Cochrane Library.3
We aimed to describe radiologist's attitudes and perspectives on learning and applying EBM in practice, and in doing so, inform strategies for improving EBM training, and assist development of resources to facilitate greater use of EBM. This may help to foster appropriate decisions regarding imaging and interventional procedures for improved patient outcomes.
Participant selection and practice setting
Radiologists in Australia and New Zealand were purposively selected to capture a range of age, gender, years of clinical experience, radiology subspecialties, EBM training, and practice locations. Invitations were emailed to radiologists enrolled in the critical skills appraisal programme at The University of Sydney (July 2013) or were attending an Annual Scientific Meeting (October 2013). Radiologists known to the investigators were also invited and participants could nominate other radiologists who could offer a different perspective about EBM. Interviews were conducted in meeting rooms, clinic offices and conference venue. All participants provided informed consent.
The interview guide was based on a review of the literature3–8 ,13 ,14 and discussion among the research team (box 1). AT conducted a face-to-face semistructured interview with each participant from July to November 2013. Participant recruitment ceased when theoretical saturation was reached that is, when little or no new concepts were being raised in subsequent interviews. All interviews were audio-recorded and transcribed.
1. Role as a radiologist/radiology trainee
A. What is your current position and how many years have you been working in that role or what year of training?
B. Working as a radiologist/trainee, what are the some of the most difficult, common, or complex issues/decisions you have faced and how do you deal with them (eg, when interpreting images)?
2. Knowledge about evidence-based medicine (EBM)
A. How would you define EBM or what is the first thing that comes to mind when thinking about EBM?
B. How you rate your knowledge or understanding about EBM from 0 (lowest) to 10 (highest—most knowledgeable)—why?
C. What educational sources do you use to inform your practice? What resources or how do you go about interpreting diagnostic tests? (STATdx, up-to-date, guidelines and protocols, experience and intuition, other colleagues’ opinions, decision aids or online resources such as calculators)
D. Can you comment on the level of trust you have in these resources—what makes you trust it more/less? (journal reputation, authors, sample size)
E. What aspect of EBM do you find most challenging or difficult—why? (asking answerable questions, searching, appraising, analysis or synthesis, interpreting the data, applicability to individual patients or applying EBM in your day-to-day practice)
F. What EBM concepts of terms do you feel you understand the least/most?
3. EBM training
A. What was the most/least interesting or valuable thing you learnt in EBM—why?
B. How you rate the importance of EBM training to your—clinical practice from 0 (least) to 10 (most important)—why?
C. How would you improve EBM teaching that is, what teaching methods do you believe would be most effective in helping radiologists apply EBM in their practice—why? (journal club, study appraisal)
4. Applying EBM in practice
A. Can you describe the role EBM has in your overall clinical decision-making? Have you applied EBM in practice—how/give an example?
B. Do you believe there are benefits in applying EBM in radiology practice—why? (service efficiency, costs, patient care, develop guidelines)
C. What are the challenges or barriers in applying EBM in day-to-day practice? (service provision vs consultative, not sure where to find resources to guide its use (access), unsure of the validity of a checklist compared with clinical reasoning, competing priorities, concern about the trustworthiness of the evidence, threat to clinical autonomy, financial interests, potential to cause conflict, contradictory data, information overload)
D. What can you suggest would facilitate the application of EBM into day-to-day practice? (Evidence summaries, technology and tools)
E. Do you think EBM is more relevant for the clinical aspects of radiology such as interventional radiology rather than diagnosis—why?
F. Compared with other medical specialties, do you think radiology lags behind, is equal or, or is more advanced in terms of EBM knowledge and application—why? (research culture, limited research evidence)
G. Does learning about EBM make you more or less likely to undertake research to answer clinical issues you have encountered—why?
The transcripts were entered into HyperRESEARCH software (ResearchWare, US V.3.0, Randolph Massachusetts, USA) for coding qualitative data. Based on the principles of grounded theory (to develop a theory or explanation ie, grounded in the data collected)15 and thematic analysis, AT read the transcripts, conceptualised and coded all sections relating to radiologists’ perspectives on EBM into concepts identified inductively in the data; and created new codes when necessary. These were reviewed by SEM who also read the transcripts. This can help to ensure that data were captured in the preliminary codes. Similar concepts were grouped into themes and subthemes. Patterns and conceptual links between themes were mapped into a thematic schema. Member checking was conducted whereby participants were sent a copy of the preliminary analysis and given 2 weeks to suggest additional opinions. These were integrated into the final thematic analysis. This ensures that the findings reflect and captures the full breadth of data from the participants.
Twenty-five radiologists from 24 institutions across six Australian states and New Zealand participated. Non-participation (n=6) was due to travel and clinical commitments. The mean duration of interviews was 35 min. Participant characteristics are provided in table 1.
We identified six major themes: legitimising decisions, optimising outcomes, availability of access, over-riding pragmatism, limited confidence, and competing powers. Illustrative quotations for each theme are provided in table 2/online supplementary file 1. A thematic schema illustrating the conceptual links among themes is shown in figure 1. EBM was believed to support clinical decision-making for optimal patient outcomes and service efficiency, but radiologists’ capacities to assimilate and apply EBM were limited by barriers to accessing and appraising the evidence, perceived need for pragmatism and gaining practical experience, and contending with power hierarchies with referring physicians and commercial interests. A description of the themes and subthemes are provided in the following section. Most of the themes apply to interventional and diagnostic radiology; however results that were specific to either interventional (therapeutic) or diagnostic radiology will be indicated.
EBM provided a framework to make clinical decisions based on science, rather than anecdotal data. EBM “added weight, added experience and evidence behind decisions.” And some participants felt reassured when research “validated their own experiences”, and EBM was regarded as an opportunity “to borrow information and techniques from other people who have been using them more.”
Prioritising patient preferences
Shared decision-making was regarded as important though some felt that patient preferences contradicted EBM. For example, they felt unable to deny patients who wanted treatment even when the evidence suggested that the therapeutic intervention would not be cost-effective or beneficial.
EBM was regarded as useful for developing evidence-based protocols and guidelines. This was particularly relevant in directing decisional pathways for inexperienced referrers. Protocols were seen to protect radiologist's decisions as “you get in trouble for missing something, you don't get in trouble for over-investigating something.”
Ensuring patient safety
Some were convinced that EBM had demonstrable impact on ensuring safe patient care. There was reference to studies assessing safe dosages of gadolinium in patients with renal impairment, reducing the incidence of nephrogenic systemic fibrosis. Participants remarked on the “increasing number of normal examinations which meant they were scanning more people, and that they were not selecting them correctly.” They felt a burden of responsibility—“we definitely do cause the public harm, we just don't know how many [malignancies] is due to us yet” and believed that applying EBM in diagnostic radiology could reduce overexposing patients to unnecessary ionising radiation and risk of malignancies.
Participants felt certain that the financial costs to public hospitals incurred by performing excessive radiological tests and procedures could be minimised by applying EBM. Some suggested that evidence-based referral guidelines may reduce the number of unnecessary diagnostic tests ordered by referrers and improve economic and service efficiency.
Availability of access
Simple and direct access to information was important. Most used Google or PubMed as their primary search engine. Some relied on email updates or review articles to keep up-to-date. Evidence summaries offered information that “had already been critically appraised, filtered out so most of the rubbish wasn't there.” Being unable to access journals due to institutional restrictions “added an extra element of difficulty” however, some participants who had received EBM training felt confident about searching for high-quality scientific research using MEDLINE and the Cochrane Library.
Inadequacy of evidence
Radiological technologies often “progressed before the evidence could come out.” Some observed that diagnostic procedures (eg, CT angiograms for vertebral artery dissection) were used without supporting evidence. In radiological research, some felt that relevant outcomes could not be feasibly measured, for example if it necessitated surgical or autopsy proof. Research in diagnostic imaging was unavailable particularly for rare clinical cases. They felt that, “there's no culture of [radiology] research except in little pockets and enclaves in different institutions few and far between.”
The ‘fast-growing field’ of radiology meant participants felt without capacity to be “a true academic,” engage in research, and “to go through everything we do and find the evidence to prove that that's the best method or way of doing something.” Instead of an appointment-based patient caseload, radiologists had to manage a continuous inflow of patients, which was more difficult to keep manageable particularly if working in a smaller radiology department, or in the private sector. Some reviewed the literature only when required to, for example when preparing for presentations.
Proximity of peer networks
Participants relied on their colleagues to discuss and resolve cases. In larger radiology departments, participants could readily contact experienced colleagues for advice. One participant stated, “My approach to medicine is a very practical approach and based on my experience and the knowledge of others, my skill set is complementary to others so I use their skills.” Interventional radiologists valued practical suggestions from colleagues about procedures and participants attended conferences to learn about new procedures and protocols, then turned to “selected articles on the nuts and bolts, assuming that somebody has looked at the utility of the procedure.”
Grasping information dispersion
The field of radiology was described as broad, “dynamic,” and all-encompassing of different subspecialties which augmented the challenge of keeping up with the literature—“radiologists in most places are generalists, you have to be able to do neurology, gastroenterology, intervention, it's not actually possible to be at the top level of science in all those fields.” They felt “confounded by the plethora of information that you can't filter anymore.” Radiologists practising within a narrow subspecialty felt better able to remain aware of the current literature in their area.
From a practical perspective, some judged the clinical relevance of a research article rather than the methodology. One participant reflected, “I'm not an academic, I'm much more practical, I'm more operational.” They would “look at what they do and the outcomes measures more than analysing the way they get the outcomes.” Research results that were too broad or excluded relevant patient groups made it difficult to extrapolate or assess the transferability of the findings to their own patient population or to an individual patient—“like meta-analysis, you have to spend more energy on trying to pick out tiny fragments of useful data, so most of the time the article was justifying itself and talking about itself, I just want to get to the crux of the matter.”
Preserving the art of medicine
There was anxiety that “if everything gets based on evidence based medicine, we lose the art of radiology.” Participants with more years of experience believed in learning from practice, observing senior colleagues, and developing expert intuition, more so than ‘reading about things’ as EBM could ‘never capture the whole story.’
The technological advances in radiology placed demands on participants to prioritise their technical competence and knowledge of anatomy and pathology. They had to “keep on learning new tricks, as well as refine what they knew before” and study textbooks and ‘didactic’ articles rather than scientific research publications—“What does such a condition look like on ultrasound? You don't need a research article.”
The perceived complexity of concepts, mainly relating to critical appraisal and statistical analysis, was overwhelming. Many described their judgement about study validity as ‘superficial’ and felt they lacked a framework for critically appraising an article—“we all aspire to practice EBM but we don't necessarily know how to and I think there's a lot of quasi EBM going on.” Some assessed articles based on sample size, participant characteristics, and blinding. Also, some believed a high-level understanding of statistics was required for EBM, perceived to be daunting, “we've come out of it with this monster that most of the radiologists don't know how to cope with, most of us left statistics behind in high school, we don't have this analysis in our brains.”
Participants trusted journals with high impact factors, and articles from reputable institutions with experienced authors. They placed confidence in editorial integrity and expertise to ensure that only high-quality and valid research articles were published.
Demands constant practice
EBM was a skill that required ongoing practice. Participants suggested regular EBM training, and journal clubs to maintain EBM proficiencies, such as conducting critical appraisal.
Suspicion and cynicism
Some were suspicious of authors who might be misusing research to push their own agenda, “People usually decide what they want the reality to be and then harness the appropriate evidence that they want to support it.” For example, in diagnostic cardiac imaging, one participant observed nuclear physicians advocating for nuclear medicine while cardiologists were promoting stress echocardiogram, and both presented compelling arguments supported by research. Contradictory results also perpetuated cynicism of EBM, “I've seen a few cases where different meta-analyses will draw completely different conclusions from the same set of data analysing the same papers.” Some were wary of academic competition, and the ‘politics of journals, and personal egos.’ However, multiple independent studies which demonstrated confirmatory findings, provided reassurance.
Disempowerment prevented the practice of EBM. Some radiologists felt that referring physicians perceived them as service providers rather than as consultants. They viewed that “referrers don't feel that radiology should be gate keepers.” Being ‘secondary referrals’ most radiologists did not control patient management and lacked clinical information about the patient. Some resigned themselves to ‘defeat’ and ‘dogmatism’ as they continued to perform tests they regarded as unnecessary. At times, there was a palpable tension between keeping referrers satisfied and advocating for the patient's safety and preventing them from being “irradiated just for expediency rather than a clinical indication”. Some tried to ‘battle’ with referrers but withdrew from the “uneven playing field”. More senior participants felt that referrers respected their recommendations.
In certain radiology subspecialties including paediatrics, oncology, and obstetrics, radiologists participated in multidisciplinary meetings and valued the active engagement in patient management where their expert opinion contributed to the broader decision-making. Participants appreciated this ‘cross-pollination’ of information and clinical history.
Prevailing commercial interests
Private radiology centres faced the pressure of “generating revenue to keep the practice going”, which was perceived to nullify any impetus to implement EBM. Some believed that “evidence-based medicine will never work in an item for service based medical culture” and there was “no real incentives for doctors to do the right thing” in referring patients for radiological diagnostics tests. To protect a thriving business, they kept referrers satisfied by fulfilling their radiological requests, even when it was not evidence based.
Although radiologists appreciate the role of EBM in improving patient care, misperceptions of the definition of EBM, a lack of critical appraisal skills and an underappreciation of how EBM could help resolve common tensions within daily practice limited its optimal use. EBM is defined as the integration of best research evidence with clinical expertise and patient values however some participants thought that EBM supplanted clinical expertise and therefore rejected it as being exclusive of clinical wisdom. A common tension cited by many participants was the performance of unnecessary tests, contributing to excess cost and increased exposure to radiation, however many felt helpless to refuse the request. However, when evidence-based guidelines were available to support appropriate imaging pathways, radiologists felt more confident in negotiating referrals.
Some of the barriers to implementing EBM we identified have been reported in other areas of medicine and health. Studies conducted in internal medicine and surgery found that confusion about EBM terminology, team dynamics, staff disapproval, and time constraints prevented residents from practising EBM.7 ,8 In primary care, EBM was perceived by some physicians as devaluing the ‘art’ of medicine and a threat to their professional autonomy, and were concerned about industry influence.5 Another study found that healthcare providers preferred tested, convenient and respected evidence sources including professional societies and expert colleagues.4 There are unique features in radiological practice: the limitations of being perceived as a service provider rather than an ‘expert consultant’, the demands of maintaining technical competence, the requirement for detailed but stable knowledge of anatomy and pathology, and the challenges of keeping up in a field of rapid technological advances.
While understanding and use of EBM is widely accepted as a core competency of clinical practice, this is the first study to explore understanding and barriers to use in radiology. We conducted interviews until little or no new concepts were emerging from subsequent interviews (theoretical saturation), and included participants from a range of demographic characteristics, years of practice in radiology, and EBM training. Also, participants were asked to provide feedback on the preliminary findings (member checking). However, our study has potential limitations. Participants were recruited from Australia and New Zealand therefore the transferability of the findings to other regions may be limited, although similar barriers have been identified in studies conducted in different settings,16 suggesting broader applicability.
The acquisition and application of EBM skills including literature searching, critical appraisal of articles and interpretation of diagnostic tests and their limitations is essential to competent clinical care.12 Several resources have been published in radiology literature.12 ,17 However, barriers related to the availability and access to evidence, unmet education and training needs, pragmatic and structural difficulties that need to be addressed. Based on our findings, we suggest key target areas, strategies and actions for promoting EBM awareness and implementation (table 3).
Moving EBM teaching from the classroom to clinical practice settings has been strongly advocated to improve knowledge, critical appraisal skills, attitudes and behaviour.18 The few strategies to clinically integrate EBM teaching which have been evaluated include daily EBM teaching rounds in which searches and study appraisals are based on cases presented at clinical rounds,19 journal clubs,20 and EBM ward round sessions led by a clinical specialist and epidemiologist to develop clinical questions, literature search, critical appraisal and development of evidence summaries.21 While these teaching methods are effective, more specific evaluation of these strategies is needed in radiology.
It has been recognised that evidence-based practice should be taught from an early stage in medical and radiological education.22 ,23 Current models for educating evidence-based practice include having trained epidemiologists to deliver regular teaching and interactive sessions which encompass theory, self-directed learning, and application to clinically relevant questions; or providing training workshops or teaching EBM in postgraduate meetings which cover the principles of EBM.22 Our study has highlighted important implications for teaching EBM within radiology. Skills that require greater emphasis included literature searching and critical appraisal. A recent survey found that radiologists and trainees preferred Google, customised radiology-focused produces and apps, and online resources to solve imaging questions.16 Evidence summaries or EBM guidelines could be developed and disseminated via these channels. Confidence in appraisal skills could be improved with wider use of EBM tools.24 For most EBM skills, a mix of educational strategies is likely to be most effective in increasing skills including interactive online courses, journal clubs and seminars.25
While our findings are likely to have some commonality across geographic regions, further studies on barriers to EBM in different areas would be enlightening. Teaching strategies that are most helpful to radiologists should be clarified, as these may not be the same as those for bedside practitioners. Studies of implementation of evidence-based guidelines for imaging pathways and whether these improve patient's important outcomes and cost are also needed.
Better access to evidence, ongoing education and training supplemented with practical tools for appraising evidence; and developing evidence-based guidelines and protocols may promote optimal use of EBM within radiology, and ultimately translate to better patient care.
The authors sincerely thank all the participants for their time and efforts in sharing their perspectives. With written consent, they acknowledge: Roger Bain, Lourens Bester, Roger Bodley, Timothy Cain, Kwang Chin, Craig Hacking, Robin Harle, Albert Lam, Lance Lawler, Melissa Lea, Wai-Kit Lee, Philip Lew, Arthur McKenna, Sabaratnum Muthukumaraswamy, Anthony Peduto, Alex Rhodes, Umesh Shetty, Rohan van den Driesen, Pamela Walsh.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Contributors AT participated in the design of the study, did the interviews, transcribed the interviews, carried out the thematic analysis and drafted the manuscript. SEM, JCC and CL designed the study, participated in the thematic analysis, and provided critical review of manuscript drafts. GL and AJP participated in the design of the study, assisted with the analysis, and provided intellectual input into subsequent manuscript drafts. All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; drafting the article or revising it critically for important intellectual content; and provided final approval of the version to be published. All authors had full access to all of the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding This research is funded by an internal grant from the Sydney School of Public Health, The University of Sydney.
Competing interests None.
Ethics approval The study was approved by the University of Sydney Ethics Committee (20130616).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.