Article Text
PDF
Abstract
Introduction A journal club (JC) is a commonly used medical educational tool. Videoconferencing technology can facilitate the delivery of JCs, however, there remains no evidence on the role of web-based virtual JCs in promoting the acquisition and retention of medical knowledge. The Web-Ed trial aims to evaluate the educational benefits, feasibility and acceptability of web-based virtual JCs compared with traditional face-to-face ones.
Methods and analysis Web-Ed is a multicentre pragmatic parallel-group randomised trial across teaching hospitals within the UK National Health Service (NHS). We will enrol qualified doctors or medical students who are >18 years old, proficient in English and able to use online videoconferencing software. Block randomisation will be used to allocate participants in 1:1 ratio to either intervention group. Both groups will be presented with the same educational material and follow a standardised JC structure hosted by nominated moderators and medical faculty members.
The primary outcome is the difference in participants’ knowledge acquisition and retention 7 days after the JCs evaluated using standardised multiple-choice questions. We will report secondarily on the feasibility and acceptability of the JCs using Likert scale questionnaires. Assuming a 30% drop-out rate, we aim to enrol 75 participants to detect a 20% improvement in knowledge acquisition at 80% power and 5% significance. We will report using mean difference or risk ratio with 95% CIs and assess significance using parametric/non-parametric testing. Where relevant, we will adjust for predetermined characteristics (age, grade of training and session duration) using multivariate regression analyses.
Ethics and dissemination Web-Ed was designed by doctors in training to address their learning needs and evaluate the preferred mode of learning. The trial results will be published in peer-reviewed journals and presented at relevant scientific conferences. The trial has been approved by the NHS Health Regulation Authority (21/HRA/3361).
Trial registration number ISRCTN18036769.
- MEDICAL EDUCATION & TRAINING
- Clinical trials
- World Wide Web technology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The Web-Ed trial is designed by doctors in training to evaluate the role of web-based virtual journal clubs in modern medical education.
The trial multicentre design and use of standardised interventions in both comparison groups will increase the generalisability of its findings.
The trial results will help to inform the future provision of postgraduate medical education using modern videoconferencing technology.
The trial findings might be limited by subtle variations in intervention delivery across groups and participating sites.
Introduction
Journal clubs (JCs) are commonly used in medical education as dedicated events where clinicians meet to share medical knowledge and critically appraise recent updates in the medical literature.1 JCs were first introduced by Sir James Page in London between 1835 and 1854 and Sir William Osler at McGill University in 1875 as a mean to share periodicals with physicians in training since texts were very expensive and the literature could not be easily accessed.2 3
Since then, the educational goals and format of JCs have evolved from simple dissemination of new knowledge to focus on relevant skills acquisition to encourage the application of new evidence into the practice of evidence-based medicine such as critical appraisal and evidence grading skills.4 5
Unlike other static educational tools, the JC facilitates interactive learning via structured and informal exchange among the attendees. Hosting this safe exchange medium can increase knowledge retention and the integration of knowledge into everyday clinical practice.6 While attractive, the optimal use, format and structure of a JC in modern medical education remains unknown.
Several features were proposed to increase the educational benefits of a JC such as presenting original articles, having clear objectives, nominating a facilitator, holding frequent predetermined sessions (eg, monthly), maintaining high attendance and fostering a conducive safe environment to enable interactive debate among the participants.5–9 In the UK, the majority of JCs take the format of in-person group discussions held in-house, during working hours and focused on specified content chosen by local faculty members. However, the educational benefits of this format are often limited by poor attendance and disruptions due to increasing clinical service demands on both the attendees and the faculty. Additionally, the hierarchical delivery of JC contents limits trainees’ input to address their knowledge gap and desired learning objectives.10
The use of online telecommunication and videoconferencing platforms to host and deliver educational materials is gaining momentum, especially during the COVID-19 global pandemic.11 Online education is now readily available thanks to rapid advances in videoconferencing technology with interactive features such as online polling, live questions and answers, breakout rooms and peer-to-peer sharing.6 The last couple of years saw rapid adoption of various online JC formats based on Twitter or other social media platforms.5 The uptake in web-based JCs is likely to grow as the videoconferencing technology continues to improve offering a more conducive online meeting environment. However, there remains no evidence on the role of web-based virtual JCs in promoting the acquisition and retention of knowledge in modern medical education. We designed the Web-Ed multicentre randomised trial to address this knowledge gap and provide guidance on the optimal design and delivery of educational JCs in medical training.
Aim and objectives
We aim to evaluate the effectiveness, feasibility and acceptability of a web-based virtual JC compared with face-to-face group-based JC as an educational tool among medical trainees.
Our primary objective is to assess the acquisition and retention of medical knowledge among participants in both groups. Secondarily, we will evaluate the feasibility and participants’ acceptability of the web-based virtual JC as an educational tool.
Methods and analysis
Study design
Web-Ed is a multicentre pragmatic parallel-group randomised trial. Recruitment is planned between March and October 2022 with a predicted trial end date in April 2023.
Setting
Ten accredited medical teaching hospitals within the UK National Health Service (NHS).
Participants
Qualified doctors undertaking formal clinical training in an accredited medical training programme within the NHS. Medical students who are affiliated with an accredited UK medical university and undertaking a clinical attachment within the NHS are eligible to participate in Web-Ed. Participants must be over 18 years of age at the time of enrolment, able to provide written consent, have a good command of the English language and be proficient in using simple online meeting and videoconferencing software.
Screening, consenting and enrolment
The Web-Ed trial will be coordinated by each site’s primary investigator (PI). All PIs are members of the UK Audit and Research Collaborative in Obstetrics and Gynaecology (UKARCOG) and will receive formal training in Good Clinical Practice. Local PIs will screen and identify eligible trainees within their department to enrol them to the trial after providing the electronic participant information sheet (ePIS). Trainees will be given sufficient time to consider the trial and provide consent after reading the ePIS. All participants will receive clear information on the use and storage of any personal data including policies for safeguarding information as per the European Union General Data Protection Regulation. All participants will be provided with a personal identifying digit and will be asked to complete written consent forms. Following consent, local PIs will collect the participants’ baseline information and randomise them to either intervention group.
An independent research assistant will perform block randomisation using online computer software (randomise.com) in 1:1 ratio. Concealed participant allocation will be performed using an electronic messaging service. Participants will be informed immediately of their allocated group and will be provided with electronic diary invitations and instructions to attend their allocated JC group.
Procedures
Participants in both groups will receive an electronic copy of a scientific article published in a peer-reviewed medical journal selected a priori. The selected article was chosen in consensus among the Trial Management Group (TMG) based on its relevance and suitability for the trial target cohort. The article will be sent to all participants 2 weeks before the planned JC. An additional reminder will be sent to prompt participants to read the shared article 1 week before their allocated JC date.
We aim to keep the size of each JC session limited to 5–10 participants to ensure equal participation among attendees, prompt debates and boost interactions. The duration of JC sessions in both arms of the study will also be limited to 30–40 min. Local PIs will have the flexibility to amend the date and time of the face-to-face JCs according to the local availability (eg, venue availability, participant preference) to ensure maximum participation. The face-to-face JC will be hosted at each participating site in a venue dedicated for education with facilities for electronic slide presentations. The local PIs will try to host each face-to-face JC within protected teaching time to minimise interruptions to participants on active clinical duty.
Participants allocated to the virtual JC will be provided with login details and instructions on how to operate the trial’s online video-conferencing software. The chosen software enables live video and audio streaming with slide presentation abilities, live question and answer, and live polling features. Video and camera sharing will be encouraged to promote active discussion and participation, but participants will have the option of keeping their video camera turned off if desired (table 1).
Structure and format of the web-based journal club compared with the face-to-face journal club
Both JCs types will be hosted by nominated moderators who will receive training by the TMG to ensure they maintain fidelity with the trial’s procedures, published protocol, and strictly follow a JC structure as highlighted in figure 1 and table 1. Where possible, JCs will be run by the same group of moderators to minimise variations across planned JC sessions. Faculty members will consist of volunteer trainers with experience in clinical teaching and advanced knowledge in the clinic topic of chosen article. To standardise both the intervention and control all JC sessions will follow the same format outlined below in figure 1. Moderators and faculty will promote discussion and dialogue among the participants using set open ended questions based on elements of critical appraisal based on the CASP tool12 (table 1).
Flow chart of the Web-Ed randomised trial to evaluate the educational benefits, feasibility and acceptability of a web-based virtual journal club compared with face-to-face journal club (JC). MCQ, multiple-choice question.
Outcome measures
We will report primarily on the participants’ knowledge acquisition evaluated immediately after the JCs using standardised multiple-choice questions. The assessment will include five questions each including four true or false statements. The questions were developed by the trial TMG to test direct knowledge acquisition based on the discussed article as well as elements of critical appraisal based on the CASP tool.12 The questions were piloted among volunteers from the UKARCOG network to ensure their face and content validity as well as suitability to the trial’s target cohort. We will also evaluate knowledge retention 7 days post the JC using the same multiple-choice questions, however, with different wording to reduce memory bias and accurately evaluate knowledge retention.
To assess feasibility, we will report on the attendance rate at each JC, the number of questions asked by the participants, the duration of each JC session and the participant attrition rate from the start to the end of the session. Finally, we will report on participants’ satisfaction and acceptability of the intervention using Likert scale questionnaires. The questionnaire will assess if participants had sufficient time to attend the JC, if they found the experience easy and feasible, if the educational content was useful to them, if they are likely to attend the same JC format in the future, and if they are happy to participate in a focus group to explore their JC experience further. Each questionnaire will include five statements anchored between (strongly disagree) and (strongly agree) with an option to choose (not sure).
Data collection, handling and confidentiality
All submitted case record forms will be collected electronically using an online case submission database. Participants will receive email notifications to complete the knowledge tests and acceptability outcomes using their personal computer or smart mobile phone. Feasibility outcomes will be completed by the JC moderators directly into the trial electronic database.
All data collected during the study will be entered onto a dedicated, password-protected, electronic database using a secure computer and internet connection. Only prespecified members of the trials study team will be able to access the electronic database. Regular automated backups will be conducted to ensure a trail of data entry into the database. All participants’ data will be pseudoanonymised for the analysis stage using unique trial personal identification digits. We will not publish any data which could lead to the identification of any study participants. PIs at all sites will share the same duty to prevent unauthorised disclosure of personal information to any unauthorised body. During the study, all records collected from participants are the responsibility of the chief investigator and will be kept in secure conditions. On completion of the study, all records will be kept securely and confidentially in accordance with the Data Protection Act 1998-UK, NHS Caldicott Guardian principles, The Research Governance Framework for Health and Social Care and the Health Research Authority by the study sponsor (The Shrewsbury and Telford Hospital NHS Trust). The chief investigator has the overall responsibility to maintain and protect the participants’ anonymity throughout the study.
Quality assurance and auditing
The chief investigator will ensure the trial is conducted in compliance with the principles of the Declaration of Helsinki (1996) and the regulatory requirements including, but not limited to, the Research Governance Framework, Good Clinical Practice. Non-compliance may be captured from a variety of different sources including intervention session monitoring, data entry auditing, communications and updates. The sponsor will maintain a log of any non-compliance to ascertain if any trends are developing or escalating. The sponsor of the study will assess the non-compliances and action a time frame in which they need to be dealt with. Each action will be given a different time frame dependent on the severity of the event. If the actions are not dealt with accordingly, the sponsor will agree on appropriate action, including an on-site audit.
Sample size
Previous studies evaluating the educational value of JCs report a mean difference in knowledge acquisition scores between cohorts ranging between 8% and 25%.13–15 Assuming a 10% mean difference between groups, we need to enrol 32 participants to detect a 20% improvement in knowledge acquisition immediately after the JC. Similar numbers are expected to be sufficient to detect a meaningful mean difference for knowledge retention 1 week post-JC after allowing for a 30% drop-out rate. Therefore, we need to randomise 75 participants to achieve 80% power at 5% significance.
Statistical analysis
Data normality will be assessed using a Kolmogorov-Smirnov test. We will compare the mean difference of continuous outcomes between groups using a t-test for parametric data and a Mann-Whitney U test for non-parametric data. We will compare the distribution of categorical data between groups using a χ2 test with a significant p value set at <0.05. We will report using mean difference for continuous outcomes with 95% CIs and risk ratio for dichotomous outcomes. Where relevant we will adjust the mean difference to predetermined population characteristics (eg, age, grade of training and session duration) using multivariate linear regression analysis.
Ethics and dissemination
The trial has been approved by the Health Regulation Authority in the UK and is exempt from Research Ethics Committee approval. The trial is registered prospectively and will report any amendment of its protocol. At the conclusion of the Web-Ed trial, the steering and management committees will hold a meeting to discuss the main results and dissemination policy. The success of this study depends on the participation of trainees and local faculty members within the UKARCOG network. As such, the credit for publishing the study’s results will be dedicated to all collaborators equally. The TMG will be responsible for publishing the findings of the Web-Ed trial in a peer-reviewed journal as per the ICMJE guidelines. Open-access publication will be sought where possible to maximise impact. We will also aim to disseminate the trial’s findings via oral and poster presentations at relevant national and international conferences.
Patient and public involvement
Key stakeholders (medical trainees and trainers in the NHS) were consulted on the trial design, research objective and outcome reporting to address the highlighted training priorities. The trial findings will be communicated to all participants to scope their feedback before publication in peer-reviewed journals. Patients and members of the public were not involved in the trial design, conduct or reporting.
Discussion
Postgraduate medical education is a cornerstone of clinical training globally and it ensures clinicians can respond to the fast-changing field of medical practice.16 The medical literature is changing rapidly with daily updates and about 50% of evidence-based recommendations being reversed or updated within 5 years.17 Unlike the undergraduate setting where there is dedicated time for personal study and provision of up-to-date learning materials, the postgraduate clinical practice offers little time outside clinical duties to review up to date medical literature and implement newly available evidence. Traditionally, undergraduate training programmes make use of ‘spiral learning’ where the material is covered multiple times throughout the course to reinforce key new concepts and medical knowledge.18 This runs in contrast to the sporadic and fragmented nature of learning materials presented in a JC. As such, a structured JC format with clear learning objectives is essential to maximise the learning benefit for those attending.
The Web-Ed trial is specifically designed to test knowledge retention between in-person and online JCs which will inform future practice on how to deliver a JC. It is anticipated that the results of Web-Ed will inform future provision of postgraduate medical education.
The educational value of locally lead JCs may be limited by the lack of practitioners with adequate teaching skills, or indeed sufficient time to deliver regular JCs to a high quality. Enabling virtual delivery of JCs could facilitate centralised wide-scale participation in specialised JCs delivered by content experts to enrich the learning experience and maximise value. We hope that our findings will provide the much-needed evidence base to formalise the use of virtual teaching technology in medical education. The use of other online social media platforms may also have a role in the advertisement of online JCs as well as the dissemination of their learning outcomes to those who were unable to attend.
Several elements may limit the generalisability of the Web-Ed findings. Variations between intervention arms and across sites in content delivery may affect performance bias. We aim to minimise this risk by implementing a structured JCs format using standardised educational materials. The knowledge sharing and debate will invariably differ within and across groups depending on each group’s specific dynamics and participants background knowledge on the topic of discussion. Specifically, participants’ familiarity and affinity with the use of web-based video conferencing software may affect groups’ interactions. Still, we adopted a pragmatic design for our trial to provide evidence sought from real-life examples of the educational process in clinical settings.
Ethics statements
Patient consent for publication
References
Footnotes
Twitter @JackTeh96
Contributors MRi wrote the first manuscript, contributed to drafting the protocol and will coordinate the trial conduct. WP-S contributed to drafting the protocol and assist as cochief investigator. BHAW conceived the idea, drafted the protocol and will oversee the trial conduct as chief investigator. NE, CD, DC, NC, OR, JJT, MF, SM, NM, MRa, MA, YJ and MW will contribute equally to the study conduct and provided critical input to the protocol and final manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.