Article Text
PDF
Abstract
Introduction Mounting evidence has suggested that novel teaching strategies have a positive impact on the quality and efficiency of medical education. However, the comprehensive evidence about the superiority among various strategies is not clear. To address this issue, we aim to conduct a systematic review and network meta-analysis (NMA) to evaluate the effects of six main strategies on medical education, including case-based learning, problem-based learning, team-based learning, flipped classrooms, simulation-based education and bridge-in, objective, preassessment, participatory learning, postassessment and summary.
Methods and analysis A systematic search will be conducted in PubMed, Embase, Web of Science and the Cochrane Library, covering studies published from database inception to November 2023. Randomised controlled trials which evaluated the different teaching methods and meet the eligibility criteria will be included. The effectiveness of medical students’ learning, which is evaluated by theoretical test score, experimental or practical test score, will be analysed as the primary outcomes. Besides, the secondary outcomes consist of learning satisfaction of students and formative evaluation score. The study selection and data extraction will be independently performed by two authors. The risk of bias in each study will be assessed using V.2 of the Cochrane risk-of-bias tool for randomised controlled trials. To compare the effects of six teaching strategies, pairwise meta-analysis and NMA will be performed using Rev Man, STATA and R software. Statistical analyses including homogeneity tests, sensitivity analysis, consistency tests, subgroup analysis, Egger’s test and publication bias will also be completed.
Ethics and dissemination No formal research ethics approval is required because this study is a meta-analysis based on published studies. The results will be disseminated to a peer-reviewed journal for publication.
Protocol registration number CRD42023456050.
- Meta-Analysis
- Systematic Review
- EDUCATION & TRAINING (see Medical Education & Training)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This comprehensive review and network meta-analysis will encompass randomised controlled trials related to a wide range of clinical and basic medical courses.
The study selection, data extraction and quality assessment will be performed by two authors independently.
In cases where the original investigators are unable or unwilling to share their data, the missing information will be supplemented using the multiple imputation by chain equation method.
Despite conducting both electronic and manual searches, it is possible that some relevant studies or unpublished data may not be retrieved.
This study may not fully capture other important qualities and skills of medical students in ‘type of outcomes’ section, such as their ability to analyse cases, social and communication skills, problem-solving and self-learning abilities, and subjective enthusiasm.
Introduction
Medical education is inherently challenging and demanding, leading to chronic physical and mental stress for medical students.1 This stress is caused by various factors, including vastness of academic curriculum, dissatisfaction with class lectures, frequency of examinations, fear of failure and high parental expectations.2–4 The teaching strategy employed by the medical colleges plays a crucial role in determining the effectiveness of their students.5 Traditionally, medical education has relied heavily on lecture-based learning (LBL), which is a teacher-centred approach that provides a systematic and comprehensive understanding of the curriculum.6 7 However, in the current era of abundant information, this passive method is insufficient to inspire the enthusiasm for learning among medical students. Additionally, it hinders the mastery of knowledge in terms of depth and breadth, as well as the flexible application of acquired knowledge.8 Hence, the current teaching strategies need to be modernised to enhance academic performance and learning quality and meet the emerging healthcare demands.9–11
In recent years, multiple novel teaching strategies, including problem-based learning (PBL),12 case-based learning (CBL),13 14 team-based learning (TBL),15 flipped classrooms (FC),16 simulation-based education (SBE)17 and bridge-in, objective, preassessment, participatory learning, postassessment and summary (BOPPPS),18 have been adopted by medical colleges and showed positive effects and higher acceptance for both teachers and students. These innovative teaching methods have been shown to be excellent in improving academic performance, subjective enthusiasm and integrated development of medical students. Comparative studies have demonstrated that PBL methodology, in particular, has garnered high levels of student satisfaction and has been associated with superior examination marks.6 19 Additional, CBL pedagogy is regarded as an active teaching method that effectively educates medical students and helps to improve their academic performance, the mastery of professional knowledge and course satisfaction.20 21 Results of an RCT conclude that TBL has been recognised as an important pedagogic tool, as medical students show a higher gain in theoretical and practical test of basic musculoskeletal ultrasound skills compared with the conventional learning.22 Moreover, FC approaches have gained popularity as an educational method that optimises classroom time for knowledge acquisition and retention and long-term benefits of high scoring.23 24 SBE is a rapidly developing medical strategy, which supplements and enhances clinical education, especially the clinical skills scores of medical students.17 25 26 A previous meta-analysis revealed that BOPPPS contributed to increase knowledge examination scores, skill scores and satisfaction of medical students compared with the LBL.18
Although previous evidence from several pairwise meta-analyses favoured the specific novel teaching methods’ efficacy compared with LBL, contradictory results also existed among different studies. For instance, Townsend et al27 and Polyzois et al28 demonstrated that PBL failed to increase the test scores compared with traditional teaching in class teaching. However, another meta-analysis from Zhang et al29 revealed significantly higher examination scores and examination pass rate after applying PBL in class teaching. Similar conflicting results were also found in clinical teaching from various studies. Luke et al30 demonstrated that PBL helps to improve the acquisition of radiographic interpretation skills, while another study argued that PBL and traditional methods did not differ on tests of factual knowledge and clinical knowledge.31 These discrepancies may limit the widespread adoption of novel teaching methods in both classroom and clinical settings. Furthermore, while multiple teaching methods, such as CBL,20 TBL32 and FC,33 have showen higher academic performance than LBL, the superiority of different novel teaching methods in different medical courses has not been well concluded.
Unlike pairwise meta-analysis, network meta-analysis (NMA) provides a way to simultaneously compare various interventions and the ranking of interventions based on relative parameter.34 35 To address these gaps in knowledge, we plan to perform a systematic review and NMA by retrieving studies using six main teaching strategies, including CBL, PBL, TBL, FC, SBE and BOPPPS. By examining the possible advantages of the six strategies on test scores and other skills or abilities in various medical subjects excluding pharmacology,36 which has been well determined, we aim to determine the optimal teaching strategy for medical curriculum designers and educators.
Materials and methods
This systematic review and NMA aim to compare the effectiveness and acceptability of six main teaching strategies among medical students. No ethical approval is required for the present NMA because only available data are retrieved and analysed. This protocol is registered in the PROSPERO International prospective register of systematic reviews (registration number: CRD42023456050) and reported based on Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) statement.37 Study search and data extraction will start after the registration. The study is expected to start on 30 November 2023 and be completed before 31 May 2024. The results will be published in a peer-reviewed journal.
Eligibility criteria
Types of participants
Studies that accessed medical students, including undergraduate and graduate students majoring in healthcare/clinical professions (including students of clinical medicine, foundation medicine, biomedical engineering, anaesthesiology and any other receiving medical education) will be retrieved. The inclusion criteria will not discriminate based on gender, age, grade, major, ethnicity and nationality. However, studies that evaluated veterinarians, residents or other on-the-job healthcare professionals will not be considered in the present study. Since prior studies have evaluated teaching strategies for pharmacology36 courses, our NMA will focus on other clinical and basic medicine courses.
Types of interventions
Six main teaching strategies, TBL, PBL, CBL SBE, FC and BOPPPS, used in the experimental group for curriculums of medical teaching will be included. Emphatically, the studies in which teaching methods do not use one major strategy and instead two or more methods in the same course will be excluded. Additionally, we will not include studies that use teaching strategies in only a single class/lecture or less than 10 studious hours. Control conditions include traditional LBL or any other teaching methods like textbooks-based learning, self-study module and lab-based learning.38 39 Each of these control conditions will be regarded as an independent node in this NMA.
Types of outcomes
The primary outcomes are the effectiveness of medical students’ academic performance, measured by theoretical test score, experimental or practical test score. The theoretical test scores refer to the grades of closed-book paper test, which are commonly used to evaluate the levels of understanding and mastery of knowledge from textbooks. Experimental or practical test scores mean the grades of operational examination, indicating the ability of basic medical experiments and clinical skills. The secondary outcomes will include students’ satisfaction and formative evaluation score.
Types of study
RCTs, including cross-over trials and cluster-randomised trials, of teaching methods for medical undergraduate and graduate students are eligible for inclusion. To reduce heterogeneity among included studies, non-RCTs, quasi-RCTs (eg, allocation according to the last number of the date of birth), and uncontrolled trials will be excluded. We will also exclude secondary analyses based on RCTs and studies in which the sample size of students is less than 10 per study. Only studies in the English language will be included.
Search methods and the identification of studies
Electronic searches
The following electronic databases will be systematically searched: PubMed, Embase, Web of Science and the Cochrane Library. The search will cover the period from inception of the databases to November 2023, when this protocol is supposed to be completed. The search strategy will be performed based on the PICO principle, which mainly includes three parts: populations (medical undergraduate and graduate students), interventions (six main teaching strategies) and comparison (traditional learning methods). Different search strategies adapted for each database will be completed by the combination of medical subject headings (MeSH) and free-text words. The reproducible search strategies for electronic databases are shown in online supplemental file 1.
Supplemental material
Searching other resources
Furthermore, we will manually identify the relevant studies through the reference lists of included studies and relevant reviews. Additional relevant RCTs will be obtained by hand-searching relevant key medical education journals (eg, Medical Education, Perspectives on Medical Education, BMC Medical Education, and Biochemistry and Molecular Biology Education) and corresponding authors will be contacted for incomplete data.
Data collection and analysis
Selection of studies
The retrieved studies will be imported into ENDNOTE 20 literature management software and duplicate studies will be removed. After that, through browsing the title and abstract of studies, a preliminary selection of potential trials will be completed by two authors (HG and S-LZ) based on inclusion and exclusion criteria independently. Next, the full texts of these potential trials will be read seriously to further evaluate and select the eligible studies independently by two authors (HG and S-LZ). The exclusion criteria include (1) duplicated publications; (2) non-RCTs or quasi-RCTs studies; (3) studies without full-text or valid data; (4) studies not written in English and (5) reviews, conference abstracts, case reports and meta-analysis. The studies which are considered ineligible by both authors will be excluded, and if two authors disagree, the studies will be assessed again by consensus. The specific process and the result of searching and screening are shown in figure 1.
PRISMA flow chart of study selection.
Data extraction and management
A standard table for data extraction has been created using Microsoft Word 2020 (table 1). Two authors (HG and S-LZ) will independently collect main information of eligible studies using this table. The extracted information includes the following aspects: eligible studies characteristics, intervention characteristics, participant characteristics, outcomes, the source of bias and any other information relevant to this review.
Data items to be requested for individual participant data meta-analysis
In cases where data are missing, we will make efforts to contact all corresponding authors or sponsor organisations of the original studies and request their cooperation in providing the necessary data for our project. Contact information of corresponding authors will be obtained from the papers, online research profiles or other available ways. We will send emails to the authors explaining our purpose and requesting them to provide complete data and contact the corresponding authors by phone or other personal contacts if no response. A similar process will be followed for interaction with sponsor organisations. If they do not reply or refuse to offer the data for us, the missing data will be managed through the multiple imputation by chained equation (MICE) using the MICE package in R software. Sensitivity analyses will be performed to evaluate the potential effect of this method on the results of NMA.
The detailed items of primary information and data to be obtained from papers and the original authors are shown in table 1. The raw data will be provided in any convenient manner (such as by email) in Excel of electronic format and will be securely stored on a server at Third Military Medical University. To ensure the validity of data information, then this work will be crosschecked again by another author (D-MZ), and any disagreements will be resolved through consensus. Only authorised members of the research team will have access to this dataset to maintain confidentiality and data integrity.
Risk of bias and quality assessment of included studies
Two authors will independently perform the risk of bias and quality assessment of each study in accordance with V.2 of the Cochrane risk-of-bias tool (RoB2) for randomised controlled trials. The tool assesses the following bias domains: (1) bias arising from the randomization process; (2) bias due to deviations from intended interventions; (3) bias due to missing outcome data; (4) bias in the measurement of the outcome; (5) bias in selection of the reported result and (6) overall bias. The assessment of each item will be scored and classified as low risk, unclear risk or high risk. Then, the quality of each study will be rated as high-risk study (two or more items rated as high risk of bias), low-risk study (four or more items rated as low risk and no more than one as high risk) or unclear risk study (all remaining situations). Any discrepancies between the authors’ assessments will be resolved through consensus. Review Manager V.5.4.1 and R software will be used to make the risk of bias diagram.
Statistical analyses
Pairwise meta-analyses
The analysis of four outcome measurements will be statistically analysed separately in our study. Effects will be estimated as standardised mean difference (SMD) for continuous data and OR for dichotomous data, along with 95% CI for continuous data. Statistical heterogeneity in each pairwise comparison will be assessed with I2 and p value. According to the result, a fixed-effect model (p<0.10, I2 >50%) or random-effects model (p>0.10, I2 <50%) will be used for analysis.
Network meta-analyses
The network meta-analyses will be conducted using R software and STATA V.17.0. The network plot will be generated using STATA V.17.0 software to show the direct and indirect comparative relationship among six novel teaching methods. Heterogeneity will be tested in the same way as pairwise meta-analysis. The pooled estimates of network meta-analysis will be obtained using the Markov Chain Monte Carlo method. Afterwards, in the case of closed loops of interventions, a consistency test will be conducted by node-splitting analysis, and the result will be determined based on the p values. The consistency model will be applied (p>0.05) when there are no significant differences between direct and indirect comparisons. Besides, the surface under the cumulative ranking curve will be used to estimate the probability of ranking each teaching strategy. The larger the area under the curve, the higher the ranking.
Subgroup analyses
When applicable, meta-regression or subgroup analyses will be performed to address the potential heterogeneity and inconsistency. Subgroup analyses can evaluate the influence of the following potential factors: (1) age, (2) gender, (3) grade, (4) intervention time, (5) time of duration, (6) curriculum and (7) the country in which the study was conducted.
Sensitivity analysis
To verify the robustness of the study conclusions, sensitivity analysis of primary outcomes will be carried out using the one-by-one elimination method, assessing the impact of study quality, sample size and the effect of missing data as well as the analysis methods on the result of this review. If the results of heterogeneity before and after sensitivity analysis are consistent, the results are regarded as robust.
Other analyses
Funnel plots and Egger’s test will be used to examine whether there is a dominant publication bias in this network meta-analysis. If the publication bias of two methods appears different, the latter will be used. In addition, we will grade the quality of evidence for primary outcome by using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) system, which characterises the quality of a body of evidence based on the study limitations, imprecision, inconsistency, indirectness and publication bias for network estimates. Through it, the quality of evidence will be divided into four levels: high, moderate, low and very low.
Patient and public involvement
No patients and/or the public were involved in the design of this systematic review protocol and NMA. The results will be disseminated in a peer-reviewed journal and presented at national and international conferences.
Ethics and dissemination
No formal research ethics approval is required because this study is a meta-analysis based on published studies. The results will be disseminated to a peer-reviewed journal for publication.
Ethics statements
Patient consent for publication
Acknowledgments
The authors thank Prof. Qian Luo (Third Military Medical University) for linguistic assistance in this protocol.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
SZ and DZ contributed equally.
Contributors HG, XF and DZ conceived the study. SZ and HG drafted the protocol. SZ, DZ and HG revised the protocol. HG, TL, LW and DZ designed and updated the strategy development. HG, SZ and DZ will perform the literature search and will finish most work of study selection, quality assessment and data collection. DZ and XW will help to resolve the disagreements and check the data. SZ, TL and HG participated in the design of data synthesis and analysis. SZ, LW and XW will contact the authors and conduct the statistical analyses. All authors revised and approved the final version of the protocol.
Funding This study was funded by the National Natural Science Foundation of China (Grant No. 82201687, 82071544) and the Open Fund of Guangxi Key Laboratory of Brain and Cognitive Neuroscience (Guilin Medical University) (Grant No. GKLBCN-202301-04).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.