You are here

  • Home
  • Archive
  • Volume 9, Issue 11
  • Is computer-assisted instruction more effective than other educational methods in achieving ECG competence amongst medical students and residents? A systematic review and meta-analysis

Article Text

Article menu

Download PDFPDF

Medical education and training
Original research
Is computer-assisted instruction more effective than other educational methods in achieving ECG competence amongst medical students and residents? A systematic review and meta-analysis
  1. Charle André Viljoen1,
  2. Rob Scott Millar1,
  3. Mark E Engel2,
  4. Mary Shelton3,
  5. Vanessa Burch2
  1. 1 Cardiology, University of Cape Town, Cape Town, South Africa
  2. 2 Medicine, Unversity of Cape Town, Cape Town, South Africa
  3. 3 Health Sciences Library, University of Cape Town, Cape Town, South Africa
  1. Correspondence to Dr Charle André Viljoen; charle.viljoen{at}uct.ac.za

Abstract

Objectives It remains unclear whether computer-assisted instruction (CAI) is more effective than other teaching methods in acquiring and retaining ECG competence among medical students and residents.

Design This systematic review and meta-analysis followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

Data sources Electronic literature searches of PubMed, databases via EBSCOhost, Scopus, Web of Science, Google Scholar and grey literature were conducted on 28 November 2017. We subsequently reviewed the citation indexes for articles identified by the search.

Eligibility criteria Studies were included if a comparative research design was used to evaluate the efficacy of CAI versus other methods of ECG instruction, as determined by the acquisition and/or retention of ECG competence of medical students and/or residents.

Data extraction and synthesis Two reviewers independently extracted data from all eligible studies and assessed the risk of bias. After duplicates were removed, 559 papers were screened. Thirteen studies met the eligibility criteria. Eight studies reported sufficient data to be included in the meta-analysis.

Results In all studies, CAI was compared with face-to-face ECG instruction. There was a wide range of computer-assisted and face-to-face teaching methods. Overall, the meta-analysis found no significant difference in acquired ECG competence between those who received computer-assisted or face-to-face instruction. However, subanalyses showed that CAI in a blended learning context was better than face-to-face teaching alone, especially if trainees had unlimited access to teaching materials and/or deliberate practice with feedback. There was no conclusive evidence that CAI was better than face-to-face teaching for longer-term retention of ECG competence.

Conclusion CAI was not better than face-to-face ECG teaching. However, this meta-analysis was constrained by significant heterogeneity amongst studies. Nevertheless, the finding that blended learning is more effective than face-to-face ECG teaching is important in the era of increased implementation of e-learning.

PROSPERO registration number CRD42017067054.

  • ECG
  • computer-assisted instruction
  • web-based learning
  • e-learning
  • systematic review

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2018-028800

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • To the best of our knowledge, this is the first systematic review and meta-analysis comparing the efficacy of computer-assisted instruction to other methods of ECG instruction among medical students and residents.

    • Systematic reviews provide robust evidence because they follow a rigorous method of search, selection and appraisal of articles.

    • We used the Medical Education Research Study Quality Instrument (MERSQI) to assess the quality of studies included in this systematic review.

    • The interpretation of the meta-analysis results is constrained by significant heterogeneity among the studies.

    • This systematic review with its meta-analysis and subanalyses identified valuable information about the educational approaches and types of computer-assisted learning material that were beneficial in acquiring ECG competence.

    Introduction

    The ECG is an indispensable diagnostic modality in cardiac disease.1 2 Although knowledge of, and skills in ECG analysis and interpretation, hereafter referred to as ECG competence, are desired learning outcomes of undergraduate and postgraduate medical training programmes, there is ongoing concern that graduating medical trainees lack adequate ECG competence.3–12 Many reasons account for this observation. First, electrocardiography is a difficult subject to teach and to learn.13 14 Second, although clinical exposure is important to gain experience in ECG analysis and interpretation,15 experiential learning alone does not guarantee ECG competence unless it is supplemented by structured teaching.16 Third, medical knowledge is ever-expanding,17 and there is limited time allocated to the teaching of electrocardiography in medical curricula.18–22 Alternative methods of instruction are therefore being sought to improve ECG training.

    Technology-enhanced methods of instruction are increasingly being implemented in the training of healthcare professionals.23–25 It remains important to review whether these novel teaching and learning methods are effective.26 Previous studies have shown that students’ knowledge of, and skills in the analysis and interpretation of ECGs improve with computer-assisted instruction (CAI).27–34 However, these studies did not compare CAI to other methods of instruction and thus it cannot be concluded that CAI is better than traditional methods of ECG teaching.

    To the best of our knowledge, there is no published systematic review comparing the efficacy of CAI with other methods of ECG instruction for training medical students and residents. Systematic reviews are important in the era of best evidence health professions education,35 because they follow a rigorous process of searching, selecting and appraising eligible articles.36 37 Reviewer bias is limited by applying strict criteria when appraising the articles and summarising the strengths and weaknesses of the studies evaluated.36–38

    Objectives

    The objectives of this systematic review were to:

    • establish whether CAI (on its own or in a blended learning setting) achieves better acquisition of ECG competence among medical students and residents than other methods of ECG instruction do;

    • establish whether CAI (on its own or in a blended learning setting) achieves better retention of ECG competence among medical students and residents than other methods of ECG instruction do;

    • establish whether there is a difference in the effectiveness of computer-assisted ECG instruction between medical students and residents enrolled for specialty training;

    • identify the types of learning material and/or activities that are used in computer-assisted ECG instruction, and to establish which CAI material and/or activities are associated with better outcomes;

    • identify the educational approaches used in computer-assisted ECG instruction, and to establish which of these are associated with better outcomes;

    • identify learning theories that may underpin computer-assisted ECG instruction.

    Methods

    A protocol was developed in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) guidelines39 and registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 6 July 2017 with registration number CRD42017067054.40

    Search strategy

    By using the search strategy described in the protocol,40 and shown in online supplementary file 1, we searched for relevant studies on 28 November 2017 using the following electronic databases: PubMed, EBSCOhost (which searched Academic Search Premier, CINAHL, PsycINFO, Education Resources Information Center, Africa-Wide Information, Teacher Reference Center), Scopus, Web of Science and Google Scholar. Citation indexes and reference lists were reviewed, and a grey literature search was also conducted.

    Supplemental material

    Eligibility criteria

    As summarised in table 1, all studies that compared the efficacy of CAI with other methods of ECG instruction were eligible for inclusion in this review. Studies were excluded if the teaching methods were not exclusively used to teach ECGs, or if the subject of teaching was not the conventional 12-lead ECG. We included studies in which the participants were medical students and/or residents enrolled for specialty training. Studies were excluded if the data for medical students or residents could not be separately identified from students other than medical students, healthcare professionals who were not medical doctors or qualified doctors who were not in training. We excluded studies that did not assess ECG knowledge and analysis and interpretation skills (ECG competence). There were no language or geographical restrictions. All eligible articles published before 1 July 2017 were included.40

    View this table:
    Table 1

    Eligibility criteria

    Study selection

    Two reviewers (CAV and RSM) independently screened all the articles identified by the search. All titles and abstracts were screened for eligibility and full-text articles of all studies potentially meeting inclusion criteria were retrieved. Both reviewers (CAV and RSM) individually evaluated the full text articles using a predesigned form evaluating each study’s eligibility. Where there was no consensus, the reviewers (CAV and RSM) discussed uncertainties pertaining to inclusion eligibility and a third reviewer (VCB) acted as an adjudicator.

    Data abstraction

    Two reviewers (CAV and RSM) independently extracted data from all eligible studies using a standardised electronic data abstraction form hosted on Research Electronic Data Capture (REDCap),41 which was subsequently crosschecked (CAV and RSM). Data extraction included study design, study duration, study population, ECGs used during teaching, teaching methods (CAI and non-CAI methods), type of digital learning material, educational approaches, learning theories underpinning instructional methods (using a classification proposed by Taylor).42 ECG competencies measured, testing times and results, as well as the validity and reliability of results with psychometric properties of the assessment tools (eg, Cronbach’s α coefficient) where reported.

    In the event of missing or unreported data, corresponding authors were contacted. Following two email messages, a delay of 6 weeks was allowed to receive a response.

    Quality of included studies and risk of bias assessment

    The Medical Education Research Study Quality Instrument (MERSQI) was used to assess the quality of studies included in this systematic review. The MERSQI is a validated quality assessment tool used in health professions education to evaluate the quality of experimental, quasi-experimental and observational studies.36 43

    As recommended by the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA),44 two reviewers (CAV and RSM) independently assessed each included study for risk of selection, performance, attrition, detection and/or reporting bias.

    Data synthesis

    Tests scores (pre-intervention test, post-intervention test and delayed post-intervention test) reported in the studies were used as objective measures of teaching method effectiveness.34 40 45 Where the mean or SD results were not reported, these were requested from the authors or, in the absence of a reply, calculated using the formula of Wan et al.46 The mean and SD results for the CAI and non-CAI groups in each study were converted to a standardised mean difference (effect size, Cohen’s d).47–49 Random-effects models were used to pool weighted effect sizes for all studies, as well as for the planned subanalyses. Planned subanalyses were conducted based on the level of training of participants (students or residents), the different educational approaches reported in the studies (eg, blended learning or not, massed or distributed instruction, restricted or unrestricted access to CAI, online or offline use of CAI), as well as learning materials (eg, real patient ECGs, case scenarios, images, animations) and learning activities (eg, online chat rooms, self-administered quizzes with automated feedback) used with CAI. The consistency in results was determined by visualising the forest plots and calculating the I2 statistic.50Statistical analyses were performed on Stata (V.14.2, StataCorp, College Station, Texas, USA) and Review Manager (RevMan, V.5.3.5, Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014).

    We analysed studies for their educational impact using the modified version of the Kirkpatrick framework.35 51–53 The modified Kirkpatrick model is a widely used method of appraising the outcome of educational interventions by measuring participants’ perceptions of (reactions to) the learning experience (level 1), modification of participants’ perceptions of the intervention (level 2a), modification of their knowledge and/or skills (level 2b), transfer of learning to the workplace (level 3), change in organisational practice (level 4a) and benefits to patients (level 4b).

    Patient and public involvement

    There were no patients or public involved in this systematic review and meta-analysis.

    Results

    Trial flow

    Our search strategy identified 592 papers, that is, 129 articles in PubMed, 349 in EBSCOhost, 65 in Scopus and 49 in Web of Science. We identified an additional 32 papers by reviewing the citation indexes and reference lists of the identified articles and grey literature. After 65 duplicate publications were removed, another 437 articles were excluded by screening their titles and abstracts. From the remaining 122 articles that were assessed in full text, thirteen articles met the predefined eligibility criteria for this systematic review. The reasons for exclusion are shown in figure 1. Eight studies contained sufficient data (mean scores, SD and number of participants reported for each cohort) to be included in the meta-analysis.

    Figure 1
    Figure 1

    Trial flow. CAI, computer-assisted instruction.

    Study characteristics

    Table 2 summarises the characteristics of the nine randomised control trials and four prospective cohort studies that were included in this systematic review. Nine studies were conducted at a single centre, three studies at two centres and one study at more than two centres. Four studies were conducted in the USA,54–57 three in the UK,58–60 two in France61 62 and one each in China,63 Iran,64 India65 and Sweden.66 All the studies were published in English and included 1242 students and 86 residents in total. Of the thirteen studies, eleven focused on undergraduate students,54–58 60 61 63–66 one on residents62 and one on both students and residents.59

    View this table:
    Table 2

    Characteristics of included studies in this systematic review

    As shown in online supplementary file 2, the earliest study on the use of computer-assisted ECG instruction was published in 1965,60 followed by two studies in the mid 80s.55 56 Most of the studies were published in the last decade,54 57–59 61–66 the majority of which used online CAI (web-based instruction).54 58 61–64 66

    Supplemental material

    Study quality

    A detailed summary of the quality of the included studies as measured by the MERSQI tool is contained in online supplementary file 3. The mean MERSQI total score of all included studies was 12.73 (SD 1.76). The studies scored well in the domains that assessed the type of data and data analysis. All studies had objective outcome assessments and twelve of the thirteen studies reported appropriate analyses, which extended beyond descriptive analysis. Studies scored poorly in the sampling domain: more than two-thirds of studies were conducted at a single centre and a third had a response rate of either less than 50% or did not report their response rate.

    Supplemental material

    Risk of bias

    As elaborated in table 3, and summarised in online supplementary file 3, there was selection bias and/or performance bias in nine studies. Three studies had attrition bias and one had reporting bias.

    View this table:
    Table 3

    Summary of the study design, assessment of knowledge and outcomes of the included studies

    Educational approaches

    In all studies, CAI was compared with face-to-face teaching (refer to glossary for definitions). However, CAI and face-to-face teaching were delivered in variable formats. CAI formed part of a blended learning strategy in four studies (online supplementary file 4).54 57 63 66 In one of these studies, blended learning was applied in a ‘flipped classroom’ approach, where CAI took place before classroom teaching.63 Face-to-face teaching was facilitated by experienced lecturers or specialists in all the studies,54–57 59–66 with the exception of one study in which near-peer teaching was used.58

    Supplemental material

    The frequency of instruction in the studies was variable. In three of the thirteen studies, participants were exposed to a single learning event (massed instruction), whether assigned to CAI or face-to-face teaching, before ECG competence was assessed.58 59 64

    Learning materials and activities

    A range of learning materials were used in CAI (table 2). In most studies, the digital learning material consisted of ECG tracings with accompanying text. In addition, in some studies CAI also included the use of multimedia in the form of diagrams and images64 or animations.57 59 66 As summarised in table 2, the curricular content varied across the studies and a wide range of ECG diagnoses were included.

    Active learning (during which learners deliberately engaged with learning material)67 formed an integral part of CAI, which used ‘interactive software’ in all the studies included in this review. In addition to engaging with the learning material, some studies also reported on the use of self-administered assessments with automated feedback,54 56 58 60–62 64 66 online chat rooms61 63 and interaction with lecturers and peers during ‘flipped classroom’ activities.63 Six of the thirteen studies reported interaction between students and lecturers in the non-CAI group, for example, lecturers quizzed students or students asked questions during the face-to-face teaching activities (online supplementary file 5).54 58 59 61 62 64 In the study where CAI was compared with near-peer face-to-face teaching, there was a strong emphasis on interaction between students and tutors in the face-to-face teaching group.58

    Supplemental material

    Educational outcomes

    The outcomes of the studies are summarised in table 3. Baseline ECG competence was assessed in six of the thirteen studies.54–56 61 62 65 All studies tested ECG competence acquired after the educational intervention; only one study assessed the retention of ECG competence after a period of three months without further instruction since the acquisition of knowledge was tested.59 Five studies used multiple choice questions to assess study participants’ knowledge,58 59 61 62 65 whereas another five used short answer questions marked by the course convenors.54–56 60 63 Three studies did not report how ECG competence was assessed.

    Using the Kirkpatrick model of evaluation of educational interventions, it was found that eight studies reported participants’ reactions to CAI (Kirkpatrick level 1)54 58–61 63 64 66 and three studies reported a change in trainees’ attitudes and perceptions after exposure to CAI (Kirkpatrick level 2a).54 58 59 All the studies reported on the acquisition and/or retention of ECG competence (Kirkpatrick level 2b) since this was one of the eligibility criteria of this systematic review. None of the studies reported on outcomes at Kirkpatrick level 3 or 4.

    Kirkpatrick level 1 and 2a outcomes were variable. Though some studies reported that students had a positive attitude towards web-based learning,54 60 63 64 66 others reported less favourable attitudes towards CAI than lectures.58 59 61 In one study, all the potential participants did not want to use the e-learning platform and so some potential participants were excluded from the particular study.66 While three studies reported on students who felt that an improvement in their confidence was no better with CAI as compared with lectures,58 59 61 other studies identified students who thought that CAI improved their confidence in ECG interpretation.54 58 66 In general, students valued CAI approaches that included multimedia learning material,59 64 and self-assessment tools.66 In some studies they requested more visually-oriented learning material59 64 and applications that had a facility or method for asking questions.59 Kirkpatrick level 2b outcomes of the studies are summarised in table 3 and have already been described .

    Learning theories

    Learning theories that underpin education were infrequently mentioned or discussed in any detail. The most frequent reference to learning theories was to self-directed learning in CAI.54 59 62 63 66 One study66 referenced Kolb’s description of experiential learning,68 and another study mentioned ‘cognitive learning’ and ‘collaborative learning’.63 However, careful review of the papers included in this systematic review identified multiple examples of teaching and learning activities that were aligned with contemporary theories of learning. These are shown in table 4 using a simplified classification of learning theories described by Taylor.42

    View this table:
    Table 4

    Learning theories, based on a classification by Taylor42 that underpinned computer-assisted and face-to-face ECG instruction in the included studies

    Quantitative data synthesis

    Overall, we found that CAI was not better than face-to-face teaching for acquiring ECG competence (standardised mean difference (SMD)=0.32 (95% CI −0.09 to 0.74); eight studies, n=945; I2=88.9%) (figure 2). However, there was inconsistency among the studies and effect sizes ranged from −1.08 to 1.09 (table 5). A positive effect size (ie, CAI was better than face-to-face teaching) was found in most studies, one of which showed a large effect size (>0.8)54 and four a moderate effect size (>0.5).62 63 65 66 However, in two studies59 60 there was no significant difference between CAI and face-to-face teaching and one study showed that face-to-face teaching was better than CAI.58

    Figure 2
    Figure 2

    Overall effect of teaching methods on the acquisition of ECG knowledge and analysis and interpretation skills. CAI, computer-assisted instruction; SMD, standardised mean difference.

    View this table:
    Table 5

    Acquired and retained ECG competence according to educational approaches used in the included studies

    Only one study assessed the effect of CAI on the retention of ECG competence.59 While this study showed that there was no significant difference between the CAI and face-to-face teaching (SMD=−0.24 (95% CI −1.05 to 0.58)), the response rate was only 14% for the retention of knowledge test which was conducted three months after the educational intervention.

    Medical students compared to residents

    In the subanalysis comparing the acquisition of ECG competence with CAI and face-to-face teaching in undergraduate and postgraduate trainees separately (figure 3), there was a tendency to favour CAI over face-to-face teaching among both medical students (SMD=0.41 (95% CI −0.03 to 0.84); six studies, n=738; I2=87%) and residents (SMD=0.64 (95% CI 0 to 1.28); one study, n=19). The single study assessed the retention of ECG competence combined medical students and residents.59

    Figure 3
    Figure 3

    Pooled effect sizes according to level of training of participants, educational approaches and CAI learning materials used in the studies. CAI, computer-assisted instruction.

    Educational approaches

    A subanalysis found a large positive effect size when CAI formed part of a blended learning strategy as compared with face-to-face teaching (SMD=0.84 (95% CI 0.54 to 1.14); three studies, n=422; I2=50%) (figure 3). This systematic review did not identify any studies that evaluated the retention of ECG analysis and interpretation skills after exposure to CAI in a blended learning programme.

    In another subanalysis, studies using a distributed approach to ECG instruction (ie, more than one ECG training opportunity) showed that CAI was better than face-to-face teaching (SMD=0.65 (95% CI 0.31 to 1.00); five studies, n=538; I2=70%). Review of these studies showed that the benefit of distributed instruction was only present in studies where CAI was part of a blended learning approach (SMD=0.84 (95% CI 0.54 to 1.14); three studies, n=422; I2=50%; vs SMD=0.31 (95% CI −0.21 to 0.84); two studies, n=116; I2=46%). There was no statistically significant difference between CAI and face-to-face teaching when massed instruction strategies were used (ie, a single session of ECG teaching) (figure 3).

    Although there was no difference between online and offline CAI, four studies showed that CAI was better than face-to-face teaching when students had unlimited access (ie, 24 hours a day, 7 days a week) to CAI learning materials (SMD=0.82 (95% CI 0.57 to 1.07); four studies, n=461; I2=32%). This benefit, as shown in a subanalysis, was not apparent when access to CAI learning materials was limited (SMD=−0.34 (95% CI −0.86 to 0.18); three studies, n=284; I2=74%).

    In the study that used reminder emails to encourage the use of CAI, there was a large effect size in favour of CAI (1.09 (95% CI 0.79 to 1.4)).54

    Learning activities and materials used in CAI

    Subanalyses showed that CAI was better than face-to-face teaching when ECGs were accompanied by case scenarios (SMD=0.90 (95% CI 0.59 to 1.21); three studies, n=280; I2=24%) and if images were used to explain impulse conduction (SMD=1.09 (95% CI 0.79 to 1.40); one study, n=191). Studies in which CAI included self-administered assessments with automated feedback showed better ECG knowledge acquisition than face-to-face teaching (SMD=0.64 (95% CI 0.14 to 1.13); four studies, n=357; I2=77%) (figure 3). This effect size was larger in studies where self-administered assessment with automated feedback formed part of a blended learning approach (SMD=0.95 (95% CI 0.57 to 1.34); two studies, n=241; I2=38%). CAI was better than face-to-face teaching when students had access to online chat rooms to discuss the study material (SMD=0.68 (95% CI 0.38 to 0.98); one study, n=181) (figure 3).

    Discussion

    This systematic review and meta-analysis set out to determine whether CAI is more effective than other methods of teaching electrocardiography knowledge and analysis and interpretation skills to undergraduate and postgraduate medical trainees. All the studies included in this systematic review and meta-analysis compared CAI to face-to-face teaching. Based on the overall results of the review there is currently insufficient evidence to favour CAI over face-to-face ECG instruction. Though there was significant heterogeneity in the studies included in the meta-analysis, subanalyses of the different learning materials and educational approaches were less heterogenous. We found that CAI was better than face-to-face teaching when used in a blended learning approach. Studies also favoured computer-assisted distributed instruction with unrestricted access to learning materials; the use of case scenarios to contextualise ECG interpretation with images to explain concepts and interactive learning activities, including chat rooms, and self-assessment with automated feedback. While contemporary learning theories were not explicitly articulated in most studies, there were many examples of computer-assisted instruction strategies and activities that were aligned with these theories.

    Although self-directed, computer-assisted learning may seem attractive to busy clinicians with limited time for teaching,55 61 our systematic review and meta-analysis did not find sufficient evidence to recommend that computer-assisted ECG instruction should replace face-to-face teaching. Rather, we found that computer-assisted ECG instruction was more effective than face-to-face teaching when it formed part of a blended learning strategy. This is in keeping with the literature which shows that CAI should be used as an adjunct to face-to-face teaching in order to enhance ECG training.56 61 69 Our findings are also in keeping with the results of a recent meta-analysis published in the health professions education literature, which showed that blended learning was better than face-to-face teaching alone.70 However, as with other systematic reviews and meta-analyses that assessed the efficacy of blended learning in the training of healthcare professionals,71 72 our analyses were also limited by a small number of studies, incomplete reporting of results and significant heterogeneity among the studies.

    One of the studies included in this review demonstrated the successful use of CAI in a flipped classroom strategy for teaching ECG analysis and interpretation skills.63 Although the flipped classroom method required more preparation time, for both lecturers and students, trainees were more proactive in discussions with their peers and their lecturers during the face-to-face teaching time, resulting in better post-intervention test scores than traditional face-to-face teaching.73 Since it is accepted that ECG competence is difficult to acquire,13 14 the successful use of a flipped classroom approach is encouraging because this method allows for engagement with the learning material prior to face-to-face interaction with teachers when difficult concepts can be discussed and misunderstandings resolved.

    When evaluating the educational effect of teaching and learning methods, it is critical to review access and frequency of exposure to the learning materials. In a subanalysis, students did not benefit from computer-assisted or face-to-face massed instruction (single educational event). As has been previously found,74 CAI was only beneficial if students had multiple exposures to the learning activities and study materials (distributed instruction). In the setting of blended learning, CAI facilitates distributed instruction, because it can be used asynchronously, allowing for consolidation of knowledge acquired during face-to-face teaching.24 34 61 75

    This review found that there was a significant benefit to students when they had unrestricted access to CAI learning materials. Although we did not show a difference in outcomes between online and offline CAI, the benefit of web-based learning is that it can be accessed whenever and wherever convenient.24 34 61 75 However, the high cost of, and/or lack of access to computers with Internet facilities may be a barrier to web-based learning, particularly in developing countries.24 64 76 Health professions educators, especially in resource-limited settings should therefore be cognisant of the availability of computers and students’ access to the Internet when planning CAI with online requirements.

    A key aspect of any method of instruction is the nature of the learning materials and activities included in the programme. CAI has been shown to enhance the learning experience by using multimedia and interactive learning materials.69 In this study we confirmed that visual material was highly valued by participants and a subanalysis showed specific benefit when using images in combination with the 12-lead ECG, for example to explain cardiac impulse conduction. The value of using images in medical education is that it helps to embed knowledge in long-term memory.25 Although images are widely used to demonstrate concepts in medical education,77 it has previously been shown to be of most value when accompanied by good explanations,78 79 as was the case in the study by Nilsson et al.66 In this study we also found that there were additional educational gains when computer-assisted ECG instruction made use of clinical scenarios.54 61 62 66 This is in line with previous studies which have shown more accurate ECG analysis and interpretation when the clinical context was known.80 81

    In this systematic review we found evidence that CAI was better than face-to-face teaching in studies in which the CAI included exercises of ECG analysis and interpretation that required deliberate practice with automated feedback. This finding is in keeping with studies which have shown that practice exercises followed by feedback facilitate high levels of interactivity with educational materials and significantly enhance learning.61 82–84 In CAI there are opportunities for both self-reflection85 and repetitive practice86 because students can repeat the self-assessments, correct their errors and further improve their performance.54 60–62 66 84

    The studies included in this review demonstrated variable outcomes using the Kirkpatrick framework of evaluation. Improvement of trainees’ ECG knowledge and analysis and interpretation skills using either CAI and face-to-face instruction was an eligibility criterion for inclusion in the study. A few studies reported on the responses of participants to the methods of instruction used with no consistent preference for CAI. None of the studies evaluated CAI at the level of behavioural change (Kirkpatrick level 3), change in organisational practice (Kirkpatrick level 4a) or improved patient care (Kirkpatrick level 4b). This is consistent with studies showing that health professions education interventions rarely show impact at Kirkpatrick level 3 or 4.87 88 Indeed it is a widely recognised ongoing shortcoming of health professions education research. This systematic review endorses a plea in the literature for the evaluation of educational interventions at the level of impact on physician behaviour,89 90 organisational practice91 92 and patient care.93–95

    While learning theories were not explicitly discussed in most of the studies in this review, there were multiple examples of educational strategies that are aligned with contemporary learning theories.96 However, as this review shows, studies describing and evaluating educational interventions continue to be conducted without a firm rationale imbedded in contemporary learning theories. This highlights a significant ongoing shortcoming of health professions education research.97–99

    CAI serves as a good example of self-directed learning, whereby students plan and conduct their own learning.42 While face-to-face teaching time is limited,100 CAI allows for flexibility in learning – students can adjust the pace of their learning and spend as much time as they need to assimilate new knowledge. While face-to-face teaching is ideal for promoting collaborative learning by allowing interaction between peers and tutors,55 58 59 61 62 it is also possible in CAI when chat rooms were available61 63 or when CAI forms part of a blended learning programme.54 57 63 66

    In this review we found that participants valued learning with demonstrations and explanations (cognitivism).96 CAI-based learning opportunities had the advantage of offering multimedia learning resources, which enrich the educational content by means of animations, audio and video clips.69

    The flipped classroom method of teaching ECGs, as described in one study included in this review,63 serves as an excellent example of a learning process which focuses on actively creating meaning rather than merely acquiring knowledge (constructivism).96 In a flipped classroom approach, students used CAI to familiarise themselves with educational content, and expand their learning by using the time in class to discuss concepts that they did not understand.101 It seems that this could be a useful approach for electrocardiography, which is considered a difficult subject to teach and to learn.13 14

    Because CAI does not require attendance of class, external motivation in the form of reminder emails or summative assessments might be needed to encourage students to use the e-learning modules. In the study that made use of such external motivation strategies, CAI showed a large positive effect size.54

    Though variably applied in the studies in this review, contextualisation was possible in both CAI and face-to-face teaching settings.54 61 62 66 Where CAI made use of patient scenarios, there was a larger benefit in acquiring ECG competence.

    Reflective learning is possible with CAI when self-administered quizzes with automated feedback are used. Learning is facilitated because knowledge and/or skills gaps are highlighted.54 61 62 66

    Strengths and limitations

    The strength of this study is that it was conducted as a systematic review using a comprehensive search strategy and detailed data extraction method. However, the inferences that can be made from this systematic review and its meta-analysis are limited by high levels of bias and the heterogeneity of the included studies. There was significant variability in study design, the format, delivery and exposure time of the teaching intervention (CAI) and control (face-to-face teaching) and the topics taught and assessed.57 Many studies also did not include a baseline test of ECG knowledge and/or analysis and interpretation skills prior to the educational intervention and did not report all their data. Nevertheless, the mean MERSQI score of the studies included in this review was similar to MERSQI scores reported in other systematic reviews in medical education.23 102–104 In fact, 9 of the 13 studies in this review had a high MERSQI score (ie ≥12).105 Furthermore, most of the studies included in this systematic review were performed in well-resourced countries and the generalisability of these findings to resource-constrained settings is therefore not known.

    Implications for practice and future research

    Owing to the heterogenous nature of the studies included in this review it was not possible to provide conclusive evidence that CAI is better than face-to-face teaching of ECG knowledge and analysis and interpretation skills. However, CAI was better than face-to-face teaching in a blended learning setting where students had unrestricted access to the learning materials and opportunities for self-assessment with automated feedback.

    There are currently many aspects of CAI that need to be further explored. These include a more detailed evaluation of the efficacy of this medium of instruction in postgraduate education and its impact on the long-term retention of ECG competence in both undergraduate and postgraduate trainees. Studies are also needed to better understand the impact of CAI on clinician behaviour (ECG analysis and interpretation practices in clinical settings), changes in organisational practice and patient care.

    Conclusion

    Owing to the mixed findings of the studies included in this systematic review, there is currently insufficient evidence to favour the use of computer-assisted ECG instruction. However, CAI can be used to enhance face-to-face teaching in a blended learning setting. CAI was found to be more beneficial than face-to-face teaching when students had unrestricted access to learning materials and opportunities for self-assessment with automated feedback.

    Acknowledgments

    The authors wish to thank Ms Sylvia Dennis, Dr Nicholas Simpson, Ms Kathryn Manning and Professsor Ike Okpechi from the University of Cape Town for their valuable support and input.

    References

    1. ↵
      2. Okreglicki A ,
      3. Scott Millar R
      . ECG: PQRST morphology – clues and tips. A guide to practical pattern recognition. SA Heart Journal 2006;3:27–36.
      OpenUrl
    2. ↵
      2. Auseon AJ ,
      3. Schaal SF ,
      4. Kolibash AJ , et al
      . Methods of teaching and evaluating electrocardiogram interpretation skills among cardiology fellowship programs in the United States. J Electrocardiol 2009;42:339–44.doi:10.1016/j.jelectrocard.2009.01.004
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      2. Novotny T ,
      3. Bond RR ,
      4. Andrsova I , et al
      . Data analysis of diagnostic accuracies in 12-lead electrocardiogram interpretation by junior medical fellows. J Electrocardiol 2015;48:988–94.doi:10.1016/j.jelectrocard.2015.08.023
      OpenUrl
    4. ↵
      2. Kopeć G ,
      3. Magoń W ,
      4. Hołda M , et al
      . Competency in ECG interpretation among medical students. Med Sci Monit 2015;21:3386–94.doi:10.12659/MSM.895129
      OpenUrl
    5. ↵
      2. Jablonover RS ,
      3. Lundberg E ,
      4. Zhang Y , et al
      . Competency in electrocardiogram interpretation among graduating medical students. Teach Learn Med 2014;26:279–84.doi:10.1080/10401334.2014.918882
      OpenUrlPubMed
    6. ↵
      2. McAloon C ,
      3. Leach H ,
      4. Gill S , et al
      . Improving ECG competence in medical trainees in a UK district general hospital. Cardiol Res 2014;5:51–7.doi:10.14740/cr333e
      OpenUrl
    7. ↵
      2. de Jager J ,
      3. Wallis L ,
      4. Maritz D
      . ECG interpretation skills of South African emergency medicine residents. Int J Emerg Med 2010;3:309–14.doi:10.1007/s12245-010-0227-3
      OpenUrlCrossRefPubMed
    8. ↵
      2. Eslava D ,
      3. Dhillon S ,
      4. Berger J , et al
      . Interpretation of electrocardiograms by first-year residents: the need for change. J Electrocardiol 2009;42:693–7.doi:10.1016/j.jelectrocard.2009.07.020
      OpenUrlPubMed
    9. ↵
      2. Lever NA ,
      3. Larsen PD ,
      4. Dawes M , et al
      . Are our medical graduates in New Zealand safe and accurate in ECG interpretation? N Z Med J 2009;122:9–15.
      OpenUrlPubMed
    10. ↵
      2. Berger JS ,
      3. Eisen L ,
      4. Nozad V , et al
      . Competency in electrocardiogram interpretation among internal medicine and emergency medicine residents. Am J Med 2005;118:873–80.doi:10.1016/j.amjmed.2004.12.004
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      2. Salerno SM ,
      3. Alguire PC ,
      4. Waxman HS
      . Competency in interpretation of 12-lead electrocardiograms: a summary and appraisal of published evidence. Ann Intern Med 2003;138:751–60.doi:10.7326/0003-4819-138-9-200305060-00013
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      2. Little B ,
      3. Mainie I ,
      4. Ho KJ , et al
      . Electrocardiogram and rhythm strip interpretation by final year medical students. Ulster Med J 2001;70:108–10.
      OpenUrlPubMed
    13. ↵
      2. Hurst JW
      . Current status of clinical electrocardiography with suggestions for the improvement of the interpretive process. Am J Cardiol 2003;92:1072–9.doi:10.1016/j.amjcard.2003.07.006
      OpenUrlPubMed
    14. ↵
      2. Hurst JW
      . Methods used to interpret the 12-lead electrocardiogram: pattern memorization versus the use of vector concepts. Clin Cardiol 2000;23:4–13.doi:10.1002/clc.4960230103
      OpenUrlPubMed
    15. ↵
      2. Alinier G ,
      3. Gordon R ,
      4. Harwood C , et al
      . 12-Lead ECG training: the way forward. Nurse Educ Today 2006;26:87–92.doi:10.1016/j.nedt.2005.08.004
      OpenUrlPubMed
    16. ↵
      2. Devitt P ,
      3. Worthley S ,
      4. Palmer E , et al
      . Evaluation of a computer based package on electrocardiography. Aust N Z J Med 1998;28:432–5.doi:10.1111/j.1445-5994.1998.tb02076.x
      OpenUrlPubMedWeb of Science
    17. ↵
      2. Nilsson M ,
      3. Östergren J ,
      4. Fors U , et al
      . Does individual learning styles influence the choice to use a web-based ECG learning programme in a blended learning setting? BMC Med Educ 2012;12:5.doi:10.1186/1472-6920-12-5
    18. ↵
      2. Raupach T ,
      3. Harendza S ,
      4. Anders S , et al
      . How can we improve teaching of ECG interpretation skills? Findings from a prospective randomised trial. J Electrocardiol 2016;49:7–12.doi:10.1016/j.jelectrocard.2015.10.004
      OpenUrl
    19. ↵
      2. Moura FS ,
      3. Carvalho FVde ,
      4. Martins MdoCdeCe , et al
      . Knowledge of guidelines for cardiopulmonary resuscitation among Brazilian medical students. Rev Bras Educ Med 2016;40:77–85.doi:10.1590/1981-52712015v40n1e01772015
      OpenUrl
    20. ↵
      2. Dong R ,
      3. Yang X ,
      4. Xing B , et al
      . Use of concept maps to promote electrocardiogram diagnosis learning in undergraduate medical students. Int J Clin Exp Med 2015;8:7794–801.
      OpenUrl
    21. ↵
      2. Goy J-J ,
      3. Schlaepfer J ,
      4. Stauffer J-C
      . Competency in interpretation of 12-lead electrocardiogram among Swiss doctors. Swiss Med Wkly 2013;143:w13806.doi:10.4414/smw.2013.13806
    22. ↵
      2. Hurst JW
      . The interpretation of electrocardiograms: pretense or a well-developed skill? Cardiol Clin 2006;24:vii:305–7.doi:10.1016/j.ccl.2006.03.001
      OpenUrlCrossRef
    23. ↵
      2. Taveira-Gomes T ,
      3. Ferreira P ,
      4. Taveira-Gomes I , et al
      . What are we looking for in computer-based learning interventions in medical education? A systematic review. J Med Internet Res 2016;18:e204.doi:10.2196/jmir.5461
    24. ↵
      2. Greenhalgh T
      . Computer assisted learning in undergraduate medical education. BMJ 2001;322:40–4.doi:10.1136/bmj.322.7277.40
      OpenUrlFREE Full Text
    25. ↵
      2. Swanwick T
      . Understanding medical education: evidence, theory and practice. 2nd edn. Wiley Blackwell, 2013.
    26. ↵
      2. Cook DA ,
      3. West CP
      . Conducting systematic reviews in medical education: a stepwise approach. Med Educ 2012;46:943–52.doi:10.1111/j.1365-2923.2012.04328.x
      OpenUrlCrossRefPubMed
    27. ↵
      2. Burke JF ,
      3. Gnall E ,
      4. Umrudden Z , et al
      . Critical analysis of a computer-assisted tutorial on ECG interpretation and its ability to determine competency. Med Teach 2008;30:e41–8.doi:10.1080/01421590801972471
      OpenUrlPubMed
    28. ↵
      2. DeBonis K ,
      3. Blair TR ,
      4. Payne ST , et al
      . Viability of a web-based module for teaching electrocardiogram reading skills to psychiatry residents: learning outcomes and trainee interest. Academic Psychiatry 2015;39:645–8.doi:10.1007/s40596-014-0249-x
      OpenUrl
    29. ↵
      2. Jheeta JS ,
      3. Narayan O ,
      4. Krasemann T
      . Republished: accuracy in interpreting the paediatric ECG: a UK-wide study and the need for improvement. Postgrad Med J 2015;91:436–8.doi:10.1136/postgradmedj-2013-305788rep
      OpenUrlAbstract/FREE Full Text
    30. ↵
      2. Liu SS ,
      3. Zakaria S ,
      4. Vaidya D , et al
      . Electrocardiogram training for residents: a curriculum based on Facebook and Twitter. J Electrocardiol 2017;50:646–51.doi:10.1016/j.jelectrocard.2017.04.010
      OpenUrl
    31. ↵
      2. Porras L ,
      3. Drezner J ,
      4. Dotson A , et al
      . Novice interpretation of screening electrocardiograms and impact of online training. J Electrocardiol 2016;49:462–6.doi:10.1016/j.jelectrocard.2016.02.004
      OpenUrl
    32. ↵
      2. Pourmand A ,
      3. Yadav K ,
      4. Shokoohi H , et al
      . 191 can lecture capture technology affects accuracy of EKG interpretation among emergency medicine residents? Ann Emerg Med 2012;60:S68–9.doi:10.1016/j.annemergmed.2012.06.168
      OpenUrl
    33. ↵
      2. Pourmand A ,
      3. Tanski M ,
      4. Davis S , et al
      . Educational technology improves ECG interpretation of acute myocardial infarction among medical students and emergency medicine residents. West J Emerg Med 2015;16:133–7.doi:10.5811/westjem.2014.12.23706
      OpenUrl
    34. ↵
      2. Rolskov Bojsen S ,
      3. Räder SBEW ,
      4. Holst AG , et al
      . The acquisition and retention of ECG interpretation skills after a standardized web-based ECG tutorial – a randomised study. BMC Med Educ 2015;15:36.doi:10.1186/s12909-015-0319-0
    35. ↵
      2. Harden RM ,
      3. Grant J ,
      4. Buckley G , et al
      . BEME guide No. 1: best evidence medical education. Med Teach 1999;21:553–62.doi:10.1080/01421599978960
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      2. Sharma R ,
      3. Gordon M ,
      4. Dharamsi S , et al
      . Systematic reviews in medical education: a practical approach: AMEE guide 94. Med Teach 2015;37:108–24.doi:10.3109/0142159X.2014.970996
      OpenUrlCrossRefPubMed
    37. ↵
      2. Yuan Y ,
      3. Hunt RH
      . Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol 2009;104:1086–92.doi:10.1038/ajg.2009.118
      OpenUrlCrossRefPubMed
    38. ↵
      2. Cook DJ ,
      3. Mulrow CD ,
      4. Haynes RB
      . Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997;126:376–80.doi:10.7326/0003-4819-126-5-199703010-00006
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      2. Shamseer L ,
      3. Moher D ,
      4. Clarke M , et al
      . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647.doi:10.1136/bmj.g7647
    40. ↵
      2. Viljoen CA ,
      3. Scott Millar R ,
      4. Engel ME , et al
      . Is computer-assisted instruction more effective than other educational methods in achieving ECG competence among medical students and residents? Protocol for a systematic review and meta-analysis. BMJ Open 2017;7:e018811.doi:10.1136/bmjopen-2017-018811
    41. ↵
      2. Harris PA ,
      3. Taylor R ,
      4. Thielke R , et al
      . Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81.doi:10.1016/j.jbi.2008.08.010
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      2. Taylor DCM ,
      3. Hamdy H
      . Adult learning theories: implications for learning and teaching in medical education: AMEE guide No. 83. Med Teach 2013;35:e1561–72.doi:10.3109/0142159X.2013.828153
      OpenUrlPubMed
    43. ↵
      2. Reed DA ,
      3. Cook DA ,
      4. Beckman TJ , et al
      . Association between funding and quality of published medical education research. JAMA 2007;298:1002–9.doi:10.1001/jama.298.9.1002
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      2. Moher D ,
      3. Liberati A ,
      4. Tetzlaff J , et al
      . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009;151:W64.:264–9.doi:10.7326/0003-4819-151-4-200908180-00135
      OpenUrl
    45. ↵
      2. Losco CD ,
      3. Grant WD ,
      4. Armson A , et al
      . Effective methods of teaching and learning in anatomy as a basic science: a BEME systematic review: BEME guide No. 44. Med Teach 2017;39:234–43.doi:10.1080/0142159X.2016.1271944
      OpenUrl
    46. ↵
      2. Wan X ,
      3. Wang W ,
      4. Liu J , et al
      . Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol 2014;14:135.doi:10.1186/1471-2288-14-135
    47. ↵
      2. Sullivan GM ,
      3. Feinn R
      . Using effect Size – or why the P value is not enough. J Grad Med Educ 2012;4:279–82.doi:10.4300/JGME-D-12-00156.1
      OpenUrlCrossRefPubMed
    48. ↵
      2. Hojat M ,
      3. Xu G
      . A visitor's guide to effect sizes: statistical significance versus practical (clinical) importance of research findings. Adv Health Sci Educ Theory Pract 2004;9:241–9.doi:10.1023/B:AHSE.0000038173.00909.f6
      OpenUrlCrossRefPubMed
    49. ↵
      2. Faraone SV
      . Interpreting estimates of treatment effects: implications for managed care. P T 2008;33:700–11.
      OpenUrlPubMed
    50. ↵
      2. Higgins JPT ,
      3. Thompson SG ,
      4. Deeks JJ , et al
      . Measuring inconsistency in meta-analyses. BMJ 2003;327:557–60.doi:10.1136/bmj.327.7414.557
      OpenUrlFREE Full Text
    51. ↵
      2. Lameris AL ,
      3. Hoenderop JGJ ,
      4. Bindels RJM , et al
      . The impact of formative testing on study behaviour and study performance of (bio)medical students: a smartphone application intervention study. BMC Med Educ 2015;15:72.doi:10.1186/s12909-015-0351-0
    52. ↵
      2. Hammick M ,
      3. Dornan T ,
      4. Steinert Y
      . Conducting a best evidence systematic review. Part 1: from idea to data coding. BEME guide No. 13. Med Teach 2010;32:3–15.doi:10.3109/01421590903414245
      OpenUrlCrossRefPubMed
    53. ↵
      2. Curran VR ,
      3. Fleet L
      . A review of evaluation outcomes of web-based continuing medical education. Med Educ 2005;39:561–7.doi:10.1111/j.1365-2929.2005.02173.x
      OpenUrlCrossRefPubMedWeb of Science
    54. ↵
      2. Chudgar SM ,
      3. Engle DL ,
      4. Grochowski CO , et al
      . Teaching crucial skills: an electrocardiogram teaching module for medical students. J Electrocardiol 2016;49:490–5.doi:10.1016/j.jelectrocard.2016.03.021
      OpenUrl
    55. ↵
      2. Fincher RM ,
      3. Abdulla AM ,
      4. Sridharan MR , et al
      . Comparison of computer-assisted and seminar learning of electrocardiogram interpretation by third-year students. J Med Educ 1987;62:693–5.doi:10.1097/00001888-198708000-00015
      OpenUrlPubMed
    56. ↵
      2. Fincher RE ,
      3. Abdulla AM ,
      4. Sridharan MR , et al
      . Computer-assisted learning compared with weekly seminars for teaching fundamental electrocardiography to junior medical students. South Med J 1988;81:1291–4.doi:10.1097/00007611-198810000-00020
      OpenUrlPubMed
    57. ↵
      2. Patuwo T ,
      3. Wagner G ,
      4. Ajijola O
      . Comparison of teaching basic electrocardiographic concepts with and without ECGSIM, an interactive program for electrocardiography. In: Computer cardiology. IEEE, 2007.
    58. ↵
      2. Davies A ,
      3. Macleod R ,
      4. Bennett-Britton I , et al
      . E-learning and near-peer teaching in electrocardiogram education: a randomised trial. Clin Teach 2016;13:227–30.doi:10.1111/tct.12421
      OpenUrl
    59. ↵
      2. Fent G ,
      3. Gosai J ,
      4. Purva M
      . A randomized control trial comparing use of a novel electrocardiogram simulator with traditional teaching in the acquisition of electrocardiogram interpretation skill. J Electrocardiol 2016;49:112–6.doi:10.1016/j.jelectrocard.2015.11.005
      OpenUrl
    60. ↵
      2. Owen SG ,
      3. Hall R ,
      4. Anderson J , et al
      . Programmed learning in medical education. An experimental comparison of programmed instruction by teaching machine with conventional lecturing in the teaching of electrocardiography to final year medical students. Postgrad Med J 1965;41:201–5.doi:10.1136/pgmj.41.474.201
      OpenUrlFREE Full Text
    61. ↵
      2. Montassier E ,
      3. Hardouin J-B ,
      4. Segard J , et al
      . E-Learning versus lecture-based courses in ECG interpretation for undergraduate medical students: a randomized noninferiority study. Eur J Emerg Med 2016;23:108–13.doi:10.1097/MEJ.0000000000000215
      OpenUrl
    62. ↵
      2. Barthelemy FX ,
      3. Segard J ,
      4. Fradin P , et al
      . ECG interpretation in emergency department residents: an update and e-learning as a resource to improve skills. Eur J Emerg Med 2017;24:149–56.doi:10.1097/MEJ.0000000000000312
      OpenUrl
    63. ↵
      2. Rui Z ,
      3. Lian-Rui X ,
      4. Rong-Zheng Y , et al
      . Friend or foe? Flipped classroom for undergraduate electrocardiogram learning: a randomized controlled study. BMC Med Educ 2017;17:53.doi:10.1186/s12909-017-0881-8
    64. ↵
      2. Akbarzadeh F ,
      3. Arbat BK ,
      4. Alizadeh A , et al
      . The efficacy of web-based multimedia education of normal electrocardiogram in junior and senior medical students. Res Develop Med Educ 2012;1:77–9.
      OpenUrl
    65. ↵
      2. Sonali N ,
      3. Limaye RP ,
      4. Madhushree K , et al
      . Assessing impact of computer assisted learning (CAL) on cognitive perception – a study in medical college students. Res J Pharm Biol Chem Sci 2014;5:600–4.
      OpenUrl
    66. ↵
      2. Nilsson M ,
      3. Bolinder G ,
      4. Held C , et al
      . Evaluation of a web-based ECG-interpretation programme for undergraduate medical students. BMC Med Educ 2008;8:25.doi:10.1186/1472-6920-8-25
    67. ↵
      2. Gleason BL ,
      3. Peeters MJ ,
      4. Resman-Targoff BH , et al
      . An active-learning strategies primer for achieving ability-based educational outcomes. Am J Pharm Educ 2011;75:186.doi:10.5688/ajpe759186
    68. ↵
      2. Kolb D
      . Experiential learning. Englewood cliffs: Prentice Hall, 1984.
    69. ↵
      2. Choules AP
      . The use of elearning in medical education: a review of the current situation. Postgrad Med J 2007;83:212–6.doi:10.1136/pgmj.2006.054189
      OpenUrlAbstract/FREE Full Text
    70. ↵
      2. Liu Q ,
      3. Peng W ,
      4. Zhang F , et al
      . The effectiveness of blended learning in health professions: systematic review and meta-analysis. J Med Internet Res 2016;18:e2.doi:10.2196/jmir.4807
    71. ↵
      2. Rowe M ,
      3. Frantz J ,
      4. Bozalek V
      . The role of blended learning in the clinical education of healthcare students: a systematic review. Med Teach 2012;34:e216–21.doi:10.3109/0142159X.2012.642831
      OpenUrlCrossRefPubMed
    72. ↵
      2. McCutcheon K ,
      3. Lohan M ,
      4. Traynor M , et al
      . A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. J Adv Nurs 2015;71:255–70.doi:10.1111/jan.12509
      OpenUrl
    73. ↵
      2. Chen KS ,
      3. Monrouxe L ,
      4. YH L , et al
      . Academic outcomes of flipped classroom learning: a meta-analysis. Med Educ 2018.
    74. ↵
      2. Monteiro S ,
      3. Melvin L ,
      4. Manolakos J , et al
      . Evaluating the effect of instruction and practice schedule on the acquisition of ECG interpretation skills. Perspect Med Educ 2017;6:237–45.doi:10.1007/s40037-017-0365-x
      OpenUrl
    75. ↵
      2. Cook DA
      . Web-based learning: pros, cons and controversies. Clin Med 2007;7:37–42.doi:10.7861/clinmedicine.7-1-37
      OpenUrlAbstract/FREE Full Text
    76. ↵
      2. Healy DG ,
      3. Fleming FJ ,
      4. Gilhooley D , et al
      . Electronic learning can facilitate student performance in undergraduate surgical education: a prospective observational study. BMC Med Educ 2005;5:23.doi:10.1186/1472-6920-5-23
    77. ↵
      2. Ellaway R ,
      3. Masters K
      . AMEE guide 32: e-learning in medical education Part 1: learning, teaching and assessment. Med Teach 2008;30:455–73.doi:10.1080/01421590802108331
      OpenUrlCrossRefPubMedWeb of Science
    78. ↵
      2. Milner RE
      . Learner differences and learning outcomes in an introductory biochemistry class: attitude toward images, visual cognitive skills, and learning approach. Biochem Mol Biol Educ 2014;42:285–98.doi:10.1002/bmb.20658
      OpenUrl
    79. ↵
      2. Mayer RE
      . Applying the science of learning to medical education. Med Educ 2010;44:543–9.doi:10.1111/j.1365-2923.2010.03624.x
      OpenUrlCrossRefPubMed
    80. ↵
      2. Grum CM ,
      3. Gruppen LD ,
      4. Woolliscroft JO
      . The influence of vignettes on EKG interpretation by third-year students. Acad Med 1993;68:S61–3.doi:10.1097/00001888-199310000-00047
      OpenUrlPubMedWeb of Science
    81. ↵
      2. Hatala R ,
      3. Norman GR ,
      4. Brooks LR
      . Impact of a clinical scenario on accuracy of electrocardiogram interpretation. J Gen Intern Med 1999;14:126–9.doi:10.1046/j.1525-1497.1999.00298.x
      OpenUrlCrossRefPubMedWeb of Science
    82. ↵
      2. Kandasamy T ,
      3. Fung K
      . Interactive Internet-based cases for undergraduate otolaryngology education. Otolaryngology–Head and Neck Surgery 2009;140:398–402.doi:10.1016/j.otohns.2008.11.033
      OpenUrlPubMed
    83. ↵
      2. Cook DA ,
      3. Levinson AJ ,
      4. Garside S , et al
      . Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad Med 2010;85:909–22.doi:10.1097/ACM.0b013e3181d6c319
      OpenUrlCrossRefPubMedWeb of Science
    84. ↵
      2. Issenberg SB ,
      3. McGaghie WC ,
      4. Petrusa ER , et al
      . Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005;27:10–28.doi:10.1080/01421590500046924
      OpenUrlCrossRefPubMedWeb of Science
    85. ↵
      2. Moulaert V ,
      3. Verwijnen MGM ,
      4. Rikers R , et al
      . The effects of deliberate practice in undergraduate medical education. Med Educ 2004;38:1044–52.doi:10.1111/j.1365-2929.2004.01954.x
      OpenUrlCrossRefPubMedWeb of Science
    86. ↵
      2. McGaghie WC ,
      3. Issenberg SB ,
      4. Cohen ER , et al
      . Medical education featuring mastery learning with deliberate practice can lead to better health for individuals and populations. Acad Med 2011;86:e8–9.doi:10.1097/ACM.0b013e3182308d37
      OpenUrlCrossRefPubMedWeb of Science
    87. ↵
      2. Yardley S ,
      3. Dornan T
      . Kirkpatrick’s levels and education ‘evidence’. Med Educ 2012;46:97–106.doi:10.1111/j.1365-2923.2011.04076.x
      OpenUrlCrossRefPubMed
    88. ↵
      2. Steinert Y ,
      3. Mann K ,
      4. Centeno A , et al
      . A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME guide No. 8. Med Teach 2006;28:497–526.doi:10.1080/01421590600902976
      OpenUrlCrossRefPubMedWeb of Science
    89. ↵
      2. Davis D ,
      3. O’Brien MA ,
      4. Freemantle N , et al
      . Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282:867–74.doi:10.1001/jama.282.9.867
      OpenUrlCrossRefPubMedWeb of Science
    90. ↵
      2. Sampath J ,
      3. Dietze DT ,
      4. Toth PP , et al
      . Are continuing medical education activities effective in improving the competence and performance of clinicians? Evidence from activities for primary care clinicians who manage patients with acute coronary syndromes. Crit Pathw Cardiol 2012;11:1–9.doi:10.1097/HPC.0b013e318242e6cd
      OpenUrlPubMed
    91. ↵
      2. Clausen C ,
      3. Cummins K ,
      4. Dionne K
      . Educational interventions to enhance competencies for interprofessional collaboration among nurse and physician managers: an integrative review. J Interprof Care 2017;31:685–95.doi:10.1080/13561820.2017.1347153
      OpenUrl
    92. ↵
      2. Grol R ,
      3. Grimshaw J
      . From best evidence to best practice: effective implementation of change in patients' care. Lancet 2003;362:1225–30.doi:10.1016/S0140-6736(03)14546-1
      OpenUrlCrossRefPubMedWeb of Science
    93. ↵
      2. Zendejas B ,
      3. Brydges R ,
      4. Wang AT , et al
      . Patient outcomes in simulation-based medical education: a systematic review. J Gen Intern Med 2013;28:1078–89.doi:10.1007/s11606-012-2264-5
      OpenUrlCrossRefPubMed
    94. ↵
      2. Cheng A ,
      3. Nadkarni VM ,
      4. Mancini MB , et al
      . Resuscitation education science: educational strategies to improve outcomes from cardiac arrest: a scientific statement from the American Heart Association. Circulation 2018;138:e82–122.doi:10.1161/CIR.0000000000000583
      OpenUrl
    95. ↵
      2. Bisgaard CH ,
      3. Rubak SLM ,
      4. Rodt SA , et al
      . The effects of graduate competency-based education and mastery learning on patient care and return on investment: a narrative review of basic anesthetic procedures. BMC Med Educ 2018;18:154.doi:10.1186/s12909-018-1262-7
    96. ↵
      2. Ertmer PA ,
      3. Newby TJ
      . Behaviorism, cognitivism, constructivism: comparing critical features from an instructional design perspective. Perform Improv Q 1993;6:50–72.doi:10.1111/j.1937-8327.1993.tb00605.x
      OpenUrl
    97. ↵
      2. Lau KHV
      . Computer-based teaching module design: principles derived from learning theories. Med Educ 2014;48:247–54.doi:10.1111/medu.12357
      OpenUrl
    98. ↵
      2. DeCaporale-Ryan LN ,
      3. Dadiz R ,
      4. Peyre SE
      . Simulation-based learning: from theory to practice. Fam Syst Health 2016;34:159–62.doi:10.1037/fsh0000203
      OpenUrl
    99. ↵
      2. Lau Y ,
      3. Nyoe RSS ,
      4. Wong SN , et al
      . Effectiveness of digital resuscitation training in improving knowledge and skills: a systematic review and meta-analysis of randomised controlled trials. Resuscitation 2018;131:14–23.doi:10.1016/j.resuscitation.2018.07.033
      OpenUrl
    100. ↵
      2. Moffett J ,
      3. Berezowski J ,
      4. Spencer D , et al
      . An investigation into the factors that encourage learner participation in a large group medical classroom. Adv Med Educ Pract 2014;5:65–71.doi:10.2147/AMEP.S55323
      OpenUrl
    101. ↵
      2. Eaton M
      . The flipped classroom. Clin Teach 2017;14:301–2.doi:10.1111/tct.12685
      OpenUrl
    102. ↵
      2. Cook DA ,
      3. Hamstra SJ ,
      4. Brydges R , et al
      . Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis. Med Teach 2013;35:e867–98.doi:10.3109/0142159X.2012.714886
      OpenUrlCrossRefPubMed
    103. ↵
      2. Keifenheim KE ,
      3. Teufel M ,
      4. Ip J , et al
      . Teaching history taking to medical students: a systematic review. BMC Med Educ 2015;15:159.doi:10.1186/s12909-015-0443-x
    104. ↵
      2. Rees EL ,
      3. Quinn PJ ,
      4. Davies B , et al
      . How does peer teaching compare to faculty teaching? A systematic review and meta-analysis. Med Teach 2016;38:829–37.doi:10.3109/0142159X.2015.1112888
      OpenUrl
    105. ↵
      2. Cook DA ,
      3. Hatala R ,
      4. Brydges R , et al
      . Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA 2011;306:978–88.doi:10.1001/jama.2011.1234
      OpenUrlCrossRefPubMedWeb of Science
      2. Torre DM ,
      3. Daley BJ ,
      4. Sebastian JL , et al
      . Overview of current learning theories for medical educators. Am J Med 2006;119:903–7.doi:10.1016/j.amjmed.2006.06.037
      OpenUrlCrossRefPubMed
      2. Kaufman DM
      . Applying educational theory in practice. BMJ 2003;326:213–6.doi:10.1136/bmj.326.7382.213
      OpenUrlFREE Full Text
      2. Ryan RM ,
      3. Deci EL ,
      4. Intrinsic DEL
      . Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 2000;25:54–67.doi:10.1006/ceps.1999.1020
      OpenUrlCrossRefPubMedWeb of Science
      2. Young JQ ,
      3. Van Merrienboer J ,
      4. Durning S , et al
      . Cognitive load theory: implications for medical education: AMEE guide No. 86. Med Teach 2014;36:371–84.doi:10.3109/0142159X.2014.889290
      OpenUrlCrossRefPubMed
      2. Neufeld VR ,
      3. Barrows HS
      . The ‘McMaster Philosophy’: an approach to medical education. J Med Educ 1974;49:1040–50.
      OpenUrlPubMedWeb of Science
      2. Warschauer M
      . Computer-mediated collaborative learning: theory and practice. Mod Lang J 1997;81:470–81.doi:10.1111/j.1540-4781.1997.tb05514.x
      OpenUrl
      2. Ruiz JG ,
      3. Mintzer MJ ,
      4. Leipzig RM
      . The impact of e-learning in medical education. Acad Med 2006;81:207–12.doi:10.1097/00001888-200603000-00002
      OpenUrlCrossRefPubMedWeb of Science
      2. Hatala R ,
      3. Norman GR ,
      4. Brooks LR
      . Influence of a single example on subsequent electrocardiogram interpretation. Teach Learn Med 1999;11:110–7.doi:10.1207/S15328015TL110210
      OpenUrlCrossRef
      2. Bergman EM ,
      3. Sieben JM ,
      4. Smailbegovic I , et al
      . Constructive, collaborative, contextual, and self-directed learning in surface anatomy education. Anat Sci Educ 2013;6:114–24.doi:10.1002/ase.1306
      OpenUrl
      2. Kadish AH ,
      3. Buxton AE ,
      4. Kennedy HL , et al
      . ACC/AHA clinical competence statement on electrocardiography and ambulatory electrocardiography: a report of the ACC/AHA/ACP-ASIM Task force on clinical competence (ACC/AHA Committee to develop a clinical competence statement on electrocardiography and ambulatory electrocardiography) endorsed by the International Society for Holter and Noninvasive Electrocardiology. Circulation 2001;104:3169–78.
      OpenUrlFREE Full Text
      2. Cook DA ,
      3. Levinson AJ ,
      4. Garside S , et al
      . Internet-based learning in the health professions: a meta-analysis. JAMA 2008;300:1181–96.doi:10.1001/jama.300.10.1181
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Twitter @charleviljoen

    • Contributors CAV is a PhD student. RSM and VB are his supervisors. CAV conceived of the review and undertook the drafting of the manuscript. CAV and MS undertook a scoping search and developed the search strategy. CAV and RSM screened all the articles. CAV, RSM and VB were involved in data acquisition. CAV and MEE analysed the data. The results were interpreted by CAV, RSM, VB and MEE. All authors read the manuscript and gave their approval for publication.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests RSM is a lecturer and host of the AO Memorial Advanced ECG and Arrhythmia Course and receives an honorarium from Medtronic Africa.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available via the Dryad data repository at https://datadryad.org/stash/dataset/doi:10.5061/dryad.2dt037b