Article Text

Download PDFPDF

Is computer-assisted instruction more effective than other educational methods in achieving ECG competence amongst medical students and residents? A systematic review and meta-analysis
  1. Charle André Viljoen1,
  2. Rob Scott Millar1,
  3. Mark E Engel2,
  4. Mary Shelton3,
  5. Vanessa Burch2
  1. 1 Cardiology, University of Cape Town, Cape Town, South Africa
  2. 2 Medicine, Unversity of Cape Town, Cape Town, South Africa
  3. 3 Health Sciences Library, University of Cape Town, Cape Town, South Africa
  1. Correspondence to Dr Charle André Viljoen; charle.viljoen{at}uct.ac.za

Abstract

Objectives It remains unclear whether computer-assisted instruction (CAI) is more effective than other teaching methods in acquiring and retaining ECG competence among medical students and residents.

Design This systematic review and meta-analysis followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

Data sources Electronic literature searches of PubMed, databases via EBSCOhost, Scopus, Web of Science, Google Scholar and grey literature were conducted on 28 November 2017. We subsequently reviewed the citation indexes for articles identified by the search.

Eligibility criteria Studies were included if a comparative research design was used to evaluate the efficacy of CAI versus other methods of ECG instruction, as determined by the acquisition and/or retention of ECG competence of medical students and/or residents.

Data extraction and synthesis Two reviewers independently extracted data from all eligible studies and assessed the risk of bias. After duplicates were removed, 559 papers were screened. Thirteen studies met the eligibility criteria. Eight studies reported sufficient data to be included in the meta-analysis.

Results In all studies, CAI was compared with face-to-face ECG instruction. There was a wide range of computer-assisted and face-to-face teaching methods. Overall, the meta-analysis found no significant difference in acquired ECG competence between those who received computer-assisted or face-to-face instruction. However, subanalyses showed that CAI in a blended learning context was better than face-to-face teaching alone, especially if trainees had unlimited access to teaching materials and/or deliberate practice with feedback. There was no conclusive evidence that CAI was better than face-to-face teaching for longer-term retention of ECG competence.

Conclusion CAI was not better than face-to-face ECG teaching. However, this meta-analysis was constrained by significant heterogeneity amongst studies. Nevertheless, the finding that blended learning is more effective than face-to-face ECG teaching is important in the era of increased implementation of e-learning.

PROSPERO registration number CRD42017067054.

  • ECG
  • computer-assisted instruction
  • web-based learning
  • e-learning
  • systematic review

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

View Full Text

Statistics from Altmetric.com

Strengths and limitations of this study

  • To the best of our knowledge, this is the first systematic review and meta-analysis comparing the efficacy of computer-assisted instruction to other methods of ECG instruction among medical students and residents.

  • Systematic reviews provide robust evidence because they follow a rigorous method of search, selection and appraisal of articles.

  • We used the Medical Education Research Study Quality Instrument (MERSQI) to assess the quality of studies included in this systematic review.

  • The interpretation of the meta-analysis results is constrained by significant heterogeneity among the studies.

  • This systematic review with its meta-analysis and subanalyses identified valuable information about the educational approaches and types of computer-assisted learning material that were beneficial in acquiring ECG competence.

Introduction

The ECG is an indispensable diagnostic modality in cardiac disease.1 2 Although knowledge of, and skills in ECG analysis and interpretation, hereafter referred to as ECG competence, are desired learning outcomes of undergraduate and postgraduate medical training programmes, there is ongoing concern that graduating medical trainees lack adequate ECG competence.3–12 Many reasons account for this observation. First, electrocardiography is a difficult subject to teach and to learn.13 14 Second, although clinical exposure is important to gain experience in ECG analysis and interpretation,15 experiential learning alone does not guarantee ECG competence unless it is supplemented by structured teaching.16 Third, medical knowledge is ever-expanding,17 and there is limited time allocated to the teaching of electrocardiography in medical curricula.18–22 Alternative methods of instruction are therefore being sought to improve ECG training.

Technology-enhanced methods of instruction are increasingly being implemented in the training of healthcare professionals.23–25 It remains important to review whether these novel teaching and learning methods are effective.26 Previous studies have shown that students’ knowledge of, and skills in the analysis and interpretation of ECGs improve with computer-assisted instruction (CAI).27–34 However, these studies did not compare CAI to other methods of instruction and thus it cannot be concluded that CAI is better than traditional methods of ECG teaching.

To the best of our knowledge, there is no published systematic review comparing the efficacy of CAI with other methods of ECG instruction for training medical students and residents. Systematic reviews are important in the era of best evidence health professions education,35 because they follow a rigorous process of searching, selecting and appraising eligible articles.36 37 Reviewer bias is limited by applying strict criteria when appraising the articles and summarising the strengths and weaknesses of the studies evaluated.36–38

Objectives

The objectives of this systematic review were to:

  • establish whether CAI (on its own or in a blended learning setting) achieves better acquisition of ECG competence among medical students and residents than other methods of ECG instruction do;

  • establish whether CAI (on its own or in a blended learning setting) achieves better retention of ECG competence among medical students and residents than other methods of ECG instruction do;

  • establish whether there is a difference in the effectiveness of computer-assisted ECG instruction between medical students and residents enrolled for specialty training;

  • identify the types of learning material and/or activities that are used in computer-assisted ECG instruction, and to establish which CAI material and/or activities are associated with better outcomes;

  • identify the educational approaches used in computer-assisted ECG instruction, and to establish which of these are associated with better outcomes;

  • identify learning theories that may underpin computer-assisted ECG instruction.

Methods

A protocol was developed in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) guidelines39 and registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 6 July 2017 with registration number CRD42017067054.40

Search strategy

By using the search strategy described in the protocol,40 and shown in online supplementary file 1, we searched for relevant studies on 28 November 2017 using the following electronic databases: PubMed, EBSCOhost (which searched Academic Search Premier, CINAHL, PsycINFO, Education Resources Information Center, Africa-Wide Information, Teacher Reference Center), Scopus, Web of Science and Google Scholar. Citation indexes and reference lists were reviewed, and a grey literature search was also conducted.

Eligibility criteria

As summarised in table 1, all studies that compared the efficacy of CAI with other methods of ECG instruction were eligible for inclusion in this review. Studies were excluded if the teaching methods were not exclusively used to teach ECGs, or if the subject of teaching was not the conventional 12-lead ECG. We included studies in which the participants were medical students and/or residents enrolled for specialty training. Studies were excluded if the data for medical students or residents could not be separately identified from students other than medical students, healthcare professionals who were not medical doctors or qualified doctors who were not in training. We excluded studies that did not assess ECG knowledge and analysis and interpretation skills (ECG competence). There were no language or geographical restrictions. All eligible articles published before 1 July 2017 were included.40

Table 1

Eligibility criteria

Study selection

Two reviewers (CAV and RSM) independently screened all the articles identified by the search. All titles and abstracts were screened for eligibility and full-text articles of all studies potentially meeting inclusion criteria were retrieved. Both reviewers (CAV and RSM) individually evaluated the full text articles using a predesigned form evaluating each study’s eligibility. Where there was no consensus, the reviewers (CAV and RSM) discussed uncertainties pertaining to inclusion eligibility and a third reviewer (VCB) acted as an adjudicator.

Data abstraction

Two reviewers (CAV and RSM) independently extracted data from all eligible studies using a standardised electronic data abstraction form hosted on Research Electronic Data Capture (REDCap),41 which was subsequently crosschecked (CAV and RSM). Data extraction included study design, study duration, study population, ECGs used during teaching, teaching methods (CAI and non-CAI methods), type of digital learning material, educational approaches, learning theories underpinning instructional methods (using a classification proposed by Taylor).42 ECG competencies measured, testing times and results, as well as the validity and reliability of results with psychometric properties of the assessment tools (eg, Cronbach’s α coefficient) where reported.

In the event of missing or unreported data, corresponding authors were contacted. Following two email messages, a delay of 6 weeks was allowed to receive a response.

Quality of included studies and risk of bias assessment

The Medical Education Research Study Quality Instrument (MERSQI) was used to assess the quality of studies included in this systematic review. The MERSQI is a validated quality assessment tool used in health professions education to evaluate the quality of experimental, quasi-experimental and observational studies.36 43

As recommended by the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA),44 two reviewers (CAV and RSM) independently assessed each included study for risk of selection, performance, attrition, detection and/or reporting bias.

Data synthesis

Tests scores (pre-intervention test, post-intervention test and delayed post-intervention test) reported in the studies were used as objective measures of teaching method effectiveness.34 40 45 Where the mean or SD results were not reported, these were requested from the authors or, in the absence of a reply, calculated using the formula of Wan et al.46 The mean and SD results for the CAI and non-CAI groups in each study were converted to a standardised mean difference (effect size, Cohen’s d).47–49 Random-effects models were used to pool weighted effect sizes for all studies, as well as for the planned subanalyses. Planned subanalyses were conducted based on the level of training of participants (students or residents), the different educational approaches reported in the studies (eg, blended learning or not, massed or distributed instruction, restricted or unrestricted access to CAI, online or offline use of CAI), as well as learning materials (eg, real patient ECGs, case scenarios, images, animations) and learning activities (eg, online chat rooms, self-administered quizzes with automated feedback) used with CAI. The consistency in results was determined by visualising the forest plots and calculating the I2 statistic.50Statistical analyses were performed on Stata (V.14.2, StataCorp, College Station, Texas, USA) and Review Manager (RevMan, V.5.3.5, Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014).

We analysed studies for their educational impact using the modified version of the Kirkpatrick framework.35 51–53 The modified Kirkpatrick model is a widely used method of appraising the outcome of educational interventions by measuring participants’ perceptions of (reactions to) the learning experience (level 1), modification of participants’ perceptions of the intervention (level 2a), modification of their knowledge and/or skills (level 2b), transfer of learning to the workplace (level 3), change in organisational practice (level 4a) and benefits to patients (level 4b).

Patient and public involvement

There were no patients or public involved in this systematic review and meta-analysis.

Results

Trial flow

Our search strategy identified 592 papers, that is, 129 articles in PubMed, 349 in EBSCOhost, 65 in Scopus and 49 in Web of Science. We identified an additional 32 papers by reviewing the citation indexes and reference lists of the identified articles and grey literature. After 65 duplicate publications were removed, another 437 articles were excluded by screening their titles and abstracts. From the remaining 122 articles that were assessed in full text, thirteen articles met the predefined eligibility criteria for this systematic review. The reasons for exclusion are shown in figure 1. Eight studies contained sufficient data (mean scores, SD and number of participants reported for each cohort) to be included in the meta-analysis.

Figure 1

Trial flow. CAI, computer-assisted instruction.

Study characteristics

Table 2 summarises the characteristics of the nine randomised control trials and four prospective cohort studies that were included in this systematic review. Nine studies were conducted at a single centre, three studies at two centres and one study at more than two centres. Four studies were conducted in the USA,54–57 three in the UK,58–60 two in France61 62 and one each in China,63 Iran,64 India65 and Sweden.66 All the studies were published in English and included 1242 students and 86 residents in total. Of the thirteen studies, eleven focused on undergraduate students,54–58 60 61 63–66 one on residents62 and one on both students and residents.59

Table 2

Characteristics of included studies in this systematic review

As shown in online supplementary file 2, the earliest study on the use of computer-assisted ECG instruction was published in 1965,60 followed by two studies in the mid 80s.55 56 Most of the studies were published in the last decade,54 57–59 61–66 the majority of which used online CAI (web-based instruction).54 58 61–64 66

Study quality

A detailed summary of the quality of the included studies as measured by the MERSQI tool is contained in online supplementary file 3. The mean MERSQI total score of all included studies was 12.73 (SD 1.76). The studies scored well in the domains that assessed the type of data and data analysis. All studies had objective outcome assessments and twelve of the thirteen studies reported appropriate analyses, which extended beyond descriptive analysis. Studies scored poorly in the sampling domain: more than two-thirds of studies were conducted at a single centre and a third had a response rate of either less than 50% or did not report their response rate.

Risk of bias

As elaborated in table 3, and summarised in online supplementary file 3, there was selection bias and/or performance bias in nine studies. Three studies had attrition bias and one had reporting bias.

Table 3

Summary of the study design, assessment of knowledge and outcomes of the included studies

Educational approaches

In all studies, CAI was compared with face-to-face teaching (refer to glossary for definitions). However, CAI and face-to-face teaching were delivered in variable formats. CAI formed part of a blended learning strategy in four studies (online supplementary file 4).54 57 63 66 In one of these studies, blended learning was applied in a ‘flipped classroom’ approach, where CAI took place before classroom teaching.63 Face-to-face teaching was facilitated by experienced lecturers or specialists in all the studies,54–57 59–66 with the exception of one study in which near-peer teaching was used.58

The frequency of instruction in the studies was variable. In three of the thirteen studies, participants were exposed to a single learning event (massed instruction), whether assigned to CAI or face-to-face teaching, before ECG competence was assessed.58 59 64

Learning materials and activities

A range of learning materials were used in CAI (table 2). In most studies, the digital learning material consisted of ECG tracings with accompanying text. In addition, in some studies CAI also included the use of multimedia in the form of diagrams and images64 or animations.57 59 66 As summarised in table 2, the curricular content varied across the studies and a wide range of ECG diagnoses were included.

Active learning (during which learners deliberately engaged with learning material)67 formed an integral part of CAI, which used ‘interactive software’ in all the studies included in this review. In addition to engaging with the learning material, some studies also reported on the use of self-administered assessments with automated feedback,54 56 58 60–62 64 66 online chat rooms61 63 and interaction with lecturers and peers during ‘flipped classroom’ activities.63 Six of the thirteen studies reported interaction between students and lecturers in the non-CAI group, for example, lecturers quizzed students or students asked questions during the face-to-face teaching activities (online supplementary file 5).54 58 59 61 62 64 In the study where CAI was compared with near-peer face-to-face teaching, there was a strong emphasis on interaction between students and tutors in the face-to-face teaching group.58

Educational outcomes

The outcomes of the studies are summarised in table 3. Baseline ECG competence was assessed in six of the thirteen studies.54–56 61 62 65 All studies tested ECG competence acquired after the educational intervention; only one study assessed the retention of ECG competence after a period of three months without further instruction since the acquisition of knowledge was tested.59 Five studies used multiple choice questions to assess study participants’ knowledge,58 59 61 62 65 whereas another five used short answer questions marked by the course convenors.54–56 60 63 Three studies did not report how ECG competence was assessed.

Using the Kirkpatrick model of evaluation of educational interventions, it was found that eight studies reported participants’ reactions to CAI (Kirkpatrick level 1)54 58–61 63 64 66 and three studies reported a change in trainees’ attitudes and perceptions after exposure to CAI (Kirkpatrick level 2a).54 58 59 All the studies reported on the acquisition and/or retention of ECG competence (Kirkpatrick level 2b) since this was one of the eligibility criteria of this systematic review. None of the studies reported on outcomes at Kirkpatrick level 3 or 4.

Kirkpatrick level 1 and 2a outcomes were variable. Though some studies reported that students had a positive attitude towards web-based learning,54 60 63 64 66 others reported less favourable attitudes towards CAI than lectures.58 59 61 In one study, all the potential participants did not want to use the e-learning platform and so some potential participants were excluded from the particular study.66 While three studies reported on students who felt that an improvement in their confidence was no better with CAI as compared with lectures,58 59 61 other studies identified students who thought that CAI improved their confidence in ECG interpretation.54 58 66 In general, students valued CAI approaches that included multimedia learning material,59 64 and self-assessment tools.66 In some studies they requested more visually-oriented learning material59 64 and applications that had a facility or method for asking questions.59 Kirkpatrick level 2b outcomes of the studies are summarised in table 3 and have already been described .

Learning theories

Learning theories that underpin education were infrequently mentioned or discussed in any detail. The most frequent reference to learning theories was to self-directed learning in CAI.54 59 62 63 66 One study66 referenced Kolb’s description of experiential learning,68 and another study mentioned ‘cognitive learning’ and ‘collaborative learning’.63 However, careful review of the papers included in this systematic review identified multiple examples of teaching and learning activities that were aligned with contemporary theories of learning. These are shown in table 4 using a simplified classification of learning theories described by Taylor.42

Table 4

Learning theories, based on a classification by Taylor42 that underpinned computer-assisted and face-to-face ECG instruction in the included studies

Quantitative data synthesis

Overall, we found that CAI was not better than face-to-face teaching for acquiring ECG competence (standardised mean difference (SMD)=0.32 (95% CI −0.09 to 0.74); eight studies, n=945; I2=88.9%) (figure 2). However, there was inconsistency among the studies and effect sizes ranged from −1.08 to 1.09 (table 5). A positive effect size (ie, CAI was better than face-to-face teaching) was found in most studies, one of which showed a large effect size (>0.8)54 and four a moderate effect size (>0.5).62 63 65 66 However, in two studies59 60 there was no significant difference between CAI and face-to-face teaching and one study showed that face-to-face teaching was better than CAI.58

Figure 2

Overall effect of teaching methods on the acquisition of ECG knowledge and analysis and interpretation skills. CAI, computer-assisted instruction; SMD, standardised mean difference.

Table 5

Acquired and retained ECG competence according to educational approaches used in the included studies

Only one study assessed the effect of CAI on the retention of ECG competence.59 While this study showed that there was no significant difference between the CAI and face-to-face teaching (SMD=−0.24 (95% CI −1.05 to 0.58)), the response rate was only 14% for the retention of knowledge test which was conducted three months after the educational intervention.

Medical students compared to residents

In the subanalysis comparing the acquisition of ECG competence with CAI and face-to-face teaching in undergraduate and postgraduate trainees separately (figure 3), there was a tendency to favour CAI over face-to-face teaching among both medical students (SMD=0.41 (95% CI −0.03 to 0.84); six studies, n=738; I2=87%) and residents (SMD=0.64 (95% CI 0 to 1.28); one study, n=19). The single study assessed the retention of ECG competence combined medical students and residents.59

Figure 3

Pooled effect sizes according to level of training of participants, educational approaches and CAI learning materials used in the studies. CAI, computer-assisted instruction.

Educational approaches

A subanalysis found a large positive effect size when CAI formed part of a blended learning strategy as compared with face-to-face teaching (SMD=0.84 (95% CI 0.54 to 1.14); three studies, n=422; I2=50%) (figure 3). This systematic review did not identify any studies that evaluated the retention of ECG analysis and interpretation skills after exposure to CAI in a blended learning programme.

In another subanalysis, studies using a distributed approach to ECG instruction (ie, more than one ECG training opportunity) showed that CAI was better than face-to-face teaching (SMD=0.65 (95% CI 0.31 to 1.00); five studies, n=538; I2=70%). Review of these studies showed that the benefit of distributed instruction was only present in studies where CAI was part of a blended learning approach (SMD=0.84 (95% CI 0.54 to 1.14); three studies, n=422; I2=50%; vs SMD=0.31 (95% CI −0.21 to 0.84); two studies, n=116; I2=46%). There was no statistically significant difference between CAI and face-to-face teaching when massed instruction strategies were used (ie, a single session of ECG teaching) (figure 3).

Although there was no difference between online and offline CAI, four studies showed that CAI was better than face-to-face teaching when students had unlimited access (ie, 24 hours a day, 7 days a week) to CAI learning materials (SMD=0.82 (95% CI 0.57 to 1.07); four studies, n=461; I2=32%). This benefit, as shown in a subanalysis, was not apparent when access to CAI learning materials was limited (SMD=−0.34 (95% CI −0.86 to 0.18); three studies, n=284; I2=74%).

In the study that used reminder emails to encourage the use of CAI, there was a large effect size in favour of CAI (1.09 (95% CI 0.79 to 1.4)).54

Learning activities and materials used in CAI

Subanalyses showed that CAI was better than face-to-face teaching when ECGs were accompanied by case scenarios (SMD=0.90 (95% CI 0.59 to 1.21); three studies, n=280; I2=24%) and if images were used to explain impulse conduction (SMD=1.09 (95% CI 0.79 to 1.40); one study, n=191). Studies in which CAI included self-administered assessments with automated feedback showed better ECG knowledge acquisition than face-to-face teaching (SMD=0.64 (95% CI 0.14 to 1.13); four studies, n=357; I2=77%) (figure 3). This effect size was larger in studies where self-administered assessment with automated feedback formed part of a blended learning approach (SMD=0.95 (95% CI 0.57 to 1.34); two studies, n=241; I2=38%). CAI was better than face-to-face teaching when students had access to online chat rooms to discuss the study material (SMD=0.68 (95% CI 0.38 to 0.98); one study, n=181) (figure 3).

Discussion

This systematic review and meta-analysis set out to determine whether CAI is more effective than other methods of teaching electrocardiography knowledge and analysis and interpretation skills to undergraduate and postgraduate medical trainees. All the studies included in this systematic review and meta-analysis compared CAI to face-to-face teaching. Based on the overall results of the review there is currently insufficient evidence to favour CAI over face-to-face ECG instruction. Though there was significant heterogeneity in the studies included in the meta-analysis, subanalyses of the different learning materials and educational approaches were less heterogenous. We found that CAI was better than face-to-face teaching when used in a blended learning approach. Studies also favoured computer-assisted distributed instruction with unrestricted access to learning materials; the use of case scenarios to contextualise ECG interpretation with images to explain concepts and interactive learning activities, including chat rooms, and self-assessment with automated feedback. While contemporary learning theories were not explicitly articulated in most studies, there were many examples of computer-assisted instruction strategies and activities that were aligned with these theories.

Although self-directed, computer-assisted learning may seem attractive to busy clinicians with limited time for teaching,55 61 our systematic review and meta-analysis did not find sufficient evidence to recommend that computer-assisted ECG instruction should replace face-to-face teaching. Rather, we found that computer-assisted ECG instruction was more effective than face-to-face teaching when it formed part of a blended learning strategy. This is in keeping with the literature which shows that CAI should be used as an adjunct to face-to-face teaching in order to enhance ECG training.56 61 69 Our findings are also in keeping with the results of a recent meta-analysis published in the health professions education literature, which showed that blended learning was better than face-to-face teaching alone.70 However, as with other systematic reviews and meta-analyses that assessed the efficacy of blended learning in the training of healthcare professionals,71 72 our analyses were also limited by a small number of studies, incomplete reporting of results and significant heterogeneity among the studies.

One of the studies included in this review demonstrated the successful use of CAI in a flipped classroom strategy for teaching ECG analysis and interpretation skills.63 Although the flipped classroom method required more preparation time, for both lecturers and students, trainees were more proactive in discussions with their peers and their lecturers during the face-to-face teaching time, resulting in better post-intervention test scores than traditional face-to-face teaching.73 Since it is accepted that ECG competence is difficult to acquire,13 14 the successful use of a flipped classroom approach is encouraging because this method allows for engagement with the learning material prior to face-to-face interaction with teachers when difficult concepts can be discussed and misunderstandings resolved.

When evaluating the educational effect of teaching and learning methods, it is critical to review access and frequency of exposure to the learning materials. In a subanalysis, students did not benefit from computer-assisted or face-to-face massed instruction (single educational event). As has been previously found,74 CAI was only beneficial if students had multiple exposures to the learning activities and study materials (distributed instruction). In the setting of blended learning, CAI facilitates distributed instruction, because it can be used asynchronously, allowing for consolidation of knowledge acquired during face-to-face teaching.24 34 61 75

This review found that there was a significant benefit to students when they had unrestricted access to CAI learning materials. Although we did not show a difference in outcomes between online and offline CAI, the benefit of web-based learning is that it can be accessed whenever and wherever convenient.24 34 61 75 However, the high cost of, and/or lack of access to computers with Internet facilities may be a barrier to web-based learning, particularly in developing countries.24 64 76 Health professions educators, especially in resource-limited settings should therefore be cognisant of the availability of computers and students’ access to the Internet when planning CAI with online requirements.

A key aspect of any method of instruction is the nature of the learning materials and activities included in the programme. CAI has been shown to enhance the learning experience by using multimedia and interactive learning materials.69 In this study we confirmed that visual material was highly valued by participants and a subanalysis showed specific benefit when using images in combination with the 12-lead ECG, for example to explain cardiac impulse conduction. The value of using images in medical education is that it helps to embed knowledge in long-term memory.25 Although images are widely used to demonstrate concepts in medical education,77 it has previously been shown to be of most value when accompanied by good explanations,78 79 as was the case in the study by Nilsson et al.66 In this study we also found that there were additional educational gains when computer-assisted ECG instruction made use of clinical scenarios.54 61 62 66 This is in line with previous studies which have shown more accurate ECG analysis and interpretation when the clinical context was known.80 81

In this systematic review we found evidence that CAI was better than face-to-face teaching in studies in which the CAI included exercises of ECG analysis and interpretation that required deliberate practice with automated feedback. This finding is in keeping with studies which have shown that practice exercises followed by feedback facilitate high levels of interactivity with educational materials and significantly enhance learning.61 82–84 In CAI there are opportunities for both self-reflection85 and repetitive practice86 because students can repeat the self-assessments, correct their errors and further improve their performance.54 60–62 66 84

The studies included in this review demonstrated variable outcomes using the Kirkpatrick framework of evaluation. Improvement of trainees’ ECG knowledge and analysis and interpretation skills using either CAI and face-to-face instruction was an eligibility criterion for inclusion in the study. A few studies reported on the responses of participants to the methods of instruction used with no consistent preference for CAI. None of the studies evaluated CAI at the level of behavioural change (Kirkpatrick level 3), change in organisational practice (Kirkpatrick level 4a) or improved patient care (Kirkpatrick level 4b). This is consistent with studies showing that health professions education interventions rarely show impact at Kirkpatrick level 3 or 4.87 88 Indeed it is a widely recognised ongoing shortcoming of health professions education research. This systematic review endorses a plea in the literature for the evaluation of educational interventions at the level of impact on physician behaviour,89 90 organisational practice91 92 and patient care.93–95

While learning theories were not explicitly discussed in most of the studies in this review, there were multiple examples of educational strategies that are aligned with contemporary learning theories.96 However, as this review shows, studies describing and evaluating educational interventions continue to be conducted without a firm rationale imbedded in contemporary learning theories. This highlights a significant ongoing shortcoming of health professions education research.97–99

CAI serves as a good example of self-directed learning, whereby students plan and conduct their own learning.42 While face-to-face teaching time is limited,100 CAI allows for flexibility in learning – students can adjust the pace of their learning and spend as much time as they need to assimilate new knowledge. While face-to-face teaching is ideal for promoting collaborative learning by allowing interaction between peers and tutors,55 58 59 61 62 it is also possible in CAI when chat rooms were available61 63 or when CAI forms part of a blended learning programme.54 57 63 66

In this review we found that participants valued learning with demonstrations and explanations (cognitivism).96 CAI-based learning opportunities had the advantage of offering multimedia learning resources, which enrich the educational content by means of animations, audio and video clips.69

The flipped classroom method of teaching ECGs, as described in one study included in this review,63 serves as an excellent example of a learning process which focuses on actively creating meaning rather than merely acquiring knowledge (constructivism).96 In a flipped classroom approach, students used CAI to familiarise themselves with educational content, and expand their learning by using the time in class to discuss concepts that they did not understand.101 It seems that this could be a useful approach for electrocardiography, which is considered a difficult subject to teach and to learn.13 14

Because CAI does not require attendance of class, external motivation in the form of reminder emails or summative assessments might be needed to encourage students to use the e-learning modules. In the study that made use of such external motivation strategies, CAI showed a large positive effect size.54

Though variably applied in the studies in this review, contextualisation was possible in both CAI and face-to-face teaching settings.54 61 62 66 Where CAI made use of patient scenarios, there was a larger benefit in acquiring ECG competence.

Reflective learning is possible with CAI when self-administered quizzes with automated feedback are used. Learning is facilitated because knowledge and/or skills gaps are highlighted.54 61 62 66

Strengths and limitations

The strength of this study is that it was conducted as a systematic review using a comprehensive search strategy and detailed data extraction method. However, the inferences that can be made from this systematic review and its meta-analysis are limited by high levels of bias and the heterogeneity of the included studies. There was significant variability in study design, the format, delivery and exposure time of the teaching intervention (CAI) and control (face-to-face teaching) and the topics taught and assessed.57 Many studies also did not include a baseline test of ECG knowledge and/or analysis and interpretation skills prior to the educational intervention and did not report all their data. Nevertheless, the mean MERSQI score of the studies included in this review was similar to MERSQI scores reported in other systematic reviews in medical education.23 102–104 In fact, 9 of the 13 studies in this review had a high MERSQI score (ie ≥12).105 Furthermore, most of the studies included in this systematic review were performed in well-resourced countries and the generalisability of these findings to resource-constrained settings is therefore not known.

Implications for practice and future research

Owing to the heterogenous nature of the studies included in this review it was not possible to provide conclusive evidence that CAI is better than face-to-face teaching of ECG knowledge and analysis and interpretation skills. However, CAI was better than face-to-face teaching in a blended learning setting where students had unrestricted access to the learning materials and opportunities for self-assessment with automated feedback.

There are currently many aspects of CAI that need to be further explored. These include a more detailed evaluation of the efficacy of this medium of instruction in postgraduate education and its impact on the long-term retention of ECG competence in both undergraduate and postgraduate trainees. Studies are also needed to better understand the impact of CAI on clinician behaviour (ECG analysis and interpretation practices in clinical settings), changes in organisational practice and patient care.

Conclusion

Owing to the mixed findings of the studies included in this systematic review, there is currently insufficient evidence to favour the use of computer-assisted ECG instruction. However, CAI can be used to enhance face-to-face teaching in a blended learning setting. CAI was found to be more beneficial than face-to-face teaching when students had unrestricted access to learning materials and opportunities for self-assessment with automated feedback.

Acknowledgments

The authors wish to thank Ms Sylvia Dennis, Dr Nicholas Simpson, Ms Kathryn Manning and Professsor Ike Okpechi from the University of Cape Town for their valuable support and input.

References

View Abstract

Footnotes

  • Twitter @charleviljoen

  • Contributors CAV is a PhD student. RSM and VB are his supervisors. CAV conceived of the review and undertook the drafting of the manuscript. CAV and MS undertook a scoping search and developed the search strategy. CAV and RSM screened all the articles. CAV, RSM and VB were involved in data acquisition. CAV and MEE analysed the data. The results were interpreted by CAV, RSM, VB and MEE. All authors read the manuscript and gave their approval for publication.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests RSM is a lecturer and host of the AO Memorial Advanced ECG and Arrhythmia Course and receives an honorarium from Medtronic Africa.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available via the Dryad data repository at https://datadryad.org/stash/dataset/doi:10.5061/dryad.2dt037b

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.