Article Text

Download PDFPDF

Is computer-assisted instruction more effective than other educational methods in achieving ECG competence among medical students and residents? Protocol for a systematic review and meta-analysis
  1. Charle André Viljoen1,
  2. Rob Scott Millar1,
  3. Mark E Engel2,
  4. Mary Shelton3,
  5. Vanessa Burch2
  1. 1Division of Cardiology, Groote Schuur Hospital, University of Cape Town, Cape Town, South Africa
  2. 2Department of Medicine, Groote Schuur Hospital, University of Cape Town, Cape Town, South Africa
  3. 3Health Sciences Library, University of Cape Town, Cape Town, South Africa
  1. Correspondence to Dr Charle André Viljoen; charle.viljoen{at}uct.ac.za

Abstract

Introduction Although ECG interpretation is an essential skill in clinical medicine, medical students and residents often lack ECG competence. Novel teaching methods are increasingly being implemented and investigated to improve ECG training. Computer-assisted instruction is one such method under investigation; however, its efficacy in achieving better ECG competence among medical students and residents remains uncertain.

Methods and analysis This article describes the protocol for a systematic review and meta-analysis that will compare the effectiveness of computer-assisted instruction with other teaching methods used for the ECG training of medical students and residents. Only studies with a comparative research design will be considered. Articles will be searched for in electronic databases (PubMed, Scopus, Web of Science, Academic Search Premier, CINAHL, PsycINFO, Education Resources Information Center, Africa-Wide Information and Teacher Reference Center). In addition, we will review citation indexes and conduct a grey literature search. Data extraction will be done on articles that met the predefined eligibility criteria. A descriptive analysis of the different teaching modalities will be provided and their educational impact will be assessed in terms of effect size and the modified version of Kirkpatrick framework for the evaluation of educational interventions. This systematic review aims to provide evidence as to whether computer-assisted instruction is an effective teaching modality for ECG training. It is hoped that the information garnered from this systematic review will assist in future curricular development and improve ECG training.

Ethics and dissemination As this research is a systematic review of published literature, ethical approval is not required. The results will be reported according to the Preferred Reporting Items for Systematic Review and Meta-Analysis statement and will be submitted to a peer-reviewed journal. The protocol and systematic review will be included in a PhD dissertation.

PROSPERO registration number CRD42017067054; Pre-results.

  • electrocardiogram (ecg)
  • computer-assisted instruction
  • web-based learning
  • medical student(s)
  • resident(s)

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • In the face of inadequate ECG competence among graduating medical students and residents worldwide, it is important to review how Electrocardiography is taught.

  • This systematic review will evaluate the effectiveness of computer-assisted instruction compared with other teaching methods used in the ECG training of medical students and residents.

  • The protocol describes a comprehensive search strategy a eligibility criteria which have no geographical or language restrictions.

  • The systematic review might be limited by the presence of selection and/or performance bias inherent in some of the selected studies.

  • A meta-analysis will only be possible in the absence of heterogeneous data among included studies.

Introduction

The electrocardiogram (ECG) remains one of the most frequently performed diagnostic procedures in clinical practice.1 2 ECG interpretation is, therefore, considered an essential learning outcome in undergraduate medical curricula.3 Incorrect interpretation of an ECG, however, can lead to inappropriate clinical decisions with serious adverse outcomes, especially in the realms of arrhythmias and myocardial infarction.4 5 Previous studies have found that the majority of medical students lack confidence when interpreting ECGs, as they find it a difficult skill to master and retain.6–10 Of greater concern is the finding that graduating medical students are often unable to accurately interpret ECGs, particularly when dealing with life-threatening conditions such as complete heart block and atrial fibrillation.7–10 Suboptimal ECG competence has also been shown in residents in cardiology, internal medicine and emergency medicine, all of which are specialties where the ECG is considered a core skill of daily practice.11–16

‘ECG analysis’ refers to the detailed examination of an ECG tracing, which requires the measurement of intervals and the evaluation of the rhythm and each waveform, whereas ‘ECG interpretation’ refers to the conclusion reached after careful ECG analysis, that is, making a diagnosis of an arrhythmia, ischaemia and so on.17 ‘ECG competence’ refers to the ability to accurately analyse and interpret the ECG,7 18 whereas ‘ECG knowledge’ refers to the understanding of ECG concepts, for example, knowing that transmural ischaemia or pericarditis can cause ST-segment elevation.6 19

It is well known that ECG analysis and interpretation are difficult and require significant training.20 The reasons for this are multifold. To start, students are required to have sound prior knowledge of the anatomy and physiology of the cardiac conduction system before they can begin to study ECGs.21 ECG analysis also requires a good understanding of vectors and how these are influenced by lead placement and pathology.17 21 Furthermore, ECG interpretation requires two types of reasoning: the non-analytical pattern recognition of abnormal waveforms and rhythms, and the analytical, systematic analysis of the entire 12-lead ECG.22 23 The best clinical results are attained when both non-analytical pattern recognition and analytic systematic analysis of the ECG are used simultaneously; however, most medical students and postgraduate trainees find this overwhelming.22 23

Although a large deal of experience in ECG interpretation depends on clinical exposure,24 clinical exposure alone does not improve ECG diagnostic accuracy if it is not supplemented by a structured form of teaching.25 In undergraduate and postgraduate courses, ECGs are commonly taught by means of large group teaching,2 7 20 25 26 where a teacher or expert transfers ECG knowledge to a group of learners in the format of a lecture.27–29 Lectures are an effective and cost-efficient method of tuition, as they allow for large groups of students to be taught at once.30 31 However, large group teaching facilitates passive learning, as didactic lectures often offer students little opportunity for interactive discussion with the lecturer.27 30–32 ECG teaching also frequently occurs in the small group setting, that is, during ward rounds and bedside tutorials.29 31 Small group teaching allows for free communication and interaction between the learner and the teacher, or among the learners themselves.33

Alternative teaching methods are increasingly being implemented and investigated to improve ECG training, and the following are some examples of these. The ‘flipped classroom’ refers to the teaching method where students are required to watch short video lectures or study written material at their own pace, before attending a classroom lecture.34 35 Instead of didactic tuition, lecture time is devoted to a more interactive discussion between the student and lecturer, which allows for problem solving and knowledge application in the classroom.35 36 ‘Peer teaching’ refers to the teaching method in which students are taught by fellow students of the same academic year, whereas ‘near-peer teaching’ refers to the teaching method in which students are taught by more senior students from the same curriculum.37 ‘Reciprocal peer teaching’ allows for students to alternate between the roles of tutor and learner.38 The tutoring role promotes self-learning by teaching others,37 38 whereas the learner role has been shown to be as effective as instruction by lecturers.38 39 ‘Problem-based learning’ refers to the student-centred teaching method where a clinical problem is assigned to students, who then need to identify what they need to learn from the clinical case and apply their knowledge to solve a clinical problem.40 Apart from the face-to-face tuition by experts or peers, ECG knowledge can also be acquired by means of self-directed learning (SDL), which refers to the independent study of textbooks or other designated study material.41

‘Computer-assisted instruction’ (CAI) has been used as an ECG teaching modality since the 1960s.42 CAI or ‘computer-assisted learning’ (CAL) refers to any teaching method that uses a digital platform as an SDL technique, which includes both online and offline learning opportunities.43 Although CAI is the broadest term as it encompasses both online and offline modalities, newer terminology specifically referring to online learning modalities includes terms such as ‘web-based learning’, ‘web-based training’ and ‘e-learning’.44–47 CAI or web-based learning typically provides the student with text, illustrations and other multimedia material to study. Additional educational features such as practice fields and test-enhanced learning (eg, online multiple choice questions with immediate feedback) can also be provided by the digital platform.43 47–49 CAI is increasingly being used as a possible solution for the increasing numbers of medical students that lecturers need to teach and the insufficient time allocated for ECG instruction in undergraduate and postgraduate curricula.50–53 Web-based learning allows for flexibility in learning, as the student can access the material wherever and whenever convenient, outside the constraints of time allocated for formal instruction.46–48

It is worth reviewing the value of computer-based training in medical education, as the current generation of medical students and residents, who are known as ‘millennials’, are computer literate and often seek technologically enhanced means of education.54–56 These students and residents grew up during the advent of the world wide web, smartphones and social media and are used to obtaining immediate access to unlimited information through mobile devices and desktop computers.56 57 Although today’s medical student prefers podcasts and interactive multimedia to conventional classroom teaching and textbooks,56 there is not enough evidence to suggest that the digital platform should replace traditional teaching methods. Although a meta-analysis showed that web-based learning was as effective as conventional teaching methods in health professionals,58 more recent subject-specific systematic reviews in anatomy and orthopaedics favoured CAI, especially in the setting of blended learning.43 59 However, it cannot be extrapolated that the effectiveness of CAI in other domains holds true for teaching ECG.

The objective measure of a teaching method’s effectiveness is the assessment of students’ competence after being exposed to the educational intervention.43 ECG competence is measured by assessing the student’s ECG analysis and/or interpretation skills. An assessment shortly after an educational intervention tests the acquisition of ECG competence, whereas delayed testing assesses the retention of ECG competence.46 More comprehensively, the modified Kirkpatrick model is a widely accepted method of appraising an educational intervention’s outcome, as it measures learners’ views on the learning experience (level 1), modification of learners’ perception of the intervention (level 2a), modification of knowledge or skills (level 2b), transfer of learning to the workplace (level 3), change in organisational practice (level 4a) and benefits to patients (level 4b).60–63

However, the effectiveness of an instructional method should not be interpreted in isolation, as there are several educational approaches that also have a significant impact on learning. The learning environment (ie, whether instruction occurs in the classroom, computer laboratory or clinical setting)64 and the spacing of instructional events (ie, massed vs distributed instruction)65–67 should be borne in mind when assessing the efficacy of instructional methods. It should also be considered whether provision was made for deliberate practice (eg, paper-based or computer-based ECG analysis)68–70 and whether the instruction included any formative or summative assessment with feedback.71 72

A distinction should be made between the method of instruction (how knowledge is transferred from the expert to the learner) and learning theories (how knowledge is acquired and assimilated by the learner).73 Different learning theories underpin a range of instructional methods.74 Although there is some overlap, learning theories can be categorised as:

  • instrumental learning theories,74 which include behaviourism (learning through practice, feedback and reinforcement),73–75 cognitivism (learning with demonstrations and explanations, understanding concepts),74–76 constructivism (critical thinking and elaboration)75–78 and experiential learning (learning through experience)64 74 79 80;

  • humanistic learning theories,74 which include andragogy (adult learning driven by intrinsic instead of extrinsic motivation)74 81 82 and SDL (self-regulated learning, where the learner plans and monitors their own learning)83 84;

  • the transformative learning theory (critical reflection)74;

  • social learning theories,74 which include collaborative learning (interaction with peers and tutors)85 86 and contextual learning (with case scenarios or multiple examples with different perspectives).87 88

In the face of inadequate ECG competence among graduating medical students and residents worldwide,7–10 it is time to review the way that ECG analysis and interpretation are taught. Are conventional ECG teaching methods achieving the necessary ECG competence? Are teaching methods on the digital platform better than the ways that ECGs have traditionally been taught? Or should a blended learning strategy (ie, the combination of CAI and other teaching modalities) be implemented for ECG teaching? And which learning theories underpin computer-assisted ECG instruction? To the best of our knowledge, there is no systematic review of the effectiveness of CAI compared with other teaching methods used in the ECG training of medical students and residents.

Objectives

The objectives of this systematic review are to:

  • establish whether CAI (on its own or in a blended learning setting) achieves better acquisition of ECG competence among medical students and residents than other non-CAI ECG teaching methods do;

  • establish whether CAI (on its own or in a blended learning setting) achieves better retention of ECG competence among medical students and residents than other non-CAI ECG teaching methods do;

  • establish whether there is a difference in the effectiveness of computer-assisted ECG instruction between medical students and residents enrolled for specialty training;

  • identify the types of learning material or activities (eg, reading material, case scenarios, illustrations, videos, test-enhanced learning tools, etc) in which CAI is delivered for ECG teaching, and to establish which CAL material or activities are associated with better outcomes;

  • identify the educational approaches that are possible with computer-assisted ECG instruction, and to establish which of these are associated with better outcomes;

  • identify the learning theories that may underpin computer-assisted ECG instruction.

Methods and design

In accordance with the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) guidelines,89 this systematic review protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 6 July 2017 with registration number CRD42017067054.

Criteria for considering studies for this review

A study will be deemed eligible to be included in this systematic review only if it fulfils all inclusion criteria and does not meet any of the exclusion criteria, as outlined in table 1.

Table 1

Criteria to assess a study’s eligibility to be included in this systematic review

Types of studies

All studies with a comparative research design, that is, randomised controlled trial, cohort study, case–control study, before-and-after study or cross-sectional research will be included.

Types of participants

We will include studies in which the participants were medical students or residents enrolled for specialty training (eg, cardiology, internal medicine, emergency medicine, family medicine, paediatrics and anaesthetics). In studies where the participants were not limited to medical students or residents, only data pertaining to the medical students and residents will be extracted.

Types of interventions

Studies must include CAI as an educational intervention, in either an online or an offline format. The comparator education intervention may include any other teaching method to which CAI was compared. We will exclude studies in which teaching modalities were not primarily and solely used to teach ECGs, or if the subject of teaching was not the conventional 12-lead ECG.

Types of outcome measures

Results must include quantitative data in which ECG competence was measured. We will include assessments of the acquisition of ECG competence (measured shortly after educational intervention) and/or assessments of the retention of ECG competence (delayed testing after educational intervention).

Language and years of publication

All articles published before July 2017 will be included. Publications in languages other than English will be translated, wherever possible.

Primary outcomes

The primary outcome of this systematic review is to determine whether CAI, on its own or in a blended learning setting, is more effective than non-CAI teaching methods in achieving acquisition and retention of ECG competence among medical students and residents.

ECG competence will be measured by extracting the mean scores and SD of assessments before and after exposure to CAI and non-CAI teaching methods, as well as the P values, CIs and effect sizes (Cohen’s d). If the Cohen’s d is not reported in the study, this will be calculated using the mean difference between the groups exposed to CAI and non-CAI teaching methods, divided by the SD of the group exposed to non-CIA teaching methods90 91:

Embedded Image

An effect size of greater than 0.8 will be considered of significant practical importance, whereas effect sizes of 0.5 and 0.2 will be considered as moderate and negligible practical importance, respectively.90 91

The effect of the different ECG teaching modalities will also be scored according to a modified version of Kirkpatrick framework for the evaluation of educational interventions, as shown in table 2.60–63

Table 2

The modified Kirkpatrick framework for the evaluation of educational interventions

Secondary outcomes

The secondary outcomes of this study are to:

  • determine whether there is a difference in the effectiveness of computer-assisted ECG instruction between medical students and residents enrolled for specialty training;

  • identify the types of learning material or activities that are possible with computer-assisted ECG instruction (eg, annotated ECGs, text, illustrations, videos, case scenarios, worked examples, deliberate practice tools) and to establish which CAI learning material or activities were associated with better outcomes;

  • identify the educational approaches (combined or implemented separately) that are possible with computer-assisted ECG instruction and to establish whether these are more successful when used with CAI, conventional teaching methods or in a blended learning setting;

  • identify the learning theories that underpin the methods of ECG instruction, that is, CAI and other methods used for ECG teaching.

Search methods for identification of studies

The lead reviewer (CAV) and an expert librarian (MS) from the University of Cape Town’s Faculty of Health Sciences will conduct an extensive search for peer-reviewed articles.

Electronic searches

The following electronic databases will be used for the search of articles for this systematic review: PubMed, Scopus, Web of Science, Academic Search Premier, CINAHL, PsycINFO, Education Resources Information Center, Africa-Wide Information, Teacher Reference Center and Google Scholar. A combination of Medical Subject Heading terms and free text terms will be used to search for articles. Table 3 shows the main search strategy that we will use.

Table 3

PubMed search strategy, modified as needed for other electronic databases

Searching other sources

Citation indexes and reference lists of all articles found through the database search will be reviewed for any articles that were not identified during the database search. A grey literature search will also be conducted.

Data collection and analysis

The screening process and study selection will be done according to the guidelines of the Cochrane Handbook for Systematic Reviews of Interventions.92

Selection of studies

Two reviewers (CAV and RSM) will independently screen all articles identified by the search. The reviewers will complete a standardised coding sheet that will indicate whether an article meets all the inclusion criteria or what the reason for exclusion is.

Duplicate publications of articles will be removed. The more recent publication with the most complete dataset will be used where duplicate publications for the same data are reported.

The screening process will occur in two phases:

  • Phase 1: screening of title and abstract

    All titles and abstracts of articles identified in the search will be screened for eligibility. If it is not apparent from the title or abstract whether an article meets eligibility criteria, or if both reviewers (CAV and RSM) do not exclude the article, the full text of the article will be reviewed.

  • Phase 2: screening of full-text article

    The full text will be reviewed of all potentially eligible articles. A kappa coefficient will be calculated to measure the consistency between the reviewers (CAV and RSM).30 Where there are discrepancies between the reviewers, this will be discussed with a third reviewer (VB) who will act as an adjudicator. Reasons for exclusion will be documented and presented in a table of excluded studies.

Data extraction and management

References will be managed using EndNote V.X8 software (Clarivate Analytics).93 Two reviewers (CAV and RSM) will independently extract data from all articles meeting eligibility criteria. The reviewers will use a standardised electronic data collection form on Research Electronic Data Capture (REDCap),94 which is a secure online database manager hosted at the University of Cape Town. Collected data will be exported from REDCap database to Stata V.14.2 (Stata) for statistical analysis.

Data extraction will include, but will not be limited to:

  • citation information

  • study design

  • total study duration

  • study population

  • ECGs used during teaching

  • teaching methods (CAI vs other teaching methods)

  • digital learning material

  • educational approaches in study

  • learning theories underpinning instructional methods

  • ECG competencies measured

  • testing times

  • results

  • validity and reliability of results

  • psychometric properties of the assessment tools (eg, Cronbach’s α coefficient).95

A more detailed data extraction set is included in the online supplementary material.

Supplementary file 1

Quality assessment

The Medical Education Research Study Quality Instrument (MERSQI) will be used to assess the quality of studies in this systematic review.96 Designed to evaluate the quality of experimental, quasi-experimental and observational studies, the MERSQI is a validated quality assessment tool in medical education.97

Assessment of risk of bias

Two reviewers (CAV and RSM) will independently assess each included study for risk of bias92:

  • Selection bias, that is, different baseline characteristics among the different groups.

  • Performance bias, that is, different exposure to factors other than intervention that may have influenced outcome among different groups.

  • Attrition bias, that is, differences between groups in withdrawal of participants.

  • Detection bias, that is, differences between groups in how outcomes are determined.

  • Reporting bias, that is, differences in outcome reporting.

Measures of effectiveness of educational intervention

The practical significance of the educational interventions will be determined by reviewing their effect sizes. The effectiveness of ECG teaching modalities used in the articles will also be scored according to a modified version of Kirkpatrick framework for the evaluation of educational interventions. This framework is the internationally preferred framework for evaluation of educational interventions.60–62 The framework is composed of four levels, as shown in table 2.

Dealing with missing data

Corresponding authors will be contacted in the event of absent or incomplete evidence in the included studies. A delay of 6 weeks will be allowed to receive a response following two email attempts.

Data synthesis

Systematic review

We will provide a descriptive analysis of CAI and the comparator teaching modalities used for teaching ECGs. The educational impact of the different teaching modalities used for ECG training (CAI and other methods) will be evaluated by the modified version of Kirkpatrick framework for the evaluation of educational interventions, as shown in table 2.60–62

Meta-analysis

Heterogeneity of the data will be tested by means of the I2 and χ2 tests, as well as by visual inspection of the forest plot. Where found, the possible reasons for any heterogeneity will be explored, and if unexplainable, findings will be reported in a narrative review. In the absence of heterogeneity, the effects of different teaching modalities will be quantitatively analysed. The relative risk and/or the OR will be used to determine the strength of effects among dichotomous variables, and weighted mean difference will be calculated for continuous variables. The statistical significance will be evaluated through inspection of the 95% CIs.

We will do subanalyses of the efficacy of CAI and conventional teaching methods on medical students versus residents. In addition, we will consider subanalyses of studies in terms of teaching methods (ie, CAI, non-CAI and blended learning), different learning material or activities used by CAI (where sufficient data exist), different educational approaches and different learning theories underpinning CAI and other teaching methods.

Mapping review

A mapping review will be done to characterise the quality, quantity and focus of current medical education literature on CAI of ECGs.

Sensitivity analysis

A sensitivity analysis will be undertaken to evaluate the effect of the risk of bias score on the overall result.92 Should any further arbitrary or unclear characteristics arise from the data extraction, a sensitivity analysis will also be applied.

Presenting and reporting of results

Results will be discussed in the text and summarised in table format, an example of which is given in table 4.

Table 4

Results will be summarised in table format

Discussion

Expected significance of the study

This systematic review aims to explore the pedagogical value of CAI compared with other instructional methods used in the teaching of ECGs. The findings of this systematic review will be important in the review of  medical curricula. If gaps are identified in the literature, this will inform future research in the field of ECG teaching. The goal is to provide evidence of best teaching practices, as patient care will ultimately benefit from improved ECG competence among graduating medical students and residents.

Ethics and dissemination

This research does not require ethical approval, as the study is a systematic review of published literature. Any changes to the current protocol will be considered protocol amendment, and this will be communicated to the journal, along with a motivation and justification for the protocol amendment. The status of the systematic review will be updated regularly in PROSPERO. We aim to submit the results of this systematic review to a peer-reviewed journal. The protocol and systematic review will be included in a PhD dissertation.

Acknowledgments

The authors wish to thank Dr Nicholas Simpson, Dr Mahmoud Werfalli and Ms Kathryn Manning from the University of Cape Town for their valuable support and input during the preparation of this protocol.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61.
  62. 62.
  63. 63.
  64. 64.
  65. 65.
  66. 66.
  67. 67.
  68. 68.
  69. 69.
  70. 70.
  71. 71.
  72. 72.
  73. 73.
  74. 74.
  75. 75.
  76. 76.
  77. 77.
  78. 78.
  79. 79.
  80. 80.
  81. 81.
  82. 82.
  83. 83.
  84. 84.
  85. 85.
  86. 86.
  87. 87.
  88. 88.
  89. 89.
  90. 90.
  91. 91.
  92. 92.
  93. 93.
  94. 94.
  95. 95.
  96. 96.
  97. 97.

Footnotes

  • Contributors CAV is a PhD student. RSM and VB are his supervisors. CAV conceived of the review and undertook the drafting of the manuscript. CAV and MS undertook a scoping search and developed the search strategy. CAV, RSM and VB will be involved in data acquisition. CAV and MEE will analyse the data and participate in the interpretation of the results. All authors have read the manuscript and have given their approval for publication.

  • Competing interests RSM is a lecturer and host of the AO Memorial Advanced ECG and Arrhythmia Course, and receives an honorarium from Medtronic Africa.

  • Provenance and peer review Not commissioned; externally peer reviewed.