Table 3

Data extraction table

Author year countryAimStudy designPopulationPalliative care teaching interventionComparatorAssessment methodOutcomesKirkpatrick model levelStudy qualityStrengths and limitationsFurther research
Auret 2008
Identify if a structured clinical instruction module improves students self-rated confidencePre and post test design.91 sixth year medical students
(91/106 students: response rate=86%)
Follow-up questionnaire at end of academic year: 30/109 students (response rate=28%)
2-hour Structured Clinical Instruction Module - nine 15 min stations. Four groups of 30 to 35 students (in groups of four).
Taught by one palliative care consultant + team of nurses/doctors/ pharmacist
Acted as own comparator, pre and post testQuestionnaire – 6-point Likert scale. Pre workshop, immediately post workshop + follow up at end of academic yearImproved knowledge and skill post workshop. Poor rate of completion of follow-up, but sustained improvement.2aAssessed using Cochrane risk of bias tool, medium risk of bias. Risk of attrition bias- 86% initial completion rate dropped to 28% completion at end of academic year. Reasons not fully explored.Strengths - required less facilitators than some other interventions, ‘practical feel’. Limitations - no statistical reporting, poor response to longer-term follow-up minimising evaluation of knowledge retention.To formally test knowledge and skill competence following workshop
Brand et al 2012
Evaluate students’ knowledge, attitudes and experience of a palliative care education programme in a graduate entry medical settingPre and post test design knowledge and self-efficacy in palliative care62 second year graduate med students.
40/62 (64.5%) completed both the pre and the post test
Taught by four palliative care consultants + four registrars
8 hours palliative teaching within 100 hours oncology curriculum. 5 week oncology/palliative care block. Lectures, PBL sessions, bedside/clinic tutorials, visit to inpatient unit, self-directed reading.Acted as own comparator, pre test and post testMultiple choice question knowledge test, two validated attitudinal scales, student feedback survey (Likert scale + open ended questions)No statistical significance in mean improvement in knowledge. Subset statistical improvement in symptom management (p=0.001). improvements in attitudes towards communication, symptom management and MDT care2bAssessed using mixed methods tool- passed all components.Strengths -
mixed methods study.
Limitations - possible selection bias – only 64.5% completed pre and post tests and 42% response rate to student evaluation questionnaire, multiple choice questions weren’t independently validated.
Examine the feasibility of a 1-week palliative care course incorporated into the medicine clerkship; knowledge and attitudinal changes in students who had completed the course.Pre and post test design84 third year medical students.
(63%) students completed both pre and post tests
1-week palliative care curriculum during a 1-year period.
Included in-patient and out-patient care, MDT rounds, reflection and didactic teaching around core clinical topics.
Acted as own comparator, pre test and post testSurvey of attitudes towards palliative care and pre and post course measurements of knowledge.Statistically significant improvement in knowledge scores (pre-course mean scores
145/230 and 175/230 post-intervention
Improved attitudes.
2bAssessed using mixed methods tool- passed all components.Strengths - mixed methods study.
Limitations - 63% response rate even to knowledge tests response bias.
Chang et al 2009
China (including Taiwan)31
Evaluate the effect of a multimodal teaching programme on preclinical medical students’ knowledge of palliative care and their beliefs relating to ethical decision-making.Pre and post test design118 third year medical students ‘pre-clinical’ in Taiwan as medicine is a 6-year degree.
Voluntary participation.
Taught by palliative care doctors, clinical social workers, chaplain, nurse practitioner/nurse lead for palliative care.
1 week, end of life care curriculum developed. Three learning modules. Included bedside teaching, lecture series and small group discussion.Acted as own comparator, pre test and post testAssessed knowledge + beliefs regarding decision-making.
Instrument constructed based on literature review and national guidance. Validated for use by content expert and tested for reliability- items not meeting reliability statistical cut-off excluded.
Improved knowledge following intervention by 14.7% (p<0.001). Clinical management knowledge improved the most.
Some improvement in beliefs regarding decision-making but not universal.
2bAssessed using Cochrane risk of bias tool Medium risk of bias.
Selection bias likely to be present students were volunteers, risking self-selection bias.
100% response rate no attrition bias.
Strengths - validated test tool.
Limitations - only 18/32 knowledge items reliable enough for inclusion. Knowledge questions were true/false/not sure.
Follow-up immediately after intervention, not testing long-term retention of knowledge.
None discussed
Day et al
Compare the effect of eLearning versus small-group learningQuasi-randomised controlled trial of web-based interactive education (eDoctoring) compared with small-group education (Doctoring)119 Third year medical students. eDoctoring (n=48) or small-group Doctoring (n=71).Interactive e-learning: eDoctoring on palliative care clinical content over 2 months.
No faculty input while taking the course.
Small group teaching for 3 hours on communication skills. Year-long course.
26 Small group sessions on palliative/end of life care.
Small group teaching for 3 hours on communication skills. Year-long course (same as intervention arm).
Pre-test and post-test questionnaires.
27 self-efficacy questions rating confidence.
six single best answer knowledge questions relating to curriculum covered by modules completed in eDoctoring.
Both groups- knowledge questions improved post-test, non-statistical trend present favouring the eDoctoring students.
in self-efficacy ratings in both intervention and control, with no differences in improved between the groups.
2bAssessed using Cochrane risk of bias tool. Medium risk of bias.
Quasi-randomised, low selection bias.
Attrition bias possible- more dropouts in Doctoring arm (results excluded from analysis) - reasons not explored.
Strengths - quasi-randomised
Limitations - no long-term measures in knowledge retention. Randomisation did not include technology fluency or viewpoints.
Dorner et al 2014
Explore the feasibility of peer teaching for communication skills training.Pre and post test study37/49 (76%) medical students in the fourth to sixth of medical school. Voluntary participation open to all medical students.
Tutors - fifth and sixth year medical students trained by faculty to deliver teaching.
90 min peer taught workshop teaching nine core communication skills regarding palliative and end of life care, particularly within the intensive care unit. Case based discussions and role play both used.Own comparator, pre test and post testExternal ‘intensivist’ rated students based on a taped role play they conducted with another student.
Qualitative analysis of transcripts to see how students spoke about death. Self-rated skills scores.
Self-rating scores improved following intervention (p<0.001).
Mean expert ratings did not differ from student’s own assessment of performance or skills except in one domain
2bAssessed using mixed methods tool- passed all points.Strengths - peer teaching affordable and easily scalable.
Limitations - lacks long-term evaluation
Further work required regarding student’s ability to use the word ‘death’.
Ellman et al 2016
Evaluate 4-year curriculum in palliative care.Mixed method evaluationFirst to fourth year medical students. 95 students in the implementation year4-year longitudinal, integrated curriculum. Included workshops, hospice experience, modules, communication skills and a year four palliative care observed structured clinical examination (OSCE) station.
2 hours in first year; 4 hours in second year; 15 to 23 hours in third year; 4 hours in fourth year.
Comparator only for graduating student surveys- compared with national Association of American
Medical Colleges questionnaire of US medical schools regarding confidence with palliative care.
Competency in a palliative care OSCE station at the end of the curriculum. Analysis of student written reflections.
Graduating student surveys regarding how prepared students felt following course.
In implementation year, average score 74% in OSCE palliative care station- lower than average score for other OSCE stations (84%) but felt to be ‘acceptable’. Students undertaking 4-year curriculum felt more prepared in palliative care compared with other US medical schools.2bAssessed using mixed methods tool - passed all points.Strengths - mixed-methods study, curriculum is well integrated and longitudinal.
Limitations - no long-term follow-up data, OSCE station on palliative care scored lower still than OSCE stations on other subjects
In order to evaluate longer-term effect of curriculum, team are planning a survey of former students now in postgraduate training.
Gerlach et al 2015
Evaluate the effects of the Mainz undergraduate palliative care education on students’ self-confidence regarding important domains in palliative care.Prospective questionnaire-based cohort study with a pre–post design. Knowledge test only at end of module.329 fifth year medical students. All students took knowledge test.
156 (47%) students completed matched surveys at both points of measurement
Facilitators: physicians, palliative care nurses, bereaved family members.
Mandatory palliative care module over one term.
7×90 min sessions.
Included pain lecture hospice home care through use of videoed live interview with bereaved family member. Small group discussion.
Knowledge scores: historic test scores from before the intervention within Mainz examined same test so comparison is likely acceptable.
Self-confidence scores, comparison with cohort from 2011 in Mainz who did not receive module.
Multiple choice electronic knowledge exam after module 21 item, single best choice answer.
Pre and post testing of students’ self-confidence.
All passed knowledge exam, average scores >90%. Compared with historic cohort: increased in correct answers for pain (40%), symptom control (69%), and psychosocial knowledge (33%).
Self-reported confidence improved.
2bAssessed using Cochrane risk of bias tool. Medium risk of bias.
Attrition bias: 47% of surveys matched for pre and post test results, so data lost. Reasons are clear - incomplete form completion, effect of this is unclear.
Strengths - Intervention acceptable, enjoyed interdisciplinary input.
Limitations - only 47% of surveys pre and post intervention matched and used (due to local policy), unknown if increases in knowledge and self-confidence are linked.
Whether or not the course provided only an instant or a long-term effect - research underway.
Further research needed regarding any effect on patient outcomes.
Goldberg et al 2011
To assess the effect of a required clinical rotation in palliative medicineHistorical control trial117 fourth year medical students (month prior to graduation)
Taught by two interdisciplinary teams, each with an: attending physician, fellow in palliative medicine/geriatrics/oncology, a nurse practitioner, and a social worker staff (clinical team portion) + social worker, chaplain, & massage therapist
n=59 (51% of students from class of 2008) Addition of a required 1 week clinical rotation in palliative medicine (integrated in 12 week IM-Geriatrics clerkship) – multiple venues, time spent with consult team + formal didactic lectures on palliative care issuesn=58 (55% of students from class of 2007)=historical control group (received didactics but no clinical rotation in palliative medicine)Survey: self-rated skills performance and interest, student educational experience, 30-question MCQ exam
2008 cohort also had two open-ended questions
Components of Association of American
Medical Colleges annual graduate questionnaire
No statistical difference in mean scores for knowledge questions
Higher skills self-ratings in 2008 cohort
Association of American
Medical Colleges questionnaire: 2008 cohort more experience in palliative care
2bAssessed using mixed methods tool - passed all points.Strengths - mixed methods study, utilised historical control group
Limitations - diversity of exposure with clinical rotations, not controllable
Further research into qualitative findings - how might reported skills be applied
Exploring different venues of palliative care (outpatient) for clinical rotations
Pilot study evaluating the effectiveness of a computer-based decision aid for teaching medical students about advance care planning.Prospective, randomised controlled design133 second year medical students.
121/133 (91%) of students agreed to have their data used in study - 60 in the
Decision Aid Group and 61 in the Standard Group.
Computer-based decision aid for student use to help patients with advanced care planning (to help patient complete advance directive).
Multimedia tool, uses educational material and exercises to help patient clarify values and priorities, help students explain end of life conditions and treatments and then helps synthesise this into an advance directive.
Prior to intervention all students received instruction in advanced care planning lectures, reading material, small group discussion.Knowledge assessed using a 17-item true/false and MCQ.
Self-rated satisfaction, confidence and perceived knowledge of patient wishes.
Patients’ evaluation of student assessed using 12-items addressing students’ communication skills, helpfulness and perceived understanding of their wishes.
Patients’ satisfaction assessed by measure of global satisfaction
High baseline knowledge for advance care planning. Students in decision aid group more improved (84% to 88%, p<0.01)
Student confidence increased following interventions in both groups but more in decision aid group.
Student satisfaction higher in decision aid group.
Patients significantly more satisfied with student performance and global impression in decision aid group.
3Assessed using Cochrane risk of bias tool. High risk of bias.
No discussion of how students were randomised so unclear if selection bias is present.
patient bias - students were responsible for recruiting patients and these were eligible to be family/friends- patient rating scales may well be biassed.
Strengths - tool easy to roll out and applicable within other institutions. High levels of student and patient satisfaction.
Limitations - pilot study so not powered. Selection of patients determined by students. No full data regarding student interactions with patients, time spent. Confounding factors within this that could have impacted results. Measures used within the study not validated.
National study comparing this computer programme with current approaches to advance care planning.
Evaluate a palliative medicine curriculum developed for medical students in the required third-year clerkship in family medicine at the University of Tennessee.Pre and post test design with the post-test assessment 7 weeks later.69 third year medical students on their family medicine clerkshipFour-hour curriculum.
Prior to session students were sent reading concerning palliative care. During session- discussion, role play, information giving via PowerPoint and lecture.
Acted as own comparator, pre test and post test20 item pre-test and post-test for palliative care knowledge.
One item confidence question regarding palliative care clinical skills.
Significant knowledge gain post-test (37% pre-test to 55% post-test); (p<0.0001).
Small but statistically significant increase in self-reported confidence (p=0.031).
2bAssessed using Cochrane risk of bias tool.
No bias noted in any domains.
Low risk of bias.
Strengths -Popular with students on course evaluation.
Limitations - other palliative care education at institution.
long-term retention of knowledge and the development of instruments to measure the translation of a theoretical knowledge base into actual clinical skill sets.
Paneduro et al 2014
Develop and evaluate a pain management and palliative care seminar for medical students during surgical clerkshipPre and post test design with the post-test assessment at 1 year292 third and fourth year medical students in surgical clerkship
(n=277) completed post test immediately following the seminar and 31% (n=90) completed the follow-up test via email.
4-hour seminar on pain management and palliative care
Taught by faculty from pain medicine, surgery and palliative care
Acted as own comparator, pre test and post test10-item knowledge test
Comments on seminar
Significant knowledge improved; maintained at 1 year. mean pre-test, post-test and 1 year follow-up test scores were 51%, 75% and 73%, respectively.
No difference between third and fourth year students
2bAssessed using mixed methods tool - passed all points.Strength - relatively short items to respond to in order to facilitate participant, collaboratively designed seminar
Limitations - high attrition rate at 1 year. Hard to control for seminar impact specifically, at long-term follow-up
Modify seminar to better target attitudes/beliefs
Porter-Williamson et al 2004
Assess impact of a hospice curriculum for medical students, in terms of knowledge, skills and attitudesPre and post test study127 third year medical students32 hours, 4 day curriculumActed as own comparator, pre test and post test26-item self-assessment of competency, a 20-item self-report of concerns, a 50-item MCQ knowledge test and qualitative assessment of course curriculum23% improved knowledge
56% improved competence
29% improved for concerns
(all p<0.0001).
No changes for attitudes (p=0.35) (already had appropriate attitudes)
2bAssessed using mixed methods tool- passed all points.Strength - multiple measures of curricular evaluation, curriculum could be applied at other universities
Limitations - no long-term follow-up
Link specific clinical encounters with clinical knowledge changes, for explanation; longitudinal re-examining
Schulz-Quach et al 2018
Evaluate an eLearning course ‘Palliative Care Basics’ in terms of student acceptance, exam performance and competenceCross-sectional study670 undergraduate medical students (three cohorts). 569 (96%) used eLearning as preparation for the exam; 23 did not.eLearning course (five teachings domains over 10 teaching units). Virtual patient contact, didactic teaching, e-lectures, patient case vignettesStudents who did not access the eLearning course. 23 studentsQuestionnaire of self-assessment
Course evaluation, with ratings and free response section
20-item MCQ exam
Knowledge improved (p=0.02). High approval of eLearning tool – easy to approach topics, increased interest2bAssessed using mixed methods tool- passed all points.Strength - mixed methods
Limitations - no baseline measurements, very small comparator group
Further assessment of eLearning tools in blended curriculum
Tai et al
Assess whether a 1-week palliative care placement improves student performance and knowledge. Explore student views on palliative care rotation, particular for building confidenceConsecutive cohort retrospective analysis, pre and post test mixed methodology84 fifth year medical students (who enrolled in palliative care placement).
72 (86%) completing both pre and post course multiple-choice questions
1 week palliative care placement
Combination of didactic and interactive tutorials with experiential attachment such as ward rounds
Acted as own comparator, pre test and post test/courseKnowledge-based questions (16 MCQs)
Post-course satisfaction ratings (10-closed item questions + 2 open-ended questions)
Improved knowledge: average 58% to 74% (p<0.001).
Most reported value of course and wanted more palliative care education
2bAssessed using mixed methods tool- passed all points.Strength - mixed methods.
Limitations - measures not validated; reduced sample size due to exclusion of students who did not complete both parts of study
Assess value of different length palliative care placements (1 week might not be enough)
Tan et al 2013
Determine whether virtual patient case in palliative care could offer students acceptable alternative to real-life experiencesMixed methods pre and post survey137 third year medical students
95% (130/137) consented to have their results analysed. knowledge score assessed in 127
Virtual patient clinical case, mandatory exercise in family medicine rotation
Average time spent with virtual patient case=0.93 hours, SD=0.65
Acted as own comparator, pre test and post testKnowledge test and level-of-preparedness survey (self-assessment of clinical skills), plus student feedback on virtual patient case/usage and general feedbackKnowledge scores increased (48%–63%: p<0.001) virtual patient case was realistic (91%), and educational (86%)
Students spending >20 min on case reported more engagement
2bAssessed using mixed methods tool- passed all points.Strength - mixed methods approach for evaluation
Limitations - hard to correlate time spent on case with outcomes, limited info about students’ experiences with real patients
Expanding knowledge component of study to better understand specific changes in knowledge
Tsai et al 2008
Assess the impact of a 4-hour multimodule curriculum on knowledge and attitudes of end of life careProspective cross-sectional pre and post test survey259 fifth year medical students4-hour course included: 1 hour lecture by specialist, 1 hour patient visit at unit, 1 hour literature reading, 1 hour discussionActed as own comparator, pre test and post testQuestions on knowledge, demographics and ethical beliefsKnowledge improved (55% to 70%) (p<0.0001).
Principles of palliative care scores improved (58% to 73%). Clinical management improved (59% to 68%)
2bCochrane risk of bias tool - low risk of bias, no bias evident in any domains.Strength - easy to implement curriculum. correlation analysis across items
Limitations - hard to control for confounding variables like maturation effect
Further assessment of medical training (residency and clinical practice) – follow-up studies
Longitudinal study to better understand changes over time
Explore the application of online learning tool with hospice experienceRandomised prospective pre and post study152 second year medical students completed the survey
(response rate 51%)
56% (n=85) completed the online module
Addition of 30 min online module to hospice experience.
Taught by hospice care physician or nurse) in hospice setting
Randomised to receive module prior to hospice experience (YES module) versus after experience (NO module)23-item electronic survey: 10 attitude-assessing statements from FATCOD, 8 multiple choice knowledge questionsHigher scores on knowledge questions for students completing the online module (p=0.006).
No statistical difference in attitudes
2bAssessed using Cochrane risk of bias tool, medium risk of bias. Self-selection bias as voluntary participation, could suggest already motivated regarding palliative care. Randomisation not describedStrengths - mixed methods study, focussed on assessing blended learning experiences
Limitations - single site study, survey was relatively few items
Expanding scope of study for more institutions (generalisability)
More survey items → more comprehensive assessment
von Gunten et al 2012
Assess impact, retention and magnitude of effect of a required didactic and experiential palliative care curriculumProspective pre and post study487 third year medical studentsSpecified palliative care curriculum designed for 1 day/week for 4 weeks (during the ambulatory block of the 12 week IM clerkship)
Taught by IM faculty. participation was compulsory
Self-comparator over time (pre test and post test).
knowledge compared with national cross-sectional study comparing residents at progressive training levels
36-item knowledge test, self-assessment of competency, & self-assessment of attitudes + written surveysKnowledge: improved 52% to 67% (national residents, average score 62%).
56% improved confidence (higher than resident national averages).
29% decrease in concern. (All p<0.001).
All maintained at 1 year.
2bAssessed using mixed methods tool- passed all points.Strength - mixed methods, assess various levels of effect, national comparison
Limitations - evaluation instruments designed for specific learning objectives of course. Documentation of long-term follow-up unclear
None outlined by study
  • IM, Internal Medicine; MDT, Multidisciplinary Team Meeting; PBL, Problem Based Learning.