Data extraction table
| Author year country | Aim | Study design | Population | Palliative care teaching intervention | Comparator | Assessment method | Outcomes | Kirkpatrick model level | Study quality | Strengths and limitations | Further research |
| Auret 2008 Australia35 | Identify if a structured clinical instruction module improves students self-rated confidence | Pre and post test design. | 91 sixth year medical students (91/106 students: response rate=86%) Follow-up questionnaire at end of academic year: 30/109 students (response rate=28%) | 2-hour Structured Clinical Instruction Module - nine 15 min stations. Four groups of 30 to 35 students (in groups of four). Taught by one palliative care consultant + team of nurses/doctors/ pharmacist | Acted as own comparator, pre and post test | Questionnaire – 6-point Likert scale. Pre workshop, immediately post workshop + follow up at end of academic year | Improved knowledge and skill post workshop. Poor rate of completion of follow-up, but sustained improvement. | 2a | Assessed using Cochrane risk of bias tool, medium risk of bias. Risk of attrition bias- 86% initial completion rate dropped to 28% completion at end of academic year. Reasons not fully explored. | Strengths - required less facilitators than some other interventions, ‘practical feel’. Limitations - no statistical reporting, poor response to longer-term follow-up minimising evaluation of knowledge retention. | To formally test knowledge and skill competence following workshop |
| Brand et al 2012 Australia19 | Evaluate students’ knowledge, attitudes and experience of a palliative care education programme in a graduate entry medical setting | Pre and post test design knowledge and self-efficacy in palliative care | 62 second year graduate med students. 40/62 (64.5%) completed both the pre and the post test Taught by four palliative care consultants + four registrars | 8 hours palliative teaching within 100 hours oncology curriculum. 5 week oncology/palliative care block. Lectures, PBL sessions, bedside/clinic tutorials, visit to inpatient unit, self-directed reading. | Acted as own comparator, pre test and post test | Multiple choice question knowledge test, two validated attitudinal scales, student feedback survey (Likert scale + open ended questions) | No statistical significance in mean improvement in knowledge. Subset statistical improvement in symptom management (p=0.001). improvements in attitudes towards communication, symptom management and MDT care | 2b | Assessed using mixed methods tool- passed all components. | Strengths - mixed methods study. Limitations - possible selection bias – only 64.5% completed pre and post tests and 42% response rate to student evaluation questionnaire, multiple choice questions weren’t independently validated. | No |
| Brownfield 2009 USA30 | Examine the feasibility of a 1-week palliative care course incorporated into the medicine clerkship; knowledge and attitudinal changes in students who had completed the course. | Pre and post test design | 84 third year medical students. 53/84 (63%) students completed both pre and post tests | 1-week palliative care curriculum during a 1-year period. Included in-patient and out-patient care, MDT rounds, reflection and didactic teaching around core clinical topics. | Acted as own comparator, pre test and post test | Survey of attitudes towards palliative care and pre and post course measurements of knowledge. | Statistically significant improvement in knowledge scores (pre-course mean scores 145/230 and 175/230 post-intervention (p<0.01). Improved attitudes. | 2b | Assessed using mixed methods tool- passed all components. | Strengths - mixed methods study. Limitations - 63% response rate even to knowledge tests response bias. | No |
| Chang et al 2009 China (including Taiwan)31 | Evaluate the effect of a multimodal teaching programme on preclinical medical students’ knowledge of palliative care and their beliefs relating to ethical decision-making. | Pre and post test design | 118 third year medical students ‘pre-clinical’ in Taiwan as medicine is a 6-year degree. Voluntary participation. Taught by palliative care doctors, clinical social workers, chaplain, nurse practitioner/nurse lead for palliative care. | 1 week, end of life care curriculum developed. Three learning modules. Included bedside teaching, lecture series and small group discussion. | Acted as own comparator, pre test and post test | Assessed knowledge + beliefs regarding decision-making. Instrument constructed based on literature review and national guidance. Validated for use by content expert and tested for reliability- items not meeting reliability statistical cut-off excluded. | Improved knowledge following intervention by 14.7% (p<0.001). Clinical management knowledge improved the most. Some improvement in beliefs regarding decision-making but not universal. | 2b | Assessed using Cochrane risk of bias tool Medium risk of bias. Selection bias likely to be present students were volunteers, risking self-selection bias. 100% response rate no attrition bias. | Strengths - validated test tool. Limitations - only 18/32 knowledge items reliable enough for inclusion. Knowledge questions were true/false/not sure. Follow-up immediately after intervention, not testing long-term retention of knowledge. | None discussed |
| Day et al 2015 USA26 | Compare the effect of eLearning versus small-group learning | Quasi-randomised controlled trial of web-based interactive education (eDoctoring) compared with small-group education (Doctoring) | 119 Third year medical students. eDoctoring (n=48) or small-group Doctoring (n=71). | Interactive e-learning: eDoctoring on palliative care clinical content over 2 months. No faculty input while taking the course. Small group teaching for 3 hours on communication skills. Year-long course. | 26 Small group sessions on palliative/end of life care. Small group teaching for 3 hours on communication skills. Year-long course (same as intervention arm). | Pre-test and post-test questionnaires. 27 self-efficacy questions rating confidence. six single best answer knowledge questions relating to curriculum covered by modules completed in eDoctoring. | Both groups- knowledge questions improved post-test, non-statistical trend present favouring the eDoctoring students. in self-efficacy ratings in both intervention and control, with no differences in improved between the groups. | 2b | Assessed using Cochrane risk of bias tool. Medium risk of bias. Quasi-randomised, low selection bias. Attrition bias possible- more dropouts in Doctoring arm (results excluded from analysis) - reasons not explored. | Strengths - quasi-randomised Limitations - no long-term measures in knowledge retention. Randomisation did not include technology fluency or viewpoints. | No |
| Dorner et al 2014 Germany28 | Explore the feasibility of peer teaching for communication skills training. | Pre and post test study | 37/49 (76%) medical students in the fourth to sixth of medical school. Voluntary participation open to all medical students. Tutors - fifth and sixth year medical students trained by faculty to deliver teaching. | 90 min peer taught workshop teaching nine core communication skills regarding palliative and end of life care, particularly within the intensive care unit. Case based discussions and role play both used. | Own comparator, pre test and post test | External ‘intensivist’ rated students based on a taped role play they conducted with another student. Qualitative analysis of transcripts to see how students spoke about death. Self-rated skills scores. | Self-rating scores improved following intervention (p<0.001). Mean expert ratings did not differ from student’s own assessment of performance or skills except in one domain | 2b | Assessed using mixed methods tool- passed all points. | Strengths - peer teaching affordable and easily scalable. Limitations - lacks long-term evaluation | Further work required regarding student’s ability to use the word ‘death’. |
| Ellman et al 2016 USA29 | Evaluate 4-year curriculum in palliative care. | Mixed method evaluation | First to fourth year medical students. 95 students in the implementation year | 4-year longitudinal, integrated curriculum. Included workshops, hospice experience, modules, communication skills and a year four palliative care observed structured clinical examination (OSCE) station. 2 hours in first year; 4 hours in second year; 15 to 23 hours in third year; 4 hours in fourth year. | Comparator only for graduating student surveys- compared with national Association of American Medical Colleges questionnaire of US medical schools regarding confidence with palliative care. | Competency in a palliative care OSCE station at the end of the curriculum. Analysis of student written reflections. Graduating student surveys regarding how prepared students felt following course. | In implementation year, average score 74% in OSCE palliative care station- lower than average score for other OSCE stations (84%) but felt to be ‘acceptable’. Students undertaking 4-year curriculum felt more prepared in palliative care compared with other US medical schools. | 2b | Assessed using mixed methods tool - passed all points. | Strengths - mixed-methods study, curriculum is well integrated and longitudinal. Limitations - no long-term follow-up data, OSCE station on palliative care scored lower still than OSCE stations on other subjects | In order to evaluate longer-term effect of curriculum, team are planning a survey of former students now in postgraduate training. |
| Gerlach et al 2015 Germany20 | Evaluate the effects of the Mainz undergraduate palliative care education on students’ self-confidence regarding important domains in palliative care. | Prospective questionnaire-based cohort study with a pre–post design. Knowledge test only at end of module. | 329 fifth year medical students. All students took knowledge test. 156 (47%) students completed matched surveys at both points of measurement Facilitators: physicians, palliative care nurses, bereaved family members. | Mandatory palliative care module over one term. 7×90 min sessions. Included pain lecture hospice home care through use of videoed live interview with bereaved family member. Small group discussion. | Knowledge scores: historic test scores from before the intervention within Mainz examined same test so comparison is likely acceptable. Self-confidence scores, comparison with cohort from 2011 in Mainz who did not receive module. | Multiple choice electronic knowledge exam after module 21 item, single best choice answer. Pre and post testing of students’ self-confidence. | All passed knowledge exam, average scores >90%. Compared with historic cohort: increased in correct answers for pain (40%), symptom control (69%), and psychosocial knowledge (33%). Self-reported confidence improved. | 2b | Assessed using Cochrane risk of bias tool. Medium risk of bias. Attrition bias: 47% of surveys matched for pre and post test results, so data lost. Reasons are clear - incomplete form completion, effect of this is unclear. | Strengths - Intervention acceptable, enjoyed interdisciplinary input. Limitations - only 47% of surveys pre and post intervention matched and used (due to local policy), unknown if increases in knowledge and self-confidence are linked. | Whether or not the course provided only an instant or a long-term effect - research underway. Further research needed regarding any effect on patient outcomes. |
| Goldberg et al 2011 USA18 | To assess the effect of a required clinical rotation in palliative medicine | Historical control trial | 117 fourth year medical students (month prior to graduation) Taught by two interdisciplinary teams, each with an: attending physician, fellow in palliative medicine/geriatrics/oncology, a nurse practitioner, and a social worker staff (clinical team portion) + social worker, chaplain, & massage therapist | n=59 (51% of students from class of 2008) Addition of a required 1 week clinical rotation in palliative medicine (integrated in 12 week IM-Geriatrics clerkship) – multiple venues, time spent with consult team + formal didactic lectures on palliative care issues | n=58 (55% of students from class of 2007)=historical control group (received didactics but no clinical rotation in palliative medicine) | Survey: self-rated skills performance and interest, student educational experience, 30-question MCQ exam 2008 cohort also had two open-ended questions Components of Association of American Medical Colleges annual graduate questionnaire | No statistical difference in mean scores for knowledge questions Higher skills self-ratings in 2008 cohort Association of American Medical Colleges questionnaire: 2008 cohort more experience in palliative care | 2b | Assessed using mixed methods tool - passed all points. | Strengths - mixed methods study, utilised historical control group Limitations - diversity of exposure with clinical rotations, not controllable | Further research into qualitative findings - how might reported skills be applied Exploring different venues of palliative care (outpatient) for clinical rotations |
| Green 2010 USA25 | Pilot study evaluating the effectiveness of a computer-based decision aid for teaching medical students about advance care planning. | Prospective, randomised controlled design | 133 second year medical students. 121/133 (91%) of students agreed to have their data used in study - 60 in the Decision Aid Group and 61 in the Standard Group. | Computer-based decision aid for student use to help patients with advanced care planning (to help patient complete advance directive). Multimedia tool, uses educational material and exercises to help patient clarify values and priorities, help students explain end of life conditions and treatments and then helps synthesise this into an advance directive. | Prior to intervention all students received instruction in advanced care planning lectures, reading material, small group discussion. | Knowledge assessed using a 17-item true/false and MCQ. Self-rated satisfaction, confidence and perceived knowledge of patient wishes. Patients’ evaluation of student assessed using 12-items addressing students’ communication skills, helpfulness and perceived understanding of their wishes. Patients’ satisfaction assessed by measure of global satisfaction | High baseline knowledge for advance care planning. Students in decision aid group more improved (84% to 88%, p<0.01) Student confidence increased following interventions in both groups but more in decision aid group. Student satisfaction higher in decision aid group. Patients significantly more satisfied with student performance and global impression in decision aid group. | 3 | Assessed using Cochrane risk of bias tool. High risk of bias. No discussion of how students were randomised so unclear if selection bias is present. patient bias - students were responsible for recruiting patients and these were eligible to be family/friends- patient rating scales may well be biassed. | Strengths - tool easy to roll out and applicable within other institutions. High levels of student and patient satisfaction. Limitations - pilot study so not powered. Selection of patients determined by students. No full data regarding student interactions with patients, time spent. Confounding factors within this that could have impacted results. Measures used within the study not validated. | National study comparing this computer programme with current approaches to advance care planning. |
| Jackson 2002 USA34 | Evaluate a palliative medicine curriculum developed for medical students in the required third-year clerkship in family medicine at the University of Tennessee. | Pre and post test design with the post-test assessment 7 weeks later. | 69 third year medical students on their family medicine clerkship | Four-hour curriculum. Prior to session students were sent reading concerning palliative care. During session- discussion, role play, information giving via PowerPoint and lecture. | Acted as own comparator, pre test and post test | 20 item pre-test and post-test for palliative care knowledge. One item confidence question regarding palliative care clinical skills. | Significant knowledge gain post-test (37% pre-test to 55% post-test); (p<0.0001). Small but statistically significant increase in self-reported confidence (p=0.031). | 2b | Assessed using Cochrane risk of bias tool. No bias noted in any domains. Low risk of bias. | Strengths -Popular with students on course evaluation. Limitations - other palliative care education at institution. | long-term retention of knowledge and the development of instruments to measure the translation of a theoretical knowledge base into actual clinical skill sets. |
| Paneduro et al 2014 Canada33 | Develop and evaluate a pain management and palliative care seminar for medical students during surgical clerkship | Pre and post test design with the post-test assessment at 1 year | 292 third and fourth year medical students in surgical clerkship 95% (n=277) completed post test immediately following the seminar and 31% (n=90) completed the follow-up test via email. | 4-hour seminar on pain management and palliative care Taught by faculty from pain medicine, surgery and palliative care | Acted as own comparator, pre test and post test | 10-item knowledge test Comments on seminar | Significant knowledge improved; maintained at 1 year. mean pre-test, post-test and 1 year follow-up test scores were 51%, 75% and 73%, respectively. No difference between third and fourth year students | 2b | Assessed using mixed methods tool - passed all points. | Strength - relatively short items to respond to in order to facilitate participant, collaboratively designed seminar Limitations - high attrition rate at 1 year. Hard to control for seminar impact specifically, at long-term follow-up | Modify seminar to better target attitudes/beliefs |
| Porter-Williamson et al 2004 USA23 | Assess impact of a hospice curriculum for medical students, in terms of knowledge, skills and attitudes | Pre and post test study | 127 third year medical students | 32 hours, 4 day curriculum | Acted as own comparator, pre test and post test | 26-item self-assessment of competency, a 20-item self-report of concerns, a 50-item MCQ knowledge test and qualitative assessment of course curriculum | 23% improved knowledge 56% improved competence 29% improved for concerns (all p<0.0001). No changes for attitudes (p=0.35) (already had appropriate attitudes) | 2b | Assessed using mixed methods tool- passed all points. | Strength - multiple measures of curricular evaluation, curriculum could be applied at other universities Limitations - no long-term follow-up | Link specific clinical encounters with clinical knowledge changes, for explanation; longitudinal re-examining |
| Schulz-Quach et al 2018 Germany22 | Evaluate an eLearning course ‘Palliative Care Basics’ in terms of student acceptance, exam performance and competence | Cross-sectional study | 670 undergraduate medical students (three cohorts). 569 (96%) used eLearning as preparation for the exam; 23 did not. | eLearning course (five teachings domains over 10 teaching units). Virtual patient contact, didactic teaching, e-lectures, patient case vignettes | Students who did not access the eLearning course. 23 students | Questionnaire of self-assessment Course evaluation, with ratings and free response section 20-item MCQ exam | Knowledge improved (p=0.02). High approval of eLearning tool – easy to approach topics, increased interest | 2b | Assessed using mixed methods tool- passed all points. | Strength - mixed methods Limitations - no baseline measurements, very small comparator group | Further assessment of eLearning tools in blended curriculum |
| Tai et al 2014 Australia21 | Assess whether a 1-week palliative care placement improves student performance and knowledge. Explore student views on palliative care rotation, particular for building confidence | Consecutive cohort retrospective analysis, pre and post test mixed methodology | 84 fifth year medical students (who enrolled in palliative care placement). 72 (86%) completing both pre and post course multiple-choice questions | 1 week palliative care placement Combination of didactic and interactive tutorials with experiential attachment such as ward rounds | Acted as own comparator, pre test and post test/course | Knowledge-based questions (16 MCQs) Post-course satisfaction ratings (10-closed item questions + 2 open-ended questions) | Improved knowledge: average 58% to 74% (p<0.001). Most reported value of course and wanted more palliative care education | 2b | Assessed using mixed methods tool- passed all points. | Strength - mixed methods. Limitations - measures not validated; reduced sample size due to exclusion of students who did not complete both parts of study | Assess value of different length palliative care placements (1 week might not be enough) |
| Tan et al 2013 Canada36 | Determine whether virtual patient case in palliative care could offer students acceptable alternative to real-life experiences | Mixed methods pre and post survey | 137 third year medical students 95% (130/137) consented to have their results analysed. knowledge score assessed in 127 | Virtual patient clinical case, mandatory exercise in family medicine rotation Average time spent with virtual patient case=0.93 hours, SD=0.65 | Acted as own comparator, pre test and post test | Knowledge test and level-of-preparedness survey (self-assessment of clinical skills), plus student feedback on virtual patient case/usage and general feedback | Knowledge scores increased (48%–63%: p<0.001) virtual patient case was realistic (91%), and educational (86%) Students spending >20 min on case reported more engagement | 2b | Assessed using mixed methods tool- passed all points. | Strength - mixed methods approach for evaluation Limitations - hard to correlate time spent on case with outcomes, limited info about students’ experiences with real patients | Expanding knowledge component of study to better understand specific changes in knowledge |
| Tsai et al 2008 China32 | Assess the impact of a 4-hour multimodule curriculum on knowledge and attitudes of end of life care | Prospective cross-sectional pre and post test survey | 259 fifth year medical students | 4-hour course included: 1 hour lecture by specialist, 1 hour patient visit at unit, 1 hour literature reading, 1 hour discussion | Acted as own comparator, pre test and post test | Questions on knowledge, demographics and ethical beliefs | Knowledge improved (55% to 70%) (p<0.0001). Principles of palliative care scores improved (58% to 73%). Clinical management improved (59% to 68%) | 2b | Cochrane risk of bias tool - low risk of bias, no bias evident in any domains. | Strength - easy to implement curriculum. correlation analysis across items Limitations - hard to control for confounding variables like maturation effect | Further assessment of medical training (residency and clinical practice) – follow-up studies Longitudinal study to better understand changes over time |
| Tse 2017 USA27 | Explore the application of online learning tool with hospice experience | Randomised prospective pre and post study | 152 second year medical students completed the survey (response rate 51%) 56% (n=85) completed the online module | Addition of 30 min online module to hospice experience. Taught by hospice care physician or nurse) in hospice setting | Randomised to receive module prior to hospice experience (YES module) versus after experience (NO module) | 23-item electronic survey: 10 attitude-assessing statements from FATCOD, 8 multiple choice knowledge questions | Higher scores on knowledge questions for students completing the online module (p=0.006). No statistical difference in attitudes | 2b | Assessed using Cochrane risk of bias tool, medium risk of bias. Self-selection bias as voluntary participation, could suggest already motivated regarding palliative care. Randomisation not described | Strengths - mixed methods study, focussed on assessing blended learning experiences Limitations - single site study, survey was relatively few items | Expanding scope of study for more institutions (generalisability) More survey items → more comprehensive assessment |
| von Gunten et al 2012 USA24 | Assess impact, retention and magnitude of effect of a required didactic and experiential palliative care curriculum | Prospective pre and post study | 487 third year medical students | Specified palliative care curriculum designed for 1 day/week for 4 weeks (during the ambulatory block of the 12 week IM clerkship) Taught by IM faculty. participation was compulsory | Self-comparator over time (pre test and post test). knowledge compared with national cross-sectional study comparing residents at progressive training levels | 36-item knowledge test, self-assessment of competency, & self-assessment of attitudes + written surveys | Knowledge: improved 52% to 67% (national residents, average score 62%). 56% improved confidence (higher than resident national averages). 29% decrease in concern. (All p<0.001). All maintained at 1 year. | 2b | Assessed using mixed methods tool- passed all points. | Strength - mixed methods, assess various levels of effect, national comparison Limitations - evaluation instruments designed for specific learning objectives of course. Documentation of long-term follow-up unclear | None outlined by study |
IM, Internal Medicine; MDT, Multidisciplinary Team Meeting; PBL, Problem Based Learning.