Article Text
PDF
Abstract
Palliative care is central to the role of all clinical doctors. There is variability in the amount and type of teaching about palliative care at undergraduate level. Time allocated for such teaching within the undergraduate medical curricula remains scarce. Given this, the effectiveness of palliative care teaching needs to be known.
Objectives To evaluate the effectiveness of palliative care teaching for undergraduate medical students.
Design A systematic review was prepared according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidance. Screening, data extraction and quality assessment (mixed methods and Cochrane risk of bias tool) were performed in duplicate.
Data sources Embase, MEDLINE, PsycINFO, Web of Science, ClinicalTrials.gov, Cochrane and grey literature in August 2019. Studies evaluating palliative care teaching interventions with medical students were included.
Results 1446 titles/abstracts and 122 full-text articles were screened. 19 studies were included with 3253 participants. 17 of the varied methods palliative care teaching interventions improved knowledge outcomes. The effect of teaching on clinical practice and patient outcomes was not evaluated in any study.
Conclusions The majority of palliative care teaching interventions reviewed improved knowledge of medical students. The studies did not show one type of teaching method to be better than others, and thus no ‘best way’ to provide teaching about palliative care was identified. High quality, comparative research is needed to further understand effectiveness of palliative care teaching on patient care/clinical practice/outcomes in the short-term and longer-term.
PROSPERO registration number CRD42018115257.
- adult palliative care
- education & training (see medical education & training)
- general medicine (see internal medicine)
- medical education & training
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- adult palliative care
- education & training (see medical education & training)
- general medicine (see internal medicine)
- medical education & training
Strengths and limitations of the study
This was a rigorously conducted systematic review, including ‘grey’ literature, which evaluated the quality of included studies.
Studies using objective measures of assessment were included; with studies only reporting subjective assessments, self-reports and opinions of participants being excluded. Studies using external ratings as assessment of students were included.
Even using a systematic approach, it remains possible that some studies might have been missed.
Publication bias is possible, as studies yielding negative results are less likely to be published and, although ‘grey’ literature was searched, this may not have fully captured unpublished works.
In view of the variability in interventions and outcomes between included studies, a meta-analysis was not possible.
Background
Palliative care is the holistic care of people with advanced, incurable illnesses, and their families.1 The spectrum of patients receiving palliative care is wide reaching, and ranges from care at the point of incurable illness diagnosis, to the care of dying patients.1 Palliative care is interdisciplinary in nature and involves: symptom control; information sharing with patients; advance care planning; coordination of interdisciplinary input; and care for the families of patients.2 The literature informs us these are the key areas which are deemed important to patients when diagnosed with an advanced and incurable illness.
Medical students and doctors require the appropriate knowledge, skills and attitudes to care for patients who have an advanced and incurable illness. For example, in the UK, it is estimated in their first year of working, newly qualified Foundation Year 1 (FY1) doctors will care for approximately 40 dying patients, and a further 120 patients who are in the last months of life.3 The ability to care for, and communicate appropriately with these patients and their families is an essential skill for all doctors.4
Current medical curricula are saturated,5 and competition for teaching time is fierce. There is an increased drive to incorporate palliative care teaching into medical schools,6 in the hope to improve care for patients. Greater integration of palliative care teaching represents the acknowledgement that care of these patients and those who are dying has room for improvement. Furthermore, an ageing, multimorbid population and a growth in the diversity of palliative treatment options also contribute to the surge in recognition of palliative care’s importance.7 8 Given this increased drive to incorporate palliative care teaching, we need to ensure there is an evidence-base around its effectiveness as justification for its inclusion and/or how to best use this time. Despite this, no contemporary examination of palliative care-related teaching methods exists. The efficacy of various methods has not been recently evaluated, and it is therefore difficult to conclude which methods infer the most benefit on medical students.
Aim
The overall aim of this review was to evaluate the effectiveness of palliative care teaching on medical students.
Methods
This systematic review was designed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Protocol 2015 guidance,9 and registered with International Prospective Register of Systematic Reviews (PROSPERO, CRD42018115257). It is reported according to PRISMA guidelines.10
Search strategy
A search and associated terms were developed with an information science specialist to determine the best search strategy. Studies of palliative care teaching were searched using the terms ‘palliative care’, ‘medical student’, ‘Education, Medical, Undergraduate’ and ‘teaching’. To increase sensitivity, Medical Subject Headings (MeSH) terms and free-text terms were used in searches using the electronic databases Embase (Ovid); Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations; PsycINFO (Ovid); Conference Proceedings Citation Index–Science (Web Of Science; Thomson Reuters, New York City, New York); ClinicalTrials.gov (US NIH); ISRCTN registry (BMC); Cochrane Database of Systematic Reviews (Wiley); Cochrane Central Register of Controlled Trials (Wiley); and Health Management Information Consortium (HMIC) (Ovid). Searches were also conducted for grey literature using the following online databases: the Bielefeld Academic Search Engine (BASE) (https://www.base-search.net/), OpenGrey (http://www.opengrey.eu/) and Mednar (https://mednar.com/). The Embase search strategy is included as a online supplemental file. Search strategies from all other databases are available on request from the authors. Searches were carried out on 06 August 2019.
Supplemental material
Reference lists of relevant articles (included studies and reviews) were hand searched.11 Authors’ personal files were also searched to make sure that all relevant material has been captured. Finally, we circulated a bibliography of the included articles to the systematic review team, as well as to scholarship palliative care clinicians’ experts identified by the team, to ensure any relevant literature was not missed.
Eligibility criteria
Studies evaluating a palliative care teaching intervention directed towards medical students were included (table 1). Where there were mixed study populations and data, studies were only included if data on medical students could be individually extrapolated. To be included, studies needed to demonstrate an objective measure of knowledge or skills (eg, a test score); studies with only self-opinion/self-perspective, reflective essays and qualitative outcomes were excluded.
Eligibility criteria for inclusion or exclusion based on key study criteria
Titles/abstracts and full-text papers were independently screened against pre-defined eligibility criteria (table 1) by two reviewers (JB and either AD/MB). Disagreement at all stages was resolved by consensus and/or with a third reviewer (either JB or AD/MB). The results of the searches were shown in a PRISMA flow diagram (figure 1).
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.
Data extraction
Data were extracted in duplicate (JB and either AD/MB) for the aim, study setting, design, population included, educational intervention and comparator, assessment method used, outcomes, Kirkpatrick Model level,12 study quality, strengths/limitations and ideas for further research (determined by the study authors and reviewers) onto pre-prepared templates.
Quality assessment
The methodological quality of each study was independently assessed by at least two reviewers (JB and either AD/MB). Disagreement was resolved by consensus and/or with a third reviewer (either AD or MB). The mixed methods appraisal tool (MMAT) was used if the study was mixed methods13 and Cochrane risk of bias tool was used if a study was quantitative.14
The MMAT is a critical appraisal tool developed to evaluate studies using both qualitative and quantitative data.15 MMAT was used in line with its original purpose, to appraise mixed methods research and to evaluate non-randomised quantitative research. Two screening questions are asked, before progression to more detailed analysis:
Are there clear research questions?
Do the collected data allow to address the research questions?
In this review, the answer to both of these questions had to be ‘yes’ for a study to qualify for inclusion. Evaluation using MMAT subsequently focussed most heavily on appraising methodology, assessing five core criteria for each study type. These core criteria can be reviewed in detail, with additional usage guidance, using the 2018 iteration of the MMAT tool.15 To aid interpretation of what was meant by the core quality criteria, the research team referred to this expanded guidance. A summary of the core criteria for mixed methods research and non-randomised quantitative research, the ways in which the MMAT was used in this work, are listed in table 2.
Summary of mixed methods appraisal tool (MMAT) core quality criteria for mixed methods and non-randomised quantitative research, adapted from Hong et al15
The Cochrane risk of bias tool was used to appraise any randomised trial studies; as it is the gold-standard for such evaluation.14 The Cochrane risk of bias tool has more stringent appraisal criteria, focussing on evaluating the presence of several types of bias: selection bias; performance bias; detection bias; attrition bias; reporting bias; and other bias. The plausible bias within studies deemed ‘low risk’ were unlikely to seriously alter results and therefore be accepted. Studies at medium risk of bias imply ‘some confidence that the results represent true effect’. Despite medium risk, the issues with these studies are ‘not sufficient to invalidate results’; these studies were therefore included in our review unproblematically.16 Studies rated as high risk of bias should be considered sceptically.
Data analysis and synthesis
Due to the heterogeneity of results, a narrative data synthesis was performed. A team of researchers were involved in the synthesis and development of themes, and analysis of potential biasses and quality. Four stages took part with all members of the research team: (1) development of a theoretical model, (2) preliminary synthesis, (3) exploration of relationships in the data and (4) assessing the robustness of the final synthesis.17
Patient and public involvement
Patients and public were not involved in the planning or design of this systematic review.
Results
The search identified 1446 titles and abstracts for initial screening against the study’s eligibility criteria. Following this, 122 full-text articles were screened in detail for eligibility. Nineteen studies were included (figure 1). The total number of participants in the 19 studies was 3595, data were gained and used from 3253 participants, with long-term follow-up data (up to 1 year) in 274 participants (from three studies). Publication dates were between 2002 and 2018. The number of participants in the included studies ranged from 40 to 670; with a mean of 171.2 participants per study (table 3).
Data extraction table
Quality appraisal
The quality of mixed methods studies were assessed using the MMAT (n=11),13 and purely quantitative studies using a trial type of methodology were assessed using Cochrane risk of bias tool (n=8).14
Overall the 11 mixed method studies included met all required components of quality using the MMAT (table 3).
The Cochrane risk of bias tool was used to appraise any randomised trial studies. Included studies showed a range of bias; one was high risk of bias, five were medium risk of bias and two were low risk of bias (table 3).
Context of included studies
Demographics
The selected studies took part in many countries; nine USA, three Australia, three Germany, two Canada and two China (including Taiwan).
Study designs
Fourteen of the included studies tested knowledge before and after a teaching intervention, in a pre–post design. The post test was immediately post intervention in all but four studies, with one study conducting its post test at 7 weeks, and the other three at approximately 1 year post intervention. Most of these pre–post designed studies were cohort-type studies; one was randomised and three included a mixed methods design. The other five included studies used a randomised controlled design, quasi-randomised controlled trial, historical control trial and two cross-sectional design studies (table 3).
Types of teaching interventions
The included studies had a wide variety of teaching methods and teaching hours. The main shared descriptor of palliative care teaching interventions in the included studies was the duration. Studies could be largely summarised as ‘small’ scale teaching interventions (interventions with a duration of hours) or as ‘large’ scale teaching interventions (interventions that took place over the course of days). Included studies were categorised into these durations, and durations were decided comparatively by the researchers. In addition to these small and large interventions, a third descriptive category was determined: eLearning interventions. Because the nature of eLearning is often associated with uncertain measures of time (depending on student use outside of learning environment), eLearning interventions were considered to be different than small or large face-to-face teaching interventions. Given the variance in shared descriptors, the decision was made to synthesise results based on the type of intervention: small, large or eLearning.
Different assessment methods
The studies used different assessment methods and some studies used multiple methods of assessment (table 3); this made it difficult to assimilate study outcomes. Most commonly, multiple choice questions (MCQs) were used to test knowledge18–24 or a combination of MCQs and true/false questions.25 The number of items testing knowledge differed between studies. These ranged from 6 single best answer items,26 8 MCQs27 to 50 MCQs.23 Other methods of assessments included an ‘external intensivist’ rating student performance based on a taped role play28 and observed structured clinical examination (OSCE) station assessment.29 Some studies also assessed student attitudes and confidence in a pre–post format.19 21 24 25 27 30 31
Synthesis of results
Smaller teaching interventions
Seven of the included studies evaluated a ‘small’ palliative care teaching intervention; these included a range of interventions of different sizes, from 1.5 to 10.5 hours, with a median of 4 hours.19 20 28 32–35 Six of the seven included studies showed statistically significant improvements in knowledge assessment outcomes (table 3),20 28 32–35 and one of these studies included a 1-year follow-up, with knowledge retention demonstrated.33 Although one study did not show overall improvement in knowledge scores, it did demonstrate statistically significant improvements in symptom management scores in a subset analysis.19
Larger teaching interventions
Seven of the included studies evaluated a ‘large’ palliative care teaching intervention, with interventions ranging from 4 to 5 days, with a median of 5 days (table 3). Six of the seven large scale studies demonstrated statistically significant improvements in knowledge assessment outcomes; although one of these had a poor comparator.29 One study failed to demonstrate an improvement in knowledge from mandatory participation in a clinical palliative care module compared with didactic teaching alone.18 There were critical limitations in the comparator used in the study by Ellman et al.29 Ellman et al developed a new palliative care OSCE to assess student knowledge regarding symptom management, communication and the psychosocial, spiritual and cultural aspects of care. Competency in this OSCE station was deemed adequate by the authors (average score 74%) although the level attained at this station was below that of other OSCE stations; which was on average 84%.29 There was also no pre and post intervention testing, thus it is unclear if this intervention improved knowledge or not.
eLearning teaching interventions
Five studies evaluated the effect of eLearning on knowledge in palliative care, with all these studies demonstrating statistically significant improvements in knowledge scores (table 3). The specific type of eLearning varied, but included: a virtual patient clinical case,36 a computer-based decision aid for advance care planning content,25 a flipped classroom online module coupled with a hospice care experience,27 and an eLearning course.22 The fifth study, an interactive e-learning course, is notable because it reported equivalence in increasing knowledge scores, when compared with small-group teaching sessions.26 Of the eLearning studies included, this is the only one to provide a comparator to the eLearning resource. However, the study still considered the eLearning intervention to be ‘successful,’ as it was determined to be less faculty intensive to run but imparted the same degree of knowledge as ‘traditional’ teaching.26 Overall, all eLearning interventions offered flexibility for students.
Summary
Overall, the majority (n=17) of the included studies demonstrated an improvement in knowledge. Small amounts of specific teaching improved knowledge in six out of seven studies. Similarly, large amounts of teaching improved knowledge in six out of seven studies. All eLearning interventions improved assessment outcomes in tests of knowledge. No included study directly compared small and large teaching interventions and, as study outcomes were heterogenous, it was not possible to evaluate whether small or large interventions were ‘better.’
Discussion
This systematic review presents a contemporary overview of the literature regarding the effectiveness of palliative care teaching to medical students. All types of teaching intervention (small-scale and large-scale teaching, clinical and eLearning) improved knowledge scores for medical students. No method appeared to be superior in improving knowledge. Few studies explored knowledge retention, skills or attitudes. No studies explored the impact of teaching on clinical care for patients. Significant heterogeneity of teaching approaches continues to exist, and is increasing, as new teaching methods (such as eLearning) develop and grow in popularity. Further contributing to the heterogeneity was the inconsistency of overall teaching approaches and methods of assessment in all included studies. This leads to the hypothesis that, regardless of the style of teaching, improvement in palliative care knowledge scores is possible following teaching. Study designs, too, differed significantly, with no consistent approach to long-term follow-up. In view of the multifaceted heterogeneity evident in both study design and outcomes, the data gathered systematically were synthesised narratively.17
Outcomes and constructive alignment considerations
Examining the intervention efficacy with an educational theory lens was the logical first step in performing a narrative synthesis of included articles in this particular review. One of the first theories to consider in any study measuring knowledge via assessment is Biggs’ theory of constructive alignment.37 38 Constructive alignment argues that there needs to be alignment of learning outcomes, teaching methods and assessment measures, otherwise, true learning may not occur. For example, if an educator presents learning outcomes to students related to palliative care, but then teaches a session on dermatology, and gives an assessment with questions concerning cardiology, you would expect students to not pass their assessment, and conclude learning did not occur. However, in this admittedly bizarre example, learning might have occurred; it just may have been related to palliative care, or most likely dermatology. Yet, because these educational components are not constructively aligned, it would be impossible to actually comment on learning. This same reasoning can be applied to the studies included in this review. Many studies determined learning occurred, as exemplified by improvement in knowledge scores. However, one issue when conducting this review was the inability to know with any certainty how related teaching and assessment were to one another. It was not made clear by the analysed studies how constructively aligned their assessment was to the palliative care teaching delivered. It was clear that some short interventions were geared to improve a specific aspect of palliative care (eg, advanced care planning),25 but most larger interventions (where details were published and we could discern more exact content of the teaching), covered a range of topics in the palliative care curriculum. Poor detail regarding the content of assessment, and limited assessment regimens, makes it seem likely only some of these topics were formally assessed.
Failure to explicitly acknowledge constructive alignment within any of the included studies makes it difficult to accurately assess the efficacy of any (especially the large) teaching interventions. Reproducibility of the value of the interventions will likely largely depend on specific variables relating to constructive alignment. Utilisation of constructive alignment in teaching intervention design and assessment may have been an influencing factor as to whether an intervention improved knowledge scores. However, without discussion of this in any of the studies, it is not possible to know whether constructively aligned learning outcomes, teaching and assessment are important to effective palliative care teaching.
Impact of teaching interventions
Kirkpatrick’s Four-Level Training Evaluation Model is used to evaluate the results of educational programmes, which are divided into four levels (figure 2).12 This model was used to evaluate the impact of interventions in the included studies.
Kirkpatrick’s four-level training evaluation model. Reproduced from.41
Included studies in this review were mostly at level 2 of Kirkpatrick’s Four-Level Training Evaluation Model; what students have learnt.12 The only study to assess Behaviour (Level 3) was by Green et al25 where patient satisfaction was evaluated in an advance directive scenario. This introduces the concept that for many of these teaching interventions, their potential efficacy has really only been assessed from a limited viewpoint. Although changes in knowledge and attitude are important, they do not guarantee the educational experience will change behaviour/practice. Measuring the clinical impact of a teaching intervention requires rigorous long-term follow-up, and such follow-up was not performed by any studies within this review. Thus, no conclusions regarding the impact of these palliative care teaching interventions on clinical practice or patient outcomes can be made. This is particularly important as with growing demands and need for quality palliative care in practice, it is important to understand if medical school interventions are actually improving later clinical practice, or long-term decisions of medical students. Studies suggest there are many misconceptions by lay and healthcare professionals of what palliative care is/hospices are, and thus one of the main aims of undergraduate teaching should be to try and dispel these.39 40 This was not explored in any of the studies.
Heterogeneity might indicate wide possibilities for curricular design
While the effect of palliative care teaching on clinical practice could not be elucidated from this review, there was significant information relating to potential knowledge gain and exposure via palliative care teaching interventions. While there was significant heterogeneity in how knowledge was measured in these studies, interesting findings were identified. Both small amounts of specific teaching and larger scale interventions improved knowledge, which may support the argument that institutions should investigate integrating some level of teaching palliative care, even if small, as these can prove beneficial to the knowledge base for students. This is supported by the fact that in these studies, regardless also of the teaching method, improvement in palliative care knowledge scores was possible following instruction. Again, this provides more evidence that while there seems to be no identifiable ‘best practice’ for teaching palliative care in medical education (as no studies compared this or asked this question, and knowledge scores used by different studies was not the same), this means that institutions can adapt from a variety of methods that may work best for their curriculum. eLearning also appeared to improve knowledge scores in studies included in this review. One study demonstrated the potential value of integrated eLearning with existing clinical experiences; a small, online module provided to students prior to a hospice experience demonstrated improved knowledge among these students.27 This study, and the others relating to eLearning, contribute to the possibility that any type of palliative care teaching may be very beneficial, even with the need for more focussed and detailed research.
Strengths and limitations of the systematic review
This is a rigorously conducted systematic review designed using PRISMA Protocol 2015 guidance,9 and reported according to PRISMA guidelines.10 It included ‘grey’ literature and evaluated quality of the studies and impact on clinical practice. However, it is possible some studies might have been missed and publication bias is possible, as if studies were not available then they would not have been included. Different reviewer expertise brought diversity to the team and ensured a multi-angled perspective. The systematic review drew on the international literature studying medical student education about palliative care. As such, it is generalisable and applicable to an international audience.
In view of the variability in interventions and outcomes between included studies a meta-analysis was not possible, and a narrative synthesis was performed. Risk of bias was assessed by two different tools, depending on the study type. The mixed methods tool was used if the study was mixed methods as this tool was not applicable to purely quantitative work.13 Cochrane risk of bias tool was used if a study was purely quantitative.14 The Cochrane risk of bias is designed for randomised controlled trials so some aspects of appraisal, like allocation concealment, often weren't applicable for the included quantitative studies.14
This review primarily used objective measures of assessment and excluded subjective assessments, self-report and opinions of participants. However, studies using self-report of external people were included. External rating is still subjective but is an external outcome measure.
Limitations of included studies
The main limitation of the included studies is that none assessed effect on clinical practice and patient outcomes. Thus, the effect on clinical practice of each teaching intervention is unknown. Only three studies undertook follow-up and collected long-term data; this was on 274 students. Thus, only a small portion of participants are represented in this data. ‘Long-term’ in this sense encompasses follow-up within 1 year. No studies provided follow-up data beyond this point, a limitation of all included studies. None of the included studies compared the impact of small versus large scale interventions, meaning that, although most interventions were effective, it is unknown whether large-scale or small-scale teaching or eLearning interventions are more effective in instilling palliative care knowledge.
Future work
Our review highlights the need for future research to evaluate the differential impact of small and large interventions, whether interventions elicit behavioural changes and the impact of teaching on clinical practice during long-term follow-up. Impact of teaching on patient care also requires study and could be based on markers of clinical assessment, management and patient/family feedback.
Conclusions
Most types of palliative care teaching interventions conducted with medical students improve knowledge. This provides useful information for medical schools when considering the teaching they currently provide, or aim to provide, in the future. The effect of undergraduate palliative care teaching on clinical practice has not been studied and warrants investigation. For all teaching approaches, constructive alignment and the communication of constructive alignment in educational studies should be considered to ensure adequate teaching impact. Further research into palliative care teaching should explicitly detail this alignment to allow for evaluation as to whether constructive alignment, not the teaching method, may be responsible for any effect of palliative care teaching interventions.
Medical students can learn about palliative care using a variety of methods; there is no definitive ‘best’ way to learn about palliative care. We have the responsibility to not just train medical students to pass exams, but to be safe and knowledgeable doctors. Given this, future research needs to assess the effect of teaching on clinical practice, including some analysis of patient-related outcomes, in order to discern the real-world impact of palliative care teaching interventions.
Acknowledgments
We would like to thank Helen Elwell, clinical librarian, for her guidance and assistance on the use and refinement of search terms and searches.
References
Footnotes
Twitter @Megan_EL_Brown, @gabs_finn
Contributors JB designed the study, performed the searches, led on data collection, data analysis and drafted the article. MB and AD contributed to data collection, data analysis and writing of the article. JG and GF contributed to study design, analysis and writing of the article. All authors were responsible for approval of the final report.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.