Objective The purpose was to analyse the effectiveness of high-fidelity patient simulation (HFPS) based on life-threatening clinical condition scenarios on undergraduate and postgraduate nursing students’ learning outcomes.
Design A systematic review and meta-analysis were conducted based on the Cochrane Handbook for Systematic Reviews of Interventions and its reporting was checked against the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist.
Data sources PubMed, Scopus, CINAHL with Full Text, Wiley Online Library and Web of Science were searched until July 2017. Author contact, reference and citation lists were checked to obtain additional references.
Study selection To be included, available full-texts had to be published in English, French, Spanish or Italian and (a) involved undergraduate or postgraduate nursing students performing HFPS based on life-threatening clinical condition scenarios, (b) contained control groups not tested on the HFPS before the intervention, (c) contained data measuring learning outcomes such as performance, knowledge, self-confidence, self-efficacy or satisfaction measured just after the simulation session and (d) reported data for meta-analytic synthesis.
Review method Three independent raters screened the retrieved studies using a coding protocol to extract data in accordance with inclusion criteria.
Synthesis method For each study, outcome data were synthesised using meta-analytic procedures based on random-effect model and computing effect sizes by Cohen’s d with a 95% CI.
Results Thirty-three studies were included. HFPS sessions showed significantly larger effects sizes for knowledge (d=0.49, 95% CI [0.17 to 0.81]) and performance (d=0.50, 95% CI [0.19 to 0.81]) when compared with any other teaching method. Significant heterogeneity among studies was detected.
Conclusions Compared with other teaching methods, HFPS revealed higher effects sizes on nursing students’ knowledge and performance. Further studies are required to explore its effectiveness in improving nursing students’ competence and patient outcomes.
- high fidelity simulation training
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This meta-analysis provides data on the impact of high-fidelity patient simulation sessions based on life-threatening clinical scenarios on knowledge, performance, satisfaction, self-confidence and self-efficacy in undergraduate and postgraduate nursing students.
A structured search strategy was used across multiple databases.
Data heterogeneity and limited amount of high-quality primary studies limit the generalisability of results in nursing education practice.
Healthcare systems and health needs of general population worldwide require newly registered nurses to have adequate knowledge, skills and attitudes in order to be ‘fit for practice’.1 2 The clinical training of nursing students plays an essential role in the learning process during undergraduate courses,3 but the unpredictable nature of the clinical training environment can generate risk of error potentially harmful for both nursing students4 5 and patients.6 7 Since available evidence assume that the safety for both patients and learners rises together with the growth of students’ clinical expertise,4–8 an active learning method may allow nursing students to practice clinical procedures learnt in theory and patients to receive best-quality safe care.9 10 Unfortunately, the organisational issues and short rotations in clinical settings do not always allow nursing students to train in an interactive way especially in high-risk, low-incidence clinical events.11 All these reasons have generated the need for integrative teaching methods, such as high-fidelity patient simulation (HFPS). The HFPS, especially when performed according to acknowledged standards,12 uses technologically improved manikins that are able to breathe, talk and have both heart and lung sounds, programmed by algorithms or dynamic ‘off-the-cuff’ instructions to replicate the physiological parameters in normal or deteriorating patients.13 This method allows for giving and receiving feedback on repeated actions permitting the shift from theory to lived experience for the student within a safe learning environment rich with opportunities.14 15 The use of high-fidelity patient simulators has been shown to improve nursing students’ learning outcomes, such as satisfaction, self-confidence and self-efficacy,16 as well as knowledge and performance17 18 by means of deliberate practices, feedback opportunities and gradually augmented task difficulties.19 Moreover, the usefulness of the forgiving nature of the simulation environment is often acknowledged and appreciated by students who experience high-fidelity simulation sessions.16 Consequently, HFPS has become an important learning strategy in nursing education3 6 20 21 since it provides the opportunity to frequently experience acute clinical situations without risk to the patient or learner.20 22 23
Although primary studies widely documented the potential of HFPS to improve nursing students’ learning outcomes,18 24 25 literature did not focus on the effectiveness of the simulation when based on life-threatening clinical scenarios referred to different clinical settings. Therefore, considering the increase of published studies on the effectiveness of HFPS in academic nursing education, a systematic analysis of these studies is expected to allow the development of guidelines in this field.
The aim of this systematic review was to analyse the effectiveness of HFPS based on life-threatening clinical condition scenarios in improving the learning outcomes of knowledge, self-confidence, satisfaction, self-efficacy and performance for undergraduate and postgraduate nursing students.
A systematic review and meta-analysis were conducted based on the Cochrane Handbook for Systematic Reviews of Interventions26 and its reporting was checked against the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist.27
Eligibility and inclusion criteria
In order to be included in this analysis, the abstract had to clearly indicate the study (a) was experimental or quasi-experimental, (b) had used HFPS and (c) had involved nursing students (undergraduate or postgraduate). Available full-texts had to be published in English, French, Spanish or Italian language and studies had to include (a) HFPS based on critical care scenarios, (b) control groups not tested on the HFPS before the intervention, (c) data on the learning outcomes of performance, knowledge, self-confidence, self-efficacy or satisfaction measured just after the simulation session, and (d) data for meta-analytic synthesis. For the purpose of this systematic review, the concept of knowledge was intended as delivery of the theoretical basis of caring,28 self-confidence is defined as trusting the soundness of one’s own judgement and performance,23 satisfaction is considered the fulfilment of student’s expectations during the simulation experience,29 self-efficacy consists of the way students perceive, think and motivate themselves when learning and performing clinical training30 and finally, performance is the student’s ability to demonstrate clinical skills.31
Information sources and search
A pilot search was performed to identify keywords and Medical Subject Headings relevant for the electronic research. PubMed, Scopus, CINAHL with Full Text, Wiley Online Library and Web of Science were searched until July 2017 using the search strategies listed in the box of the online supplementary file. To perform an exhaustive search, reference and citation lists from included studies were checked for other relevant references. Thomson Reuters EndNote X7 was used for the management of the retrieved studies and references.
Titles and abstracts were screened by three raters (CLC, AD and VC) for eligibility according to the listed criteria and, for each eligible study, full-texts were retrieved by using online databases and faculty interlibrary service as well as by contacting the authors. Full-texts were analysed by two raters (CLC and AD) for their inclusion in the review based on the described criteria. Both in the eligibility and inclusion stage, the agreement among the judgements of the authors (inter-rater reliability) was estimated with the Krippendorff’s alpha coefficient ranging from 0 (totally disagree) to 1 (totally agree).32 Any disagreement between the raters was resolved by discussion until consensus was reached.
Data collection process
For the purposes of this systematic review, a coding protocol was designed by the research team and developed with a spreadsheet built with Microsoft Excel. To obtain an accurate version of the tool, the form was tested independently by two authors (CLC and AD).
Data items and quality appraisal of individual studies
Data related to year of publication, study design, country, sample size, participants characteristics, simulator features, control conditions, scenarios, outcomes and measurement tools and time of exposure to scenarios were extracted independently by two authors (AD and CLC). Krippendorff’s alpha coefficient was used to calculate inter-rater reliability and any disagreement about data extraction was resolved by discussing with a third author (LL) to gain consensus.32 The study designs were checked with ‘List of study design features’.26
The included studies were screened for their methodological quality through the Quality Appraisal Checklist for Quantitative Intervention Studies designed by the National Institute for Health and Care Excellence33 shown in the table A of the online supplementary file. To provide a global measure for both external and internal validity, the most frequent judgement was used. The quality of the studies was not deemed to be an exclusion criterion.
Synthesis of results and summary measures
For each study, the outcome data were synthesised through meta-analytic procedures using the software ProMeta V.3.0. The random-effect model was used for all studies as a conservative approach to account for different sources of variation among studies (between-studies and within-study variance).34 35 As Cohen’s d (standardised mean difference) permits meta-analysis even when studies have used different original measures, it was directly computed or derived.36 37 In this regard, standardisation has been the only way to carry out a meta-analysis, considering multiple measurement instruments found in included studies.37 Effect sizes were pooled across studies to obtain an overall effect size with the inverse-variance method. For each effect size, the corresponding 95% CI, weight and statistical significance were calculated. The pooled effect size significantly favoured the HFPS when Cohen’s d was higher than ‘0’ and its 95% CI did not overlap the 0line. Values of Cohen’s d can be interpreted as a small effect (0.2), medium effect (0.5) and large effect (0.8).37 In order to assess the significance of the difference between the means of HFPS and the other teaching methods, a Z-test was performed for each meta-analysed outcome. The historical trends from the searched databases were graphed.
Risk of bias across studies and additional analyses
In order to evaluate the influence of each study on the overall effect sizes and to verify the robustness of the results, sensitivity analysis was undertaken through the leave-one-out approach.26 Publication bias was examined by the Egger’s regression,38 Trim and Fill and the fail-safe number methods were used to assess the effect of publication bias on effect size.39 Since robust eligibility criteria were adopted and the reliability of data extraction was guaranteed by a multi-rater approach, data were presented considering any acceptable level of heterogeneity, which was checked and measured with Q-test and I2 and explored through sub-group analyses,40using the ‘scenario’, ‘manikin brand’ and ‘control intervention’ as moderators. ProMeta V.3.0 and IBM SPSS V.19.0 were used for data analysis.
Patient and public involvement
This review had no contact with patients. All information was obtained from published studies.
The search produced 2603 references from databases and 1857 studies from reference and citation searching, all published until July 2017. After removing duplicates, 2130 abstracts were screened for relevance. Consequently, 492 full-texts were analysed and 459 studies were excluded for not meeting the inclusion criteria (figure 1).
Inter-rater reliability among the authors for abstracts and full-texts was 0.84 and 1.00 (Krippendorff’s alpha coefficient), respectively, before consensus among authors was reached. The final sample of 33 studies originating 44 comparisons was included in this systematic review, as shown in the table B of the online supplementary file. It should be noted that a significant increase in the general number of studies (R2=0.835; p<0.001) occurred over the last 30 years about HFPS (figure 1 of the online supplementary file).
Detailed information about study characteristics are presented in the table C of the online supplementary file. Summaries about more significant features of included studies are presented as follows.
The overall sample of nursing students (n=3042) showed sample sizes varying from 17 to 352 participants composed of undergraduate (n=2607; 85.7%) and postgraduate students (n=435; 14.3%) and had a mean age of 25.7 (SD 5.8). Just over half of the studies (n=19; 57.6%) were conducted in North America (USA n=15, 45.5%; Canada n=4, 12.1%), three studies (9.1%) in Europe (UK n=2, 6.1%; Portugal n=1, 3.0%), five studies (15.1%) were conducted in South Korea, three studies (9.1%) in Jordan, while three studies (9.1%) in other countries (Australia, Singapore and Turkey). Students in their fourth year of undergraduate courses (n=922; 30.3%) were represented in ten studies conducted in Canada, Portugal, USA, South Korea and Jordan. Most studies did not provide descriptive statistics related to gender.
Interventions and comparisons
Studies used a variety of both HFPS (intervention group) and other teaching methods (control group). Most of the simulators used in the intervention groups by qualified instructors or tutors were Laerdal (n=16; 47.1%). Simulation sessions were based mainly on cardio-circulatory scenarios (n=30; 54.5%), followed by respiratory scenarios (n=16; 29.1%) and others (n=9; 16.4%). Among the control group interventions, more than one-third used lectures (n=14; 31.1%), no intervention (n=11; 24.4%) or low-fidelity manikin (n=5; 11.1%).
The subjective outcomes (satisfaction, self-confidence and self-efficacy) were measured by self-rating instruments (eg, Resuscitation Self-Efficacy Scale, Satisfaction with Clinical Experience Simulation Scale, etc), whereas the objective outcomes (knowledge and performance) through direct observation of performance by raters or other objective instruments (eg, Advanced Cardiac Life Support [ACLS] Mega Code Performance Score Sheet, ACLS Written Examination, etc), as shown in table C in the online supplementary file. Different types of measurement tools were detected including Likert-type scales (n=25 430.9%), multiple-choice questionnaires (n=11; 19.3%), dichotomous scales (n=7; 12.3%), checklists (n=3; 5.3%), open questions (n=1; 1.7%) and others (n=10; 17.5%).
Type of studies
Most studies included in this meta-analysis were based on a quasi-experimental design with a pseudo-randomised allocation to groups (n=29; 87.9%) while the remaining studies (n=4; 12.1%) were randomised controlled trials. The included studies were published from 2006 to 2017 and their design features and extracted data are available for consultation in the tables C and D of the online supplementary file.
Quality appraisal of individual studies
Good internal validity was reported for all included studies (table E of the online supplementary file), while 42.4% of the studies (n=14) demonstrated good external validity and just over half (n=19) depicted a scarce generalisability of the results mainly due to lack of details concerning the process of recruiting participants (57.6%).
Results of individual studies and synthesis of results
HFPS sessions showed significant larger effects sizes for knowledge (d=0.49, 95% CI [0.17 to 0.81], Z-test=3.06, p=0.003) and performance (d=0.50, 95% CI [0.19 to 0.81], Z-test=3.12, p=0.001) than any other teaching method (figures 2 and 3). No significant differences were detected between HFPS and control groups for the satisfaction (d=0.38, 95% CI [−0.01 to 0.77], Z-test=1.90, p=0.053), self-confidence (d=0.21, 95% CI [−0.02 to 0.43], Z-test=1.75, p=0.072) and self-efficacy (d=0.05, 95% CI [−0.45 to 0.55], Z-test=0.20, p=0.840) (figures 4–6).
Since Q-test highlighted a significant heterogeneity (p≤0.01) for all the outcomes (I2 from 70.09% to 89.85%), subgroup analyses were carried out to determine its source (table 1). The scenario (analysis of variance [ANOVA] Q-test 11.43, p=0.003), manikin brand (ANOVA Q-test 10.59, p=0.001), and control intervention (ANOVA Q-test 13.37, p=0.010) appeared to be the source of heterogeneity for self-efficacy. Otherwise, these moderators did not prove to be the sources of heterogeneity for the remaining learning outcomes.
In regards to the objective outcomes, such as knowledge and performance, the strength of the pooled effect sizes was still robust and significant (ranging from 0.38 to 0.58 and from 0.43 to 0.57, respectively) and did not significantly differ according to the characteristics of individual studies in the leave-one-out sensitivity analysis.
Regarding the self-rating outcome of satisfaction, the pooled effect size became significant by removing Kang,41 Luctkar-Flude42 or Luctkar-Flude (0.51, p=0.002; 0.48, p=0.018; 0.42, p=0.047; respectively). Even about the self-confidence, the pooled effect size became significant when Ahn, Brannan,43 44 Kang,41 Luctkar-Flude42 or Luctkar-Flude were removed (all 0.25, p value from 0.027 to 0.032). The last self-rating outcome, that is self-efficacy, did not show any change of the effect size that remained not significant in all cases (ranging from −0.13 to 0.26).
Risk of bias
With the exception of self-efficacy, no significant publication biases were detected on performed tests measuring knowledge, performance, satisfaction and self-confidence. For self-efficacy the Egger’s regression showed a significant risk of publication bias (intercept=−6.54, p=0.018), even if no change in the effect size was found by the Trim and Fill method between the observed and estimated values (d=0.05, 95% CI [−0.45 to 0.55]), as shown in figure 2 of the online supplementary file. The fail-safe number was 0.
In this review, a significant increase in HFPS research based on life-threatening clinical condition scenarios was detected over the years, which recognises simulation-based education as a key component of nursing education45 46 especially for life-threatening clinical conditions requiring rapid and effective interventions. Although a positive publication trend on this topic emerged, most of the research had been conducted in North America. Consequently, generalisability of results in Europe and Asia is limited given the differences in many academic and curriculum aspects.47
In accordance with global health concerns,48–50 life-threatening clinical condition scenarios used in HFPS sessions were mainly based on cardio-circulatory and respiratory clinical problems that allowed students to manage high-risk situations rarely practically faced during their clinical training.11 In this regard, to comprehend if patients will receive better and safer care due to the improvement on learning outcomes in nursing students produced by HFPS, translational research on this topic should be strengthened.
Given the emerging variety of measurement tools (eg, Likert-type, multiple-choice, etc), research methods on this topic should be more focused and rigorous. In particular, ad hoc scenario-specific instruments with reported reliability and validity should meet the minimum general requirements of global-shared guidelines in order to have comparable results.51 Standardisation of their core contents is strongly advisable.20 52
Considering these issues, this meta-analysis should be read cautiously considering that few included studies had a good external validity and adopted a randomised controlled design. Therefore, conducting high-quality replication studies on this topic is recommended.
HFPS and nursing students’ learning outcomes
This systematic review analysed the effectiveness of HFPS using life-threatening clinical condition scenarios on nursing students’ learning outcomes. In accordance with other reviews conducted on this topic,18 24 25 although with different aims and populations, HFPS seems to improve students’ knowledge19 34 36 40 41 53–59 and performance,34 42 44 55 60–69 that are considered objective outcomes in current literature.70 Considering that competence can be defined as knowledge and performance combined with psychomotor and clinical problem-solving skills,71 HFPS can be considered an important teaching method that can contribute to build nursing competence especially in the area of critical care. Engaging in simulated life-threatening clinical condition scenarios, students could improve their ability to provide appropriate and safe nursing care in patients with unstable and rapidly changing clinical conditions. However, it is not enough for nursing students to just demonstrate good knowledge and performance to completely achieve their learning outcomes as well as securely meet the needs of the critically ill patient.
In regards to subjective outcomes,70 nursing is an aid profession and patients need to feel safe and reassured, therefore, adequate levels of self-confidence and self-efficacy30 are required in order to improve the well-being of nurses that is closely linked to the quality of care provided. However, this review does not confirm the benefits of HFPS based on life-threatening clinical condition scenarios in improving nursing students’ self-efficacy,42 68 72 73 self-confidence41–44 54 56 59 60 68 72 74–78 and satisfaction.41 42 53 54 75 77 79–82 Maybe, non-significant results for these learning outcomes are due not only to the small sample sizes of some included studies, but also to the outcome measurement performed immediately after any single simulation experience, not allowing the detection of any change. To achieve significant improvements in self-efficacy and self-confidence, it may be useful to provide students with repeated exposures to the HFPS sessions in order to maintain successful performances over time and allow them to observe the success of the other students to increase encouragement and engagement.30 83 84 Hence, future studies should use repeated exposures to the HFPS with outcome evaluation during both intermediate and long-term intervals. The increased use of HFPS in nursing education programmes may result in more clinically confident and proficient nurses who are able to respond accurately and appropriately to patients’ needs.85 To better understand how the gain in performance and knowledge improves patient outcomes, more research based on translational approach is required.52
The results from this meta-analysis were affected by a high heterogeneity and was not explained by those variables except for self-efficacy and was likely due to the different application methods of HFPS across several context of the studies. Unfortunately, most studies did not provide data useful to exploring the reasons for the heterogeneity that represents both a threat to the reliability of the results86 and an opportunity to provide a quantitative proof of the methodological limitations in the current research.
The unexplained heterogeneity detected from this meta-analysis have a surprising usefulness in orienting future research to provide evidence-based responses to various unsolved questions related to the ability of HFPS to improve nursing learning outcomes. Further details are needed in regards to how long should a simulation session last? What are the best briefing and debriefing methods? What are the most effective facilitation methods to use during the simulation? What is the ideal number of participants in each session? Even if many studies have been conducted in these fields and also there are standards of best practice in simulation,12 17 25 51 the results of this meta-analysis highlighted that a high heterogeneity in simulation practice and research persists.87 Therefore, further studies using shared HFPS practice and investigation methods are needed to achieve more homogeneity in literature in order to allow the establishment of evidence-based guidelines, protocols and algorithms88 89 that interrupt the vicious circle in which the lack of homogeneity in the behaviours determines a heterogeneity of the results and vice versa.
This systematic review analysed the effectiveness of HFPS through life-threatening clinical condition scenarios on nursing students learning outcomes. The robustness of the results was confirmed for knowledge, performance and self-efficacy after sensitivity analysis; however, some limitations were revealed.
Even if a good internal validity was reported for all the included studies, only few researches were based on an experimental design. Consequently, as likely and unmeasurable confounding and selection bias could be present in no experimental included studies, the results of this meta-analysis should be cautiously considered also in the light of the relevant heterogeneity. In addition, the basic knowledge of the postgraduate students hypothetically higher than the undergraduate students could have potentially affected the effect size of the considered outcomes. Publication bias detected for self-efficacy was probably due to negative studies less likely to be published or to a more attention paid by editors to manuscripts investigating objective than self-rating outcomes; consequently, caution in the interpretation of the results is necessary. Finally, lack of data about the participants’ characteristics, measurement tools, duration of the session and briefing and debriefing modalities limit the analyses and interpretation of the results.
Results of this systematic review demonstrate HFPS is superior to other teaching methods in improving knowledge and performance of nursing students when exposed to life-threatening clinical condition scenarios, corroborating the importance of HFPS into the academic educational programmes especially for the management of clinically acute events. However, more studies are still necessary to explore the potential use of the HFPS as an effective tool to increase nursing students’ competence levels and to better understand its impact on patient outcomes.
Contributors All authors developed the protocol, interpreted the results, and approved the final version. CLC, AD, IF, EG, and VC completed the search, screened articles for inclusion, and synthesised the findings. CLC and AD extracted data and drafted the manuscript. CP, CMA, and LL critically revised the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
Patient consent for publication Not required.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.