Article Text

Original research
Extracurricular work experience and its association with training and confidence in emergency medicine procedures among medical students: a cross-sectional study from a Norwegian medical school
  1. Remi William Scott1,
  2. Knut Fredriksen1,2
  1. 1Department of Clinical Medicine, UiT The Arctic University of Norway, Tromsø, Norway
  2. 2Division of Emergency Medical Services, University Hospital of North Norway, Tromsø, Norway
  1. Correspondence to Dr. Knut Fredriksen, Department of Clinical Medicine, UiT The Arctic University of Norway, Tromsø, Norway; knut.fredriksen{at}uit.no

Abstract

Objective Proficiency in basic emergency procedures is important for junior doctors, but the amount of practical exposure may vary. We studied the association between students’ extracurricular healthcare-related (ECHR) work experience and self-reported practical training and confidence in selected emergency medicine procedures.

Study design Cross-sectional study.

Materials and methods Medical students and first-year graduates answered a Likert-based questionnaire probing self-reported amount of exposure to (‘training amount’) and confidence with selected emergency medicine procedures. Participants also reported ECHR work experience, year of study, previous healthcare-related education, military medic training and participation in the local student association for emergency medicine (Tromsø Acute Medicine Students’ Association (TAMS)). Differences within variables were analysed with independent samples t-tests, and correlation between training amount and confidence was calculated. Analysis of covariance and mixed models were applied to study associations between training amount and confidence, and work experience (primary outcomes) and the other reported factors (secondary outcomes), respectively.

Results 539 participants responded (70%). Among these, 81% had ECHR work experience. There was a strong correlation (r=0.878) between training amount and confidence. Work experience accounted for 5.9% and 3.5% of the total variance in training amount and confidence (primary outcomes), and respondents with work experience scored significantly higher than respondents without work experience. Year of study, previous education, military medic training and TAMS participation accounted for 49.3%, 8.7%, 6.8% and 23.6%, and 58.5%, 5.1%, 4.7% and 12.3% of the total variance in training amount and confidence, respectively (secondary outcomes). Cohen’s D was 0.48 for training amount and 0.32 for confidence level, suggesting medium and weak medium-sized associations with work experience, respectively.

Conclusion ECHR work experience is common among medical students and was associated with more training amount and higher confidence in the procedures. Year of study, previous relevant education and TAMS participation, but not military medic training, were also significantly associated with training amount and confidence.

  • accident & emergency medicine
  • medical education & training
  • quality in health care

Data availability statement

Data are available in a public, open access repository. Technical appendix, statistical code and dataset available from UiT Open Research Data (doi: https://doi.org/10.18710/O1ZOQ0).

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • 539 of 766 participants responded, that is, a response rate of 70% .

  • Besides extracurricular healthcare-related work experience, other factors of potential relevance (year of study, previous education, military training and participation in medical student association) were investigated; however, unknown confounders cannot be excluded.

  • The study reports self-perceived procedure proficiency—and lack of such—pointing at areas for potential improvement of practical skills training in medical schools.

  • Self-report bias might have influenced the findings as no objective evaluation of the participants’ actual performance level was performed, and self-perceived confidence does not necessarily equate with actual competence.

  • The study included a single medical school, and this may affect the external validity.

Introduction

While theoretical knowledge is thoroughly covered and assessed throughout medical school, several authors argue that practical expertise has deteriorated substantially over the last decades.1–5 This may differ between study programmes, but the overall impression is that the graduates’ clinical skills have become a matter of concern.5 Several studies within emergency medicine report student inadequacy in first aid and basic life support skills.6–10 While student confidence and competence with such procedures have been thoroughly investigated, external factors such as extracurricular healthcare-related (ECHR) work has not been in focus. Knowledge of such factors is important both for proper guidance of students longing for practical proficiency and for medical schools wanting to facilitate increased procedural competence among their students.

In 1996, the Medical School Objectives Project was established because of growing concerns regarding lack of practical competence in physicians, with the objective of developing a list of key procedures that all graduating medical students should be able to perform independently.11 Unfortunately, subsequent studies suggest that the goal of achieving undergraduate proficiency for these procedures remains unmet.12–14 Even more worrying, students’ performance level seems to be below stakeholders’ expectations,10 15–17 and students themselves report a discrepancy between desired and actual competence with performing selected procedures.12

At the University of Tromsø, the Arctic University of Norway (UiT), practical training takes place during all 6 years of medical school. In addition to full-time studies, many UiT students work ECHR jobs as well. This is often economically motivated, but it may also increase clinical experience and practical proficiency. In addition, some students have started or even finished other healthcare professional studies before medical school. Others have gone through medic training when serving with the Norwegian Armed Forces. Furthermore, a popular campus-based student organisation for emergency medicine—Tromsø Acute Medicine Students’ Association (TAMS)—provides eager students with lectures and procedural training related to emergency medicine. These arenas provide learning opportunities for some students but perhaps not for all. The individual students’ proficiency in practical skills may differ, and even more important, if much of the acquisition of skills takes place outside the study programme, some students may by chance not reach the necessary level of competence in clinical skills.

This motivated the present study. We investigated the association between ECHR work experience and self-reported training amount and confidence level among UiT medical students and first-year graduates in selected basic procedures important for emergency medicine. The associations with year of study, previous education, military medic training and TAMS participation were studied as well. We hypothesised that respondents with ECHR work experience would report more training and confidence than those without such experience.

Materials and methods

Study design

This is a cross-sectional questionnaire-based study.

Study participants and setting

All medical students enrolled at the UiT from 2013 to 2019 and 2019 graduates were invited to participate. The UiT School of Medicine is a 6-year programme. The first year covers basic sciences, while the second–fourth years gradually integrate preclinical and clinical teaching. Year 5 comprises 7 months of clinical clerkship, while year 6 is dedicated to final clinical teaching. Emergency medicine is taught specifically in year 1 (1 week of basic life support), year 4 (4 weeks of emergency medicine, anaesthesia and critical care) and in year 6 as a 4-week module. Furthermore, TAMS provides all students an opportunity to learn and maintain basic emergency medicine procedures, with practical workshops on, for example, airway control, CPR and use of defibrillators, and to teach basic life support to laypeople and other healthcare students. Experienced members also participate in workshops covering advanced topics such as advanced cardiac life support and may be assistant trainers on medical school courses.

Practical procedures recommended for medical schools were defined several years ago by a national working group, and the national medical programmes have used this consensus to define compulsory training activities. To our knowledge, such a comprehensive list has never been made publicly available, and there is no formal quality control of acquisition of procedural skills at the UiT programme, except for a limited objective structured clinical examination (OSCE) during year 3 and practical case-based trial exams throughout year 6. A complete list of practical procedures and time of expected acquisition is not known to the students.

Questionnaire and variables

We designed a questionnaire to probe the participants’ training amount for selected core procedures relevant to emergency medicine (training amount). The respondents reported the number of times having performed the procedures (0, 1–5, 6–10, 11–30 and >30 times). For some items, training amount in practice (0, 1–5, 6–10, 11–30 and >30 times) and real-life situations (0, 1, 2–5, 6–10 and >10) were probed separately. Self-perceived confidence level with each procedure was investigated with degree of agreement (strongly disagree, disagree, neither agree nor disagree, agree and strongly agree) to a statement of feeling confident to perform the respective skill. The questions were designed as 5-point Likert items (online supplemental appendices A and B).

ECHR work experience (no or yes), workplace, work length (<6 months, 6 months–1 year, 1–3 years and >3 years) and work hours (<10, 10–100, 101–200, 201–300, 301–500 and >500 hours), as well as year of study (years 1–6 and first-year graduates), previous healthcare-related education (no, commenced but unfinished and finished), previous military medic training (none, basic (mandatory for all soldiers) and advanced (military medic level)) and number of TAMS events participated in (0, 1, 2–5, 6–10 and >10) were recorded.18

For most study years, paper questionnaires were handed out in lectures between November 2019 and February 2020. Approval from lecturers and the programme was obtained in advance. The questionnaire required around 10 min to answer. Most students responded immediately, but a few finished later. Fifth-year students and graduates were off campus and received an electronic questionnaire. All students were informed about the project at least 1 day in advance through formal information channels and social media. They also received two notifications that encouraged remaining students to hand in their answers. To increase response rate, respondents were invited to a lottery with modest prizes. Participation was voluntary, and measures were taken to ensure anonymity. Prior to data collection, the questionnaire was piloted to a limited number of students from different study years and with varying experience with emergency medicine. Feedback from the pilots was integrated in the final version. We did not probe any personal medical information.

Statistics

Responses were analysed using IBM SPSS Statistics V.26.0.0.1. The alternatives for each Likert item were assigned values from 0 to 4. Missing data accounted for less than 1% in almost all variables, and no action was taken for these. Three variables were excluded from analyses due to high amounts of missing data.

Composite scores for training amount and confidence level for each respondent were defined by calculating the mean of all the items probing training amount, including items probing both practice and real-life situations, and items probing confidence level in performing the various procedures, respectively. This allowed the data to be treated as interval data. Cronbach’s alpha was calculated for both composite scores for reliability analyses. Independent samples t-tests were conducted to compare the composite scores within the various categories of the independent variables. The individual items’ median scores as well as correlational analyses between training amount and confidence level for each procedure were also calculated.

For primary outcome analyses, the composite scores of respondents with and without ECHR work experience were investigated with analysis of covariance (ANCOVA). Year of study, previous education, previous military medic training and TAMS participation were included as potential confounders. For secondary outcomes, we investigated each confounder’s contribution to the total variance in the two composite scores—total variance being the variance associated with the respective confounder after all concurrent factors have been adjusted for. Assumptions regarding homogeneity of variance between the groups were violated, confirmed by a significant Levene’s test. The sample sizes within the two predictor variable groups also differed. The ANCOVA is considered a robust test, but we supplemented it with a linear mixed model analysis to investigate potential effects of the violated assumptions. Mixed models were also applied to calculate estimated marginal means for both composite scores for ECHR workers and non-workers. Cohen’s D was calculated for the two composite scores’ association with ECHR work experience by dividing the between-group difference from mixed models with the pooled SD of ECHR workers and non-workers. This was to better illustrate the strength of the associations seen in our primary outcomes.

Patient and public involvement statement

No patient was involved.

Results

A total of 689 students and 77 recent graduates were invited to participate. Five hundred thirty-nine of these 766 responded. Response rates were high: 75%–88% for years 1–4, about 60% for years 5–6 and 23% for graduates, giving a total response rate of 70%. No less than 81% of all respondents had ECHR work experience, and 59% of respondents with work experience had worked at more than one workplace. The most frequently reported workplaces were nursing homes, hospitals and home healthcare services. Nine per cent of the respondents had previously commenced healthcare-related education, and 5% had completed a degree. Thirteen per cent had previous military medic training, and 66% had been involved with TAMS (tables 1 and 2).

Table 1

Training and confidence in practical procedures

Table 2

Workplaces for students and graduates

High Cronbach’s alpha was obtained for both composite scores (0.927 and 0.919, respectively), suggesting a high internal consistency of the items. Items 18–20 (see online supplemental appendix B) were excluded for the latter score due to high amounts of missing data—51%, 81% and 67%, respectively—reason being that respondents were instructed specifically to answer these questions only if they had relevant experience. Removal of any included items did not change Cronbach’s alpha significantly, suggesting all items to be equally important. Independent samples t-tests demonstrated significantly higher composite scores among respondents with ECHR work experience, compared with those without such experience. Further t-tests showed a gradual increase in both scores with increasing work experience, though with varying levels of significance. Experience from more than one workplace gave significantly higher scores for both outcomes. Each increase in study year gave a gradual increase in both scores: the largest gaps were observed between years 2 and 3, and years 4 and 5. The same trend was seen for previous education and TAMS engagement. Military medic training was the only included predictor variable that did not give any significant difference in outcome (table 1).

Table 3 shows median self-reported training amount and confidence for the individual procedures. Automatic blood pressure measurement was the only skill with a median training amount of >30 and the highest reported median confidence level. A positive correlation was seen between training amount and confidence level for all respondents (Pearson coefficient of .878, p<0.001). Regarding real-life situations, 55% of students with ECHR work experience reported to have checked level of consciousness at least once, compared with only 29% for students without work experience, and 43% with work experience had placed a patient in recovery position, compared with 28% without work experience. Thirty-six per cent with work experience had at least once observed cardiopulmonary resuscitation (CPR) being performed in a real-life cardiac arrest, and as many as 18% had actively participated in CPR themselves. These figures were two to four times the size of those of students without work experience (17% and 4%, respectively; see table 4). Differences in real-life exposure between respondents with and without work experience were significant for all items at p<0.05.

Table 3

Self-perceived training amount and confidence level

Table 4

Real-life exposure to selected procedures

ANCOVA and linear mixed model analyses gave almost equal results, suggesting that ANCOVA was sufficiently robust regardless of the violated assumptions. Work experience accounted for 5.9% and 3.5% of the total variance in the two composite scores. However, year of study and TAMS participation showed significantly stronger associations, accounting for 49.3% and 58.5%, and 23.6% and 12.3% of the total variance in the two composite scores, respectively. Previous education and military medic training accounted for 8.7% and 5.1%, and 6.8% and 4.7% of the total variance. All primary and secondary outcomes are displayed in table 5.

Table 5

Confounder effects on total variance

Mixed model analyses on estimated marginal means yielded significantly higher scores for ECHR workers than non-workers for both composite scores, which demonstrates that workers in average scored themselves higher in questions probing both training amount and self-perceived confidence when adjusted for the concurrent factors. The difference in mean between workers and non-workers was 0.35 (p<0.001, 95% CI 0.26 to 0.43) and 0.25 (p<0.001, 95% CI 0.14 to 0.37) for the two composite scores, which is a difference of 8.8% and 6.3%, respectively (table 5). Cohen’s D calculations yielded medium (0.48) and weak-medium (0.32)-sized associations between work experience and the two composite scores, respectively.

Discussion

Our data represent a cross section of UiT’s medical students’ and first-year graduates’ training amount and self-perceived confidence level in basic emergency medicine-related procedures necessary for a junior doctor. We show associations that support the idea that extracurricular experience may be beneficial for the acquisition of procedural proficiency. As expected, late-year students estimated their own competence as far better than early-year students. However, and quite interestingly, active participation in the student’s organisation TAMS was, together with study year, the most important contributor for both outcomes. ECHR work experience, together with previous healthcare-related education, were both associated with higher training amount and student confidence in own expertise in the included procedures as well, and both were independently important when effects of the concurrent factors were adjusted for. These results were in line with reports from other studies,8 19–22 although some have found no such relationship.12

It was somewhat surprising that only automated blood pressure measurement received the highest possible median value in both outcomes. Checking level of consciousness, recovery position and basic CPR are procedural skills introduced during the first months of the programme and retrained repeatedly during the education. We therefore expected a higher score for these procedures. However, fear of causing harm by not mastering the procedures properly might have contributed to the low confidence. Too little retraining through the education might also be important. Nevertheless, these are examples of essential, life-saving procedural skills that should be mastered by all practising physicians, and the results suggest that the programme should objectively assess the individual student’s competence with these procedures during training. For the more advanced procedures, many respondents reported low amount of training and self-perceived confidence, like results from other studies.12 23 This was expected; taking an arterial blood gas, placing an intraosseous line and using a multimonitor in an emergency situation are not introduced until year 4 and 5 at UiT. Prior to this, students would have to acquire these skills elsewhere. However, these procedures displayed several of the strongest correlations between training amount and self-perceived confidence, emphasising the importance of training volume for performance level. Similar findings have also been reported elsewhere.12 22

Some of our results differed from similar studies. Dehmer et al. studied last-year medical students at the University of North Carolina.12 Thirty-eight per cent of their students had never placed an intravenous line, while 100% of the final-year students at UiT had done so at least once. Twenty-eight per cent of the same students felt insecure with the procedure, compared with 1.4% of UiT’s students. Thirty per cent of Dehmer’s students had never done an arterial puncture and 43% felt insecure, while only 1.4% of UiT’s last-year students had never done this skill, and 4.2% felt the least confident. When comparing these results, however, it is important to note that the American students seemingly only reported practice from real patients, whereas we did not make a distinction between manikin practice and actual patients. For Dehmer’s students, total training amount including practice situations might therefore be higher. This illustrates that comparison with other studies is not straightforward. Furthermore, UiT has a 6 year medical education, whereas The University of North Carolina has a 4 year programme based on a 2 year pre-medical school, which may make comparisons even more demanding.

ECHR work experience

ECHR work experience was moderately associated to both training amount and self-perceived confidence. This was not surprising, and it strongly suggests that ECHR work positively influences training amount and confidence. It is striking that 81% of the participants reported ECHR work. Medical school is demanding, and leaves limited spare time, but ECHR work may represent a significant arena for training. However, the degree of exposure to procedures depends on type of work. Our study focused primarily on procedures important for emergency medicine, likely to be more common in ambulance and district surgeries. Our study did not allow for subgroup analyses for workplace comparison. However, all workplaces taken together, the difference in real-life exposure to important procedures between the groups with or without work experience was substantial. This was especially true for CPR; ECHR work experience was associated with twice the exposure. As many as 36% and 17% of students with work experience had real-life CPR-experience–by observation and direct participation, respectively. These numbers were higher than expected.

Regardless of the modestness, ECHR work experience displayed significant associations, even after adjusting for the included confounders. Considering our findings, it is reasonable to believe that ECHR work is associated with increased skill level for procedures not included in this study as well. It is also reasonable to believe that both amount and diversity of work matter, where students with >500 work hours and experience from more than one workplace scored themselves significantly higher for both primary outcomes, as displayed in table 1.

Year of study, military and prior education, and the TAMS

The largest increase in both training amount and confidence was seen between the fourth-year and fifth-year students. The data were gathered around the end of the fifth-year students’ clerkship period. This serves as a reasonable explanation for the observed gap and argues that the clerkship period incorporated in UiT’s study programme is an important arena for practical skill learning and development. It is also important to notice that the main body of emergency medicine is given at the end of year four. Also interesting was the almost equally large gap observed between students at the second and third study year. Third-year students at UiT receive an increased frequency of bedside teaching, and they must complete a mandatory OSCE. Exams direct student priorities and based on the present results it may be plausible that more frequent use of OSCEs could raise the awareness of procedural training among students. The predominance of theoretical exams might reduce the students’ focus on procedural training. Another interesting finding was that study year had stronger association with confidence than to training amount. This contrasted the other predictors, where associations to training amount were strongest. This might be explained by the increased theoretical knowledge and more patient encounters at higher study years.

Approximately 13% of the students reported to have military training, and the vast majority of these (85%) had advanced levels of medic training. Still, no associations were found between military training and the outcomes. This might be explained by the Dunning-Kruger effect, a cognitive bias where someone overestimates their own abilities, a bias that medical students may be prone to.24 Respondents with military medic training will likely have more experience from realistic scenario training, and thus have a better understanding of the practical demands with the procedures. This may be less well understood by other students, which might make them more prone to overestimate their own competence. It is also plausible that the former military medics are a selected group, with a cognitive bias that may be difficult to adjust for. Lastly, results might be misleading due to few participants included in these analyses, clearly shown by the large CIs for respondents with basic medic training.

A priori, we expected that respondents with a healthcare-related degree scored higher than those without such education. It was therefore surprising to find only moderate associations to the outcomes. Most participants with previous education were nurses, but physiotherapists, bioengineers, dentists, ambulance technicians, healthcare assistants, pharmacists and radiographers were also represented. Only a small proportion of the respondents reported having a degree particularly relevant for emergency medicine, and thus, we might not have been able to find existing associations.

Furthermore, the results show that a substantial proportion of the students had participated in TAMS, and this was strongly associated with higher scores of both outcomes. TAMS offers frequent and longitudinal practical training for all participating students, which is known to be important for learning and maintaining practical skill level.1 20 25 This was also supported by a strong correlation between training amount and self-perceived confidence level in the present study.

Limitations and implications for further education

The questionnaire has not been validated externally. However, Cronbach’s alpha values over 0.9 suggest that the items maintained a high internal consistency. Graduates delivered a low response rate, making interpretations of this group difficult. We present only self-reported information; actual performance was not objectively evaluated; thus, self-report bias due to the Dunning-Kruger effect cannot be excluded.24 Furthermore, self-perceived confidence does not necessarily equate with actual competence. However, practical exposure to procedures is an important prerequisite for actual competence. The study was limited to a single institution, which may limit the external validity of the findings.

We limited our study to basic emergency procedures relevant for all junior doctors. The role of extracurricular activities for practical skills acquisition may be a concern for medical education, especially if this by chance renders some students without the intended level of training. In this respect, our study may serve as a model for other important medical competencies and may be of interest for all medical educators. How much of the training is controlled by the institution, and how much competence is acquired only by students with certain extracurricular activities?

Lastly, it is difficult to assess all potential confounders, as we have only analysed four factors that could be probed with questionnaires. It is plausible that students proficient in the investigated procedures possess a strong internal motivation for seeking such proficiency, which might be a common denominator. Furthermore, we did not assess the role of personality, intellectual capacity or academic achievement, all potential confounders that are more demanding to assess.

The results still present a cross section of the students’ own perception of training and confidence. The final and intriguing question is how the UiT School of Medicine offers training in the reported procedures. Previous research on Norwegian students suggested inadequate focus on practical training. However, there is still no publicly available list of practical items that should be addressed during the different years of Norwegian medical schools. We believe that making the aforementioned competence list accessible for the students might guide them in acquiring procedural competences. We also believe that a more thorough assessment of the individual student’s ability to perform the listed procedures might increase their preparedness for postgraduate work and to benefit the study programme.

Conclusion

Our findings suggest that students and graduates with ECHR work experience have more training and more confidence in basic procedures within emergency medicine, compared with those without such experience. Year of study, previous education and TAMS participation were associated with higher levels of self-reported training and confidence as well.

Data availability statement

Data are available in a public, open access repository. Technical appendix, statistical code and dataset available from UiT Open Research Data (doi: https://doi.org/10.18710/O1ZOQ0).

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants, but It was deemed unnecessary to apply for ethical approval. The study did not include health-related data and thus did not need approval from the regional ethical committee. The study did not need approval from the UiT Data Protection Authorities as the participant consented to participate voluntarily and anonymously, and no sensitive information was collected. The participants gave informed consent to participate in the study before taking part.

Acknowledgments

The authors acknowledge the contributions to the protocol and initial draft by Dr Frode Sørensen and assistance with the statistical analyses by Professor Tom Wilsgaard.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors RWS and KF conceived the study and developed the questionnaire. RWS collected and analysed the data and drafted the manuscript, which was subsequently revised and approved by both authors. KF is the guarantor of the study.

  • Funding This work was supported by The UiT Open Access Publishing Fund (no grant number).

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.