Article Text

Download PDFPDF

Original research
Effectiveness of a simulation-based mastery learning to train clinicians on a novel cricothyrotomy procedure at an academic medical centre during a pandemic: a quasi-experimental cohort study
  1. Nabil Issa1,
  2. Whitney E Liddy2,
  3. Sandeep Samant2,
  4. David B Conley2,
  5. Robert C Kern2,
  6. Eric S Hungness1,
  7. Elaine R Cohen3,
  8. Jeffrey H Barsuk3
  1. 1 Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
  2. 2 Department of Otolaryngology, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
  3. 3 Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
  1. Correspondence to Dr Nabil Issa; nabil.issa{at}


Objectives To develop and evaluate a simulation-based mastery learning (SBML) curriculum for cricothyrotomy using wet towels to suppress aerosolisation during a pandemic.

Design Quasi-experimental, pre–post study.

Setting Tertiary care, academic medical centre in Chicago.

Participants Ear, nose and throat and general surgery residents, fellows and attendings.

Intervention Cricothyroidotomy simulation-based mastery learning curriculum.

Outcomes measure Pretest to posttest simulated cricothyrotomy skills checklist performance.

Results 37 of 41 eligible surgeons participated in the curriculum. Median pretest score was 72.5 (IQR 55.0–80.0) and 100.0 (IQR 98.8–100.0) for the posttest p<0.001. All participants scored at or above a minimum passing standard (93% checklist items correct) at posttest.

Conclusions Using SBML is effective to quickly train clinicians to competently perform simulated cricothyrotomy during a pandemic.

  • COVID-19
  • education & training (see medical education & training)
  • otolaryngology
  • accident & emergency medicine
  • adult intensive & critical care

Data availability statement

Data are available on reasonable request.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • Our study shows the important role that simulation education can play in training physicians in a timely fashion on a new technique for cricothyrotomy during a pandemic.

  • We were able to create and implement an effective cricothyrotomy simulation-based mastery learning (SBML) training curriculum within less than 1 week without compromising the rigorous nature of SBML.

  • Almost all of the surgeons in our hospital who may potentially be called to perform cricothyrotomy on patients with COVID-19 were trained using the SBML curriculum.

  • The study was performed at a single urban, academic, tertiary care institution on a relatively small cohort of participants which potentially limits generalisability.

  • Downstream clinical outcomes resulting from the SBML training were not evaluated due to the rare nature of emergent cricothyrotomy.


Emergency surgical airway or cricothyrotomy is a rare procedure that occurs in about 0.23% of all intubations.1 Cricothyrotomy is high stakes because intubation algorithms call for it as a last resort to secure the airway in cases of failed intubation.2–5 Performing this procedure during a highly contagious pandemic requires additional steps to ensure the safety of the healthcare providers.

During the COVID-19 pandemic, ear nose and throat (ENT) and surgical residents and attendings at our tertiary care, academic medical centre were asked to perform cricothyrotomy on patients with COVID-19. This caused stress among these physicians due to the potential problem of droplet splatter during the procedure. In response, a working group of ENT and general surgeons discussed this problem after a review of the literature and personal communications with colleagues. Based on findings, the group reached five guiding principles for performing cricothyrotomy in patients with COVID-19:6

  1. Use full personal protective equipment including powered air-purifying respirator or N95 masks with a face shield/goggles and training of all personnel on proper ‘Donning and Doffing’7 when performing cricothyrotomy procedures.

  2. Emphasise sedation and potential paralysis of the patient, which is in agreement with the Centers for Disease Control, the guidelines of the World Federation of Societies of Anaesthesiologists and the American Society of Anesthesiologists.

  3. Use a previously described Seldinger technique for guiding intubation of the airway using a bougie.8 9

  4. Use wet towels as a surgical drape throughout the cricothyrotomy procedure. Moistened towels are readily available and suitable to all healthcare settings. To illustrate the efficiency of moist towels suppressing droplet splatter during the performance of cricothyrotomy, we injected fluorescein dye through an atomiser in the lower end of a pigs’ trachea at the time of entering the cricothyroid membrane with a scalpel.6 10 This previously published experiment showed that moist towels effectively suppress the splatter of fluorescent dye when shining a black light onto the healthcare providers during the procedure.6

  5. Create a simulation-based programme, addressing numbers 1–4 above, to teach ENT and surgical residents and attendings cricothyrotomy. Using simulation-based training to increase the surge capacity within healthcare organisations and improve the safety and skills of healthcare providers as well as the safety of patients was advocated in the Italian experience during the COVID-19 outbreak.11

Based on those guiding principles, we created a simulation-based mastery learning (SBML) curriculum to teach the modified cricothyrotomy procedure for all those who are likely to be called to perform an emergent cricothyrotomy procedure on patients who are diagnosed with COVID-19 or those who are suspected of contracting the disease. SBML is a strict form of competency-based learning where all learners are required to meet or exceed a minimum passing standard (MPS) before completion of training. Those who do not meet the MPS participate in more deliberate practice until they can be retested and reach this standard.12 13 Typically, SBML curricula take months to develop. However, given the urgent need for training in our institution, we rapidly created and implemented the cricothyrotomy SBML curriculum in less than a week. Our aim was to develop and evaluate the effectiveness of the cricothyrotomy SBML curriculum.

Materials and methods

We performed a pretest–posttest study of the cricothyrotomy SBML intervention at Northwestern Simulation Laboratory at Northwestern University Feinberg School of Medicine in April 2020. We created a cricothyrotomy SBML curriculum in less than a week and evaluated changes in simulated skills performance resulting from the training. The Northwestern University Institutional Review Board approved this study with exempt status and therefore need for obtaining consent was waived.


A total of 41 eligible participants consisted of ENT and general surgery senior residents (post graduate year (PGY) 2–5), fellows and attendings who would be performing emergency cricothyrotomy in patients with COVID-19. Resident and fellow participation was required by their programme directors while attending participation was voluntary. Providing demographic and clinical information as well as rating the curriculum was voluntary.

Patient and public involvement

This study did not involve any patients or the public at large.


Before simulation training, all participants reviewed written materials about the indications, contraindications and complications of cricothyrotomy. They also watched an online video how to perform the procedure with Seldinger technique using a bougie. Participants then came to the simulation laboratory and engaged in a timed simulated pretest using a skills checklist created for the procedure. This was followed by 1 hour of deliberate practice on the simulator with faculty feedback. Participants focused on using tactile cues to performing the procedure since visualising the anatomical landmarks was not possible under the wet towel. Participants were taught to use the middle finger and the thumb of the non-dominant hand clinch and stabilise the thyroid cartilage complex. The index finger then palpates the thyroid notch and travels caudally until the first depression is felt, which corresponds to the cricothyroid membrane where the incision is made. Finally, all participants took a timed posttest where they were required to meet or exceed an MPS on the same skills checklist as well as procedure time. Those who were unable to reach the MPS participated in more deliberate practice until they could meet the MPS at retesting. After the MPS was reached on the posttest, participants completed an anonymous online survey about their specialty, training level, experience with cricothyrotomy, self-confidence performing the procedure and satisfaction with the course. Participants were also asked to comment on the most challenging aspects of the procedure.

A locally developed hybrid cricothyrotomy model was used for the procedure. The model consisted of a synthetic mould representing the shoulders, neck and head of a human with a central groove in the neck area in which a pig or sheep trachea could be pinned and covered with a layer of synthetic skin (figure 1 shows a demonstration of training on the simulator). The model was developed in 2009 and has been used previously to successfully train surgery residents to perform cricothyrotomy using an open technique not including a bougie.

Figure 1

Cricothyrotomy simulator consisted of a synthetic mould representing the shoulders, neck and head of a human with a central groove in the neck area in which a pig or sheep trachea can be pinned and covered with a layer of synthetic skin. (A) A surgeon incises the simulated skin overlying the cricothyroid membrane. Note: The person depicted is not a patient and the picture was taken with the participants’ knowledge. (B) The endotracheal tube is properly placed through the simulated skin into the trachea of the cricothyrotomy simulator.

We compared differences in performance on the checklist and the time required to complete the procedure from pretest to posttest.


Five board-certified surgeons (four ENT and one trauma/critical care surgeons) used the hybrid simulator to test three draping methods to suppress droplet splatter during cricothyrotomy at the Northwestern Simulation Laboratory.6 The surgeons agreed on standard procedural steps for cricothyrotomy using a wet towel drape as the best method for droplet splatter prevention.6 The group adopted a previously described technique ‘Scalpel–Bougie–Tube’ approach, which is based on Seldinger technique using a bougie as a guide. These five surgeons then developed a 20-item checklist (see online supplemental appendix) for cricothyrotomy skills based on these best practices and checklist design strategies described by Stufflebeam.14 Subsequently, eight board-certified surgeons who are experts in cricothyrotomy as well as simulation education experts (one emergency medicine physician, and four ENT and three general surgery attendings) used the modified Delphi technique to finalise the procedural checklist including selecting critical items.15 Critical items were identified which, if missed, would require the learner to automatically participate in further deliberate practice and retesting. All items in the checklists were dichotomous and graded 1 if done correctly and 0 if not done or done incorrectly. The four ENT and one trauma/critical care attending served as proctors and raters for the training. Rater calibration was performed during a half-day session using a Frame of Reference methodology.16 This method requires an iterative process where raters practised rating each other, and compared and discussed test items as a group to ensure standardisation of the scoring process.

The MPS for the checklist and time needed to complete the procedure was determined by the same eight clinical experts who created the checklist using the Mastery Angoff standard setting method.17 The Mastery Angoff technique required these experts to judge the percentage of ‘well-prepared’ trainees who would get each checklist item correct at posttest as well as their estimated time for completing the procedure. The well-prepared trainee is one who can do the procedure safely and independently during patient care. The average of each expert’s checklist items and time needed to complete the procedure represents the individual judges’ MPS which is again averaged across judges to determine the final MPS.

The time required to complete the procedure was measured from the time of room entry to the time of completing the ventilator connection. We chose to start time-measurment from the point of room entry because cricothyrotomy is a life-saving, time-sensitive, high-risk, low-frequency procedure, and during COVID situation, only the intubation team is allowed to be in the room while others including surgeons are out of the room. Self-confidence and course satisfaction surveys were evaluated on a 5-point Likert scale (1=strongly disagree; 5=strongly agree). Surveys were adapted from prior work.18–20

Data analysis

We report descriptive summaries of survey data as means with SD and counts and frequencies (%) for categorical data. To evaluate differences in pretest to posttest checklist scores, we used the paired Wilcoxon signed-rank test (data were non-parametric). We used paired t-tests to compare time required to complete the procedure between pretests and posttests (data were parametric). All statistical analyses were performed using SPSS V.26.


Thirty-seven out of the 41 (90%) eligible participants completed the training. Three general surgery residents and one ENT attending who did not attend were either on active clinical assignments or were on rotations at other institutions. Training level and clinical experience data can be found in table 1. Nine of 37 (24%) participants indicated they were never trained on performing a cricothyrotomy. Twelve (32%) had never done a cricothyrotomy, all of whom were residents and fellows. Only six (16%) participants, five faculty and one resident (PGY-5) had performed more than five cricothyrotomy procedures during their medical career.

Table 1

Training level and experience of cricothyrotomy SBML participants (n=37)

Thirteen checklist items were considered critical. The MPS on the checklist was set at 93%, requiring participants to get 19 of 20 items correct. The MPS also required participants to complete the procedure within 5 min. Only one participant reached the MPS at pretest (figure 2). All 37 participants were able to reach the MPS at posttest. Twenty-eight out of 37 (76%) learners passed the posttest on the first attempt. Five participants (14%) required a second posttest, 1 (3%) required a third posttest, while 3 (8%) required a fourth posttest, all after more rounds of deliberate practice on the simulator. Extra deliberate practice sessions were all less than 1 hour.

Figure 2

Resident, fellow and attending performance (per cent correct) on the 20-item cricothyrotomy skills checklist at pretest and posttest. Each circle represents an individual resident. MPS, minimum passing standard.

The median pretest score for all participants was 70% (IQR 55.0%–80.0%) checklist items correct and 100.0% (IQR 97.5%–100.0%) for the posttest p<0.001 (table 2). The mean time of completing the procedure was 4:25 min (SD 0:55) for the pretest compared with 3:10 min (SD 0:43) for the posttest, p<0.001 (figure 3).

Figure 3

Time (mean minutes, SD) required by residents, fellows and attending to complete the simulated cricothyrotomy procedure at pretest and posttest.

Table 2

Comparison of median (IQR) pretest to posttest checklist scores for residents, fellows and attendings who participated in the simulation-based mastery learning cricothyrotomy curriculum

A total of 35/37 (95%) participants completed the post-training satisfaction survey. The majority rated the programme highly and agreed that training addressed their concerns about being exposed to droplet splatter when operating on patients with COVID-19. Thirty-one participants (89%) indicated they felt more confident in their ability to perform cricothyrotomy on patients with COVID-19 after the curriculum (table 3). The three most common comments from the survey about the most challenging aspects of the procedure were (a) paralysing the patient before obtaining an airway, (b) performing the procedure under the wet towels without visual cues and (c) the bougie migrating and potentially injuring the trachea.

Table 3

Post-cricothyrotomy simulation-based mastery learning (SBML) training survey responses (n=35)


We believe that our study is unique and timely for two main reasons. First, we showed the important role that simulation education can play in training physicians in a timely fashion during a pandemic on a new technique for cricothyrotomy. This rapidly increases the capacity and efficiency of a healthcare system during times of demand. Supervised clinical experiences performing this new procedure (on actual patients) would not be a reliable and safe method to train a large number of physicians due to the emergent and infrequent nature of this procedure. Second, we were able to rapidly create and implement a robust SBML curriculum with 37 physicians trained within a week. We believe we were successful in rapidly developing and implementing this curriculum because the operating rooms were shut down for elective cases during the pandemic, and our surgeons had extra time to devote to creating curriculum and performing training. In addition, our faculty have extensive experience in SBML curriculum creation and our simulation laboratory was readily available due to limited use during the pandemic. This emphasises the importance of collaboration between clinicians and educators to tackle important clinical concerns in a timely fashion by using well-constructed curricula that demonstrate superior educational and clinical outcomes. In addition, our IRB prioritised studies related to COVID-19 during the pandemic and provided expedited review. The only significant barrier to rapid implementation of this curricula was the need for creative scheduling due to participant number limitations in each training room and needs for social distancing. Previous curricula evaluating SBML for teaching Advanced Cardiac Life Support,21 central venous catheter insertion,22 lumbar puncture23 and laparoscopic cholidochoscopy24 have shown the positive translational effects on patient and health system’s outcomes.25 However, these curricula took approximately 2–3 months to create and were implemented over months to years.

Of note, the learners in our study were not surgical novices; they were senior residents and fellows as well as established faculty who were considered to be surgically versed and are usually the ones called on to perform cricothyrotomy. Yet, 7 out of 24 (29%) residents and fellows and 2 out of the 13 (15%) attendings indicated they were never trained on how to perform cricothyrotomy and many participants reported low numbers of cricothyrotomy procedures performed (numbers lower than would be expected to be competent). These findings showed an important gap in our educational programme that we addressed by formally adding cricothyrotomy SBML to our resident and fellow training curriculum.

Our SBML curriculum also helped to identify three particular procedure steps that were most challenging to learn by the participants and were emphasised in our curriculum. First, paralysing patients with COVID-19 before securing a patent airway went against historical teachings that the airway should be obtained prior to paralysis and was controversial early in the pandemic. Interestingly, the decision to paralyse suspected or actual patients with COVID-19 prior to obtaining emergent airway, whether by intubation or through surgical intervention, has been advocated by national and international anaesthesia societies.26–28 This is also reflected in our own institutional guidelines that were updated for intubating proven or suspected patients with COVID-19 at the same time that we were developing this programme. Yet, the ENT and surgical residents, fellows and attendings were not aware of those guidelines. Second, they were performing key procedure steps without visual cues while the hands were under moist towels. Participants struggled with this new skill but quickly became comfortable with significant deliberate practice and all learners eventually reached the MPS within less than 1 hour of deliberate practice. This attests to the fact that participants were not surgical novices, and also the fact that cricothyrotomy is a procedure that can be completely dependent on tactile recognition of anatomical landmarks. The third concern was the possibility of the bougie migrating further into the airway during the Seldinger technique, injuring the trachea. This was addressed during the deliberate practice training portion of the programme and is now emphasised in the new cricothyrotomy curriculum resulting from this project.

Our study had several limitations. First, it was performed at a single urban, academic, tertiary care institution on a relatively small cohort of participants. Therefore, results may not be generalisable to other smaller non-academic institutions. without simulation-based education faculty and simulation laboratories. In addition, our hybrid simulator represented an average size human, and may not generalise skills to patients with larger necks or higher body mass index. Second, our SBML curriculum had slightly different features than those described in the literature due to the 1-week time frame needed to create and implement training. For instance, learners were required to perform pre-reading and watch procedure videos to shorten the time needed between training sessions. Typically, didactics and videos are provided after the pretest. Although pre-reading and watching videos may have increased the performance on the pretest, we believe it was necessary to complete all training within 2 days for 37 participants. This should not take away from the significant improvement from pretest to posttest. In addition, we did not have time to extensively pilot test the curricula as is done with other SBML curricula. During pilot testing, multiple participants are scored during calibration of raters for the checklist. This is usually confirmed by demonstrating high inter-rater agreement through a Cohen’s Kappa greater than 0.8.17–24 We used the Frame of Reference method to help rapidly calibrate our faculty raters. Third, we were unable to link survey responses back to individual pretest and posttest skills performance because the surveys were anonymous. Fourth, we did not perform formal training with the airway team members e.g. anesthesiologists, respiratory therapists, and nurses; to enhance teamwork skills and behaviours including communications, leadership, situation monitoring and mutual support.29 However, members from the airway team were able to observe the simulations of the modified cricothyrotomy technique to appreciate the minor changes in the procedural process. We plan to address this through a future interprofessional training programme with key stakeholders at our institution. Finally, we were not able to link downstream clinical outcomes to the SBML training due to the rare nature of emergent cricothyrotomy. Use of wet towels has not been definitively shown to reduce COVID-19 transmission compared with usual surgical draping techniques, potentially resulting in ‘negative’ learning. However, we do know that droplet generation was significantly lower in our atomiser experiments with wet towels, suggesting that COVID-19 exposure would be reduced to some degree during with wet towels versus usual draping techniques. Our curriculum was developed at a time when there was significant uncertainty and fear surrounding the transmission of COVID-19 to healthcare workers. Our surgeons adopted the wet towels practice based on limited evidence and changed their policies for draping during emergent cricothyrotomy. To our knowledge, there were no adverse patient events from the procedure, and no members of the surgical airway team developed COVID-19 infection. Our SBML training can easily be adapted as more data about COVID-19 becomes available (i.e., simply by substituting the draping techniques). Further research should examine the extent to which skills from our training are retained over time and how they affect patient care and provider safety from aerosolisation.


We rapidly created and demonstrated the effectiveness of SBML training on a new method to perform simulated cricothyrotomy on patients with COVID-19. We believe that this study is timely and important for all healthcare providers faced with pandemic conditions on how to quickly educate their workforce on new procedures as we were able to swiftly train almost all of the surgeons in our hospital who may potentially be called to perform cricothyrotomy. Our study also establishes evidence to the versatility, responsiveness and effectiveness of simulation education in healthcare institutions. We encourage healthcare systems to further invest in establishing and maintaining simulation-based education resources to help them adapt to unpredictable circumstances such as a pandemic.

Data availability statement

Data are available on reasonable request.

Ethics statements

Patient consent for publication

Ethics approval

The Northwestern University Institutional Review Board approved this study (STU00212651).


We would like to thank the residents, fellows and attending surgeons who participated in this study for their dedication to education and outstanding patient care. We would also like to thank the staff at Northwestern Simulation, particularly Daniel Cronin and Steven Ford, and the Instructional Design and Development team, James Brucker, Kim Le Mezo and Deon Griffin, for their roles in supporting this simulation programme.



  • Twitter @NabilIssaNU

  • Contributors All persons who meet authorship criteria are listed as authors, and all authors certify that they have participated sufficiently in the work to take public responsibility for the content. The specific contributions made by each author are as follows: Conception and design of the study: NI, WEL, SS, DBC, ERC, JHB; revising the manuscript critically for important intellectual content: all authors; acquisition of data: NI, WEL, SS, DBC, RCK, ESH; analysis and/or interpretation of data: all authors; drafting the manuscript: NI, JHB; approval of the version of manuscript to be published: all authors. NI is the guarantor author for this article with full access to the data and took full responsibility for the work and conduct of the study as well as the decision to publish.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.