Can unannounced standardized patients assess professionalism and communication skills in the emergency department?

Objectives: The authors piloted unannounced standardized patients (USPs) in an emergency medicine (EM) residency to test feasibility, acceptability, and performance assessment of professionalism and communication skills.

Methods: Fifteen postgraduate year (PGY)-2 EM residents were scheduled to be visited by two USPs while working in the emergency department (ED). Multidisciplinary support was utilized to ensure successful USP introduction. Scores (% well done) were calculated for communication and professionalism skills using a 26-item, behaviorally anchored checklist. Residents' attitudes toward USPs and USP detection were also surveyed.

Results: Of 27 USP encounters attempted, 17 (62%) were successfully completed. The detection rate was 44%. Eighty-three percent of residents who encountered a USP felt that the encounter did not hinder daily practice and did not make them uncomfortable (86%) or suspicious of patients (71%). Overall, residents received a mean score of 60% for communication items rated "well done" (SD +/- 28%, range = 23%-100%) and 53% of professionalism items "well done" (SD +/- 20%, range = 23%-85%). Residents' communication skills were weakest for patient education and counseling (mean = 43%, SD +/- 31%), compared with information gathering (68%, SD +/- 36% and relationship development (62%, SD +/- 32%). Scores of residents who detected USPs did not differ from those who had not.

Conclusions: Implementing USPs in the ED is feasible and acceptable to staff. The unpredictability of the ED, specifically resident schedules, accounted for most incomplete encounters. USPs may represent a new way to assess real-time resident physician performance without the need for faculty resources or the bias introduced by direct observation.