Article Text

Download PDFPDF

Original research
Survey of physician attitudes to using multisource feedback for competence assessment in Alberta
  1. Nigel Ashworth1,2,
  2. Nicole Allison Kain1,2,
  3. Ed Jess2,
  4. Karen Mazurek2
  1. 1Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
  2. 2Research and evaluation unit, College of Physicians and Surgeons of Alberta, Edmonton, Alberta, Canada
  1. Correspondence to Dr Nigel Ashworth; ashworth{at}ualberta.ca

Abstract

Background The use of multisource feedback (MSF) for assessing physician performance is widespread and rapidly growing. Findings from early very small research studies using highly selected participants suggest high levels of satisfaction and support. However, after nearly two decades of experience using MSF to evaluate all physicians in Alberta, we are sceptical of this.

Objectives To determine physicians’ actual opinions of MSF using the entire physician population of Alberta, Canada

Design Online survey.

Setting Alberta, Canada.

Participants All physicians with a full licence to practice in Alberta in 2015.

Interventions All participants were asked to grade how well they thought MSF was at assessing various aspects of physician performance using a 10-point Likert-type scale. There was also a text response field for written comments.

Outcomes Mean responses to quantitative questions. Qualitative content and thematic analysis of open-ended text responses.

We analysed the data using SPSS V.23 and NVivo V.11 and built a multivariate model highlighting the predictors of high and low opinions of MSF.

Results Survey response rate was high for physicians with 2215 responses (25%). The mean rating for how successful MSF was at assessing a variety of dimensions, varied from a low of 5.03/10 for medical knowledge to a high of 6.38/10 for professionalism and communication. Canadian-trained MDs rated MSF significantly lower on every dimension by approximately 20% compared with non-Canadian-trained MDs.

Conclusions Alberta physicians have much lower opinions about the ability of MSF to measure any dimension of their performance than what has been suggested in the literature. Canadian-trained MDs have a particularly low opinion of MSF for reasons that remain unclear. The results of this survey offer a serious challenge to the effectiveness of a programme that is designed to promote self-reflection and performance improvement.

  • education & training (see medical education & training)
  • quality in health care
  • medical education & training
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Footnotes

  • Contributors NA and KM had the initial concept. NA, NAK and EJ designed the study. NA and NK collected the data, performed the quantitative and qualitative analyses. NA, NAK, EJ and KM reviewed and interpreted the results. NA drafted the manuscript. All the authors wrote and edited and revised the final versions of the manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests All the authors work for CPSA who ran the physician achievement review (PAR) programme, an MSF-based assessment programme for 18 years in Alberta.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Patient consent for publication Not required.

  • Ethics approval This survey, as a component of a larger evaluation and redesign of the College of Physicians and Surgeons of Alberta (CPSA)’s continuing competence programmes, received ethical approval from the University of Alberta joint Research Ethics Board. Participants gave informed consent at the time of the survey.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available upon reasonable request. All data requests should be submitted to the corresponding author for consideration. Access to anonymised data will not be possible.