Article Text
Abstract
Objectives This study examined the risks and supports to competence discussed in the literature related to occupational therapists, pharmacists, physical therapists and physicians, using epidemiology as a conceptual model.
Design Articles from a scoping literature review, published from 1975 to 2014 inclusive, were included if they were about a risk or support to the professional or clinical competence of one of four health professions. Descriptive and regression analyses identified potential associations between risks and supports to competence and the location of study, type of health profession, competence life-cycle and the domain(s) of competence (organised around the CanMEDS framework).
Results A total of 3572 abstracts were reviewed and 943 articles analysed. Most focused on physicians (n=810, 86.0%) and ‘practice’ (n=642, 68.0%). Fewer articles discussed risks to competence (n=418, 44.3%) than supports (n=750, 79.5%). The top four risks, each discussed in over 15% of articles, were: transitions in practice, being an international graduate, lack of clinical exposure/experience (ie, insufficient volume of procedures or patients) and age. The top two supports (over 35%) were continuing education participation and educational information/programme features. About 60% of all the articles discussed medical expert and about 25% applied to all roles. Articles focusing on residents had a greater probability of reporting on risks.
Conclusions Articles about physicians were dominant. The majority of articles were written in the last decade and more discussed supports than risks to competence. An epidemiology-based conceptual model offers a helpful organising framework for exploring and explaining the competence of health professions.
- competence
- health professions
- risks
- supports
- epidemiology
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This is the largest known scoping review of the risks and supports to the clinical competence of physicians, pharmacists, physical therapists and/or occupational therapists, with over 3500 articles screened for inclusion.
This study offers a new lens for analysing the literature on risks and supports to competence by using epidemiology as a conceptual model. The patterns and proportion of articles reflected in this review may be impacted by the priorities of researchers or publishers and, as such, may not accurately reflect their impact to competent clinical practice.
The collapsing of occupational therapists, pharmacists and physical therapists into a single group, on account of their low frequencies, made it possible to compare them to physicians through cross-tabular and regression analyses; however, doing so may have diminished the uniqueness of these distinct health professions.
By understanding the potential risks and supports to competence, individual practitioners, as well as the systems in which they work, can partner with credentialing organisations and regulatory authorities to monitor and mitigate risks while enabling and facilitating supports.
It is noteworthy that while this literature review spans a 40-year time period (1975 to 2014), articles published within the last 2 years have not been included. Given that interest in risks and supports to competence has been accelerating, emerging ideas and more recent articles may not be adequately captured. In light of increasing global movements towards competency-based educational models, and the comprehensiveness of this review, the authors still deem the findings of this review to be of relevance to policy planners, educationists, faculty and residents alike.
Introduction
Competency-based education (CBE) is an outcomes-based educational model heralded as an alternative approach to the training of health professionals.1 2 Contrary to the traditional time-based model, learners of CBE progress by demonstrating specific skills and abilities relative to the stage and context of their training.1 2 CBE also extends into continuing professional development (CPD) with the goal of maintaining the competence of practising health professionals. Both CBE teaching and assessment are to be guided by clear learning objectives that help to facilitate the desired outcomes of training (eg, ‘assessment for learning’).
While the concept of CBE is not altogether new,3 the implementation of CBE is still in its infancy. With increasing global interest in CBE, further study is therefore warranted to better understand the facilitators of and barriers to acquiring competence and how these fluctuate with varying educational and clinical practice environments, with different stages of training and types of professions and under different legal and regulatory contexts.
Defining the required competencies of learners and developing teaching and assessment methods to facilitate and monitor the progress of competence are just part of the work required in developing CBE programme. Competence is multidimensional and dynamic, changing with time, experience and context.3 To effectively develop, maintain and/or support the competence of health professions, we must therefore account for the many elements of competence, including context and continuum of practice.3 The continuum of practice refers to both the evolution of expertise (ie, student, novice, competent, proficient, expert) and the ‘competence life-cycle’ of the professional (ie, student, field-based novice, independent professional, retirement).3 It is also essential to identify and study the different factors impacting competence (ie, the ‘risks’ and ‘supports’ to competence). Some of the factors associated with underperforming or dyscompetent professionals have been previously identified in the literature,4–7 however, further research is needed to better understand their associations within different contexts, and on individuals in different clinical practice continuums and professions.
Using epidemiology as a conceptual model, we sought to identify the risks and supports to competence discussed in the literature, and to explore their associations with four health professions (occupational therapists, pharmacists, physical therapists and physicians); in different stages in the competence life-cycle (eg, resident, practising clinician) and different domains of competence (ie, CanMEDS roles). Our objectives were to (1) develop an inventory of the risks and supports to competence discussed in the literature; (2) to determine the most heavily endorsed risks and supports to competence and (3) to examine the associations of risks and supports to competence with different health professions, at varying stages of training and across multiple domains of competence.
Methods
We conducted this scoping review following the criteria outlined by Arksey and O’Malley8 and our published protocol.9
Epidemiology was used as a conceptual model, in that we sought to identify the factors impacting competence and their prevalence and associations across different health professions, stages of the competence life-cycle, competence domains, geographic regions and time periods. This is similar to the approach that an epidemiologist would employ in studying a disease and its related health factors. An epidemiologist was consulted to help frame our research questions, organise our areas of interest and guide our analytical approach to the data. A glossary of key terms was defined and repeatedly refined throughout the study (see table 1 of online supplementary 1).
Supplementary file 1
Search strategy
With the assistance of an experienced medical librarian, we searched the electronic databases of Medline, Embase, Cumulative Index to Nursing and Allied Health Literature, Cochrane and Education Resources Information Center from 1975 to 2014, inclusive. We searched the literature for articles discussing health professional groups (‘Physical Therapists,’ ‘Occupation Therapists,’ ‘Pharmacists,’ ‘Physicians’), different competence life-cycle stages (‘trainee,’ ‘intern,’ ‘student,’ ‘professional,’ etc) and the assessments or measures of, and/or influences on, their competence (‘certification,’ ‘licensure,’ ‘professional competence,’ ‘clinical competence,’ ‘Employee Performance Appraisal,’ ‘self-evaluation,’ etc) (see online supplementary 2 for full search strategy examples). All article types, other than reviews, were included in our scoping literature review. Review articles were excluded to avoid ‘double counting’, as we found substantial overlap between the primary research and review articles captured. In addition to searching electronic databases, expert reviewers also identified relevant literature.
Supplementary file 2
Inclusion criteria and article selection process
We included articles if they discussed a risk or support to the professional or clinical competence of at least one of the four health professions of interest. Frank et al’s definition and description of competence informed the development of our eligibility criteria and our framework for data extraction.3
We defined a risk to competence as any factor that negatively impacts or alters any facet of competence (eg, resulting in an associated impact such as the decline of clinical reasoning, knowledge, skills, etc) either directly (eg, through lack of clinical exposure through insufficient volume) or indirectly (eg, as a consequence of a wellness issue, such as depression).
We defined a support to competence as any factor that helps to foster, develop or improve a facet of the competence of health professions (eg, improvement in clinical reasoning associated with participation in continuing education programme).
We included articles on other professions (eg, midwives, psychologists and nurses) only when they were discussed alongside one or more of our four professions of interest.
Occupational therapists (OTs), physical therapists (PTs) and physicians were chosen for study, as the principal author (SGT) has conducted research on and has had educational practice (eg, competency projects) with these groups. Thus, past educational practice and interest drove their inclusion. The profession of pharmacy was also included as it is a well-developed profession that interacts considerably with physicians, and the body of literature was manageable within the scope of this research study.
The large volume of literature on nursing was beyond the scope and resources of this research project and the principal authors have also had limited direct research or educational practice with this professional group. Thus, literature from the nursing profession was excluded from our review.
Quality assurance measures
We employed multiple quality assurance processes, including the calibration of reviewers during the screening, abstracting and coding stages. There were also quality checks at each step, consultation on discrepancies, cross-checking between qualitative and quantitative data and consensus decisions around methodology.
Data extraction
We built an online form to collect data on all included articles. Data extracted during the full-text review included demographics (reviewer, author(s), publication year and location(s) of study), data relating to our inclusion criteria (type of health profession(s), competence life-cycle(s), CanMEDS role(s) and risks and supports to competence). We also collected open text, qualitative data describing each paper’s purpose and main findings to contextualise the quantitative data.
As a starting point, an a priori list of risks and supports was used to capture and categorise data, based on an unpublished review conducted by Glover Takahashi and Herold in 2008.10 We also included an ‘other’ field on our form to capture any risks and supports that were not included in our a priori list. Table 1 lists and describes all of the study variables that we collected data on (for explicit examples of the types of articles captured for each, see table 1 of online supplementary 1).
We chose the CanMEDS framework for its comprehensive categorisation of the measurable competencies of health professions and used it to categorise the domains of competence discussed in the articles, whether or not CanMEDS roles were explicitly identified.11 This was done by comparing the competencies as they were discussed in the articles to the CanMEDS roles’ descriptions (table 1)11 and coded the CanMEDS role equivalent. For example, if an article discussed competence in terms of physicians’ abilities to communicate with patients, we would have coded this as the CanMEDS communicator role; if the article discussed results on a written clinical knowledge test or a review of appropriateness of prescriptions, this was coded as medical expert. This practice is not uncommon, as previous literature has, for example, mapped the Accreditation Council for Graduate Medical Education competencies to their CanMEDS equivalents.12 Where an equivalent was not found, the competency domain was categorised as ‘other’.
Data cleaning and recoding
We cleaned and quality checked all abstracting data and ensured all missing values were complete.
Initially, the proportion of articles in the ‘other’ response category was quite large for location of study and the risks and supports to competence categories. In referring back to the original full-text article, responses were recoded by consensus either into (1) an existing category; (2) newly generated categories or (3) remained in ‘other’. A ‘new’ category was generated if a theme was represented in large enough frequency (ie, 15 or more articles) and was distinguished enough conceptually from existing categories.
Statistical analysis
All quantitative analyses were conducted using IBM SPSS Statistical Software V.20.0 (IBM Corp).
Descriptive statistics were generated to inventory the types of risks and supports to competence discussed and their prevalence in the literature.
We conducted cross-tabular analyses to identify potential associations between risks and supports to competence, the article publication year and location and different types of health professions, competence life-cycles and competence domains discussed in our reviewed articles.
We used univariable logistic regression models to assess whether each of the different characteristics of an article affected the probability of reporting on risks or supports to competence. Characteristics tested in the regression models included: country (Canada vs other), year of publication (treated as a continuous variable), competence life-cycle and each of the seven CanMEDS roles.
Due to the low frequencies of articles for OTs, pharmacists and PTs (n=48, n=63 and n=44, respectively), these three professions were collapsed into one category (‘three health professions’) to permit the examination of inter-relationships between other study variables through logistic regression.
Multivariable logistic regression models were developed to evaluate the adjusted association between the characteristics found to be nominally statistically significant (p<0.05) in the respective univariable analyses and the dependent variables (the risks or supports to competence discussed).
Results
Overview
We identified 943 eligible articles for review (see figure 1). Articles reporting on the risks and supports to competence of the four health professions have increased dramatically since 1980, with the majority of articles (n=617, 65.4%) published from 2005 to 2014 inclusive.
The vast majority of articles were primary research studies (n=736, 78.0%) followed by commentaries or letters to the editor (n=168, 17.8%), grey literature (n=26, 2.8%; eg, conference proceedings and materials found in organisational reports) and historical articles (n=13, 1.4%).
Most (n=472, 50.1%) articles were either published by authors in the USA, or the USA served as the primary study location. A quarter of the articles were from Europe (n=237) and 13.7% were from Canada (n=129). Few articles were conducted multinationally.
Health professions
Physicians were the most frequently discussed profession, with 80.0% (n=754) of the 943 articles focusing solely on physicians and 5.9% (n=56) discussing physicians and one or more of the other three health professions under study (figure 2).
Competence life-cycle
Most articles focused, at least in part, on the clinical practice competence life-cycle (n=645, 68.4%), with 84.8% (n=547) of these referring to physicians in clinical practice. Resident physicians were another large demographic discussed, at 34.9% (n=329).
Competency domains: CanMEDS roles
The proportion of articles on the medical expert role (n=561, 59.5%) far surpassed that of all other CanMEDS roles, regardless of the type(s) of risks and/or supports to competence discussed in association. This trend has not varied since the CanMEDS framework was published in 2005. The proportion of articles focusing on the medical expert role has remained relatively steady over the past 35 years, experiencing a slight proportional increase over the last 10 years studied.
About a quarter of the articles discussed the competence of health professions more broadly in relation to all of the CanMEDS roles (n=245, 26.0%). Few articles focused specifically on the intrinsic (ie, non-medical expert) CanMEDS roles. With the exception of the communicator role (n=123, 13.0%), intrinsic roles were each discussed in less than 10% of articles. Manager and health advocate were the least discussed at only 3.3% each (n=31).
Risks and supports to competence
In analysing the results, four ‘new’ categories of risks to competence were revealed (ie, in addition to those included in our a priori list), representing 27.3% (n=114) of all risks discussed. These were: adequacy of clinical practice or education (quality of educational programme), area of specialty, lack of clinical exposure/experience (insufficient volume) and resources (including people, money and time). Five new categories of supports to competence were revealed in the analysis, representing 20.9% (n=157) of all supports. These were: assessment and feedback through tools, adequate clinical exposure/experience, reflection and self-assessment, support through structure or organisation and technology.
Of the 943 articles, 418 (44.3%) discussed one or more risks to competence, and 750 (79.5%) discussed one or more supports to competence. A total of 226 articles (24.0%) discussed both risks and supports to competence.
The most discussed risks to competence were: experiencing transitions (including change in status, change in focus of practice and new graduate); being an international graduate; lack of clinical exposure/experience (insufficient volume) and age (table 2).
The most discussed supports to competence were: participation in continuing education; educational information/programme features and personal support and feedback (including mentorship and peer performance) (table 2).
A synopsis of key articles about risks and supports to competence is presented in online supplementary 3.
Supplementary file 3
Associations
In general, we found that risks to competence were less studied than supports to competence and that there were differences in the foci of different professions and professional life-cycles to the study of competence.
Proportionately, more articles about physicians focused on risks to competence than articles on the other three health professions studied. Almost half of the articles about physicians were about risks to competence (n=386, 47.7%), which is twofold greater than that of the ‘three health professions’. Articles about the ‘three health professions’ focused far more on supports to competence (n=157, 90.8%) than on risks to competence (n=41, 23.7%).
Some risks varied over the health professions’ competence life-cycle. Risks relating to transitions and wellness were more common earlier on in training (ie, during field training and residency, but primarily the latter); while age, gender, no certification, clinical practice features, previous discipline, area of specialty, lack of experience and resources were more common risks later on in clinical practice. Being an international medical graduate was consistently found as a risk across the entire professional life-cycle, regardless of geographic origin.
Some supports varied over the health profession life-cycle, with educational information/programme features, adequate clinical exposure/experience, reflection and self-assessment and technology being more common early on (ie, during field training and residency); whereas continuing education participation, personal support and feedback, professional organisation and quality assurance participation were more common supports in clinical practice. Assessment and feedback, performance review and support through structure or organisation were consistently found as supports across the entire professional life-cycle.
We also identified differences in the study of risks to competence by study location. Being an ‘international graduate’ was the most frequently discussed risk to competence in articles from Canada, whereas articles from the USA most frequently discussed age as a risk to competence. The risk of ‘wellness’ was discussed in articles originating from the USA, Europe and Canada. Interestingly, none of the articles from Asia and Australia/New Zealand discussed ‘wellness’ as a risk to competence.
The proportions of articles addressing different supports to competence were quite similar regardless of location of study. Continuing education participation and educational information/programme features were the top two most discussed supports across all five geographic locations and were the top supports mentioned in association with each of the CanMEDS roles.
Observations from the regression analysis
Using regression analyses, we examined the relationship among and between study variables. The dependent variables were the risks and supports to competence, and the independent or predictor variables were location, year, competence life-cycle, CanMEDS role and health profession.
In univariable analyses, articles reporting on health professions other than physicians and on the Scholar CanMEDS role predicted a lesser probability of reporting on risks associated with competence, whereas articles focusing on resident physicians (compared with articles focusing on other physicians and health professions) predicted a greater probability of reporting on risks associated with competence. The same characteristics remained as statistically significant predictors of reporting on risks in the adjusted multivariable model (table 3).
In univariable analyses, articles reporting on field-based practitioners and resident physicians predicted a lesser probability of reporting on supports to improving competence, whereas articles focusing on the other three health professions (compared with any physicians) and on the communicator CanMEDS role predicted a greater probability of reporting on supports for improving competence. The same characteristics, with the exception of field-based practitioners, remained as statistically significant predictors of reporting on supports in our adjusted multivariable model (table 3).
Discussion
Primary findings
Epidemiology functioned as a useful and unique approach to identifying patterns among the features of and factors impacting competence that we explored (ie, risks to competence, supports to competence, competence life-cycle and the CanMEDS roles). The results offer an organising framework for those interested in evidence-informed competency-based education, regulation and continuing professional development.
We found the CanMEDS framework to be useful in categorising the domains of competence discussed in our reviewed articles. This approach borrows from that of previous literature, such as Mickelson and MacNeily, who drew parallels between the CanMEDS roles and the ACGME competencies.12
Our findings demonstrate a predilection for studying competency deficits related to the medical expert role over other competency domains. This is not surprising given the difficulty of implementing and assessing non-medical expert competencies.13 14 However, given the importance of non-medical expert competencies (such as communication, professionalism, leadership, health advocacy and scholarship) for improving quality of care, patient safety14 and patient satisfaction,4 5 15 further research on intrinsic competencies in clinical practice would be beneficial.
Communication was the second most commonly discussed competency, next to medical expert, and was more often discussed in relation to supports than risks to competence. This is reassuring, as patient complaints, dyscompetence15 and academic difficulty in clinical practice4 16 have all been linked to inadequacies in the communication abilities of health professionals. A deeper dive into this literature could help to identify the most effective strategies for mitigating these issues in clinical practice. For example, Beckman et al discussed the effective implementation of a mindful communication programme where practising health professionals shared their clinical experiences and participated in activities to improve their listening skills and develop greater self-awareness.17
It is not surprising that the majority of articles on wellness as a risk to competence pertained to residents. Burnout and depression have been linked to sleep deprivation from long, irregular duty hours and from the stress caused by heavy workloads and clinical responsibilities during residency.18–21 Longer work hours have been shown to inversely impact residents’ acquisition of knowledge.19 Wellness issues such as burnout and depression have been associated with an increased risk of committing serious medical and diagnostic errors.19 21–23 While many countries have sought to improve resident wellness and mitigate the potential threat it carries for patient safety by implementing work hour restrictions, the evidence-base for these restrictions is inconsistent.24 Thus, further efforts may be required to support resident well-being.
It is notable that the risks to competence discussed varied by study location. Cultural or sociological differences, differing clinical practice contexts and environments and/or the differing interests of authors and publishers may account for these geographical differences.
It is interesting that articles on physicians reported more on risks and articles on occupational therapists, physical therapists and pharmacists combined, reported more on supports to competence. It is difficult to tease out whether the observed differences between professional groups reflect a difference in factors impacting competence in clinical practice or are a result of the differing foci/interests of academics and professional bodies.
Strengths and limitations of this study
This is the largest known scoping review of competence of physicians, pharmacists, physical therapists and occupational therapists—either individually or collectively. We screened over 3500 articles for inclusion, covering a 40-year period (1975–2014), with almost 1000 articles included for review.
Epidemiology offers a unique approach to identifying patterns among the features of and factors impacting competence. The results of this study, however, are limited by the appropriateness of the model, topic, organisation, definitions and any inappropriate conceptualisation of this research study.
Scoping reviews offer a ‘bird’s-eye’ view of a topic of interest. As such, this study can identify trends and inventory key areas and draw associations between them, but it is not designed to provide deep complex conclusions as to their cause or effects.17 Rather, this study is best used to help guide and stimulate more in depth research in the topic areas discussed.
We cannot discern for certain whether variation in the topic areas covered (ie, supports, risks) is partly or wholly attributable to the interests of and funding for authors and publishers or to their actual prevalence in clinical practice.
As noted, the four health professions studied were chosen due to personal interest and experience of the authors.
Due to the low number of articles for occupational therapists, pharmacists and physical therapists, they were collapsed into a single professional grouping, which made it possible to compare them to physicians through cross-tabular and regression analyses. Some would view this grouping as diminishing the uniqueness of these distinct health professions and feel that it constrains the utility of these results to some users. Our view was that more would be gained by including occupational therapists, pharmacists and physical therapists and studying them as a group rather than studying only physicians.
Implications for clinicians and policy-makers
Our reviewed literature on the epidemiology of competence is strongest in its application to physicians. It will be important for the other health professions (eg, pharmacists, physical therapists, occupational therapists and others) to further explore the extent to which these findings apply to their profession.
Other research could move beyond the inventorying of risks and supports to competence into quantifying their relative importance (eg, through surveying and interviewing variable groups of registrants and analysing regulatory data).
The conceptual model of epidemiology offers a helpful organising framework for exploring and explaining the complexity of competence of health professionals. Further, it has promise in: empowering registrants about self-management of risks and supports (ie, putting ‘self’ back into regulation); facilitating discussion internally and externally about risks and identifying the means for mitigating, moderating and monitoring them and/or to organise programme based on the relative risks of different factors for different registrant groups and to support the implementation of CBE.
Future work
More targeted and in-depth analysis of these data may offer insights into defining or describing further associations or relationships related to risks or supports to competence (eg, Why are transitions a risk to competence? Does education support competence because of knowledge development or is it because of community of clinical practice?). This could also offer insights about the effects of different risks or supports on competence (eg, what is the effect of age or the participation in quality assurance activities on competence?). Many questions about the risks and supports to competence in health professions remain unanswered.
Opportunities for further study include: validation of the model of ‘epidemiology’ as applied to the risks to competence and supports of competence; focused analysis of one or more of the topics included in the study (eg, age, transitions, quality assurance participation or reflection/self-assessment); studies to compare the relative risks and supports of health practitioners in other data sources (eg, apply the ‘theory’ of epidemiology to real data sets held by medico-legal organisations, regulatory authorities or certification bodies) and investigating whether the impact of supports differ for the different competence life-cycles (eg, is reflection/self-assessment more useful in field-based education or in clinical practice).
Acknowledgements
The authors would like to acknowledge Dr Salvatore Spadafora, Vice Dean of Post MD Education, University of Toronto and Dr Glen Bandiera, Associate Dean of Post MD Education, University of Toronto for all of their support in conducting this research. Specific thanks to Shanil Ebrahim, assistant professor in the Department of Clinical Epidemiology and Biostatistics at McMaster University for his role as the consulting epidemiologist; Sheena Bance, PhD candidate in Clinical Psychology at the University of Toronto, Ontario Institute for Studies in Education, for her assistance with project coordination and abstracting and to our colleague Melissa Hynes, PhD(c) for her assistance with project management and abstracting. We appreciate the funding that supported the final stages of completing this research through the unrestricted educational grant from the Physician Factors Steering Committee and their cochairs Dan Faulkner (College of Physicians and Surgeons of Ontario, CPSO) and Dr Karen Mazurek (College of Physicians and Surgeons of Alberta, CPSA). A special note of appreciation to Wendy Yen and Niels Thakkar of CPSO for their attention to the descriptions of study variables.
References
Footnotes
Contributors SGT, MN and LMMS made substantial contributions to the conception and design of this work, as well as the analysis and interpretation of data for this work; helped draft and critically revise the manuscript and gave final approval of the version to be published. All authors agree to be held accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding This work was supported by unrestricted educational grants received from the College of Physicians and Surgeons of Alberta (CPSA) and the College of Physicians and Surgeons of Ontario (CPSO) in 2014. The funding sources had no involvement in study design, data collection, analysis, synthesis or manuscript review and approval.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement If readers are interested in the complete inventory of articles on each risk and support to competence, or any other study data, please contact the corresponding author.