Article Text
PDF
Abstract
Objectives The formative aspect of the mini-clinical evaluation exercise (mini-CEX) in postgraduate medical workplace-based assessment is intended to afford opportunities for active learning. Yet, there is little understanding of the perceived relationship between the mini-CEX and how trainees self-regulate their learning. Our objective was to explore trainees’ perceptions of their mini-CEX experiences from a learning perspective, using Zimmerman’s self-regulated learning theoretical framework as an interpretive lens.
Design Qualitative, using semi-structured interviews conducted in 2017. The interviews were analysed thematically.
Setting Geriatric medicine training.
Participants Purposive sampling was employed to recruit geriatric medicine trainees in Melbourne, Australia. Twelve advanced trainees participated in the interviews.
Results Four themes were found with a cyclical inter-relationship between three of these themes: namely, goal setting, task translation and perceived outcome. These themes reflect the phases of the self-regulated learning framework. Each phase was influenced by the fourth theme, supervisor co-regulation. Goal setting had motivational properties that had significant impact on the later phases of the cycle. A ‘tick box’ goal aligned with an opportunistic approach and poorer perceived educational outcomes. Participants reported that external feedback following assessment was critical for their self-evaluation, affective responses and perceived outcomes.
Conclusions Trainees perceived the performance of a mini-CEX as a complex, inter-related cyclical process, influenced at all stages by the supervisor. Based on our trainee perspectives of the mini-CEX, we conclude that supervisor engagement is essential to support trainees to individually regulate their learning in the clinical environment.
- workplace based assessment
- geriatric medicine
- mini-cex
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Zimmerman’s self-regulated learning framework provided an interpretive lens on participants’ perspectives of the mini-clinical evaluation exercise.
This study brings to light the personal, interpersonal and contextual influences on a trainee’s self-regulated learning.
Interview data were only collected in Melbourne with geriatric medicine trainees whose experiences may not fully represent the experiences of other specialist trainees.
The study relied on trainees’ recalled experiences of their workplace-based assessment experiences.
Introduction
One promise of competency-based medical education is that learning is emphasised in the assessment process.1 2 Consequently, formative assessment is a key component in the design of competency-based assessment in postgraduate training.3–5 Assessment should fulfil three key purposes: certify achievement, facilitate feedback to the learner to drive improvement and foster self-assessment to promote life-long learning.6 7 It has been argued by Sandars and Cleary and Nichol and Macfarlane‐Dick that effective, formative assessment should incorporate these principles to enhance trainees’ self-regulated learning (SRL).8 9
Norcini et al developed the mini-clinical evaluation exercise (mini-CEX) assessment tool to facilitate observation-based feedback and promote trainee self-assessment in the workplace10; it is a widely used tool in postgraduate medical education.11 12 Mini-CEXs are conceptualised as an encounter that involves a supervisor observing a trainee performing a nominated, routine clinical task for approximately 10–15 min, with the supervisor providing verbal and written feedback using a standardised marking template.10
Studies have explored the experiences and perceptions of trainees with a focus on the educational effect of the individual assessment tool.13 14 Since 2004, there have been >40 studies examining both trainee and assessor perceptions of workplace-based assessment (WBA).15 These studies have evaluated the educational effect by eliciting learners’ attitudes towards the use of the instruments.16 Many of the studies have been conducted in the UK, using surveys to explore trainee perceptions.4 17–22 The efficacy of feedback following the assessment is a prominent theme in most studies. Trainees have reported that the use of the mini-CEX has increased the quantity of feedback,3 18–20 22 while in other studies both the quantity and quality of feedback is reportedly improved.19 21 23 Other studies report that the provision of feedback does not always encourage self-assessment, nor translate to an action plan for improvement.20 22 24
Overall, the literature conveys a mixed picture of the influence of the mini-CEX tool on feedback. Trainees have expressed a lack of understanding regarding the purpose of the mini-CEX, reporting difficulty distinguishing the formative nature from the evaluative aspects of the assessment.5 25 26 The confusion possibly explains affective aspects such as trainee anxiety and stress attributed to these assessments, although being observed by a supervisor may evoke these emotions.18 25 27 Furthermore, trainees have reported valuing the close contact with the assessor and the reassurance of competency that assessment provides, but they are reluctant to display their weaknesses.18 22 25 The influence of workplace culture is also a strong theme, with negativity centred on the time commitment and the difficulties associated with organising an assessor.3 17 18 20 22 28 A systematic review of user perceptions reported stakeholder disengagement and low morale impacting assessment performance.15 The perceived assessment burden has contributed to a ‘tick box culture’.20 29 This approach leads to WBA tools being misused and trivialised by stakeholders, and the potential loss of a useful learning opportunity.15 30
According to Brydges and Butler, a different approach to examine the educational effect of the mini-CEX is to focus on active learning, since trainees’ learning in the clinical environment is highly influenced by their capacity for SRL.31 Self-regulation refers to the ‘self-generated thoughts, feelings and behaviours that are planned and cyclically adapted to the attainment of personal goals’.32 Self-regulation is cyclical because the feedback from prior performance is used to make adjustments during current or future efforts.32 However, the relationship between the mini-CEX and the complex interactive process of a trainee’s SRL has not yet been elucidated.
Self-regulation, according to Zimmerman, is an interaction of personal, behavioural and environmental triadic processes.33 From a social cognitive perspective, self-regulatory learning processes comprise the forethought phase, the performance phase and the self-reflection phase. Although there have been many previous studies examining the perceptions of postgraduate trainees of the mini-CEX in WBA,34 35 to the best of our knowledge, none have used the SRL framework as an interpretive lens. In particular, the covert integration of personal, interpersonal and contextual influences on a trainee’s SRL has not been explored. This study adopts Zimmerman’s model of SRL32 to examine the experiences and perceptions of physician trainees in geriatric medicine with the mini-CEX assessment. The research questions were:
What are the perceptions and experiences of specialist trainees in geriatric medicine of the mini-CEX exercise?
What is the perceived relationship between the mini-CEX and trainees’ SRL?
Method
Design
The study has a qualitative design to systematically explore how trainees experience the mini-CEX assessment in order to understand their meaning.36 We used individual semi-structured interviews to explore participants’ lived experiences and perceptions. The interpretation of the data is underpinned by a social constructivist theory, which acknowledges that reality is co-constructed between the researcher and the researched and shaped by experiences and context.37 The research team consisted of a specialist clinician in geriatric medicine who has experience as a supervisor with the mini-CEX, and two medical education researchers, one of whom is also a clinician with supervisory experience. Reporting of the research has been guided by the consolidated criteria for reporting qualitative research (COREQ) checklist within the constraints of the word limit.38
Setting and participants
The study was conducted in 2017 in Melbourne, Australia, with trainees in geriatric medicine. Geriatric medicine training is a 7-year postgraduate programme, overseen by supervisors who are specialist clinicians. In the final 3 years, these senior trainees are described as advanced. Each advanced trainee must complete four formative mini-CEX assessments per year.
Participants who were invited to participate in the study were advanced trainees who had completed a minimum of two mini-CEXs. There are 80 advanced trainees in Melbourne. Purposive sampling39 was adopted to include trainees across the final 2 years of advanced training in order to ensure that potential participants had enough experience with the mini-CEX. Trainees who were currently working alongside or being directly supervised by author 1 were excluded from the study to avoid any unequal relationships between trainees and the researcher. Author 1 introduced the study at a training programme in April 2017, and interested participants were invited to contact the first author. Potential participants received a plain language statement. Written consent was obtained from participants. No patients were involved in this research.
Patient and public involvement
There were no patients in this study. There was also no patient information (medical records) accessed for this study. Participants were medical doctors who gave their permission to be interviewed.
Interview guide
A semi-structured interview guide was developed, informed by both the existing literature and the SRL framework, to explore the trainee’s perceptions and experiences of the mini-CEX.32 The interview guide consists of a number of bullet points, including questions, for each topic area. These bullet points functioned as potential questions and prompts. The interview guide was piloted with two trainees who had recently completed their training. There was further iterative refinement of the guide by the project team and by an external expert in medical education research (see online supplementary appenidx 1).
Supplemental material
Data collection and analysis
All interviews were conducted in a private office at each participant’s workplace by the first author. The interviews were audio-taped, transcribed, then de-identified. Author 1 checked the accuracy of the transcripts and also familiarised herself with the data. She then manually performed the first open coding and categorisation. There was separate independent open coding and categorisation analysis done by the other two researchers, who sampled a cross-section of five transcribed interviews. This procedure found only minor disagreement among the research team, with discussion resolving any discrepancies. The analysis of themes was an iterative, inductive, collaborative process with exploration and discussion by the group. The interpretation of the data was sensitised conceptually by the Zimmerman’s SRL theoretical framework.32
Researchers agreed that data saturation was reached after the eighth participant, with no new themes emerging with further data collection and analysis. The data analysis was thematic, following the systematic six phases approach described by Braun and Clarke.40
Results
Twelve advanced trainees participated in the project. All interviews were between 30 and 45 min in duration. There was an equal distribution of male and female participants, with the age ranging from 29 to 38 years. Most had completed an undergraduate medical education course, with only one participant completing a postgraduate programme. Nine of the participants were born in Australia, and three participants born overseas (Asia and Central America). The trainees had completed between 4 and 15 mini-CEX assessments. The broad range of activities that the trainees reported performing in a mini-CEX are shown in table 1. As can be seen from this table, trainees’ experience of Mini-CEX assessments go beyond patient care to include research activities, such as running a journal club.
Reported tasks performed by trainees in geriatric medicine for a mini-CEX assessment
Four major themes emerged in the interviews with participants. Three of the themes cluster around the three phases of the SRL framework. The first theme was goal setting; this aligns with the forethought phase of SRL. The second theme, task translation, resonates with the performance phase of the SRL cycle. This second theme described trainees’ reports of how they translated their forethought goal into the clinical setting, rather than the meta-cognitive self-monitoring of performance. The third theme, perceived outcome, aligns with the self-reflection phase. These three themes were influenced by the fourth theme, supervisor co-regulation.
Goal setting and goal orientation
The theme of goal setting refers to the desired result of the assessment. The goals set by participants included educational goals as well as the goal of fulfilling the postgraduate training requirements. The participants with an educational focus reported that the principal aim of the mini-CEX was to receive valued observation and feedback from a supervisor.
And how I chose it (task) was because it’s one of those areas that I find hard and it’s also an area that you really have to be seen doing it to get feedback; you can’t sort of get feedback from general day-to-day patient care about that sort of skill. (Trainee 6)
Other trainees saw the goal as gathering and establishing evidence of their progression through training, as well as identifying areas of weakness for supervisor attention.
I think all of our assessments, including mini-CEXs, is a chance to basically have a check-mark, to make sure that the expected level of learning is being achieved and that deficiencies can be identified, and that appropriate … learning can occur following it. (Trainee 11)
Some of the trainees described the goal of a mini-CEX primarily as a task to tick a box and fulfil the training requirements.
How could I put it? It’s, it’s often a box to be ticked, a requirement for your training purposes that you tick. (Trainee 2)
A subtheme that was aligned to goal setting was that of goal orientation, which had a learning as well as a performance dimension. A learning goal orientation is defined as valuing the process of learning in order to increase mastery.
… everyone wants to improve in their weaknesses and this is the time we can approach the consultant. We have only 3 years—very limited—and within 3 years like we have to approach the consultants ‘In this area I want to learn. Can you supervise me please?’ (Trainee 7)
Participants with a performance goal orientation reported setting goals that documented the achievement of training milestones, seeking tasks that would yield favourable judgements of their competence.
Cause I want to have a record of glowing mini-CEXs because all record of your performance is important, ‘cause I want to, you know, get a, get a job in the future. I know that consultants can’t see the mini-CEXs but the College can. (Trainee 3)
Most participants described either a learning or a performance goal preference when selecting tasks; however, several were aware of a perceived conflict and tension in their orientation. These participants were reluctant to select tasks that were challenging but more beneficial for their learning, preferring to demonstrate their strengths in a competitive employment environment.
Task translation
The second theme, task translation, describes the conversion by participants of the conceptualised goal from the first phase, into the real-time clinical setting. The universal word used by participants in describing the translation process was ‘opportunistic’: the activity is performed when the right set of circumstances occur, including the supervisor being available. Seizing an opportunity also occurred when completion deadlines approached.
Most of my mini-CEXs were spur of the moment, to be honest. I just would keep the forms with me and then, if I thought this was a good opportunity to maybe try and get it done, I would just utilise it at that time. (Trainee 10)
Some participants described a more strategic approach and this was aligned with a learning goal orientation. These participants described identifying the goal, selecting an appropriate patient and context, notifying the supervisor and allocating time for preparation and feedback.
I felt it was something that I could manage but I just wanted feedback about it. So, yeah, I chose it and I tried to prepare a little bit ‘cause I knew I was going to be supervised and also ‘cause it’s kind of a … What is it? It is a challenging thing to do so I did a little bit of reading and thinking about the person in particular, and what kind of questions we’d need to go through. (Trainee 6)
Perceived outcome
The theme of perceived outcome refers to the participants’ view of the overall consequences of the assessment for their learning. In this theme, there was a diversity of views that often reflected and aligned with those described in goal setting. Overall, most participants had a positive impression of the benefits of the assessment but often qualified their remark. Several participants who had a learning goal orientation described the assessment as being particularly useful when certain conditions were met.
I think when I’ve chosen the activity, when I’ve set aside time for it and we’ve actually allocated time, and when the consultant is engaged in the process and, and has, has taken interest in, in giving me feedback. (Trainee 6)
In the subtheme of feedback, participants were clear in their descriptions of feedback quality. Feedback that was perceived as meaningful was delivered in a private space with an engaged supervisor in an unhurried manner. Feedback that was perceived by participants as unhelpful was brief, generic and delivered by supervisors in the workflow, without much forethought or time.
Just, just some, yeah, it was verbal feedback. I, like I said, it was pretty, it was pretty minimal. I guess would you use the word ‘token’? (Trainee 12)
According to the participants, feedback was reportedly critically important in self-reflection, assisting with calibrating self-evaluation and self-reaction.
How can I put this? I think the, what the words that the, words that mean a lot to me, so the words that the consultant is using probably makes me feel better or worse about myself as a clinician. It affects how I feel about myself as a doctor I think because I think words are quite powerful. (Trainee 4)
Supervisor co-regulation
The final theme refers to the influence of the supervisor in the co-regulation of all phases of the assessment. In the goal setting phase, participants described supervisors identifying suitable learning goals especially in the earlier years of training.
That’s been said to me. ‘This is a rotation where capacity is going to come up all the time. You need to be good at it. Let’s do a capacity assessment mini-CEX and then we can go from there.’ So I think it’s a way of jumping off and it’s not threatening for them to say it that way because we all know we need to do mini-CEXs. (Trainee 4)
In the translation phase, the collaboration of supervisors was essential for the performance of the assessment, with participants displaying sensitivity to the engagement of the supervisor.
Many times they may not want to do it because of whatever reasons and things. I guess the clinical, on the wards, wherever, it’s not, you have to make that time. They find it a burden to do it with you. (Trainee 10)
The perceived relationship with the supervisor reportedly impacted significantly on the goals that participants chose, the discomfort they felt during the performance and their receptivity to feedback.
And I think I wouldn’t put myself in that situation if I felt the consultant wasn’t somebody who was good at understanding, you know, what it’s like to be in your shoes. (Trainee 6)
Discussion
This study of trainees’ perceptions of the mini-CEX has been informed by the SRL theoretical framework. The findings suggest that the performance of a mini-CEX was perceived by trainees as a complex process that was influenced at all stages by the supervisor. Additionally, there was a self-reported range in ability among the participants to optimally self-regulate their learning when performing a mini-CEX. The three themes of goal setting, task translation and the perceived outcome, resonate with the three phases described earlier by Zimmerman’s SRL framework33 with elaboration on the significance of goals, and feedback following the assessment. The theme of supervisor co-regulation for trainees’ reports of SRL behaviour may have been influenced by the relational roles of the interview context (supervisor-trainee roles). These elements are discussed below.
For the participants, the approach to the mini-CEX was influenced by their perceived individual goal setting and goal orientation. Goal setting and goal orientation are seen as powerful motivational factors in SRL theory.41 42 The motivational properties of goals contribute to our understanding of the findings, as well as those previously reported. In our study, trainees who reported approaching the tasks with a specific learning goal also described selecting more challenging tasks for the assessment. They also described expending more effort in the strategic planning of the assessment during the task translation phase to ensure attainment of their goals. These participants were more likely to report positive outcomes from their assessment experiences, with the important proviso that the feedback had effectively targeted their goals. There is evidence that process goal attainment can be intrinsically motivating in its own right for these learners.33 This finding underscores the cyclical nature of the process, with the forethought phase determining significant downstream effects. These ‘proactive’ participants reported behaviours that resonated with more developed self-regulatory processes as described in the Zimmerman’s SRL framework. Zimmerman believes that proactive learners self-regulate more effectively because they engage in high-quality forethought, which in turn improves their self-regulatory functioning during subsequent phases.43 The ‘tick box’ approach by trainees to WBA20 44 was evident in this study. The significance of this goal and its implications for learning is more fully appreciated when seen as part of the SRL forethought processes. Those trainees who described adopting a ‘tick box’ goal, generally took an opportunistic approach in the translation phase and consequently perceived less positive outcomes of the mini-CEX.
Previous studies have found that goal orientation can influence trainees’ perceptions of formative assessment.45 A study exploring medical students’ SRL in the clinical environment also reported goals as an influential motivational factor for learning.46 This study illustrates the dual significance of trainees’ goal setting and goal orientation to the approach of a mini-CEX and the impact on the subsequent phases of the SRL cycle. In addition, feedback from the supervisor to our participants was a key factor in all the processes that occurred after the performance. These processes included self-evaluation, affective responses and self-reflection. Although the literature has focused on perceived feedback, it has not fully captured the power of feedback as contextualised in the SRL framework. For goals set in the first phase of SRL to be effective, learners need summary feedback that reveals progress in relation to their goals.47 Summary feedback is believed to be a moderator of goal effects, in that the combination of goals plus feedback is accepted to be more effective than goals alone.47 Some participants in this study reported difficulty obtaining observation-based feedback from supervisors, a finding previously reported.48 The mini-CEX was a means of securing this desirable information for progress towards their goals. The multifaceted nature of effective feedback has been previously described, including its benefit in formative assessment.9 49 50
The central role of the supervisor in facilitating all aspects of the SRL cycle emerged as a major theme in the study, a finding which resonates with previous studies of WBA.20 22 51 In addition, a key feature of a social cognitive model of self-regulation is the roles of social, environmental and self-influences.32 Educational research has established that an ideal form of self-regulation cannot be assumed when opportunities for self-regulation are provided.31 The engagement of supervisors is necessary to support the challenges that arise as individual trainees learn how to focus their self-regulated activities to meet the demands of the clinical environment.31 52 The relationship with supervisors and the perception by trainees of the lack of supervisor engagement in WBA have both been previously reported in the literature.15 51 This study provides new insights because it identifies the co-regulation at all stages of a trainee’s SRL cycle by the supervisor. The supervisor’s role is also contextualised as a support to the SRL of each individual trainee as required.
A further finding of this study was the range of activities that the participants reportedly included as part of their mini-CEX assessments. Some of these activities such as journal club participation were outside the scope of a clinical skill assessment. It should be noted that there are no guidelines currently in postgraduate physician education in Australasia to inform the selection of clinical skills to be observed during a mini-CEX encounter.
While the interviewer was reflexive and endeavoured to bracket herself,53 her role as a practising physician in geriatric medicine and as a supervisor is a source of potential bias in the interpretation of the results, and is an accepted part of the constructivist paradigm.37
Conclusions
By employing the SRL theoretical lens, new insights into the covert, complex interaction between a trainee’s SRL and the performance of a mini-CEX in the clinical environment have been revealed. The findings highlight the supervisor as a key co-regulator of all stages of the SRL cycle. Our participants were informed consumers of feedback with feedback quality contributing significantly to the perception of outcome and preparation for future efforts of WBA. The SRL theoretical framework has also facilitated a re-interpretation of the existing literature in a holistic, contextualised approach.
This study has important implications for both trainees and supervisors involved with postgraduate training in CBME. With further implementation of this educational approach, trainees will also require a deeper understanding of their individual SRL to participate meaningfully in formative learning opportunities. Supervisors may benefit from targeted education regarding their role in supporting a trainee’s SRL and improving feedback quality.
Acknowledgments
The authors would like to thank the geriatric medicine trainees who participated in the study, and also the support of the Director of the Victorian Geriatric Medicine Program, Associate Professor Benny Katz. The authors would like to thank Professor Elizabeth Molloy for her comments in preparing the manuscript.
References
Footnotes
Contributors EK contributed to conceptualising the study and study design, and preparing the ethics application. EK recruited participants, piloted the interview questions and conducted the interviews. EK conducted the initial analysis of the data and subsequent analysis. EK took the lead on writing the manuscript. EF advised on conceptualising the study and assisted with review of data analysis and analysis discussion. EK provided feedback on various versions of the manuscript. RW-K advised on conceptualising the study and study design. She submitted the ethics application, advised about interview questions and incorporating the theoretical framework and designing the coding framework. RW-K reviewed the coding and thematic analysis, assisted with drafting the article and reviewed the final version.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Ethics approval Ethics approval for the study was provided by the Human Ethics Advisory Group, University of Melbourne (ethics ID: 1748832).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement In compliance with the ethics approval for this project, the interview data are not available for sharing.
Patient consent for publication Not required.