Article Text
PDF
Abstract
Objective As a result of a curriculum reform launched in 2012 at our institution, preclinical training was shortened to 2 years instead of the traditional 3 years, creating additional incentives to optimise teaching methods. In accordance with the new curriculum, a semester-long preclinical module of clinical skills (CS) laboratory training takes place in the second year of study, while an introductory clinical course (ie, brief introductory clerkships) is scheduled for the Fall semester of the third year. Objective structured clinical examinations (OSCEs) are carried out at the conclusion of both the preclinical module and the introductory clinical course. Our aim was to compare the scores at physical examination stations between the first and second matriculating classes of a newly reformed curriculum on preclinical second-year OSCEs and early clinical third-year OSCEs.
Design Analysis of routinely collected data.
Setting One Polish medical school.
Participants Complete OSCE records for 462 second-year students and 445 third-year students.
Outcome measures OSCE scores by matriculation year.
Results In comparison to the first class of the newly reformed curriculum, significantly higher (ie, better) OSCE scores were observed for those students who matriculated in 2013, a year after implementing the reformed curriculum. This finding was consistent for both second-year and third-year cohorts. Additionally, the magnitude of the improvement in median third-year OSCE scores was proportional to the corresponding advancement in preceding second-year preclinical OSCE scores for each of two different sets of physical examination tasks. In contrast, no significant difference was noted between the academic years in the ability to interpret laboratory data or ECG — tasks which had not been included in the second-year preclinical training.
Conclusion Our results suggest the importance of preclinical training in a CS laboratory to improve students’ competence in physical examination at the completion of introductory clinical clerkships during the first clinical year.
- clinical skills laboratory training
- physical examination
- bedside teaching
- curriculum reform
- OSCE
- undergraduate education
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- clinical skills laboratory training
- physical examination
- bedside teaching
- curriculum reform
- OSCE
- undergraduate education
Strengths and limitations of this study
We retrospectively compared objective structured clinical examination (OSCE) scores at physical examination stations between the first and second matriculating classes of a reformed undergraduate curriculum on preclinical 462 second-year OSCEs and 445 early clinical third-year OSCEs at the completion of introductory clinical clerkships.
Stations in both OSCEs were highly standardised and identical checklists were used throughout the analysed period.
That we analysed OSCE records from only one medical school limits the generalisability of our findings.
Extension of the observation period into later clinical years and a longitudinal assessment of individual students’ performance are lacking.
Introduction
Recent decades have witnessed a well-recognised international decline in physical examination skills among medical students and residents.1–6 This has largely been ascribed to an increasing reliance on advanced imaging technologies and laboratory markers. Notably, the inability to properly perform and interpret a physical examination can expose the patient to redundant and costly procedures and, more importantly, may lead to a missed or delayed diagnosis with potential deadly consequences.3 Therefore, in order to prevent the physical examination from becoming merely a lost art, a remedial intervention is necessary. This intervention should be planned early, preferentially already at undergraduate level,7 keeping in mind that junior doctors—engaged in administrative tasks and paperwork—spend three to five times more time in front of a computer screen than in direct contact with patients.8 9
‘To resuscitate clinical skills among clinicians’, Ramani7 proposed—among ‘Twelve tips for excellent physical examination teaching’—integration of simulation with bedside learning as well as systematic assessment of clinical skills (CS), the latter elegantly summarised in a lapidary phrase ‘assessment drives curriculum’. Objective structured clinical examinations (OSCEs) are a recognised assessment tool in medical education. OSCEs are increasingly valued for their ability to predict students’ future performance in the clinical setting.10–15 The approach of using OSCEs has practical implications, providing a basis for the optimisation of clinical education and offering insight into remedial strategies to improve students’ poor clinical performance.10 16 17
Of note, although scores on OSCEs done in the second and third years of study were related to performance on the US Medical Licensing Examination (USMLE) Step 2 CS component,14 this association was not strong, and the OSCE scores in years 2 and 3 were only weakly inter-related.12 Additionally, USMLE Step 2 CS scores and second-year OSCE scores correlated moderately with each other, but this relationship lost significance in a multivariate analysis.11
On the other hand, of the OSCE components taken at the end of the first clinical year (year 3), skills in physical examination and data interpretation exhibited the highest ability to predict students’ performance in five subsequent clinical examinations during the fourth and fifth years of study.10 Scores on an OSCE in the first clinical year have a unique property: they can be linked to future clinical competence and may be used to estimate the contributions of preclinical training in a CS laboratory and subsequent bedside teaching to early clinical competence. Surprisingly, there is limited data available comparing second-year and third-year OSCE scores between various academic years. Chima and Dallaghan15 recently compared OSCE scores for graduates of 2013 and 2014 classes and described a discordance between class-to-class variation in scores obtained during second-year preclinical OSCEs and OSCEs completed at the conclusion of the third-year internal medicine clerkship.
In 2012, a new curriculum was launched at our institution, where the preclinical course is scheduled for a period of 2 years, instead of the traditional 3 years. Our final year of the medical curriculum (year 6) is dedicated to internships in teaching hospitals during which final-year students assist junior doctors by performing similar tasks under direct clinical supervision. The new curriculum includes a preclinical module of CS laboratory training in year 2, supplemented with bedside teaching of basic CS in the Fall semester of year 3, as an introduction to further clinical exposure. The curricular reform has created an additional incentive to make the best possible use of existing educational resources within a limited timeframe. To reach our ultimate goal of maximising early clinical proficiency, continuous optimisation of teaching methods based on an ongoing assessment of the effects of our curriculum reform is necessary.
Our aim was to compare the scores obtained by medical students at physical examination stations between the first and second matriculating classes of the reformed curriculum on preclinical second-year OSCEs and third-year OSCEs at the completion of an introductory clinical course. We hypothesised that differences in the performance between classes on preclinical OSCEs may be reflected in the results of early clinical OSCEs.
Methods
Characteristics of the redesigned curriculum
Within the new curriculum, a 30 hours preclinical module of training in a CS laboratory takes place in the Department of Medical Education of our university in either the Fall or Spring semester of year 2 (15 weeks; 2 hours per week) (table 1). This module includes practical exercises with simulated patients and manikin-based learning. In the Fall semester of year 3, students enter a 12-week module in bedside teaching of basic CS (ie, mini-clerkships in the departments of Internal Medicine, Surgery, Paediatrics and Obstetrics/Gynaecology for 3 weeks each) as an introduction to the core clinical rotations in years 3–6 (table 1).
Traditional and reformed medical curriculum at our university
An OSCE was carried out at the conclusion of both teaching modules, starting from the academic year 2013–2014 and onwards. Each OSCE was composed of several stations covering history taking, physical examination and students’ skills in cardiac/pulmonary auscultation. Additionally, the third-year OSCE included stations assessing students’ ability to interpret laboratory data and a typical ECG, as well as two surgical stations (assessing suturing skills). Our highly-standardised physical examination stations did not differ between the second-year and third-year OSCEs, and they remained unchanged throughout the analysed period, including all tasks randomly chosen from a set of 19 (stations set I) and those from a different set of 16 tasks (stations set II).
Data analysis
We analysed previously collected examination data from second-year OSCEs (February/June 2014 and February/June 2015 examination sessions) and third-year OSCEs (February 2015 and February 2016 examination sessions). As a data source, we used examination records stored in the Department of Medical Education at our university using existing institutional protocols. For the purpose of our analysis, fully anonymised data sets were used in order to ensure personal data protection. Because data sets were anonymous, we were not able to longitudinally estimate individual student performance on the second-year and third-year OSCEs. An individual OSCE score for each physical examination station and data station was calculated from OSCE grades as a relative value, with the reference being an optimal result for the given task, assumed to be 100%.
The accordance of OSCE scores with a normal distribution was estimated by means of the Shapiro-Wilk test. Owing to the non-normal distribution, the data were presented as medians and IQRs. Then, OSCE scores were compared separately between the classes who matriculated in 2012 and 2013 for preclinical second-year OSCEs and third-year early clinical OSCEs, respectively. Between-class differences in OSCE scores were assessed by the Mann-Whitney U test. In order to deal with missing data, the analysis was first performed for OSCE records with complete data points and then repeated including also incomplete OSCE records with at least one available data point. A p value below 0.05 was considered significant. The analysis was performed using STATISTICA (data analysis software system), V.12 (StatSoft, Tulsa, Oklahoma, USA).
Results
Out of potentially eligible 513 second-year and 466 third-year OSCE records, we had excluded 51 and 21 incomplete records, respectively, due to missing data. OSCE records with complete data points were available for 462 second-year students and 445 third-year students from the first two matriculating classes of the reformed curriculum, for a total of 907 OSCEs that entered our final analysis.
Compared with the first class of the new curriculum who matriculated in 2012, higher (ie, better) OSCE scores in physical examination skills were observed for students who matriculated 1 year later in 2013. Improved OSCE scores were noted during both the second year of study (February/June 2015 vs February/June 2014 examination sessions) and the third year (February 2016 vs February 2015 examination sessions) (table 2).
Comparison of OSCE scores (%) between the classes who matriculated into the new curriculum in 2012 and 2013
Additionally, the magnitude of the improvement in median third-year OSCE scores was proportional to the corresponding changes between academic years in the preceding second-year preclinical OSCE for each of two different sets of physical examination tasks (stations set I: 4% and 3%; stations set II: 9% and 8%; for third-year OSCEs and second-year OSCEs, respectively) (table 2). In contrast, no significant changes between academic years were found for the ability to interpret laboratory data or ECGs (ie, tasks which had not been included in preclinical teaching during the second year of the curriculum) (table 2). In regards to auscultation skills, the only significant between-class change was an improved competence in pulmonary auscultation for the second matriculating class of the new curriculum during their third year of study (table 2).
The results were substantially unchanged either on adjustment for different timings of second-year OSCEs in the academic year (ie, separate analyses for OSCEs scheduled after the Fall or Spring semester) or after inclusion of incomplete OSCE records with one or more available data points.
Discussion
Our most salient finding was that OSCE scores at physical examination stations were higher for students matriculating into the newly reformed curriculum in 2013 compared with those matriculating in 2012. A proportional improvement was noticed between 2012 and 2013 cohorts in both preclinical second-year OSCE scores and early clinical third-year OSCE scores. Additionally, the magnitude of the improvements in physical examination competence between classes during the early clinical year correlated with the differences in scores attained by students in 2012 and 2013 matriculating classes during the preclinical second-year OSCE for each of two different sets of physical examination tasks.
The observed association differs from the results of a recent study reporting significantly higher internal medicine clerkship OSCE scores in the first clinical year (year 3 of study) despite a trend of lower second-year preclinical OSCE scores for graduates of the class of 2014 compared with the class of 2013.15 Additionally, the authors observed no association between student performance on preclinical OSCEs and OSCEs completed after an internal medicine clerkship.15 Admittedly, similar to the previously mentioned report,15 it would be appropriate to estimate the effects of preclinical OSCE scores on the results of early clinical OSCEs. However, since our data sets were anonymised, we were not able to analyse individual students’ performance; therefore, a longitudinal assessment of student performance was not possible.
Our observation has several potential explanations. First, inconsistencies in OSCE administration and grading between academic years could account for the observed differences in OSCE scores, as suggested previously.15 However, stations in both OSCEs were highly standardised and identical checklists were used throughout the analysed period. Second, the OSCEs were monitored and supervised by different teams of faculty members affiliated with either the Department of Medical Education (second-year OSCE) or the departments supervising the introductory clinical courses (third-year OSCE). Moreover, at equivalent OSCEs, the performance of students matriculating in 2012 and 2013 was assessed by virtually the same teams of examiners, including only lecturers—previously trained by senior teachers in OSCE planning, administration and grading—with a wide and proven experience in the scoring of OSCE stations. Third, even when considering the possibility of non-uniform grading across the study period, hypothetical year-to-year differences in OSCE scores might be expected for all OSCE components. Nevertheless, we observed a significant year-to-year variation exclusively in OSCE scores reflecting physical examination skills. Finally, the previously described influence of the timing of clinical clerkships in the academic year15 could be excluded because the introductory clinical course was scheduled in the Fall semester for both 2012 and 2013 matriculating class.
In conclusion, the association of year-to-year improvements in scores at physical examination stations in preclinical OSCEs and OSCEs in the middle of the first clinical year is suggestive of the importance of preclinical training in a CS laboratory to improve competence in basic physical examination at the completion of early bedside teaching. A preclinical laboratory teaching module appears to be easier to standardise and more responsive to quality-oriented interventions in comparison to the traditional clinical bedside teaching. Additionally, as to second-year students, it was suggested that an incorporation of formal clinical instruction to their training could be easier compared with those who have already begun clinical clerkships and elective rotations.3 Moreover, the effectiveness of clinical bedside teaching is known to depend on multiple factors, and studies on the relationship between clinical exposure and early clinical OSCE scores have brought conflicting results.18–21 Of note, Martin et al 19 reported no correlation between self-reported clinical exposure to patients and students’ performance on an OSCE taken at the end of the first clinical year. Importantly, Kim and Myung21 described a high variation in the number of patients for whom a medical history was taken or physical examination was performed during clerkships, which is probably indicative of a limited efficacy of bedside teaching in some departments. This observation could also be responsible for the differences between 2012 and 2013 matriculating classes in cardiac and pulmonary auscultation skills after the introductory clinical course—probably due to interclinic variation in the characteristics of patients hospitalised in individual internal medicine departments.
Whether the observed trends will be maintained in later clinical years, requires further investigations with a prolonged follow-up. Additionally, that we analysed OSCE records from only one medical school, poses another limitation to the interpretation and generalisability of our results. Nevertheless, even if seems premature to draw any far-going conclusions for the time being, our findings might have practical implications before future data become available. The results of this assessment can serve as a stimulus for further improvements in teaching physical examination skills, OSCE planning and implementing a remedial intervention for low-scoring students. Our curriculum reform offers a promising and realistic opportunity to put these plans into practice as the new curriculum promotes a continuous optimisation of preclinical and clinical education based on an ongoing assessment of teaching effects. Improved undergraduate education is the starting point to interrupt a vicious cycle of undervaluation and underuse of the physical examination in clinical decision-making with regard to real-world patients.
References
Footnotes
An.Su. and MN are joint senior authors on this work.
Contributors Study conception and design: JŚ, MN, An.Su. Study supervision: MN, An.Su. Data entry: JŚ, AS-P, KJ, TC, Ag.Sk., KC. Data analysis and interpretation: JŚ, An.Su., MN, EW-S, OK, BC, MK. Manuscript drafting: JŚ, An.Su. Contribution to discussion and manuscript revision: MN, MK, EW-S, OK, AS-P, KJ, TC, Ag.Sk., BC, KC. Approval of the final version of the manuscript: all the coauthors.
Funding This study was supported in part by the Faculty of Medicine, Jagiellonian University Medical College (grant no K/ZDS/006370).
Competing interests None declared.
Ethics approval We retrospectively analysed routinely collected administrative data, that is, examination records stored in the Department of Medical Education of our university using existing institutional protocols under the supervision of the Head of the Department (MN), one of the senior authors on this work. For the purpose of our analysis, fully anonymised data sets were used to ensure personal data protection.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement A fully anonymous data set is available from the authors (surdacki.andreas@gmx.net) on request.