Objectives Currently relative performance at medical school (educational performance measure (EPM) decile), additional educational achievements and the score on a situational judgement test (SJT) are used to rank applicants to the UK Foundation Years postgraduate medical training programme. We sought to evaluate whether these three measures were predictive of subsequent successful completion of the programme, and thus were valid selection criteria.
Methods Data were obtained from the UK Medical Education Database (UKMED) on 14 131 UK applicants to the foundation programme starting in 2013 and 2014. These data included training outcomes in the form of Annual Reviews of Competency Progression (ARCPs), which indicated whether the programme was successfully completed. The relationship between applicants’ performance on the three selection measures to the odds of successful programme completion were modelled.
Results On univariable analyses, all three measures were associated with the odds of successful completion of the programme. Converting the SJT score to deciles to compare the effect sizes suggested that one decile increase in the EPM increased the odds of completing the programme by approximately 15%, whereas the equivalent value was 8% for the SJT scores. On multivariable analyses (with all three measures included in the model), these effects were only independently and statistically significant for EPM decile (OR 1.14, 95% CI 1.10 to 1.18, p<0.001) and SJT z-score decile (OR 1.05, 95% CI 1.01 to 1.09, p=0.02).
Conclusions The EPM decile and SJT scores may be effective selection measures for the foundation programme. However, educational achievements does not add value to the other two measures when predicting programme completion. Thus, its usefulness in this context is less clear. Moreover, our findings suggest that the weighting for the EPM decile score, relative to SJT performance, should be increased.
- foundation programme
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Contributors DTS wrote the first draft of the paper and completed the analysis. PAT contributed to the supervision of the analyses, edited and reviewed and critically appraised the content of the paper.
Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. DS is employed by the GMC as a data analyst working on the UKMED project. The views expressed here are his views and not the views of the GMC. PAT is supported in his research by an NIHR Career Development Fellowship. This paper presents independent research part-funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health." PAT also wishes to acknowledge funding from the Worldwide Universities Network (WUN) that supports an academic collaboration furthering research into selection into the professions.
Disclaimer The views expressed are those of the authors and not necessarily those of the GMC, the NHS, the NIHR or the Department of Health.
Competing interests DTS is employed by the GMC as a data analyst working on the UKMED project. PAT is supported in his research by an NIHR Career Development Fellowship. This paper presents independent research part-funded by the National Institute for Health Research (NIHR). PAT has previously received research funding from the ESRC, the EPSRC, the DH for England, the UKCAT Board and the GMC. In addition, PAT has previously performed consultancy work on behalf of his employing University for the UKCAT Board and Work Psychology Group and has received travel and subsistence expenses for attendance at the UKCAT Research Group.
Patient consent Not required.
Ethics approval The authors did not need to seek formal NHS ethical approval for this study as it was a secondary data analysis of existing data. UKMED has received a letter from Queen Marys University of London Ethics of Research Committee on behalf of all UK medical schools to confirm ethics exemption for projects using exclusively UKMED data.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Researchers wishing to re-analyse the data used for this study can apply for access to the same data via UKMED.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.