Article Text

Original research
Changing trends in ethnicity and academic performance: observational cohort data from a UK medical school
  1. Prokriti Mukherji1,
  2. Maryam A Adas2,
  3. Benjamin Clarke3,
  4. James B Galloway2,
  5. Thomas Mulvey4,
  6. Sam Norton5,
  7. Jonathan Turner4,
  8. Mark D Russell2,
  9. Heidi Lempp2,
  10. Shuangyu Li6
  1. 1Marketing Department, King's College London, London, UK
  2. 2Centre for Rheumatic Diseases, King's College London, London, UK
  3. 3Postgraduate Medical & Dental Education Centre, King's College Hospital NHS Foundation Trust, London, UK
  4. 4Clinical Education Department, King's College London, London, UK
  5. 5Psychology Department, Insitute of Psychiatry, King's College London, London, UK
  6. 6Division of Medical Education, King's College London, London, UK
  1. Correspondence to Dr James B Galloway; james.galloway{at}kcl.ac.uk

Abstract

Objective Quantify differential attainment by ethnicity in undergraduate medical assessments and evaluate whether institutional efforts to reduce the attainment gap have had impact.

Design Observational cohort study.

Setting A single UK MBBS medical programme.

Participants Pseudonymised data of adults aged ≥18 years enrolled in one of the UK MBBS medical programmes between 2012 and 2018. Ethnicity was self-declared during enrolment as White, Asian, Black, mixed and other.

Main outcome measure Module mark (distinction, merit, pass, fail) graded according to a variety of assessments, including single best answer examinations, objective structured clinical examinations and coursework submissions. All modular assessments are graded as a percentage. Logistic regression models were used to calculate relative risk ratio to study the association between ethnicity and attainment gap over a calendar and scholastic year. Models were adjusted for age, gender, social deprivation and scholastic year of study.

Results 3714 student records were included. In the sample, 2134 students (57%) were non-white. The proportion of non-white students increased from 2007 (49%) to 2018 (70%). Mean age was 18 (IQR 18–21) and 56.6% were females. Higher proportion of non-white students 218 (24.8%) were from more deprived backgrounds versus white 76 (14.8%). Compared with non-white, there were no significant differences in the proportion of students failing assessments. However, white students were more likely to achieve merit (relative risk ratio 1.29 (95% CI 1.08 to 1.45)) or distinction (1.69 (95% CI 1.37 to 2.08)). Differences in attainment gap have remained unchanged over time, and for black students, attainment gap grew between their first and final year of study.

Conclusion A similar proportion (97%) of non-white and white students had a passing score, but attainment gap for higher grades persists over years despite widespread efforts in medical schools to diminish the attainment gap linked to ethnicity. Our findings are from a single institution, thus affecting generalisability.

  • medical education & training
  • education & training (see medical education & training)
  • epidemiology

Data availability statement

Data are available upon reasonable request. Data used in this study were collected from students records and are under the control of the data holders at the medical school. Only aggregated data are available upon reasonable request. Personal data cannot be shared due to confidentiality reasons. All data relevant to the study are included in the article. All figures and tables included in this article are original.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This study uses a large cohort to assess the deferential attainment across students.

  • More than half of the students analysed come from ethnic minority backgrounds.

  • We accounted for possible confounders in our analyses but need to acknowledge that there remains likely unmeasured confounding.

  • We focused only on undergraduate students in a 5-year MBBS programme, thus affecting generalisability to other programmes.

Introduction

During the pandemic, there has been substantial publicity around ethnicity and inequality. Differences in health outcomes in almost every level of society can be linked to ethnicity, and medical education is no exception.1–4 A landmark systematic review by Woolf et al, described the magnitude of differential attainment in medical education at undergraduate and postgraduate levels.5 The review demonstrated worse performance for non-white students, with a summary estimate (using Cohen’s d of 0.42) indicating a moderate effect size for the difference.6 The effects were comparable for computer-marked assessments and viva-based exams.5

Many studies have previously explored the reasons for differential attainment by ethnicity. Although there may be related to prior education attainment, age, gender and socioeconomic position (SEP), these factors do not fully explain the observed differences.7–10

Over the last decade, there have been concerted efforts to reduce differential attainment by ethnicity in healthcare education. Examples include expanding access to medical school for non-white students at both entry into and through the programme. At the entry stage, the changes include Widening Participation programmes, and during the programme, through a process of (1) rethinking and reframing the medical curricula in collaboration with students (eg, decolonising the curriculum programmes) and (2) training in cultural competency and (3) unconscious bias education for faculty and examiners.11–17

We included data from one of the largest UK medical schools, with a typical year group size of 400 medical students. In Central London, medical schools attract a diverse population, with high proportions of non-white students. We sought to quantify differential attainment in exam performance by ethnicity in our student population and evaluate if this has reduced over the last 10 years. In parallel, we investigated whether differential attainment changed between entry to, or exit from, medical school.

Methods

Data source

The medical degree course is a 5-year programme, which accepts entrance directly after secondary school as well as graduate entry. To be accepted into the Bachelor of Medicine, Bachelor of Surgery (MBBS) programme, applicants must have school-level science qualifications. UK-resident and international students are accepted into the programme. The first year of study is based on preclinical sciences and is assessed only with computer-marked written exams. Years 2–5 are based on clinically focused content and are assessed through a combination of computer-assessed exams (40% contribution), Objective structured clinical examinations (OSCEs) (40% contribution) and projects (20% contribution). Assessments are graded as a percentage, with grades 70% and above classified as distinction, 60%–69% as merit, 50%–59% as pass and below 50% as fail.

Study population

Pseudonymised data were extracted from the student records system for individuals who were enrolled on one of the UK MBBS programme. Data were available from September 2012 to July 2019, irrespective of year of study for individual students. These years corresponded to students enrolled from 2007 (who would only contribute their final year of examination data) to 2018 (who would only contribute their first year of examination data). If students intercalated during their programme, their results linked only to the MBBS programme were included.

We did not include results from 2020 due to the COVID-19 pandemic.

Self-declared ethnicity status was defined by the 2011 UK census five main ethnic groups classification.18 That is Asian (Bangladeshi, Indian, Pakistani, Chinese and other Asian); black (Black African, Black Caribbean and other Black); mixed (white and black Caribbean/white and black African/white and Asian/any other mixed or multiple ethnic backgrounds); other (Arab/any other ethnic group) and white (white, white Irish and other white).

Outcomes

The primary outcome of our analyses was exam grade boundaries (categorical variable: distinction, merit, pass, fail). A secondary outcome explored was the summary year module marks (continuous variable). Module marks relate to a variety of assessments, and include single best answer examinations, OSCEs and coursework submissions. All these modular assessments are graded as a percentage. Data were not separated by type of assessment within each year as this information was not accessible to our research group. We included only results from first attempts at any assessment (ie, not resit examinations).

Predictors

The first predictor we investigated was a calendar year of study, to investigate whether the attainment gap has changed over time. The second predictor was a scholastic year in a study for individual students, to evaluate whether attainment gaps change during their medical course.

Covariates

Confounders included in our models were age on course entry, gender and SEP. The SEP information was available from 2013 onwards. SEP was defined by entry via the Widening Participation programme. Students were listed as being in the Widening Participation group if, at the time of application to the course, they: (1) lived in geographic regions with historically low participation in higher education; (2) lived in regions described as less prosperous in the Acorn geodemographic segmentation system (https://acorn.caci.co.uk/) and (3) had been a participant in a locally organised university outreach programme. One of the main aims of the Widening Participation programme in England is to encourage students from different socioeconomic backgrounds to apply for higher education.19

Statistical analyses

Descriptive comparisons by ethnicity were tabulated with tests for statistically significant differences (χ2 for categorical variables; Mann-Whitney U for non-parametric and continuous variables).20 21

Multinomial logistic regression22 was performed to estimate the association between ethnicity and grade boundary attainment across all assessments, to provide context for our main analyses. Linear regression23 was used to estimate the association between ethnicity and examination marks. We used linear regression to describe the change in attainment gap by calendar year and scholastic year. Models incorporate unique student number as a clustering variable to account for multiple results per student. Models were adjusted for age, gender, widening participation and scholastic year of study. Results were presented graphically and as marginal means (with 95% CIs) from the regression outputs.

Sensitivity analyses explored the ethnicity subcategories. Our analyses used a complete-case approach. Missing data categories are described in the results tables. No imputation for missing data was performed as the assumptions of missingness could not be verified. Analyses were performed Stata V.17.0 (StataCorp).

Patient and public involvement

There was no patient or public involvement in the design, conduction, reporting and dissemination plans of this study.

Results

A total of 3825 students were enrolled in the 5-year MBBS programme from 2007 to 2018 and undertook assessments between September 2012 and July 2019. Only 3714 students had known ethnicity information and these students were included in further analyses: this comprising 19 247 individual assessment outcomes.

Baseline characteristics are described by ethnicity in table 1, with a breakdown by individual subgroups in online supplemental table 1. A total of 2134/3714 students (57%) were non-white. The proportion of non-white students increased over time, from 49% of those enrolled in 2007 to 70% in 2018, with the trend more apparent in recent years (see figure 1). Median age across the cohort was 19 (IQR 18–21). A smaller proportion of non-white students 67 (3.1%) were in older age groups compared with white students 143 (9.1%). A total of 2097 (56.6%) of all students were females, with no differences across ethnic groups. More non-white 218 (24.8%) than white students 76 (14.8%) were enrolled via the Widening Participation programme.

Figure 1

Percentage of white and non-white students enrolment. Percentage of the white and non-White students enrolled in the 5-year MBBS programme from 2007 to 2018. MBBS, Bachelor of Medicine, Bachelor of Surgery.

Table 1

Baseline characteristics

The regression analyses to estimate the association between ethnicity and assessment outcomes (see table 2) confirmed the presence of an attainment gap, which was driven by differences in students achieving the highest grades. There was no difference by ethnicity in proportion of students who were unsuccessful in their assessment attempts, neither at White vs non-White, or when comparing by ethnicity subgroups (see online supplemental table 2).

Table 2

The associations between ethnicity and assessment results

Compared with non-white students, white students were more likely to achieve a merit (adjusted relative risk (aRR) 1.29 (95% CI 1.08 to 1.45) or distinction (aRR 1.69 (95% CI 1.37 to 2.08)). The associations were determined by differences for the Asian and Black ethnic subgroups, with mixed and other ethnic groups not performing significantly differently from the white group. Non-white student scores were 1.9% lower in assessments compared with white students. This equates to a standardised mean difference of 0.19 (95% CI 0.12 to 0.26).

The analysis of the changing attainment gap over calendar year is shown in figure 2. Over the time studied, there is evidence of average mark improvement in all groups. Yet, across every calendar year there were significantly lower marks awarded to non-white students. The absolute difference between the groups grew over time. There was no evidence that the pattern of change over time varied within the ethnicity subgroups.

Figure 2

Attainment gap over the calendar year for students with different ethnic backgrounds. Attainment gap change over the calendar year for students enrolled in the 5-year MBBS programme from 2007 to 2018 and undertook assessments between September 2012 and July 2019. We used linear regression for this analysis. Models incorporate unique student numbers as a clustering variable to account for multiple results per student. Models were adjusted for age, gender, widening participation and scholastic year of study. Results are presented as mean in circles and 95% CI in vertical lines. MBBS, Bachelor of Medicine, Bachelor of Surgery.

The analysis of performance within scholastic year of study in figure 3 shows that the attainment gap is visible in year 1 of study and persists throughout the course. The breakdown by ethnicity subgroup shows a pattern that is different for the black student group, for whom the attainment gap worsens during the 5 years of study. The differential attainment increases across the years of study (but interpretation of the exact cause is somehow speculative, as explanations of attainment differences were not explored in this study).

Figure 3

Attainment over the scholastic year for students with different ethnic backgrounds. Attainments over the scholastic year for students enrolled in the 5-year MBBS programme from 2007 to 2018 and undertook assessments between September 2012 and July 2019. We used linear regression for this analysis. Models incorporate unique student number as a clustering variable to account for multiple results per student. Models were adjusted for age, gender, widening participation and scholastic year of study. Results are presented as mean in circles and 95% CI in vertical lines. MBBS, Bachelor of Medicine, Bachelor of Surgery .

Discussion

The attainment gap between white and non-white students has not changed over time, despite institutional efforts to reduce inequality in assessment outcomes associated with ethnicity using data from a single institution. In addition, we have demonstrated that during their medical school studies, the gap in attainment for students of black ethnicity worsens.

The possible explanations for the attainment gap itself are subject of many previous publications8–10 and were not the focus of this study. Our goal was to evaluate temporal changes over calendar year and by scholastic year of study.

The lack of any discernible reduction in attainment gap over time is cause for concern. There are several possible explanations for the failure of the current strategies.

First, we need to acknowledge that the efforts to reduce attainment gaps started prior to 2012(our data span 2012–2019). It is notably that the attainment gap we have described is lower than what has been reported previously.5 In our analysis, the standardised mean difference was 0.19 (95% CI 0.12 to 0.26) compared with the pooled standardised mean difference of 0.42 (0.35–0.49) from Woolf et al, which used data from 16 reports of UK undergraduate medical degree courses spanning 1970–2009.5 We cannot rule out the historical attainment gap being larger in magnitude and then narrowing prior to data available for this study. However, it is clear that in the last decade there has been no further substantive reducing in the gradient with non-white students scoring on average 2% lower, and around 40% less likely to be awarded a distinction.

Second, it is possible that the current strategies are not sufficiently effective to have rea-world impact. Unconscious bias training has been an important role in addressing inequality in education and was rolled out to all staff between 2014 and 2016. However, a large meta-analysis, of interventions to reduce implicit bias, demonstrated only weak effects, with mainly short-term changes being seen.24 A further systematic review focusing specifically on interventions to reduce implicit prejudice reached similar conclusions.25

The other major focus of our analysis explored the change in the attainment gap during the medical degree course. A single previous study has reported on risk factors for poor performance during a medical degree course. Using an adjusted regression model it identified non-white ethnicity as a significant factor and highlighted that the effects were greatest during clinical assessments such as OSCEs.9

As students move into more senior years during their medical studies the methods of assessment change, with a shift from computer marked assignments to projects and OSCEs. The nature of these assessments, for which markers are not blinded to student ethnicity, increases the potential for unconscious bias to influence grades, although not all studies indicate the same.26 In addition, during clinical years interaction (both nature and quality) with teaching staff may differ. Teaching becomes more dependent on clinical placements and there is a greater emphasis on small-group teaching. Experiences of discrimination that may disadvantage non-white students, whether conscious or unconscious, may be more pronounced in clinical years.27

Limitations

Our research has some important limitations. Our study is from a single institution in London and our focus has been only on undergraduate students in a 5-year MBBS programme. Also, as attainment gap causes are complicated and multifactorial, thus, our findings are not necessary generalisable to other universities or healthcare programmes. Our adjustment for confounders was limited, and further, the information on SEP was a single metric, Widening Participation status, and was missing for some of the year groups. It is important to acknowledge that widening participation is an imperfect surrogate for deprivation. Unfortunately, we had no other metric available.

We were unable to control for prior attainment (secondary school grades), although this may have been of limited value because medical school entry requires uniformly high A-level outcomes. Finally, we acknowledge that the categorisation of students by ethnic groups into broad categories may mask important differences in people from ethnic minority communities.

Conclusions

We present robust quantitative evidence of persistent attainment gaps that are associated with ethnicity. The root causes of the attainment gap between White and non-white students are complex and multifactorial. Clearly not all the solutions will lie within the educational system. Strategies to actively address differential attainment are urgently needed in collaboration with all stakeholders including students and patients. Structural approaches such as anonymous marking or written submissions, video recording and monitoring of OSCEs, and positive discrimination strategies should be explored. In addition, further qualitative research has been conducted in parallel to this study, which sheds further detailed light on the possible causes of the attainment gap from the perspective of students and medical faculty staff, and possible solutions.

Data availability statement

Data are available upon reasonable request. Data used in this study were collected from students records and are under the control of the data holders at the medical school. Only aggregated data are available upon reasonable request. Personal data cannot be shared due to confidentiality reasons. All data relevant to the study are included in the article. All figures and tables included in this article are original.

Ethics statements

Patient consent for publication

Ethics approval

The study was reviewed by a local research ethics committee, and as pseudonymised data were being used, no formal ethics approval was required.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • PM, MAA, BC and JBG are joint first authors.

  • HL and SL are joint senior authors.

  • Twitter @No, @bennyleeshuang

  • Contributors JBG, PM, BC, TM, JT, HL and SL. involved in the design. JBG, MAA, SN and MDR analysed the data. All authors interpreted the results. JBG, PM, MAA and BC wrote the report with contributions from all other authors. PM, MAA, BC and JBG are joint first authors. HL and SL are joint senior authors. JBG, SN, MAA and BC accessed and verified the underlying data. MAA and JBG had final responsibility for the decision to submit for publication. JBG accepts full responsibility for the finished work and the conduct of the study, had access to the data, and controlled the decision to publish. JBG attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding This work was supported by the King’s Together Multi and Interdisciplinary Research Scheme (Wellcome Trust Institutional Strategic Support Fund) grant reference (204823/Z/16/Z). All funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. JBG affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as originally planned (and, if relevant, registered) have been explained.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.