Article Text
PDF
Abstract
Background There has been limited research on the positive aspects of physician wellness and to our knowledge there have been no validity studies on measures of resilience and grit among internal medicine (IM) residents.
Objectives To investigate the validity of resilience (10 items Connor-Davidson Resilience Scale (CD-RISC 10)) and grit (Short Grit Scale (GRIT-S)) scores among IM residents at a large academic centre, and assess potential associations with previously validated measures of medical knowledge, clinical performance and professionalism.
Methods We evaluated CD-RISC 10 and GRIT-S instrument scores among IM residents at the Mayo Clinic Rochester, Minnesota between July 2017 and June 2019. We analysed dimensionality, internal consistency reliability and criterion validity in terms of relationships between resilience and grit, with standardised measures of residents’ medical knowledge (in-training examination (ITE)), clinical performance (faculty and peer evaluations and Mini-Clinical Evaluation Examination (mini-CEX)) and professionalism/dutifulness (conference attendance and evaluation completion).
Results A total of 213 out of 253 (84.2%) survey-eligible IM residents provided both CD-RISC 10 and GRIT-S survey responses. Internal consistency reliability (Cronbach alpha) was excellent for CD-RISC 10 (0.93) and GRIT-S (0.82) overall, and for the GRIT subscales of consistency of interest (0.84) and perseverance of effort (0.71). CD-RISC 10 scores were negatively associated with ITE percentile (β=−3.4, 95% CI −6.2 to −0.5, p=0.02) and mini-CEX (β=−0.2, 95% CI −0.5 to −0.02, p=0.03). GRIT-S scores were positively associated with evaluation completion percentage (β=2.51, 95% CI 0.35 to 4.67, p=0.02) and conference attendance (β=2.70, 95% CI 0.11 to 5.29, p=0.04).
Conclusions This study revealed favourable validity evidence for CD-RISC 10 and GRIT-S among IM residents. Residents demonstrated resilience within a competitive training environment despite less favourable test performance and grittiness that was manifested by completing tasks. This initial validity study provides a foundation for further research on resilience and grit among physicians in training.
- internal medicine
- medical education & training
- statistics & research methods
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first validity study of 10-item Connor-Davidson Resilience Scale 10 and Short Grit Scale scores among internal medicine (IM) residents for resilience and grit respectively, and was completed at a large academic centre in the USA.
This study evaluated the dimensionality, internal consistency reliability and the criterion validity of these two measures among IM residents.
This study provides a foundation for further research on resilience and grit among physicians in training.
This was an observational study, which limits the ability to draw causal inferences about the relationships found.
The analysis did not adjust for resident age, gender or international versus US medical graduation status, as they are non-modifiable variables in terms of career development and enhancing residency curricula.
Background
The prevalence of physician burn-out in the USA is estimated to be approximately 50% among physicians in training1–4 and practising physicians.5–7 Notably, burn-out rates are higher for physicians than other professionals even after adjusting for work hours.8 9 It is unclear why burn-out rates are higher in certain residency programmes and among particular individuals within the same clinical settings.10 Since burn-out seems to begin during medical school,3 11 tackling this problem at earlier stages could help mitigate its consequences later. Furthermore, although there has been much research on correlates of burn-out,12 there is limited research on the positive aspects of physician wellness and very little known about resilience and grit among internal medicine (IM) residents.
The American Psychological Association describes resilience as adapting effectively to stressors such as relationship problems, serious health issues or workplace and financial challenges.13 That is, resilience is the capacity to respond to adversity such that goals are achieved at minimal psychological and physical cost. Essentially, resilient individuals ‘bounce back’ after challenges while also growing stronger.14 Although several models of resilience have evolved over the years,15 the dominant paradigm of resilience is dynamic, linking neurobiology, behaviour and environmental conditions.16 Resilience is considered essential for enhancing quality of medical care, empathy for patients and sustainability of the healthcare workforce as a whole.14 Moreover, low resilience may impair brain function, even resulting in post-traumatic stress disorder, depression and other psychiatric disorders.17 Yet, most people do not develop such conditions after experiencing difficult life events and are thus considered to be ‘resilient’. Resilience as a successful adaptation relies on effective responses to environmental challenges and, ultimately, resistance to the harmful effects of stress.18 Therefore, a greater understanding of the factors that promote resilience is critical.19
The most widely used assessment of resilience is the Connor-Davidson Resilience Scale (CD-RISC). Use of the CD-RISC has shown that resilience is modifiable and can be improved. A shorter version of this scale, which has similar psychometric properties, is the CD-RISC 10.20 Evidence based on the use of this assessment measure suggests that resilience can be promoted in healthcare workplaces,21 although research on physicians is scarce.
Grit is defined as the perseverance and passion for long-term goals.22–24 Rather than avoidance and shying away, grit means working towards achieving specific outcomes despite difficulty, failure or adversity.25 Individuals who remain focused on a goal or task and see it through to satisfactory completion would be described as ‘gritty’.26 Grit is a predictor of success in stressful, high-achievement fields including, but not limited to, surgical residency,27–29 emergency residency,30 military23 and pharmacy.31 The original Grit Scale (Grit-O) consists of 12 items, each rated on a 5-point scale,1–5 classified under two main domains: (1) consistency of interest and (2) perseverance of effort, with six elements each.22 Subsequently, an abbreviated (8-question) scale with improved psychometric properties was developed by the same investigators to measure trait-level perseverance and passion for long-term goals (Short Grit Scale, Grit-S).23
Although there has been ample research on relationships between burn-out and various aspects of professionalism and clinical performance among resident physicians,32–36 to our knowledge the CD-RISC 10 and Grit-S scales have not been previously validated in US IM residents. Furthermore, there remains the need for further research on positive aspects of physician wellness—such as resilience and grit—which may serve to counterbalance burn-out.
In this study, we assessed the validity of CD-RISC 10 and Grit-S scores among IM residents at a large academic medical centre. Additionally, we examined associations between resident resilience and grit based on CD-RISC 10 and Grit-S scores, respectively, with previously standardised measures of medical knowledge (the in-training examination (ITE)), professionalism (dutifulness based on conference attendance and evaluation completion) and clinical performance (validated, multisource, clinical performance evaluations).
Methods
Study design and sample
This was a longitudinal cohort study of IM residents training at Mayo Clinic Rochester between July 2017 and June 2019 who were invited to participate in the Mayo Internal Medicine Well-Being (IMWELL) Study. We used existing survey data from the IMWELL study in addition to administrative data collected routinely on IM residents at Mayo Clinic in Rochester, Minnesota, USA.
The Mayo IMWELL study
The prospective, longitudinal, Mayo IMWELL study was initiated in 2003 to evaluate IM residents’ burn-out, quality of life and empathy, along with other measures of well-being.37–40 Enrolment is voluntary and is offered during the orientation of all new interns (categorical and preliminary) in the IM residency programme. Incentives were not given. For the time frame between July 2017 and June 2019, 253 residents were eligible to be enrolled as participants and were surveyed twice per year. An additional survey was sent each spring to graduating categorical residents. Identities of participants were anonymised during data collection and before analysis using numerical codes. The CD-RISC 10 and Grit-S instruments were added to the IMWELL study surveys starting July 2017. The CD-RISC 10 and Grit-S scores from the IMWELL study for each resident were merged with each resident’s residency performance metrics during the subsequent 6 months (July to December or January to June) on the other relevant instruments described below.
Validity evidence
The validity argument for this study was based on a modern approach to validity which states that all validity is construct validity, and that validity evidence is gathered from the categories of content, internal structure, relations to other variables, response process and consequences.41 Content refers to relationships between an assessment’s wording and the construct that it purportedly measures. Internal structure refers to the degree to which instrument items fit the underlying construct and is often reported in terms of dimensionality and reliability. Relations to other variables evidence is the relationship between scores and other variables relevant to the construct being measured, such that the relationships may be positive or negative depending on the constructs being measured.42 Notably, research has indicated that commonly reported categories of validity evidence among education research studies come from the categories of content, internal structure and relations to other variables.42
Instruments and scales used
Short Grit Scale
An 8-item assessment, each rated on a 5-point scale (1=Not like me at all, 2=Not much like me, 3=Somewhat like me, 4=Mostly like me, 5=Very much like me), covering two factors,23 that measures trait-level perseverance and passion for long-term goals. It has four fewer items than the Original Grit Scale (Grit-O)22 with improved psychometric properties. The Short Grit (Grit-S) scale is a brief version of the original 12-item Grit-O scale. Previous studies have shown that it has predictive validity, consensual validity and test–retest stability. Factor analysis, and later confirmatory factor analysis, has supported a two-factor structure of the scale reflecting ‘consistency of interest’ and ‘perseverance of effort’. Both factors showed adequate internal consistency reliability.23
10 item Connor-Davidson Resilience Scale
An assessment consisting of 10 items, rated on a 5-point scales (0=Not true at all, 1=Rarely true, 2=Sometimes true, 3=Often true, 4=True nearly all of the time), with higher scores indicating greater resilience.43 It has ability to distinguish between those with greater and lesser resilience levels, and to demonstrate that resilience is modifiable and can be improved. The reliability and validity of the CD-RISC were previously evaluated and performed well in other settings. Factor analysis revealed five factors for the CD-RISC scale.43 CD-RISC 10 is a 10-item version of this scale with good internal consistency and evidence to support construct validity.20 Further validation studies have shown excellent performance of the CD-RISC 10 among the general population44 and trainees in the US Air Force.45
Knowledge measures
This included In-Training Examination (ITE) score percentiles. The ITE is administered to all US IM residents annually. Residents in this study were administered the ITE annually each fall. Validity of the ITE scores has been established in several studies.46 47
Professionalism and dutifulness measures
This included conference attendance and evaluation completion, which were reported as relations to variables validity evidence in our previous studies of residents at the Mayo Clinic.48 Conference attendance was assessed using in-person card-swipe data. Evaluation completion percentage was determined from the MedHub residency evaluation system for our study’s time frame.
Clinical performance
We selected clinical performance measures as association variables for this study, because we believed that standardised assessments of performance are among the most rigorous challenges for testing residents’ resilience and grit. Clinical performance was determined by faculty and peer evaluations and the standardised Mini Clinical Evaluation Examination (Mini-CEX).49 50 The Mini-CEX evaluates core clinical skills by trainees, namely medical interviewing, physical examination, informed decision-making/counselling and clinical judgement/reasoning. The Mini-CEX has demonstrated validity evidence among IM residents.49–52 The mini-CEX used at Mayo Clinic Rochester incorporates a 5-point scale. Multisource assessments of residents’ clinical performance at Mayo Clinic Rochester are completed by faculty, peers and senior medical residents. Items within these clinical performance assessments have shown multidimensionality and excellent internal consistency reliability.53 54 The clinical performance evaluations (peer evaluations, faculty evaluations and mini-CEX ratings) are administered by the residency evaluation platform, MedHub. Aggregate reports of evaluations can be obtained by time frame of interest, with all assessments standardised to a score in the range of 0–10.
Data analysis
Participants’ demographics were summarised using descriptive statistics. Continuous variables such as age were summarised as mean (SD). Nominal variables, such as gender, were reported using a count (percent of total). Only fully completed Grit-S and CD-RISC 10 instruments were included in the analysis. A threshold of p<0.05 was used to determine statistical significance. Statistical analyses were conducted using SAS V.9.4 (SAS Institute).
Validation of the CD-RISC 10 and Grit-S Scales among IM residents
It has been recommended to re-examine the validity of assessments when applying them to new contexts and educational settings.55 Therefore, we evaluated the internal structure validity of the CD-RISC 10 and Grit-S for this study as follows:
Factor analyses of the CD-RISC 10 and Grit-S instruments were done using principal components analysis with a minimum eigenvalue of 1 criterion. An orthogonal Varimax rotation was used to estimate item loadings. Items with factor loadings of 0.48 or more were retained.56 Internal consistency reliabilities for items comprising each factor and overall were determined using Cronbach alpha, where alpha >0.7 was considered acceptable.56 Scale values were reversed for the ‘Consistency of Interest’ factor items of Grit-S (1, 3, 5 and 6) so that 1=very much like me, 2=mostly like me, 3=somewhat like me, 4=not much like me, 5=not like me at all, so that higher item scores reflect increased ‘Grit’.
Criterion validity (relations to other variables): Unadjusted bivariate associative analyses used generalised linear models with normal response distributions and identity link functions estimated via generalised estimating equations with an exchangeable covariance matrix. The CD-RISC 10 and Grit-S scores were treated as the explanatory variables, and residency performance in the various metrics during the subsequent 6 months were the outcome variables.
Patient and public involvement
Patients and the public were not involved in this research study.
Results
Sample characteristics
From a total of 253 eligible IM residents training at Mayo Clinic Rochester between July 2017 and June 2019, 213 (84.2%) completed at least 1 IMWELL survey, resulting in 468 completed CD-RISC 10 and 472 Grit-S surveys from a total of 801 possible surveys. A total of 461 IMWELL surveys included complete responses for both scales. The distributions of overall scores were assessed visually, with Grit-S appearing approximately normal and CD-RISC 10 displaying some left-skewness. There were 193 ITE percentiles and 358 mini-CEX evaluations available for the same time period. The demographic characteristics of the participants are shown in table 1.
Baseline characteristics of internal medicine resident physicians
Medical knowledge, clinical performance and professionalism metrics
The ITE score percentiles among residents in our study (N=193) ranged from 18 to 100, with a mean (SD) of 83.3 (15.5). Clinical performance, as reflected by faculty evaluations (N=429), peer evaluations (N=362), and mini-CEX (N=358) showed mean (SD) scores of 7.84 (0.69), 8.09 (0.83), and 8.19 (1.59), respectively. Performance measure summaries are shown in table 2.
Medical knowledge, professionalism and clinical performance measures for 210 IM resident physicians providing data from July 2017 to June 2019 (n=429)
Validity analyses of the CD-RISC 10 and GRIT-S instruments
Regarding the internal structure validity evidence for the scales among IM residents, the CD-RISC 10 index demonstrated a single dimension of resilience while the GRIT-S index demonstrated two dimensions of grittiness. The internal consistency reliability for both scales overall, and for the GRIT subscales, was high (Cronbach α’s >0.7, tables 3 and 4).
IM resident physicians’ mean scores on, factor loadings of and internal consistency reliability of the CD-RISC 10 (N=468)
IM resident physicians’ mean scores on, factor loadings of and internal consistency reliability of the Grit-S (N=472)
The 468 completed CD-RISC 10 scales showed mean scores for individual items ranging from 2.74 (not easily discouraged by failure) to 3.34 (can achieve goals despite obstacles) on the 0–4 scale (table 3). The overall mean summed (SD) CD-RISC score was 31.5 (6.1). The 472 completed GRIT-S scales showed individual-item mean scores ranging from 2.99 (new ideas and projects sometimes distract me from previous ones) to 4.48 (I am a hard worker) on the 1–5 scale (table 4). The overall mean (SD) GRIT-S score was 3.72 (0.59).
Regarding relations to other variables (ie, criterion) validity evidence, the CD-RISC 10 overall summed scores correlated negatively with medical knowledge acquisition as measured by ITE score percentile (β=−0.34, 95% CI −0.62 to −0.05, p=0.02). The CD-RISC 10 overall summed scores also correlated negatively with clinical performance as measured by the mini-CEX (β=−0.02, 95% CI −0.05 to −0.002, p=0.03) (table 5).
Associations of performance measures with CD-RISC 10 and Grit-S
The GRIT-S overall mean score correlated positively with evaluation completion percentage (β=2.51, 95% CI 0.35 to 4.67, p=0.02) and in-person conference attendance (β=2.70, 95% CI 0.11 to 5.29, p=0.04) (table 5), which are measures of the dutifulness aspect of professionalism.
Discussion
The CD-RISC 10 and GRIT-S instruments have strong validity in measuring resilience and grit in several populations20 23 45 57 and there have been studies of grit in surgical28 29 58 and emergency medicine residents.30 However, to our knowledge, this is the first validity study of the CD-RISC 10 and GRIT-S among IM residents. Both instruments showed excellent internal consistency reliability, statistically significant associations with previously validated measures of resident physician performance, and dimensionality characteristics that are consistent with previous research.
We identified a negative association between residents’ CD-RISC 10 scores and measures of clinical performance (mini-CEX) and medical knowledge (ITE). This finding might reflect resilient residents’ abilities to thrive within a high-pressured IM training environment, despite performing less favourably on standardised assessments within this setting. Notably, whether a resident thrives is not determined by their performance on one of these measures, but rather, by their overall standing within our rigorous training environment. This finding may be supported by the residents’ highest score on the item, ‘achieve goals despite obstacles.’ We also identified a positive association between GRIT-S scores and evaluation completion, which is a dutifulness aspect of professionalism.22 59 This finding suggests that, as expected, residents with grittiness tend to finish tasks. Additionally, this finding corresponds to residents’ highest score on the item, ‘I am a hard worker.’ Overall, our research should inform future interventions to improve resident performance and well-being by using the CD-RISC 10 and GRIT-S as roadmaps for curricular interventions.
Compared with the general US population included in the original validation studies,43 57 60 the overall mean (SD) resilience score in our sample was comparable (31.5 (6.1) vs 32.1 (5.8) and 31.8 (5.4)). However, compared with the reference group aged 25–34 years in the original validation study, the overall mean (SD) GRIT-S score in our sample was higher (3.72 (0.59) vs 3.2 (0.7)).23 Our study participants noted strong perseverance as reflected by their highest GRIT-S score on the item ‘I am a hard worker.’ Additionally, the dimensionalities for the GRIT-S and CD-RISC 10 scales in our study were consistent with findings from research in different populations.61 62 However, despite having higher grit and comparable resilience as compared with the general population, burn-out rates among physicians and physicians-in-training appear to be greater than that of the US working population.3 8 This suggests that the medical profession selects gritty and resilient individuals, yet still manages to burn them out. Furthermore, research shows that well-being indicators are highest on matriculation to medical school, and subsequently wane throughout medical training.63 Consequently, future research should examine the interactions between burn-out, empathy, resilience and grit.
This study has several limitations. First, it was observational, which constrains the ability to draw causal inferences about the relationships that were identified. Second, the analysis did not adjust for age, gender and international versus US medical graduation status. Nonetheless, these are non-modifiable variables that would not facilitate efforts at professional development or enhancing residency curricula. Third, this study involved IM residents at a large academic medical centre, which may limit generalisation to some other specialties and settings. Fourth, although we implicate potential, counterbalancing interactions between grit and wellness with burn-out, this remains speculative until there is further research that actually examines interactions between performance on these scales among IM residents. Fifth, residents’ self-selection to participate in the study introduces the potential for selection and response biases, though it is noteworthy that the response/participation rate for this study was high.
Resilience and grit may lessen burn-out, yet these relationships remain unclear among physicians in training. Thus, research on resilience and grit could assist interventions to mitigate physician burn-out64 65 and provide a deeper understanding of dynamics between the issues at play.66 Findings from this study support of use of the CD-RISC 10 and GRIT-S among IM residents and should serve as a foundation for future research on resilience and grit in medical learners. This research should examine associations among IM residents between CD-RISC 10 and GRIT-S, with validated measures of burn-out and well-being. It is noteworthy that burn-out is prevalent within current medical education and training systems and may be an indicator of organisational health.67 68 Therefore, improved understanding of resilience and grit may enhance graduate medical education curricula36 69–72 and the well-being of physicians.
There have been ample investigations on physician burn-out and depression, yet there has been less research on positive aspects of physician wellness including resilience and grit among IM residents. Especially during this era of the COVID-19 pandemic, it is necessary to better understand characteristics of physicians that allow them to surmount adversity and thrive. Since high resiliency and grit have been correlated with positive attributes in other populations, we are hopeful that further study of these traits in residents’ will help to improve their quality of life.27 58 73
References
Footnotes
Contributors Idea conception: TB, FA, CW and VMM. Study design and methodology: FA, TB and AJH. Data management and analysis: AJH and JM. Interpretation of the data: FA, AJH, TB, MHM and BEV. Manuscript drafting: FA. Supervision: TB. Revising, editing and final approval of manuscript: all authors.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Ethics approval This study was deemed exempt by the Mayo Institutional Review Board.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available. No further data are available.