Longitudinal Associations Between Grit, Academic Outcomes, and Residency Match Rates Among Pharmacy Students

INTRODUCTION

Grit is a psychological construct that includes consistency of interest and perseverance of effort, and has gained attention for its positive association with achievement, including retention in the United States Military Academy and success in the Scripps National Spelling Bee.¹ Grit has also been associated with academic success in secondary, undergraduate, and graduate education.^2-4 These results have stimulated interest in identifying and cultivating grit in learners.⁵

Passion and resilience are qualities many would associate with health professionals. Despite difficult and stressful situations, health professionals must consistently be able to prioritize the needs of their patients. Studies in the United Kingdom and Australia suggest that grit levels are relatively high overall among hospital-based and community-based health practitioners and inversely associated with disengagement, burnout, and exhaustion.^6,7 Because grit is associated with both educational and professional resilience, programs have begun to explore the role of grit in admissions to and skills development in health professions training.^8-11 Within medical education, studies have shown that higher grit scores are linked to greater academic achievement in students and lower rates of burnout, depression, and attrition in residents.^4,12,13 Within pharmacy education, studies to date have produced conflicting results about the relationship between grit and academic achievement.^14,15

Grit has primarily been investigated via cross-sectional or prospective study designs. Duckworth and colleagues previously determined that scores on the Short Grit Scale (Grit-S) were stable over a one-year period among adolescent students.² The potential for change in an individual’s grit levels over time and the relationship between changing grit levels and academic or professional outcomes at the individual and cohort level have not been explored in health professions students.

We surveyed pharmacy students at the University of California, San Francisco (UCSF), School of Pharmacy (SOP) using the Grit-S scale twice, with testing separated by approximately one year. Our goals were to: characterize grit in UCSF pharmacy students; determine how grit changes in pharmacy students and their cohorts over time; and investigate the relationship between grit, academic outcomes (ie, grade point average [GPA] and performance on advanced pharmacy practice experiences [APPEs]), and professional outcomes (pursuit and attainment of postgraduate residency training [PGY1]) among respondents.

METHODS

This was a prospective cohort study of pharmacy students enrolled at the UCSF SOP from September 2016 to September 2017. The UCSF Institutional Review Board certified this study through an exemption.

During orientation week for fall semester 2016 and fall semester 2017, we sent individualized, internet-based survey invitations (Qualtrics, Provo, UT) to all pharmacy students enrolled in the UCSF SOP. For students in years one through three of the program (P1, P2, P3), we provided study information and allocated time to complete the survey during live classroom orientation sessions. For any student who did not have an electronic device available during this period, we provided a printed version of the survey. We emailed the study objectives and investigator contact information along with a link to the electronic survey instrument to students in year four of the program (P4). We incentivized students to complete the survey by entering participants in a drawing to receive one of two $25 gift cards per cohort.

The survey instrument included the eight-item Grit-S scale, which had previously been studied in pharmacy students.^14,15 The Grit-S scale produces a score ranging from 1=low grit to 5=high grit and is composed of two subscales that measure perseverance of effort and consistency of interest. Scoring and validation of the Grit-S scale has been described previously.²

We linked each survey response to a UCSF pharmacy student ID number to allow us to study associations between Grit-S score, demographic characteristics, academic outcomes, and professional outcomes. At the UCSF SOP, only didactic performance contributes to GPA. Thus, we defined deficient performance in an APPE as receipt of a “does not meet expectations” rating in any area for one or more APPEs. We defined APPE failure as receipt of a non-passing final evaluation on any APPE. We obtained American Society of Health-System Pharmacists (ASHP) residency application and residency match information from the UCSF SOP. These variables and outcomes were chosen based on previous association with Grit-S score and/or academic success.^2,14-18 We stored the aggregate, de-identified data in an encrypted, password-protected computer owned by one of the study investigators.

Participant demographics and outcomes were characterized using descriptive statistics, presented as mean plus or minus the standard deviation. Cronbach alpha characterized the reliability of Grit-S and its corresponding subscales. Demographic characteristics and GPAs of survey responders and nonresponders were compared using chi-square tests for categorical variables and t tests for continuous variables. Paired t tests were used to compare the change in Grit-S scores for students with two grit measurements (fall 2016 and fall 2017) during the study period. Unpaired t tests compared the Grit-S scores of cohorts at the same level (ie, P1 grit scores for the classes of 2020 and 2021). The relationships between participant characteristics and initial Grit-S score were characterized using linear regression, both individually and as a multivariable model. Students’ GPAs was categorized into three categories (<3.0, 3.0-3.49, and ≥3.5) and ordinal logistic regression was used to test the relationship between Grit-S scores and GPA, adjusting for potential confounders. Grade point average was analyzed either as a student’s performance from the initial grit measurement forward or as cumulative GPA (using all available data, including GPA for courses completed before the initial grit measurement). A logistic regression model was created to examine the relationship between Grit-S score and the likelihood of receiving a “does not meet expectations” rating on any APPE, as well as of pursuit and attainment of an ASHP-accredited residency. A p value of less than .05, unadjusted for multiple comparisons, was designated as significant a priori. All data analysis was conducted using STATA (College Station, TX).

RESULTS

Overall, 422 students (84%) completed the survey in fall 2016 and 430 students (86%) completed the survey in fall 2017. The lowest response rates during both survey administration periods (62% in fall 2016 and 70% in fall 2017) were among P4 students. Table 1 summarizes participant demographics by anticipated year of graduation and level in the program (P1, P2, P3, or P4). The mean age of all participants was 25 (SD=3) years. The majority of students identified as female (70%) and were of East Asian/Filipino/Pacific Islander descent (54%). Approximately 17% of respondents were underrepresented minorities, and almost 50% of respondents in each graduation cohort were first-generation college students. The mean undergraduate science GPA of participants ranged from 3.3 to 3.5 on a 4.0 scale. Students’ mean cumulative pharmacy GPA at the time of the survey ranged from 3.4 to 3.6 on a 4.0 scale. The only significant differences between survey respondents and nonrespondents were cumulative pharmacy GPA for P2 students and cumulative pharmacy GPA for P4 students at the time of the survey in fall 2016 and cumulative pharmacy GPA for P2 students at the time of the survey in fall 2017.

The mean Grit-S scores of each group ranged from 3.5 to 3.7 (Table 2). The P1 students who completed the survey in fall 2016 had the highest mean Grit-S scores 3.7 (SD=0.5). Across all cohorts, mean score on the perseverance of effort subscale of the Grit-S was greater than the mean score on the consistency of interest subscale. Cronbach alpha ranged from 0.63 to 0.73 depending on the scale or subscale and survey year, and in all cases revealed acceptable reliability.

Figure 1 depicts the mean Grit-S scores and changes in Grit-S scores over time by graduation cohort. The class of 2020 showed a significant decrease in paired Grit-S score from the P1 to P2 year, while the class of 2019 (P2 to P3 year) and the class of 2018 (P3 to P4 year) showed significant increases in reported Grit-S score over time (Figure 1). Analysis of Grit-S subscales revealed that significant changes in paired Grit-S responses by graduation cohort were powered by different subscales (Table 3). Grit-S scores were also compared across graduation cohorts by academic year (eg, P1 scores for the class of 2020 and P1 scores for the class of 2021) to determine whether there were curricular influences upon Grit-S scores. Across all comparisons, significant differences were detected between P1 Grit-S scores for the classes of 2020 and 2021 (difference 0.17 [95% CI: 0.03 - 0.30]; p= .01) and P3 scores for the classes of 2018 and 2019 (-0.17 [95% CI: -0.31- -0.02]; p=.02; Table 4).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Mean Grit-S Scores and Paired Differences in Grits-S Score by Graduation Cohert

The relationships between participant characteristics and initial Grit-S score were characterized to inform regression modeling. Unadjusted analysis of the relationship between student characteristics and initial Grit-S score found that underrepresented minority participants reported significantly higher Grit-S scores than non-underrepresented minority students (0.12 [95% CI: 0.01 - 0.24]). African-American students had higher initial Grit-S scores compared to both East Asian students (0.29 [95% CI: 0.06 - 0.53]) and white students (0.28 [95% CI: 0.03- 0.54]). All other comparisons between ethnicities found no significant differences. First-generation college students had significantly lower initial Grit-S scores than non-first-generation students (-0.11 [95% CI: -0.19 - -0.02]). There was not a significant difference in Grit-S scores based on gender (-0.04 [95% CI: -0.13-0.06]). Students with an undergraduate GPA less than 3.0 had numerically higher Grit-S scores compared to students with an undergraduate GPA ≥3.0, but the difference was not significant (-0.11 [95% CI: -0.28 - 0.06] and -0.18 [95% CI: -0.34 - -0.01], respectively; p=.07). On multivariable linear regression, when all these variables except for underrepresented minority status (because of the overlap with ethnicity) were included, only first-generation college student status was associated with initial Grit-S score (-0.10 [95% CI: -0.20 - -0.01]).

Initial Grit-S score was not significantly predictive of categorized pharmacy GPA (<3.0, 3.0-3.49, or ≥3.5) measured after the initial survey administration using ordinal logistic regression (0.89 [95% CI: 0.61 - 1.28]) (Table 5). When categorized GPA was cumulative, higher Grit-S scores were significantly associated with likelihood of being in a lower GPA category (0.70 [95% CI: 0.49 - 0.99]). In models including initial Grit-S score, GPA prior to initial Grit-S score (either prior pharmacy GPA or undergraduate GPA for P1 students), underrepresented minority status, and first-generation college student status, the two variables that were significantly associated with pharmacy GPA (either measured after initial Grit-S score or cumulative) were underrepresented minority status and GPA prior to obtaining initial Grit-S score. Changes in paired Grit-S scores and GPAs between the P1 to P2 and P2 to P3 years were not significantly correlated (p=.74).

Among students with APPE performance data, failure was rare (3%). The mean Grit-S scores for students with an APPE failure was 4.6 (vs 3.6 for students without an APPE failure). A deficient rating on any APPE evaluation occurred in 48% (38/79) of respondents with APPE data. There was no relationship between a deficient rating and initial Grit-S score, either alone or after adjusting for pharmacy school GPA, underrepresented minority status, and first-generation college student status (Table 6). More than 75% of P4 respondents applied to ASHP-accredited pharmacy residency programs during the 2016-2017 application period. Of those applicants, 87% of candidates matched with a residency program. Logistic regression modeling indicated that higher initial Grit-S score was associated with significantly lower odds of applying to an ASHP-accredited residency. Grit-S score did not predict attainment of an ASHP-accredited residency position among applicants or all P4 survey respondents (including those who did not apply for an ASHP-accredited residency). This effect persisted after controlling for potential confounders.

DISCUSSION

To our knowledge, this is the first study of grit in health professions students measured over time and correlated to validated covariates, academic outcomes, and professional outcomes. We found small but significant differences in individual Grit-S scores when measured in the same students approximately one year after the initial survey in the absence of any intervention. The direction and magnitude of these changes varied by student level. While these changes did not correlate to student GPA over the year, the implications of these findings are worth discussing. For example, the class of 2020 showed a significant decline on the Grit-S consistency of interest subscale from the beginning of their P1 to their P2 year. This change may have reflected the foundational topics discussed in the P1 curriculum. Interestingly, this effect did not persist among paired responses from the other cohorts. A significant increase in students’ scores on the Grit-S perseverance of effort subscale occurred from the P3 to the P4 year, which may illustrate the perseverance gained while these students were completing APPEs. The class of 2019 also showed a significant increase in the Grit-S perseverance of effort subscale from the P2 to P3 years, possibly because of the rigorous therapeutics course series that begins during the P2 year. Although grit is intended to measure perseverance and passion for long-term goals, our results suggest that a pharmacy curriculum may influence these characteristics over a short period of time.

Additionally, we detected significant differences between Grit-S scores at the cohort level. For example, two cohorts of students participated in this study at the start of their P1 year (the classes of 2020 and 2021). The mean Grit-S score for the class of 2020 cohort was significantly greater than the mean Grit-S score of the class of 2021 cohort. While grit is typically measured on the individual level, peer grit scores may offer predictive value for outcomes of interest, including academic performance.¹⁹ The social cognitive theory of learning supports the notion of group dynamics influencing individual outcomes.²⁰ While the measurement of grit on a group level was not the objective of the present study, based on our findings, future studies should investigate the effects of collective grit scores on individual academic and professional outcomes within health professions education.

Student and cohort-level variability in what may be considered a relatively fixed psychological trait has several implications for future studies in this area. First, interventional studies that would aim to influence (presumably, to increase) a student’s Grit-S score should include a control group. Our data clearly show significant changes in Grit-S scores over time periods as short as one year in the absence of any intervention. Therefore, changes in Grit-S scores could be misattributed to an intervention in the absence of a control group. Second, the changes in paired Grit-S scores by student level in pharmacy school suggest that elements within the pharmacy school environment may impact perceived “grittiness.” Future studies following multiple cohorts of students through a pharmacy curriculum are required to validate whether this is a generalizable effect. Finally, the finding of significant differences in mean Grit-S scores for different cohorts at the same level in pharmacy school cautions against drawing conclusions of differences between, for example, students at different pharmacy schools based on a single cohort.

The mean initial Grit-S score of 3.6 (SD=0.5) across all of our respondents (n=547) was lower than Grit-S scores previously reported in pharmacy students, but similar to those of medical students.^4,14,15 In contrast to a previous study by Pate and colleagues measuring grit in pharmacy students, we did not detect a significant association between students’ Grit-S score and GPA.¹⁵ Palisoc and colleagues also did not find a significant relationship between Grit-S score and academic success (defined as a GPA≥3.0 without course failures) in pharmacy students.¹⁴ Studies of grit in secondary and undergraduate education also report a lack of association between academic performance and grit.^21,22 Because grit may explain up to 6% of the variance in academic outcomes, cohorts with limited GPA spread, such as those within our study, may not be significantly influenced by grit.¹ We also hypothesize that differences in study subjects and methods may offer explanations for these results. Both the current study and study by Palisoc and colleagues obtained GPA data from student records rather than from students self-reporting, and subjects in both studies were primarily of Asian ethnicity. Datu and colleagues determined a lack of association between the Grit-S consistency of interest subscale and emotional wellbeing among Filipino students.²³ The authors suggest that the “collectivist” culture of the Filipino people, which emphasizes group goals, may explain why this specific Grit-S subscale, which emphasizes individual interests and passion, was less predictive in their study.²³ This theory supports our study’s finding that the Grit-S perseverance of effort subscale scored higher than the Grit-S consistency of interest subscale across all cohorts in this study.

Palisoc and colleagues found higher Grit-S scores among students who pursued and attained postgraduate training programs,¹⁴ whereas our study found a negative association between Grit-S score and likelihood of applying for an ASHP-accredited residency program, and no association with Grit-S score and the likelihood of achieving a residency match, either alone or after controlling for potential confounders. Comparing these findings is complicated by the fact that the prior study included two cohorts of students within the analysis and fewer than half of respondents applied for residency programs. Within our study population, 77% of respondents from the class of 2017 applied for a residency and served as the basis for our analysis. Other student characteristics that have previously been described as predictive of residency attainment and are relevant to our study cohort include: female gender, employment at an institutional pharmacy, age of the applicant, North American Pharmacist Licensure Examination (NAPLEX) pass rate, academic health center affiliation, and U.S. News & World Report rankings.^24-27 While these characteristics were unavailable to compare between studies, grit may conceivably influence a student’s odds of passing the NAPLEX or gaining admission to a high-ranking pharmacy program. However, similar to the influence of grit on GPA, the additive value of grit on residency attainment, beyond these aforementioned predictors, may be marginal at best.

Our study also differs by exploring associations between underrepresented minority status and students’ Grit-S score and pharmacy GPA. Previous research suggests that underrepresented minority undergraduate students have high Grit-S scores (4.1) and that a strong correlation exists between grit and GPA, even when controlling for preadmission GPA and standardized examination scores (R=0.38, p<.01).¹⁶ While we did not detect a significant interaction between underrepresented minority status, Grit-S score, and GPA, we did find that underrepresented minority students had significantly higher Grit-S scores than non-underrepresented minority students. The implications of these findings on other curricular and professional outcomes warrants further study.

There are limitations to this study, and important considerations should be given to its external validity. Although our overall survey response rate was high (more than 80% compared to approximately 50% for the studies conducted by Pate and Palisoc and colleagues), the potential for response bias existed, particularly among fourth-year students for which our response rate was only 67% across the two cohorts who completed the survey. While study participants were similar to nonresponders on the majority of characteristics, some nonresponders had lower GPAs than responders. If the Grit-S scores of these nonresponders were substantial outliers, there could have been an overall effect on the relationship between Grit-S scores and academic performance. Another limitation to the generalizability of our results is that, unlike some pharmacy schools, our program does not assign letter grades to experiential coursework (including APPEs). However, we saw no evidence of a relationship between students’ Grit-S scores and their performance on a separate measure of APPE performance. Finally, the demographic characteristics of our students may limit the external application of our findings. Specifically, we had a high representation of the following: first-generation college students (≥50% in each cohort), students of East Asian/Filipino/Pacific Islander ethnicity (>50% of all respondents), and female students (70%). Additionally, almost all of the students had earned a bachelor’s degree prior to matriculation, and their undergraduate science and pharmacy school GPAs were ≥3.3. Given the previously identified relationships between these characteristics and Grit-S score and/or academic success, these results may not translate to other pharmacy schools that have a different student demographic from ours.

The predictive value of grit has come into question.^28-30 Angela Duckworth, the psychologist who first characterized grit, has noted the limitations of the self-reported Grit-S questionnaire and cautions against employing the Grit-S for “diagnostic” purposes or to use in making high-stakes decisions related to students.³¹ This does not mean that the attribute of grit lacks value for better understanding the pharmacy student’s journey. The Center for the Advancement of Pharmacy Education (CAPE) 2013 Education Outcomes provides learning objectives that encompass grit: “maintain motivation…and interest (eg, habits of mind) during learning and work-related activities” and “demonstrate persistence and flexibility in all situations…”³² Grit-S scores may correlate better with other measures of professional competence and practice than with academic (and in our study, primarily didactic) course performance. Grit has more recently been associated with student productivity and engagement, two highly desired traits in a competent student.³³ Yet, when measuring behaviors related to job performance (eg, organizational citizenship, counter productiveness, and task completion), grit adds little value beyond personality traits.³⁴

Alternatively, there may be a need for other metrics encompassing this psychological construct that have better predictive value in higher education. Indeed, two modified grit scales have been developed. One scale shows greater predictive value for standardized test scores in secondary education, while the other scale is associated with measures of both academic and career self-efficacy.^35,36 Interest in grit and its role in higher education will likely continue; however, whether grit should be included within student assessment models remains to be determined. Stoffel and Cain suggest that a major limitation in assessing grit is that it may not become evident until an individual is greatly challenged.³⁷ Considering the high achievement and aptitude of students pursuing pharmacy school, pharmacy students may not be sufficiently challenged by a pharmacy core curriculum for attitudes and behaviors associated with grit to become evident. Experiential rotations and postgraduate residencies require students and graduates to have significant “grittiness” and therefore may be more appropriate settings in which to measure associations between grit and professional outcomes of interest. Until the role of grit is elucidated through additional well-designed studies of health professions students, we advise pharmacy schools against implementing this construct for high-stakes admissions or curricular decisions.

CONCLUSION

Pharmacy students’ Grit-S scores varied significantly by survey year and cohort without any study intervention. The change in direction of Grit-S scores could have resulted from curricular or extracurricular influences. Grit-S scores were higher among underrepresented minorities and African American students, but lower among first-generation college students. We did not detect a significant association between Grit-S score and measures of academic or professional achievement. The strongest predictor of pharmacy GPA was undergraduate (for P1 students) or prior pharmacy school (for P2-P4 students) GPA, and this effect was not modified by the addition of Grit-S score. Additionally, higher initial Grit-S scores were significantly associated with lower odds of applying to an ASHP-accredited residency. The demographic characteristics of our study population may explain some of these findings.