Impact of Students’ Strengths, Critical Thinking Skills and Disposition on Academic Success in the First Year of a PharmD Program

INTRODUCTION

Since 2010, the number of pharmacy school applicants has decreased by about one-third while the number of available seats for students has increased by 10% leading to increased competition for the most qualified candidates.¹ This environment creates a challenge for schools to identify students capable of successfully completing the rigor of a PharmD program. The 2011-2012 Argus Commission recommended that pharmacy schools consider and assess for more non-cognitive and affective domain skills such as critical thinking, professionalism, and inquisitiveness for both admission standards and to measure curricular outcomes.² This recommendation is reflected in the current Accreditation Council for Pharmacy Education (ACPE) Standards (Standards 16).³

Pharmacy school admission committees are accustomed to assessing applicants using cognitive measures, such as PCAT scores, undergraduate GPA and grades in key, primarily science-based, undergraduate courses, as these have been shown repeatedly to be associated with performance in PharmD programs.^4-15 Success in a PharmD program and as a future health care professional cannot be determined by just examining PCAT scores and GPAs. Furthermore, by limiting candidate assessment to these cognitive measures, schools may be missing a diverse candidate population with the non-cognitive skills needed for success. In the current competitive admissions landscape, using assessments of the affective domain may help identify candidates likely to succeed in pharmacy school and beyond. However, there is no clear guidance and little research on how best to incorporate non-cognitive assessments into the admissions process.¹⁶

Various critical thinking assessment tests, some of ability and others of disposition, have been investigated for utility in predicting performance in both didactic and experiential pharmacy education with mixed results. Critical thinking skills and disposition tools used to assess pharmacy students include the Watson-Glaser Critical Thinking Appraisal (WGCTA), the California Critical Thinking Skills Test (CCTST) and the Health Sciences Reasoning Test (HSRT).^6,9,17,18 Allen and Bond found that the CCTST was highly correlated with PCAT scores and significantly, but weakly correlated with performance in pharmacy practice and experiential courses.⁶ However, the CCTST did not correlate with performance in biomedical or pharmaceutical courses or overall course grades. Kidd and Latiff found that both CCTST and critical thinking disposition as measured by the California Critical Thinking Dispositions Inventory (CCTDI), were associated with increased GPA over the four professional years of pharmacy school.⁹ Further analysis showed that the CCTDI was associated with didactic performance; whereas the CCTST was associated with performance in experiential courses. It is unclear how much these non-cognitive assessments overlap with more traditional cognitive measures or if these assessments provide unique information about pharmacy students and applicants. There is some evidence that critical thinking skills are at least moderately correlated with PCAT scores, but this data is limited.^6,19,20

The critical thinking skills and disposition assessment tools available provide quantifiable data for comparison, though other factors that may contribute to success in pharmacy school may be less quantifiable. The Clifton StrengthsFinder 2.0 Strengths Test was designed to identify an individual’s personal talents and potential for developing strengths.²¹ This assessment has been validated as a tool for personal development in both the work and academic settings. At one pharmacy school, students take the Clifton StrengthsFinder 2.0 Strengths Test and the results are incorporated into the curriculum to help students build upon their strengths and identify career options best suited to these strengths.²² More recently, Janke and colleagues identified the signature strengths possessed by students in five Midwestern pharmacy schools and compared these to the top strengths possessed by an undergraduate student population, showing a difference in signature strengths.²³

The purpose of this study is to determine the critical thinking skills, critical thinking disposition, and personal strengths that contribute to success and excellence in the first year of a PharmD program.

METHODS

This is a cross-sectional cohort study of 260 student pharmacists enrolled at Shenandoah University’s Bernard J. Dunn School of Pharmacy (BJDSOP) from Fall 2013 to Fall 2015. This study was approved by the Shenandoah University Institutional Review Board.

Critical thinking ability was assessed using the Health Sciences Reasoning Test (HSRT), a California Critical Thinking Skills Test by Insight Assessments (San Jose, CA) designed for health science students and practitioners.²⁴ The HSRT is a 50-minute high stakes assessment that provides an overall score along with subscale category scores in analysis, inference, evaluation, induction and deduction. The overall critical thinking score is classified as superior (26-30), strong (21-25), moderate (15-20), or not manifested (0-14). The test designers recommend the use of the overall HSRT scores and not the subscale categories to evaluate and compare critical thinking skills.

Critical thinking disposition was assessed using the California Critical Thinking Disposition Inventory (CCTDI), a California Critical Thinking Skills Test by Insight Assessments designed to determine the willingness or capacity to apply critical thinking skills.²⁵ The CCTDI is a 75-item assessment that ranks the disposition of an individual toward the following subscale categories: truth seeking, analyticity, open-mindedness, systematicity, confidence in reasoning, inquisitiveness, and maturity in judgment. Each subscale score is classified as strong positive (51-60), positive (41-50), inconsistent/ambivalent (31-40), negative (21-30), or strong negative (10-20). The test designers recommend the use of the subscale categories and not the overall CCTDI to evaluate critical thinking disposition.

Student strengths were assessed using the Clifton StrengthsFinder 2.0 Strengths Test (Washington, DC) to determine the themes that affect thoughts, feelings, and behaviors.²¹ In this assessment, respondents are allowed 20 seconds to choose among 177 paired self-descriptors. The result is a list of the top five signature strengths of the 34 strengths assessed.

The HSRT and CCTDI were administered to all students during orientation of the first professional (P1) year. Students whose P1 year began in Fall 2015 completed the Clifton StrengthsFinder 2.0 during the P1 orientation. Students whose P1 year began in Fall 2014 or 2013, completed the Clifton StrengthsFinder 2.0 the beginning of their P2 and P3 year, respectively. Age, gender, and ethnicity were self-reported on the HSRT and CCTDI.

Any student who did not give consent or did not complete at least one of the three assessments was excluded from evaluation. Additional exclusion criteria included completing the HSRT in less than 15 minutes, completing the CCTDI in less than 5 minutes, and less than 60% of questions answered on either the HSRT or CCTDI. Success in P1 year was defined as a cumulative grade point average (GPA) of at least 3.0 out of 4.0 and no more than one final grade of a D in any course and no final grade of an F in any course. Excellence in P1 year was defined as a cumulative GPA of at least 3.5 out of 4.0 and no final grade lower than a C in any course.

Data were de-identified prior to statistical analysis. Student characteristics were presented using descriptive statistics. The Spearman’s Rank Correlation Coefficients were calculated for age, gender, ethnicity, top five strengths, CCTDI subscale categories, HSRT overall raw score, and HSRT categorical ranking (superior, strong, moderate, not manifested) at p≤.05. Multiple logistic regression was conducted to determine the degree of influence each positively associated variable had on academic success and excellence. All quantitative data analysis were conducted in Stata for Windows, version 12.1 (StataCorp LP, College Station, TX)

RESULTS

For all enrolled class data combined, 260 pharmacy students completed at least one of the three assessments: Clifton StrengthsFinder 2.0, HSRT, and CCTDI. The proportion of females to males was approximately 2 to 1 and there were slightly fewer students who identified as Caucasian compared to other ethnicities combined (Table 1).

At the end of the P1 year, approximately two-thirds of students achieved success and one in four achieved excellence as defined in the study. The mean HSRT overall score for those who completed the assessment was 20.4, which falls between Moderate and Strong. For the mean CCTDI subscale scores, students scored highest in Inquisitiveness and all were classified as Strong except for Truth-Seeking, which fell in the Inconsistent/Ambivalent range.

Of the 260 students assessed, 256 completed the Clifton StrengthsFinder 2.0 and the top five strengths were identified. The prevalence of each strength occurring in the top five was reported as percentage of students having the strength (Table 2). The signature strengths occurring most frequently among the students’ top five were Restorative, Achiever, Harmony, Relator, and Learner. Responsibility was the sixth most commonly occurring strength and was only slightly less prevalent than Learner

Independent predictors of success and excellence in the P1 year were calculated using Spearman’s Rank Correlation (Table 3). Both the categorical ranking and raw score on the HSRT were associated with success and excellence. None of the CCTDI subscale categories were associated with success and Open-mindedness was the only CCTDI subscale associated with excellence. The strength Consistency was associated with both success and excellence. There were no other strengths associated with success. The Learner and Achiever strengths were associated with excellence. Increasing age was negatively correlated with success, but was neither positively nor negatively correlated with excellence. The strengths Command, Ideation, Self-Assurance, and Woo were negatively associated with success. These strengths were also some of the least frequently exhibited by pharmacy students.

In a multiple logistic regression model of success in the P1 year, the HSRT overall category (superior, strong, moderate, or not manifested) and the absence of the strengths Command and Woo were still associated with success (Table 4). The presence of Consistency and absence of Ideation were not significantly associated (p<.05), but trended in those directions. Even though Consistency was not statistically significant in the model, a student with this strength was 2.3 times more likely to experience success than a student without this strength. The strength Self-Assurance (n=3) was not included in the model since <5% of students surveyed had this strength in their top five strengths.

For excellence in the P1 year, multiple logistic regression supported the significant contribution of the HSRT overall category (Table 5). None of the other student variables were significant in the excellence model. However, the presence of Achiever, Consistency, or Learner strengths trended close to significant. A student with any of these three in their top five strengths was almost twice as likely to experience excellence as a student without the corresponding strength.

DISCUSSION

Determining the characteristics that are associated with excelling in the first year of pharmacy school is a reliable indicator of future performance during the remainder of school.²⁶ Previous studies showed the association of critical thinking skills with success in experiential courses, but not in didactic courses.^6,9 In contrast, this study found that critical thinking skills are significantly associated with success and excellence in didactic coursework. One difference is that this study used the HSRT to assess critical thinking skills while other studies used the CCTST assessment. Another difference may be attributed to the changing landscape of “didactic” curricula where more active learning is taking place in the classroom. Performance on the HSRT continued to be a significant contributor to both success and excellence when used in a multiple logistic regression model.

In addition to critical thinking skills, the disposition to use these skills may also be associated with performance in pharmacy school. Previously, the CCTDI overall mean score was used to evaluate performance in pharmacy school and the overall score was predictive of GPA in the first three professional years of pharmacy school, though results of the subscale categories were not evaluated.⁹ In the most recently updated CCTDI manual, evaluators are encouraged to use the subscale category scores for analysis as these categories illustrate different skillsets.²⁵ For this study, CCTDI open-mindedness was associated with excellence in the first professional year of pharmacy school. Facione and colleagues defines open-mindedness as: “... the tendency to allow others to voice views with which one may not agree. Open-minded people act with tolerance toward the opinions of others, knowing that often we all hold beliefs which make sense only from our own perspectives. Open-mindedness, as used here, is important for harmony in a pluralistic and complex society where people approach issues from different religious, political, social, family, cultural, and personal backgrounds. The opposite of open-mindedness is intolerance.”²⁵

Another way of describing open-mindedness as it relates to learning is “a compelling or overriding desire to get to the truth.”²⁷ Students strong in open-mindedness, therefore, may have a higher propensity to evaluate information objectively and with less bias than others.

The frequency of signature strengths occurring in pharmacy students has recently been reported and compared to undergraduate students at four Midwestern schools.²³ The top five strengths occurring in P1 year students at Shenandoah University’s Bernard J. Dunn School of Pharmacy (BJDSOP) match those reported for students at the University of Michigan College of Pharmacy. Three of the top five strengths at BJDSOP, Harmony, Learner, and Achiever, are consistent with the overall top five for the Midwestern schools. Responsibility was BJDSOP’s sixth most frequently reported strength. It occurred slightly less often than the fifth, and is also in the top five for all the Midwestern schools combined and in each individual program, except for the University of Michigan. It is interesting to note that the most frequently occurring strength at BJDSOP, Restorative, is shared in the top five with University Michigan, but does not occur in any of the other schools’ top five. Because the signature strengths of students at BJDSOP are similar to those at other institutions, the data should be applicable to other PharmD programs.

The only signature strength that is independently associated with both success and excellence in the BJDSOP P1 year is Consistency. People with the Consistency theme have a need for fairness in how others are treated and prefer clear rules that they can follow.²⁸ Students with the Consistency strength may thrive in an academic environment where the rules and expectations are clearly defined and success is based on effort and not preferential treatment. Even though having the Consistency strength was almost significantly correlated with success and excellence in the multiple logistic regression models, the odds ratio for this trait was higher than for any other component in the model.

The presence of any of the signature strengths Ideation, Command, Woo, or Self-Assurance was associated with a lack of success in the P1 year, though none of these strengths were highly prevalent in students’ top five reported strengths. Command and Woo continued to be significantly associated with a lack of success when analyzed in the multiple logistic regression model, though the association was small. In this model of excellence, none of these characteristics were associated positively nor negatively with the outcome.

HSRT overall category was more highly correlated as an independent predictor of success and excellence in the P1 year than CCDTI open-mindedness or any of the signature strengths. However, in multiple logistic regression models, the strength Consistency had the greatest influence on success and excellence. The Achiever and Learner strengths had higher odds ratios than HSRT overall category in the excellence model, but neither were significant in this sample. Based on the Psuedo r² values, these logistic regression models account for 14.5% and 7.4% of the variability seen in success and excellence respectively.

One limitation to this study is that students took the StrengthsFinder 2.0 assessment during different professional years. While it is not expected that strengths would change substantially, there may be some movement of strengths in and out of the top five over time. Additional limitations include sampling from a single pharmacy school and looking at only performance in the first year of that program. Since it is expected that these non-cognitive skills are harder to assess and may not influence outcomes as prominently as cognitive measures, a larger and more diverse sample may yield stronger results. Despite these limitations, there are strengths that students possess that contribute to high performance.

This is the first study to show the influence of critical thinking skills, critical thinking disposition, and student strengths on performance in a PharmD program. Until there is more data available, the StrengthsFinder 2.0 assessment should not be used as a primary admission criteria. However, as pharmacy schools compete for quality applicants, assessing strengths, critical thinking skills and disposition may help identify more students likely to succeed. Alternatively, this information may help identify students who are likely to face obstacles or who may benefit from earlier and more rigorous advising.

CONCLUSION

Critical thinking skills, the open-mindedness critical thinking disposition, and the StrengthsFinder signature strengths of Consistency, Achiever and Learner, are associated with the highest level of performance in the first professional year of a PharmD program. Further modeling that includes cognitive measures, such as undergraduate GPA, would show if these strengths are independent predictors of performance or if correlated with other characteristics.