Abstract
Objective. To determine whether a pretest assessing algebra-based problem-solving skills could aid in identifying those who may underperform in calculations course assessments and whether this provides additional value beyond preadmission and demographic characteristics.
Methods. Student pharmacists were screened for algebraic problem-solving skills using an 18-item pretest taken the semester prior to a course containing pharmaceutical calculations content. These scores were compared to students’ later performance on pharmaceutical calculations assessments. Linear regression models were computed to determine the relationship between pretest scores and pharmaceutical calculations performance after controlling for preadmission factors and demographic characteristics.
Results. The median pretest score was 15 out of 18 possible points, with scores ranging from 5 to 18 points. After controlling for age, gender, American College Testing (ACT) scores, and high school grade point average (GPA), scores on the algebra-based, word-problem pretest were associated with performance on pharmaceutical calculation assessments.
Conclusion. This research demonstrates the ability of a pretest aimed at identifying deficiencies in algebraic problem-solving skills to identify those at risk of failing to obtain mastery of pharmaceutical calculations, even after controlling for demographics, prior grades, and prior standardized test scores. Identifying these students is a first step towards implementing tailored interventions to improve students’ algebra-based word problem skills to prevent deficiencies in pharmaceutical calculations mastery before class even begins.
INTRODUCTION
Mastery of pharmaceutical calculations is of vital importance for student pharmacists given the ramifications of errors and mistakes in pharmacy practice. At the Raabe College of Pharmacy at Ohio Northern University, we observed a greater number of students each year with deficient grades in introductory pharmaceutical calculations content. This trend, while only recently observed locally, may not be so recent. In 2002, Latif and Grillo reported a large proportion of students entering pharmacy school who seemed to be deficient in basic math skills.1
There are a variety of course designs2⇓⇓-5 and assessment models5,6 for teaching pharmaceutical calculations. We first attempted to improve student performance and reverse the trend by using evidence-based educational practices, such as incorporating a flipped classroom approach7 and repeated testing of calculations,8 but we saw no appreciable impact. The teaching team probed our experiences to identify the underlying problem. Based on our experiences, we hypothesized that some students may lack basic problem-solving and algebraic skills and could benefit from some form of basic skills development prior to taking the pharmaceutical calculations course. However, we first had to identify those students. The ability to identify students who may struggle with performing pharmaceutical calculations would assist educators in providing tailored and targeted support and interventions to improve their ability.
Previous literature has detailed several demographic and preadmission characteristics that may impact mastery of calculations content. Preadmission variables, such as overall Pharmacy College Admission Test (PCAT) scores, PCAT quantitative ability scores, and undergraduate math/science grade point average (GPA) have been identified.9⇓⇓⇓⇓-14 In addition, performance in similar past courses are good predictors of performance, and time away from material can be detrimental to performance.10
Instead of using administrative preadmission data, what if an algebra-based pretest could be given to students to identify those at risk? Indeed, such a test, the Basic Math Skills Test (BMST) was reported in the literature two decades ago. The BMST was a 50-item test measuring math ability at the eighth-grade level. The BMST scores among student pharmacists correlated well with preadmission markers linked to calculations performance1,9,11 and improved the variance explained by a linear regression model for calculations course scores beyond those markers.9 The BMST, while promising, was not considered for the present research as there is limited current information regarding this test. Instead, the notion of identifying those at risk of deficiencies, going beyond demographic and preadmission characteristics, was put into use in this research. The purpose of this study was to determine if a pretest assessing algebra-based problem-solving skills could aid in identifying those who may underperform in calculations course assessments, and if this provides additional value beyond preadmission and demographic characteristics.
METHODS
This IRB exempt observational study investigated calculations performance in a cohort of student pharmacists at a 0-6, direct-entry PharmD program. The Profession of Pharmacy (1-5) course sequence at Ohio Northern University consists of five sequential two-credit, semester long courses containing a variety of pharmacy-related content; professionalism, pharmaceutical care, the Pharmacist’s Patient Care Process, introduction to the US health care system, drug information, and calculations. The calculations content is divided between two courses, with half taught and assessed in Profession of Pharmacy 3, during students’ third semester of the program, and the remainder taught in Profession of Pharmacy 5. Topics taught and assessed in Profession of Pharmacy 3 included aliquots, dosing, expressions of concentration, dilution/concentration, reconstitutions, prefabricated dosage forms, units of potency, and alligation. This is a didactic course with students’ mastery of the content assessed through three summative examinations during Profession of Pharmacy 3, and a subsequent review examination during Profession of Pharmacy 4 (a course containing no new calculations content).
The sample consisted of students who entered the pharmacy program in the fall of 2017, took Profession of Pharmacy 3 in the fall of 2018 (n=118/123; 96%), and consented to participate in this research during a math pretest administered during Profession of Pharmacy 2. Students took an 18-item math pretest in the spring semester of their first year of the program during Profession of Pharmacy 2. This pretest contained 17 free response word problems and one multiple-choice question. The pretest was developed to evaluate algebraic problem-solving and application of proportional reasoning, unit analysis, and percentages using both pharmacy and non-pharmacy related concepts. For instance, one item assessing proportional reasoning indicated a set number of children per adult attending a movie, gave the total attendees, and asked for the number of children in attendance. The 20-minute pretest was administered to students electronically using the college’s learning management system. Students were allowed to use a calculator. No partial credit was awarded for incorrect answers.
Calculations performance was assessed in two ways. Individual student scores from three summative 50-point calculations examinations administered during Profession of Pharmacy 3 were obtained from course instructors. These examinations varied in number of questions and were not inherently cumulative, but may have required students to integrate knowlege of concepts they had previously learned to be able to answer the question correctly. Similar to the pretest, questions on the examinations were open-ended word problems. Scores from these three assessments were summed. Student scores from a 12-item, 60-point cumulative calculations review examination administered in Profession of Pharmacy 4 were also obtained from course instructors. Here again, answers were scored as correct or incorrect with no partial credit given.
Demographic and preadmission characteristics were collected from administrative records for all study participants. These included gender, age, ACT total, ACT subscores (ie, math, science, English), and high school GPA.
All statistics were computed using SPSS, version 25. Descriptive statistics were used to characterize the samples and inspect the variables. Pearson correlation coefficients were computed between pretest scores, demographic characteristics, and preadmission factors, and calculations performance. Linear regression models were used to understand the relationship between pretest scores and calculations performance after controlling for demographic and preadmission factors. For correlational and regression analyses, listwise deletion of those cases with missing values was used, resulting in an overall analytic sample of 105 students.
RESULTS
The mean age of consenting participants was 20 years during Profession of Pharmacy 3, and female was the gender listed for 63% of students. The median composite ACT score was 26, with math, science, and English subscore medians of 27, 27, and 25, respectively. The mean adjusted high school GPA for this group was 3.99. Of note, adjusted GPAs may be above 4.0 due to weighting of advanced courses. Descriptive statistics of the pretest and calculations assessments is shown in Table 1.
Pretest and Calculations Examination Scores
Bivariate correlations between all study variables are shown in Table 2. Pretest scores and preadmission factors were positively and significantly associated with calculations performance.
Correlations Between All Study Variables
The results from ordinary least squares regression models for the total points earned on the Profession of Pharmacy 3 calculations examinations and for the Profession of Pharmacy 4 cumulative review examination are displayed in Table 3. The pretest was associated with both indicators of calculations performance, net the effect of ACT subscores, high school GPA, age, and gender (Table 3).
Associations Between Student Characteristics, Pretest Scores, and Calculations Performance
DISCUSSION
Research in the early 2000s sought to identify potential low performers of pharmaceutical calculations by administering a mathematics examination.1,9,11 This present work sought to continue and reignite that inquiry to help identify students who face difficulty in mastering the calculations content within the PharmD curriculum. We found that performance on a pretest aimed at identifying deficiencies in algebraic problem-solving skills was related to calculations scores within a calculations course, as well as to scores on a review examination completed a semester later. Most notably, this relationship persisted after controlling for age, gender, earlier grades, and ACT sub-scores. This may imply there is merit to more deeply understanding the underlying skills that students lack and attempting to support students in these areas.
We found that performance on an algebra-based pretest was an indicator of students’ future success with pharmaceutical calculations. Student results from this test may also be relevant to later courses and examinations where similar skill sets are used. For instance, results of this test may help to identify students at risk of having difficulty with pharmacokinetics content, be related to performance in calculations and pharmacokinetic content on the PCOA and the NAPLEX, and most importantly, performance in the real world when it really counts for patient safety and outcomes. Future research can explore these relationships, as well as the impact of interventions aimed at mitigating the impact of preexisting deficiencies in student skills.
The present findings must be juxtaposed with limitations. First and foremost, the study was conducted at a single, private 0-6 direct-entry pharmacy program. This may limit the generalizability of our findings to other PharmD programs. Furthermore, while the present study found similar factors were associated with calculations ability at our program compared to prior research in “2 + 4” PharmD programs (ie, prior grades and standardized test scores),9⇓⇓⇓⇓⇓-15 there are possible differences that warrant caution when applying our findings en masse. For instance, students entering a four-year PharmD program would likely have a stronger pregraduate didactic base, including possible undergraduate coursework in mathematics, that may have bearing on their performance.10
The pretest used in this study was not fully validated or subjected to advanced psychometric testing. It is noteworthy that a rather simple and brief test was so strongly related to calculations performance, even after adjusting for standardized test scores and GPA. This suggests that other programs could institute their own tests developed to identify deficiencies in students, with the goal of program creation or resource allocation to help students achieve academic success. As we consider our pretest, it aided in identifying students at risk of failure but did not diagnose the root of the problem. After establishing the link between pretest and calculations performance, our institution used the pretest more broadly for two years as a self-assessment for students to identify their own learning needs while subsequently offering a mathematics mindset and problem-solving workshop series aimed at building student ability. This raises an additional limitation of the present work. Even with a variety of potential indicators, the models in this study explain only approximately 30% of the variability in calculations performance. There are a variety of additional factors beyond and outside of the pretest and preadmission characteristics that influence calculations performance. Student study habits,15 mathematical identity,16,17 mindset,18-19 motivation,20 social support,21 accountability,22 critical thinking,23 and metacognitive ability24 are just a few of the possible areas to assess and create interventions that may impact calculations performance.
As an Academy, work is needed to design methods to identify pharmacy students who require additional support to develop into excellent pharmacists. One possible method would be to assign online modules of content for those students to complete during the semester, which has been reported as successful in remediating deficiencies in broader content like prior biochemistry, mathematics, and chemistry experience.25,26 Students could be given those modules along with pharmaceutical calculations content to better support them for success. Students could also be paired with mentors who could work with them in remediating algebra-based skills as well as supporting them in learning pharmaceutical calculations.9
Future work could further expand on the pretest to target specific skills. This will allow us to hone in on specific interventions for specific students, such as algebra review, problem-solving skills, growth mindset, or other interventions. It is important to identify those who will struggle with pharmaceutical calculations, and determine how to intervene and provide support.
CONCLUSION
This work demonstrates that pharmacy student scores on a pretest aimed at identifying deficiencies in algebraic problem-solving skills are associated with calculations performance, even after controlling for age, gender, earlier grades, and standardized test scores. This study identifies the demographic and preadmission characteristics associated with low performance in calculations and provides proof of concept supporting development of a validated tool to identify possible low performers. Colleges of pharmacy have a vested interest in ensuring student mastery in calculations and can use a similar tool to identify those needing extra support in the PharmD curriculum. Once those at risk have been identified, educational interventions can then be designed to support those students.
- Received November 12, 2020.
- Accepted May 7, 2021.
- © 2022 American Association of Colleges of Pharmacy
REFERENCES
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.