Student Performance with Standalone Calculations Course vs Combined Pharmaceutics-Calculations Course

INTRODUCTION

Within the first two years of teaching the integrated curriculum, faculty noticed that when students were evaluated by OSCEs, they lacked expected proficiency in pharmacy calculations. Faculty reflection on curriculum design and study of pharmacy education literature revealed a possible cause. While pharmacy calculations textbook and teaching quality were believed to be high,¹ calculations were being taught within an integrated course containing both calculations and pharmaceutics. Assessment details revealed that students were able to pass the course without mastering calculations skills.

At Manchester University College of Pharmacy, faculty members decided to split the combination course into distinct pharmaceutics and calculations courses. The combined pharmaceutics-calculations course had been 4-credit hours in the fall and 4-credit hours in the spring. This structure was later changed into a 3-credit hour pharmaceutics course and a 1-hour calculations course, both fall and spring semesters. The total number of hour credits and timing in the first didactic year remained the same. Course outcomes and their mapping to the overall curricular plan stayed intact, and assessment items were divided proportionately. Separating the calculations material into the standalone course allowed focused time for increased homework assignments and expanded in-class practice and support. Since calculations course work was no longer combined with pharmaceutics, students were motivated to learn each calculations topic and problem type. Faculty members were interested to see whether this course restructuring would result in better student performance.

It has been reported in the literature that student retention of calculations knowledge is an important consideration in pharmacy curricula.^1-3 ACPE Standards 2016 lists pharmaceutical calculations as a separate content area from pharmaceutics. The Standards expect students to be “practice ready and that they will be able to retain, recall, build upon, and apply” knowledge to deliver quality patient care.⁴

It is with this aim that course faculty changed the calculations teaching to a standalone course. The authors’ objective was to determine whether a standalone pharmacy calculations course promoted performance and retention when compared to pharmacy calculations taught within a pharmaceutics course. To explore the impact of calculations course restructuring, instructors looked at data (course grades, OSCE scores, pre-pharmacy math GPA) and collected two more data sets, a calculations self-assessment and student opinion survey. They hypothesized that course grades, OSCE scores, and self-assessment scores would be higher in students who took the standalone calculations course. This research was conducted to test whether the described course change was effective and to provide insight to other pharmacy educators.

METHODS

Pharmaceutical calculations was offered to first-year pharmacy students either as a standalone course or integrated within a pharmaceutics course. Student performance and retention were compared to determine whether teaching calculations as a separate course was beneficial. Three consecutive classes were assessed: two were the integrated pharmaceutics course, and one was a standalone calculations course. Data compared between the course types were course grade, OSCE score, a calculations self-assessment, and pre-pharmacy math GPA. Institutional Review Board (IRB) approval was obtained prior to administration of the student survey and initiation of retrospective data analysis. At the time the IRB evaluation was sought, the self-assessment had already been given to 2 of the 3 classes as it was originally developed as a student tool, not part of a study. All other measures were obtained during normal course proceedings.

The self-assessment, developed by practice faculty and similar in content to the problem types the students had learned in their course, was intended to be a measure of performance and retention. It was not counted toward a grade in any course. This problem set was given to students prior to their starting pharmacokinetics, approximately 5 months after their pharmacy calculations coursework. Students who participated in the self-assessment were not given any advanced notice about the examination and participated voluntarily. The OSCE used in this study occurs in the fall of the second didactic year and includes calculations problems. The students took the OSCE approximately 7 months after completing their calculations course work. Before the OSCE, they were given opportunities to practice calculations but no further instruction was provided. Only scores from the calculations station of the OSCE were used in this study.

To provide qualitative data, student views were assessed using three Likert-type survey questions about their confidence in solving pharmaceutical calculations problems and their perceptions of the effectiveness of the calculations course. Each survey question had an open comments field. The survey was administered using Qualtrics software (Qualtrics, Provo, UT). Students were also asked whether they felt they had extra help available during their calculations course and whether they used or sought help. Students’ math GPAs from their pre-pharmacy course work were compared between the two course types. Statistical analyses of these data were made using SigmaPlot software (Systat Software, San Jose, CA). Because the two groups’ data were not normally distributed, nonparametric analysis was used to compare them. A Mann-Whitney U test was used to determine if there was a significant difference between the data obtained from the two teaching methods. Pearson’s correlation coefficient (r) was used to test interdependence between the dependent variables of performance and retention. Both the calculations self-assessment and the OSCE were used to measure retention, and course grade was used to measure performance. A p value less than .05 was considered significant.

RESULTS

To provide an equivalent comparison to the combined pharmaceutics-calculations course, standalone pharmacy calculations and pharmaceutics course grades were averaged together (Figure 1). The percent median course grade for the group that took the combined course was 83% and 89% for the averaged standalone calculations and pharmaceutics courses. There was a significant difference in the course grades between the two groups (p<.001). The average pre-pharmacy math GPA was 3.00 for the group who took the combined course and was 3.18 for the group who took pharmacy calculations as a standalone course. This difference was found not to be significant (p=.074).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Student Performance in a Calculations Course Taught as Combined or Standalone Course.

The percent median calculations self-assessment score for the combined course group was 38 (n=140) and for the standalone course group, 50 (n=44) (Figure 2). There was a significant difference between the two groups in how they scored on the calculation self-assessment (p<.001). The combined course group’s percent median OSCE score was 78, and it was 85 for the standalone course group (Figure 3). There was a significant difference in the OSCE scores between the two groups (p<.001). Pearson’s coefficient showed that there was a significant positive correlation in both student groups, between course grade and both measures of retention (Table 1). For the self-assessment, r=.51 (p<.001), and for OSCEs, r=.47 (p<.001).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Student Retention in a Calculations Course Taught as Combined or Standalone Course.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Student OSCE Performance in a Calculations Course Taught as Combined or Standalone Course.

Results from the student opinion survey (response rate 55%) are listed in Table 2. When asked to rate the effectiveness of their calculations course structure, 21% of the students in the combined course said it was not effective while 6% of the students in the standalone course group rated the same. Only 1% of students in the combined class rated their course very effective while 28% in the standalone course rated their course as such. Thirty-four percent of students in the standalone calculations course reported they would be “very confident” that they could work calculations problems on Introductory and Advanced Pharmacy Practice Experience (IPPE/APPE) rotations while 18% of students in the combined course reported the same. On the same question, none of those taking the standalone calculations course reported “not confident,” whereas 10% of those taking the combined course did. Student confidence in ability to work calculations problems on the North American Pharmacist Licensure Examination (NAPLEX) and in future practice were similarly distributed between the two calculations course types. When asked whether students felt they had extra help available during their calculations course, 38% from the combined course said yes; whereas 59% of those in the standalone course said yes. Both groups were almost identical (36% vs 37.5%) in the percentage of students who said they sought or used extra help.

DISCUSSION

Course grades and OSCE scores improved due to a more thorough learning of calculations concepts and processes. The standalone course allows students a better pace, which facilitates greater focus on performance in each topic. A recent study comparing flipped classroom with traditional classroom calculations teaching found that student performance was better with the flipped model but that performance difference was not retained at 6 months.⁵ While the benefits of flipped classroom design are well-established, the choice has been made at this pharmacy program to blend elements from both flipped and traditional models when teaching pharmacy calculations. Each class meeting of the standalone calculations course includes some traditional instruction but also pre-class work and student problem-solving demonstrations. Students have expressed perceived value in having homework assignments and quizzes in addition to calculation demonstrations in class. This not only reflects accommodation of different learning preferences, but also shows the benefit in the element of student responsibility for their own learning. The layers of student practice with each problem type resembles the method used over many decades in the teaching of surgical procedures, “see one, do one, teach one.”⁶ Students have had some additional help available for the times they did not understand a topic, but this available assistance did not change the number of students who sought or used this help. Extra help had a positive impact on those who wanted it, but those who used other methods (eg, peer assistance, reading other texts) continued to do so. This speaks to the spectrum of learning preferences.⁷

Students responded similarly when asked about their confidence in working calculations problems on IPPE/APPE, NAPLEX, and future practice. The authors believe this reflects student uncertainty about what they will see beyond their didactic experience. Regardless of their performance in the classroom at the time of didactic coursework, they cannot know NAPLEX or future practice challenges. The place where student opinions agreed with this study’s numerical statistics was when they were asked to rate the effectiveness of their calculations course structure. Students in the standalone course were more positive. Some students commented in the survey that they wish they had the standalone instead of the combined calculations course. Students perceived there was more extra help available with the standalone course than the combined course, which reflects the additional class time devoted to practice and in-class assistance. Interestingly, the number of students seeking or using extra help was the same in both classes, regardless of its perceived availability. Based on the similar help usage between groups and some comments received in the survey, the investigators believe this is more of a reflection of personal learning preferences.⁷

Another set of information that was not part of the study data was found to be noteworthy. The pharmacy program routinely convenes focus groups from each class of students to ask about their experiences in the curriculum. The focus groups had met approximately 1 month prior to the student survey. Qualitative information from these focus groups included statements from some students in the standalone calculations course that they felt they were learning more effectively and staying on track during the semester. They felt the calculations course faculty had incorporated student feedback into course revisions. Some students in focus groups who took the combined calculations pharmaceutics course said they did not feel well-prepared in calculations skills. While these statements are merely anecdotal and are separate from any conclusions drawn from the data analysis, they correspond with the study results.

There are a few limitations to this study. It would have been desirable to tease out and grade the calculations component of the combined pharmaceutics-calculations course assessments for better comparisons. This was not possible as the learning management system was changed during the study period and that level of detail was not available. Because of the retrospective nature part of this study design, an alternate approach of combining the standalone course grade as an average to compare against grades from the combined course was chosen. This was determined to be the best strategy to achieve a fair comparison as the credit hours and percentage of questions remained the same. The optimal measures would have been Pharmacy Curriculum Outcomes Assessment (PCOA) and NAPLEX scores and even future practice competence, but these were not available to the researchers at this time. There is no real way to objectively measure retention of calculations learning in the practice years.

Faculty teaching in the combined course and the standalone calculations and pharmaceutics courses stayed mostly consistent throughout the study period. Course material was consistent across the years of the study. Since separating calculations from the pharmaceutics course allowed adjustments of teaching methods with that material such as more homework, in-class practice, and support, it is likely that these elements increased student success. This is not necessarily a confounder, however, as the positive changes may not have been possible if calculations was still tied to pharmaceutics.

While not a limitation, the separation of courses described in this study is a step away from curricular integration, the current trend of many professional educational programs. While the authors of this study believe in the concept and benefits of integration to enhance relevance and depth of learning, they are also sensitive to the needs of student pharmacists to be APPE and practice-ready with vital skills.⁸ A 2016 survey revealed skeptical faculty comments about curricular integration, including “Full integration, however, is not practical. When everything is integrated, some of the contents get lost.”⁹ Another source noted that integrated curricula may be “created at the expense of disciplinary depth and the resulting content dilution contributes to superficial learning.¹⁰ A recent article by Davies and colleagues described another pharmacy college’s efforts to find the optimal place for calculations material in the didactic curriculum.¹¹ They described an evolution to their current model of a standalone calculations course in the first didactic year, followed by vertically integrated elements later in the didactic curriculum for reinforcement. Their model has features in common with our approach, although this study includes only the first didactic year.

CONCLUSION

Pharmaceutical calculations taught as a standalone course is more effective in achieving high student performance and retention. Student opinions gathered by survey and focus groups agree with these findings. Revisiting, reviewing, and vertically integrating calculations problems into the later didactic curriculum will help reinforce this knowledge at any program.