Student Evaluation of Online Pharmaceutical Compounding Videos

INTRODUCTION

Compounding is an essential component in pharmacy practice because the skills are used to provide individualized medications to meet unique needs of a patient. For this reason, colleges and schools of pharmacy often include compounding education as a part of their curriculum. However, a report in 2012 by the American Association of Colleges of Pharmacy (AACP) Council of Sections Task Force on Compounding Education in Schools of Pharmacy noted that major limitations existed in compounding education in US schools of pharmacy.¹ The most significant findings from the AACP task force were that no standardized compounding curriculum existed, and there were inadequate laboratory facilities, equipment, and reference materials for compounding education activities.

The Pharmaceutical Care Laboratories (PCL) at the University of North Carolina Eshelman School of Pharmacy has an open access website (“pharmlabs”) that deals exclusively with compounding education and can be used to overcome the limitation of reference materials in schools of pharmacy. The website has been listed in Basic Resources in Pharmacy Education by the American Association of Colleges of Pharmacy (AACP) since 1999. Statistics from the website also indicate that 40% of its usage is in countries outside of the United States, indicating its international utility. Resources freely available on the pharmlabs website include formulation records for a variety of preparations, overviews of common pharmaceutical dosage forms and related compounding exercises, interactive study questions, and a compounding video library with nearly 100 animations and videos.

Initially, the website contained only text material and images, but was enhanced with three animations and 39 videos when digital video cameras became readily accessible. The animations and videos detailed compounding procedures and techniques and demonstrated pharmaceutical principles applicable to compounded preparations. In the 2010-2011 academic year, the course coordinator (who was also the pharmlabs webmaster) contracted with the university’s Information Technology Services to develop version 3.0 of the website. During the upgrade to version 3.0, pharmlabs was redesigned and more than 350 study questions on the website were programmed into Adobe Captivate (Adobe Systems, Inc., San Jose, CA) to provide an interactive question-and-answer format. Additionally, 61 more high definition videos and animations were added to the video library. Currently, all of the videos on the website can be viewed on Adobe Flash Player (Adobe Systems, Inc., San Jose, CA). They vary in length, but most are typically 6-8 minutes long.

The new videos were added for two primary reasons. First, the compounding component in the laboratory sequence had significantly expanded to warrant additional website materials. Second, content delivered via the Internet allowed students to control the pace at which they consumed content and determine how, when, and where they wanted to learn the material.²

The 61 new videos and animations were posted to the website at the start of fall 2011 semester. The videos were available to first-, second-, and third-year (P1, P2, and P3) students for the entire fall 2011 and spring 2012 semesters. This availability created an opportunity to evaluate the students’ perception of the overall usefulness of the complete video library and to study the effect of the complete library (with 98 videos and 5 animations) compared to the more limited library (in version 2.0) on the same group of students as they progressed through their P1, P2, and P3 years. There is no published literature evaluating the use of compounding videos as a learning tool during compounding curriculum in US schools of pharmacy. To address these questions, the first objective of its overall usefulness was measured by surveying all three years of students when they all had access to the complete library. A secondary objective used the analytical results of the students’ compounded preparations to determine if having access to the complete video library improved students’ performance compared to when the students had access to the more limited version 2.0 library. For several years, chemical analysis has been used in the Eshelman School of Pharmacy PCL sequence as the basis for grading compounded preparations.^3,4

METHODS

A written survey instrument was developed to evaluate aspects of using online videos for compounding education. The survey contained 22 items and collected demographic (two questions), accessibility (three questions), functionality (six questions), usefulness (four questions), and student confidence (three questions) information about the videos (Table 1). Demographic and accessibility information were collected using multiple-response and dichotomous (yes or no) responses. The remaining data was collected using rating scale questions ranging from 0 centimeters=strongly disagree (or strongly agree with an item on left) to 10 centimeters=strongly agree (or strongly agree with an item on right). Figure 1 shows an example of the rating scale questions.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

An Example of a Rating Scale Question Regarding Compounding Videos.

The surveys were given as a paper form to all pharmacy students (classes of 2015, 2014, and 2013), except fourth-year pharmacy students and administered during the PCL class time in spring 2012. All analyses were done using SPSS software (IBM, Armonk, NY). Means and standard deviations were calculated for the rating scale response items. For subgroups of data (ie, between classes and campuses), the independent t test was used to determine differences between groups with significance set a priori at p<0.05. The Levene’s test of equality of variance was used to examine homogeneity of variance among three classes at independent campuses. This test was used to verify equal variances at the 0.05 significance level. The three campuses compared in the study were Chapel Hill (CH), Elizabeth City State University (ECSU), and the University of North Carolina at Asheville (ASHE). The work was exempt from IRB approval at the Eshelman School of Pharmacy.

The accuracy of the active pharmaceutical ingredient (API) preparation strength comparison was possible because the P3 students who had access to the complete video library in 2011-2012 had access to the more limited video library as P2 students in 2010-2011, and as P1 students in 2009-2010 (see Table 2). The P2 students who had access to the complete video library in 2011-2012 had access to the more limited video library as P1 students in 2010-2011. In the different P1 years, the students compounded three identical preparations analyzed by the same analytical methods: diphenhydramine syrup, metronidazole saturated solution, and phenylephrine nasal gel. In the different P2 years, the students compounded three identical preparations analyzed by the same analytical methods: metoprolol tablet triturates, hydrocortisone medication sticks, and ibuprofen effervescent powder. The analytical data were evaluated with a one-way analysis of variance (ANOVA) for statistical differences. Another measure of variation in the analytical data was the number of students who compounded the preparation accurately on the first attempt, defined as ±10% of the theoretical API strength.

RESULTS

The demographic results showed that the survey was completed by 391 students (156 P1, 144 P2, and 91 P3 students), which represents an 85% response rate. More than 50% of all students reported they spent one to two hours, on average, preparing for individual compounding laboratories (Table 3).

All but five students indicated they had used the videos on the pharmlabs website. The majority of students (88.8%) thought the videos were easily accessible. Approximately 80% of all students were satisfied with the functionality of the videos in terms of overall picture quality, narrator’s voice, and comprehensiveness. The mean rating scale for “pace of compounding video” and “length of compounding video” was 5.04 cm and 5.46 cm, respectively. The rating scale was from very slow to very fast for the pace of the video, and from too short to too long for the length of the video. The two ratings being in the middle of the scale meant students were satisfied with both aspects of the videos (ie, the pace was neither too fast nor too slow and the length was neither too long nor too short). A mean rating scale of 8.2 cm indicated that students were highly satisfied with the videos as a learning methodology in compounding education.

The vast majority of students (99.2%) indicated they had used the videos as a resource to prepare for the compounding laboratory. More than 96% of students used the formulation record as well. The video presentation was a demonstration of the compound given on the formulation record, so it would be reasonable to expect that students would predominately use both resources when preparing for the compounding laboratory. Other resources students used for compounding laboratories were asking questions of teaching assistants or peers (61.4%), textbook (60.4%), and public online resources (37.6%).

Students were also asked to compare resources in pairs (see Table 4). The first resource listed in the table cell was on the left side of the scale, and the second resource was on the right side of the scale. The majority of students preferred the compounding videos to the formulation record, the textbook (which is a course requirement), and consulting with teaching assistants.

Table 5 summarizes the helpfulness of the compounding videos and the level of interest by professional year and campus location. Students in the different professional years found the videos “very helpful” to the same extent (approximately 8.8 cm) regardless of the campus location. This strongly indicated that the accessibility and functionality of the videos were equal on all campuses. The rating scales for the “change in interest” were equivalent regardless of campus location. The ratings were in the middle of the range, suggesting that using the videos did not increase or decrease the students’ interest in compounding.

There was no statistical difference in any of the rating questions when the average of all of the students (P1, P2, and P3), regardless of campus location, was compared. However, in the subgroup analysis, the P1 and P2 students differed in their ratings of some of the rating scale questions (Table 6). The P1 class had a corresponding cohort at both branch campuses, but at this time, the P2 class did not have a similar cohort at the ASHE campus. The results suggest that overall ECSU and CH students were more satisfied with the videos than ASHE students. This was not unexpected as ECSU had been established for several years as a branch campus, CH is the main campus, and the ASHE program was more recently established.

Another analysis was performed to determine differences between the P1, P2, and P3 students who completed the survey. The mean data for each professional year was composed of all of the students in that year regardless of campus affiliation. The questions showing statistical differences are presented in Table 7. The P2 students generally gave higher ratings than other classes.

To evaluate the issue of self-confidence, three questions were included in the survey (Table 8). The level of confidence or competence was similar regardless of the professional year or the campus location. The rating of approximately 7.0 cm suggests the students felt “somewhat” confident in compounding, but not fully confident.

Student performance was investigated by reviewing the results of the three common preparations students compounded in their P1 or P2 years (Table 9). Analyses were carried out on almost all of the student compounded preparations as part of the assessment process.^3,4 As shown in Table 2, the progression of the students evaluated by the survey can be tracked back to the time they were P1 or P2 students. Three compounded preparations were repeated by all P1 students for two years when these students had access to the version 2.0 video library and one year when they had access to version 3.0. Three additional compounded preparations were repeated by the P2 students for one year using version 2.0 and one year with version 3.0 of the pharmlabs website. The analytical results of these data are shown in Table 9.

The one-way ANOVA did not show any statistical difference between compounded preparations for the P1 group. At face value, the data suggests that P1 students with access to the pharmlabs 2.0 library were as successful in compounding as P1 students that accessed version 3.0. This result was also reflected in the P2 student data where there was no significant difference using a z test analysis (as the variance of each group was known).

Another way to consider variance in the compounded preparations was to determine the number of students who correctly compounded the preparation on the first attempt (Table 9). The largest increase in this parameter was with metoprolol tablet triturates (11% to 86%) when the P2s had access to version 3.0 (all of the video library). It was unlikely that just having more videos on the website would lead to increased accuracy in compounding. It is more likely that some other factor was more influential in students achieving greater accuracy such as encouragement from instructors or teaching assistants to watch the videos. It might also suggest that the P2s in that year were better prepared to compound any preparation, but the data for the other two preparations does not bear out that conclusion.

DISCUSSION

The unique role of compounding in current pharmacy practice continues to grow, and it is critical to obtain compounding skills as a part of pharmacy curriculum at schools of pharmacy in the United States. Inadequate laboratory reference materials for compounding education activities exist in schools of pharmacy.¹ The open-access pharmlabs website can provide an interactive and reliable source of compounding materials.

Our investigation was to have a portion of the website (the compounding videos) assessed by the pharmacy students at the school. The students found the videos were easily accessible, and the functionality and usefulness of videos were highly satisfying to the majority of students in different classes and campuses. The videos were also judged to be the preferable resource compared to a required textbook, a formulation record, or asking teaching assistants or peers for information. Most of the results of the survey were expected and mirrored earlier published results from this laboratory.⁵

One of the unexpected results was the students’ perception of confidence and competence in compounding. These levels were the same (about 7.0 cm) regardless of the professional year or the campus location. It seems reasonable that P3 students would have felt more confident/competent than the P2s, and the P2s more confident/competent than the P1s because those with more semesters of compounding would be expected to have more relevant experience.

One interpretation of these results is that the online compounding videos did not provide a complete resource for the students. A more likely explanation is that repeatedly compounding specific preparations would give students more confidence, but the students did not have such an opportunity. The students compound more than 20 different preparations during their compounding years in the PCL. The only opportunity to repeat a compound is if they choose to remake a preparation that is outside of the acceptable range of API accuracy. There is no indication that students lacked confidence in basic laboratory operations.

Another unexpected result was that the P2 students were the most often statistically different class for the aspects of this investigation. Class preferences are difficult to understand, but the P2 class might have had more experience in compounding than the P1 students and probably knew how the laboratory information was organized and operated. However, that argument would not hold true for the P2-P3 comparison, as one would expect the P3 to have more experience and laboratory operations. Possibly, more emphasis was to given to the P2 students in reminding them that quiz questions given in the beginning of laboratory periods often came from the video material.

None of the results outlined above were unexpected because a significant body of literature substantiates that video learning can be as effective as in-class instruction.⁵ Some reports suggest that student performance is enhanced with online instruction,^6,7 but there is also literature that shows no difference in performance with additional online instruction.^8,9 Several studies looked specifically at the effects of technology on learner self-confidence.^10,11 The analytical assays did not indicate any differences in the overall API accuracy between groups that used the more limited video library of version 2.0 compared to the complete library in version 3.0. It was anticipated that the number of students compounding a preparation correctly on the first attempt might increase with the addition of more videos and the integration of interactive questions. Studies have looked specifically at the effects of technology on learner self-efficacy, which is a major influence on academic performance.^6,7 With the exception of metoprolol tablets, this influence was not seen. The predominate variation in the results may be from the students themselves, as each student would have brought different degrees of experience, foundational knowledge, and comfort level to the compounding laboratory. In addition, students were not compounding in a vacuum. Each academic year brought curricular changes and modification in teaching, as well as in the composition of the class.

CONCLUSION

This is the first study that describes pharmacy students’ perspectives on effectiveness of compounding videos as one of their compounding resources by measuring objective and subjective outcomes. The investigation shows that the compounding videos on the pharmlabs website were preferred and well-received by the students at the Eshelman School of Pharmacy. As such, the website could serve as a resource for other schools in the United States. The investigation also demonstrated that analysis of compounded products is a necessary corollary for any compounding laboratory teaching experience to fully understand the sources of true variance in the art.