Student Self-Analysis of Their Nonsterile Preparations and its Effect on Compounding Confidence

INTRODUCTION

Compounding is an essential component of pharmacy practice because of its ability to provide individualized medications to meet a patient’s unique needs. In a 2009 survey, independent pharmacists believed a compounding course with a laboratory component should be required in pharmacy schools to prepare student pharmacists with basic compounding competence for pharmaceutical care practice.¹ Therefore, graduating students should have the competence and confidence to compound quality preparations.

Park and Shrewsbury’s study reported on students’ opinion on the expansion of the materials on the pharmlabs website (pharmlabs.unc.edu), one of the primary reference sources used in their compounding laboratory course.^2,3 As part of that survey given at the completion of the course, students were asked about their confidence to compound acceptable preparations. Students self-reported their confidence level at 70% using a 10-point continuous Likert-type rating scale with the range of “not confident” to “very confident.” This percentage was disconcerting to those authors because these students had access to the following resources: a compounding textbook, more than 350 study questions, Formulation Records for each preparation to be compounded, in-laboratory teaching assistants to answer questions and provide additional instruction beyond the weekly pre-laboratory class, and their graded Compounding Records and prescription labels from earlier assignments. The 70% confidence level raised an important question, “Is 70% the maximum limit of a student’s confidence, or would additional resources or experiences result in a higher percent confidence?”

One assessment tool used at the Eshelman School of Pharmacy for 20 years is the pharmaceutical analysis of student preparations.^4-6 The measurement of preparation potency has been encouraged as another assessment strategy,^7,8 but the consistent application of such a strategy is lacking in many pharmacy schools and colleges.⁹ Pharmaceutical analysis provides profound insight into the students’ ability to accurately compound preparations. The analytical work also enhances student understanding of the quality of compounded preparations and encourages them to prepare better preparations. Since this resource was not evaluated in the Park and Shrewsbury study, it was thought that having the students perform their own analysis might increase their self-confidence and thereby competence in their compounding ability. The purpose of this study is to determine if including the self-analysis of nonsterile compounded preparations during a laboratory exercise will improve student pharmacists’ confidence in their ability to compound acceptable preparations.

METHODS

All students in their P1 and P3 years were eligible to participate in the study. The initial survey (pre-survey) was administered to the P1 and P3 students in paper form during the first week of their required compounding course in fall 2016. The final survey (post-survey) was the same as the initial survey and was given to the same classes in paper form during the last week of the semester. The study was conducted at the Eshelman School of Pharmacy and was approved by the University of North Carolina at Chapel Hill Institutional Review Board.

The P1 course was a new single semester course (12-week, 2-credit hours) that had been developed for the curricular transformation the school began the previous year.¹⁰ Therefore, P1 students were taking their first compounding course during their first semester. The course provided an equal amount of instruction in both nonsterile and sterile compounding. The course met weekly for an hour-long pre-laboratory class and then students had a 3-hour laboratory session for the compounding exercises.

In contrast, the P3 course was the terminal course (ie, fifth semester) in a five semester Pharmaceutical Care Laboratory (PCL) sequence that spanned 2.5 curricular years. Compounding was an integrated part of each semester in the PCL sequence; typically, students would have compounding exercises 4 to 5 times per semester. Over the course of the five semester sequence, students would compound both nonsterile and sterile preparations, but not always in the same semester. The compounding exercises included in the P1 course were a subset of all of the exercises completed by the P3 students.

Both P1 and P3 classes were included in the study for different reasons. The P3 students had previous compounding experience in this pharmacy school, and were thought to be the most likely group to again demonstrate the observed 70% maximal confidence level from the Park and Shrewsbury study. It was assumed that any increase in the confidence level in this group could be attributed to the addition of the self-analysis component to the “resources” they used in the laboratory. It was further assumed that other factors of their compounding exercise (eg, interpreting prescriptions, assigning beyond-use dates, etc.) would play a minimal role in any increase in the confidence level of this group.

For the P1 class, this was their first compounding experience in this pharmacy school. It was thought that their confidence level would be influenced by not only the self-analysis component, but also all of the other resources and factors that the P3 group had already learned. They might also show that including the self-analysis component with the other resources and factors would achieve the same level of confidence as the P3 students attained by just adding the analytical component.

The survey instrument contained 19 items on demographics (3 items), pharmacy work experience (3 items), compounding experience (2 items), and students’ perception on their ability to complete steps necessary to make an acceptable compounded preparation (11 items) (Table 1). Demographic pharmacy and compounding work experience data were collected using multiple-response, dichotomous response, and free response questions. The data for self-efficacy questions 1 through 10 were collected using an ordinal 0-10 rating scale in which 0=cannot do at all, 5=moderately certain can do, and 10=highly certain can do. Question 11 used a similar ordinal rating scale in which 0=do not agree at all, 5=moderately agree, and 10=highly agree.

Both the pre- and post-surveys asked students to provide their names so the averaged individual confidence level changes could be compared to the raw mean differences for Q10. All of the other survey items were compared using the raw mean differences due to the nature of the questions to determine if that method would provide accurately reported data for Q10. The digit scores were averaged, and a Z-test was used to determine statistical differences in the pre- and post-survey results both within and between the P1 and P3 classes. This significance test was used because the variances of each group was known, and the sample size was large (>30 samples).

Throughout the semester, students self-analyzed their own nonsterile preparations immediately after compounding using UV-VIS spectrophotometric methods and submitted their results to their teaching assistants. Results from three P1 compounds (coal tar ointment, phenol-menthol troches, and diphenhydramine capsules) and two P3 compounds (diphenhydramine capsules and stomatitis mouthwash) were collected throughout the semester. Students were provided with step-by-step instructions of how to analyze their preparations. Students could typically work-up their samples and obtain the spectrophotometric results in less than 15 minutes. These instructions were also used to prepare all standards used to create the linear standard curve for each spectrophotometer. Since spectrophotometry is a summative analytical procedure, only certain preparations could be analyzed by this method. It is important to note students compounded other preparations during the semester that were analyzed only by the laboratory staff using HPLC. The pharmlabs.unc.edu website gives the analytical methods used for both the spectrophotometric and HPLC assays used in this study under the Resources>Assays tab.³

The P1 students had completed a laboratory exercise in the first class of their course to learn the fundamentals of spectrophotometry. The P3 students had previously used spectrophotometry to analyze some of their preparations in other compounding exercises. As such, the technique of using the instrument and determining the resultant concentration of a sample from a linear standard curve was known to all students. The samples were not re-analyzed by staff, but the students’ compounding records contained all their analytical data, and were examined when the records were graded.

Teaching assistants were always available to the students during the laboratory period to answer questions about the use of the spectrophotometers or their analysis data. If a student’s preparation was unacceptable (<90% or >110% of label), the teaching assistants would review the student’s analysis procedure and calculations. When a procedural error was found in preparing the preparation sample, the sample was prepared again with extra time being available to repeat the analysis during most laboratory periods. Calculation errors were also corrected before the Compounding Record was submitted for grading. If the preparation was unacceptable, the teaching assistant offered suggestions. However, time was not available for the student to re-make the preparation in that laboratory period.

A separate data analysis was performed to determine if students who compounded preparations with an acceptable potency (ie, 90% to 110% of label) expressed higher confidence levels than students who compounded preparations with an unacceptable potency (ie, <90% or >110% of label). The data was grouped by preparation, and included only students who could be matched to their pre- and post-survey confidence scores (ie, Q10) and their analytical potency results. For each preparation, the averaged pre- and post-survey confidence scores were determined in each analysis group (ie, the acceptable group or the unacceptable compounded preparation group). The Z-test was used to test for significant differences.

RESULTS

There were 103 P1 and 98 P3 students who completed the pre-survey (85% and 75% response rates) (Table 2). The majority of respondents were female (75% and 66%, respectively). More P3 students reported having pharmacy experience compared to P1 students (95% vs 74%). The average duration of previous pharmacy experience for P3 students was 2.6 (SD 1.5) years, and they reported an average of 11% of their time was related to compounding responsibilities. The average work experience for the P1 students was 1.8 (SD 1.1) years, with approximately 5% of their time devoted to compounding activities.

The most common type of compounded preparations prepared by students at work were suspensions, creams, and capsules. Suspensions (70%) and dermatological preparations (91%) were also found to be the most common compounded preparations among independent community practitioners.¹ Very few students reported performing any analysis of compounded preparations, with “visual inspection by a pharmacist” as the most common analysis strategy.

In the pre-survey, P3 students expressed significantly higher confidence (p<.0001) in their compounding abilities compared to P1 students (Table 2, Q 1-10). Both P1 and P3 students were least confident in their ability to evaluate the appropriateness of prescriptions for nonsterile compounded preparations [P1 students 1.3 (SD 1.9); P3 students 6.6 (SD 1.8)], and were most confident in their ability to accurately weigh and measure required materials [5.3 (SD 2.8) and 9.0 (SD 1.2) for P1 and P3 students, respectively].

There were 114 P1 and 59 P3 students who completed the post-survey (94% and 45% response rates). For the P1 students, there were significant increases in Q 1-10 (p<.0001) between the pre- and post-survey, suggesting a strong improvement in their overall confidence in all areas of their compounding laboratory experience (Table 2).

P3 students reported modest, but significant increases in confidence levels in Q 1-3, 7, and 9-10 in the post-survey (Table 2, p values ranged from <.02 to <.0001). These questions dealt with understanding and following directions from a Formulation Record (Q2 and Q7), having the ability to analyze the API amount or concentration with a spectrophotometer (Q9-10), and evaluating a prescription and preparing an accurate prescription label for a compounded preparation (Q1 and Q3). Since these elements were a part of each compounding exercise, it would be expected that improvement in these skills would be seen. These students did not report increased confidence in assigning beyond-use dates, accurately weighing materials, performing pharmaceutical calculations, or completing a Compounding Record (Q4-6, Q8) which were also part of each compounding exercise.

Question 11 of the survey addressed the research objective of determining if students’ self-analyses of compounded preparations increased their confidence in their ability to compound an acceptable preparation. In the pre-survey, both P1 and P3 students had a strong perception that spending time analyzing their compounded nonsterile preparations would help improve their confidence (Q 11, 9.4 (SD 1.1) and 8.1 (SD 2.2) for P1 and P3 students, respectively). However, in the post-survey, P1 students reported a slight, but significant decrease in the belief that time spent analyzing preparations would help improve their confidence in accurately compounding preparations (Q 11, 9.4 (SD 1.1) to 9.0 (SD 1.2), p<.007). In contrast to the P1 students, the P3 students reported a significant increase in the belief that the time spent analyzing preparations would be beneficial (Table 2, Q 11, 8.1 (SD 2.2) to 8.7 (SD 1.8), p<.02).

In a separate data analysis, the confidence of students who compounded acceptable (90% to 110%) preparations was compared to students who compounded unacceptable (<90% or >110%) preparations. For this analysis, only students who had submitted pre- and post-surveys and complete analysis results were included. Table 3 shows the confidence scores for Q10 from both the pre-survey and post-survey data associated with the analytical accuracy grouping for each compounded nonsterile preparation. Students felt equally as confident (ie, no statistical difference was found) in their ability to compound an acceptable preparation regardless of their analytical accuracy grouping. This was true in both the pre- and post-survey results. As was shown in Q10 of Table 2, the P3 students reported higher confidence levels in the pre-survey data compared to the P1 students [7.6 (SD 1.6) and 2.0 (SD 2.4)] , which was also evident in Table 3. In the post-survey data of Table 2, the confidence levels were essentially the same in both the P1 and P3 students [8.5 (SD 1.3) and 8.5 (SD 1.5)]. An 85% confidence value was found to also be true in Table 3 regardless of the analytical accuracy grouping or the preparation compounded.

Evaluating the numerical change between the pre- and post-survey data for each preparation and analytical accuracy grouping showed that student confidence increased regardless of their analytical accuracy grouping. (Table 3). The magnitude of numerical change for the P1 students was in the order of 6 points, while the P3 students indicated a change of about 1 point. The only significant difference (p<.0044) in the numerical change that occurred between the analytical accuracy groupings was the difference in the phenol-menthol troche preparation where the acceptable range group had an increase of 5.9 (SD 2.9) points and the unacceptable range group’s increase was 6.5 (SD 2.7).

DISCUSSION

In the pre-survey, P3 students expressed overall higher confidence in their compounding abilities relative to P1 students. This was expected since the P3 students were already accustomed to the compounding laboratory environment and had completed approximately 20 preparations in the previous four semesters of PCL instruction. The starting confidence level for this group of students would be a composite of what was learned in their academic and work experiences, and what was forgotten because of the time span between the compounding exercises.

A direct comparison of this study’s P3 data to the Park and Shrewsbury results would be unfounded since different groups of P3 students have been involved in the two studies.² However, it was interesting to note that the P3 students rated their confidence level at 76% in this study, whereas the rating was 70% in the other report. The similarity between the two groups of P3 students is that both groups had access to all of the same “resources,” and had undergone the same compounding instructional structure. Because of these similarities, it is easy to imply that a “baseline” confidence level might be approximately 70% to75%.

P3 students were found to be least confident in their ability to evaluate the appropriateness of a compounded prescription for the patient despite their prior experience in the laboratory. This is most likely a programmatic issue in students assuming the prescription is correct when they begin the laboratory exercise. Students may lack the experience to recognize commonly prescribed strengths or dosage forms for compounded preparations, or whether the preparation should be or can be compounded at all. The legality and completeness of a prescription are emphasized more to the students, so their focus would be on those aspects of a prescription.

The post-survey results showed P1 students feeling more confident in all aspects questioned, except for the ability to evaluate the appropriateness of a compounded prescription. P3 students indicated increases in their confidence in some aspects. However, there was no increase in their confidence to weigh accurately, perform correct calculations, and assign appropriate beyond-use dates (BUDs), suggesting that they had achieved their maximum confidence in those aspects.

Comparing the P1 and P3 student results showed that Q4 and 9 were significantly different between the two classes. In both cases, the P1 students reported the higher value, indicating this class was more confident about their ability to assign BUDs and analyze their preparations using a spectrophotometer than the P3 students. Published literature has clearly demonstrated a correlation between students’ performances on assessments and increased self-confidence when students are actively involved in their learning such as a laboratory experience.^11,12 The fact that P1 students felt more confident in assigning BUDs (Q4) and using a spectrophotometer (Q9) is probably due to the fact they were doing those activities on a weekly basis compared to the P3 students who had a limited number of compounds during the semester.

The P1 students also varied from the P3 students by reporting a significant decrease in their perception that potency analysis would impact their compounding ability. What influenced this change in confidence perception is speculative, but may be more the result of the physical environment of the lab exercise time. P1 students conducted seven analyses during the semester course, while P3 students performed two analyses in their course. It would appear performing analyses multiple times would reinforce the value of that component to the overall compounding experience. However, in the student course evaluations, P1 students indicated many times there was too much to be done during the laboratory periods which were held in the morning, and they were late to their next course. Students may have rushed to perform the analysis and reported the results rather than reflecting on their results and how their work may have caused errors.^8,11,12 In comparison, the P3 laboratory time was in the afternoon, and many times students would stay later to finish their work at their own pace. The P3 students also completed analyses in the other PCL courses prior to this year, and again had time to reflect on its significance.^5,6

At the completion of the compounding laboratory courses, the P1 and P3 confidence levels across almost all areas of compounding skills were very similar. Based on these results, it appears P1 students in the school’s new curriculum one-semester course were equally as confident in their compounding skills as the P3 students in the legacy 5-semester PCL curriculum. It has been stated there is no correlation between student knowledge and confidence when measured by multiple-choice assessments, but confidence and knowledge have been clearly correlated when assessments are measured by actual performance evaluations.^13-15 Since the laboratory exercises are a performance evaluation activity (with analytical performance carried out by the students), it would appear the two course formats may achieve the same outcomes. However, the authors are hesitant to unequivocally state the one-semester compounding format used in this course is the ideal length or structure of compounding courses in all pharmacy curriculum.

For the secondary analysis, the authors expected to see a larger increase in confidence in students who were able to compound preparations within an acceptable range, believing that achieving a good analytical result would positively reinforce self-confidence in compounding abilities. However, confidence levels expressed in Q10 increased by a similar amount in both analytical accuracy groups (Table 3). Students reported their perception that time and effort in analyzing their preparations would help them gain confidence in compounding an accurate preparation (Q11, Table 2). Although there is a small statistical decrease in the P1 post-survey results, both the P1 and P3 classes reported an approximate 9.0 value for the perception. Therefore, the perception was that the analytical component should be an important contributor to their overall confidence. Yet the data suggests that this perception was not mitigated by the reality of the analytical results. This implies that analysis accuracy was not a major contributing factor in improving overall student confidence.

It is possible that a greater difference in confidence levels between groups might have been seen if students were surveyed immediately after receiving analysis results, instead of waiting until the end of the semester. However, waiting until the end of the semester would have minimized the possible student bias in their pre- and post-survey scores, which were not anonymous. It is also possible that greater differences could have been seen if the response rate between the P1 and P3 groups had been more equal, since the P3 students had an especially low post-survey response rate. Another unknown variable was the utility of the additional resources students could access during each laboratory exercise. Unanswered questions, such as were they used to the same extent between groups, or were they used to their maximum potential would be reasonable to consider.

CONCLUSION

This study showed the confidence of P1 and P3 students in preparing accurate nonsterile preparations increased to approximately 85% by the end of the semester. This increase was seen regardless of whether or not the student compounded an accurate preparation. The difference in this study and a previous study² was the self-analysis of nonsterile preparation potency. Though a direct comparison between the two studies is not warranted since two different groups of students were used, it would suggest that including a self-analysis component as a necessary laboratory resource might impact student confidence by as much as 15%. The overall programmatic conclusion is that a variety of compounding-related resources, including a self-analysis component, seem necessary for students to achieve this level of compounding confidence.