Lessons Learned Using a Demonstration in a Large Classroom of Pharmacy Students

INTRODUCTION

Acid-base chemistry serves as the basis for understanding and predicting many important phenomena related to ionizable drugs, including their solubility, bioavailability, intravenous (IV) compatibility, and physical properties. Acid-base principles also underlie the pathophysiological conditions of acidosis and alkalosis and the body’s attempt to compensate for these conditions. Although pharmacy students are often repeatedly exposed to acid-base chemistry in prerequisite coursework, many find it a challenge to recall and apply these concepts in pharmacy school, necessitating particular care in the instructional approach to this topic.¹ Based on classroom observations, students frequently have difficulty understanding that ionizable drugs can exist in either a protonated or unprotonated state and that these two states have different properties, different names or symbols associated with them, and different effects on the pH of a solution. Furthermore, the bicarbonate buffer system of blood is not a straightforward equilibrium. Students need to be able to recognize that an unprotonated molecule (carbon dioxide) is able to liberate protons and lower pH when dissolved in solution and that these acidifying carbon dioxide molecules do not enter the body through respiration, but are generated within the body as a result of metabolic processes.

Recognizing the complexity of these concepts, and noting a greater proportion of students struggling with them, the supplementation of lecture with visual demonstrations of these concepts was hypothesized to enhance student comprehension as reflected by improved performance on relevant quiz questions. While some studies have shown that demonstrations can engage student interest and illustrate concepts that cannot otherwise be explained easily,^2-4 there is also evidence that demonstrations can be ineffective.⁵ This study made year-to-year comparisons of student performance on quiz questions for classes that received demonstrations compared to those that did not. In the process, it was discovered that the delivery method played an important role in the effectiveness of the demonstrations on student learning.

METHODS

This study quantitatively compared the scores of six cohorts of P1 students, from fall 2011 to fall 2016, on a quiz related to acid-base chemistry. Any student who had been enrolled in the pharmacy program on the South Jordan, Utah campus of the Roseman University of Health Sciences was eligible for inclusion in the study. The acid-base material was covered during a single, six-hour instructional day as part of a course entitled “Fundamentals of Drug Action and Metabolism” that was scheduled near the beginning of P1 year each fall. Quizzes were administered in class using electronic polling software the day following lecture and the quiz questions remained the same from year to year. Each question related to one or more learning objectives from the acid-base teaching materials, with all learning objectives represented by at least one question. The acid-base demonstrations illustrated concepts related to several, but not all, of the acid-base learning objectives. The quiz consisted of 17 items, five of which assessed learning objectives related to the acid-base demonstrations and 10 that related to other, demonstration-unrelated objectives. Two quiz questions that related to a mix of both demonstration-related and -unrelated objectives were omitted from the analysis. Quiz items were created years prior to the use of the demonstrations to assess student achievement of the learning objectives and did not assess student understanding of the demonstrations per se. Thus, the demonstrations were designed to enhance achievement of particular learning objectives and thereby improve quiz performance on related questions. Participation in the quiz was optional and students who completed the quiz during the class quiz time and who answered at least one question were included in the analysis.

In 2011 and 2012, students did not receive pH demonstrations. In 2013, the students received live, in-class demonstrations. In 2014, the demonstrations were video recorded and posted online for students to view at least one day before the quiz and one week in advance of the summative assessment and student survey. In 2015 and 2016, the video demonstrations were shown in class. Quiz scores for the demonstration-related and -unrelated questions were calculated and compared between years. A one-way ANOVA with the Tukey post-hoc test was used to make these comparisons. Qualitative observations were collected and categorized based on responses to an open-ended survey question distributed to students who received the acid/base demonstrations. This study was evaluated and exempted from review by the Institutional Review Board of the Roseman University of Health Sciences.

Three demonstrations were developed using diluted cabbage extract as a pH indicator. First, the opposing effects of a weak acid and its conjugate base were demonstrated by comparing the effects of acetic acid and sodium acetate on the color of the cabbage extract. Next, the acidifying effects of carbon dioxide in solution were demonstrated by exhaling air into the cabbage extract, causing a red shift in the color of the solution. Finally, the resistance of a buffered solution to changes in pH was demonstrated with a cabbage extract that had been acidified with exhaled carbon dioxide and then neutralized with a sodium bicarbonate solution. The acidification of the solution with carbon dioxide, followed by its neutralization with sodium bicarbonate set up the same buffer system in the solution as is found in blood. When exhaled air was subsequently passed through the buffered solution, no acidic color shift resulted because the increase in carbonic acid was consumed by aqueous bicarbonate ions. Video recorded versions of the demonstrations were in the range of 1 to 4 minutes in length. Full versions of these videos can be accessed with this link: http://tmrose.wixsite.com/imhappyproject/ph-demonstration-videos.

RESULTS

Demographic data for each year of P1 students is provided in Table 1 along with quiz and survey participation rates. Year-to-year quiz score comparisons are shown in Table 2. There was a mild decrease in demonstration-related, but not demonstration-unrelated, quiz scores in 2012. Quiz scores on demonstration-related questions continued to decrease in 2013, reaching a low in 2014. In 2015, quiz scores increased generally, and in 2016 the scores on questions related to the pH demonstrations were significantly better than those seen in both 2013 and 2014. Beyond bouncing back from a global minimum in 2014, there were no significant trends in demonstration-unrelated quiz scores.

Variations in quiz scores are not likely to be due to shifts in the gender composition of the class, which remained constant in the range of 50% to 60% male over the time period studied. Qualitative feedback is summarized in Table 3. The most common comment from students in 2013 was that they had trouble seeing the demonstrations. There were also a handful of comments indicating students were confused or needed additional explanation. The number of comments collected from students in 2014 on the pH demonstrations dropped precipitously from the previous year. Oral feedback gathered from students informally suggested that many students in 2014 were not aware that the demonstration videos had been posted or did not use them even if they were aware. In 2015, there were many more comments from students on likeability and how to improve the videos, but no comments about being confused.

DISCUSSION

The supplementation of lecture with visual demonstrations of acid-base concepts was hypothesized to enhance student comprehension, as reflected by subsequent performance on quiz questions. Improved quiz scores on questions related to the demonstrations supported the hypothesis; however, modifying the delivery method of the demonstrations based on student feedback appears to have been the key to their effectiveness. This study therefore suggests that visual demonstrations can measurably improve student assessment performance as long as the mode of delivery complements classroom conditions, and that these mode-of-delivery refinements can be guided by student feedback.

As supported by the quantitative data in Table 2, declines in student understanding of certain learning objectives were observed anecdotally as early as 2012. Scoring declines were not uniformly observed for all quiz questions (Table 2), only for those questions related to a particular subset of learning objectives reflecting particularly difficult concepts in acid-base chemistry, and these were the concepts addressed by the demonstrations. Scores on quiz items related to these learning objectives dropped year over year and were only reversed upon incorporation of the demonstration videos into the lecture time (Table 2). The decrease in student performance on items related to demonstration-related learning objectives may be attributable to an overall decrease in academic preparedness by incoming students, as suggested by the declining entrance grade point average (GPA) means in Table 1.

The use of video for delivering demonstrations potentially provides advantages over live demonstrations. For example, after an initial time investment for their creation, videos require less effort and preparation from year to year; video is easier for the class to see; video can be posted online and reviewed by students after class; and, video frees the instructor from having to concentrate on the mechanics of the demonstration, which allows a more lucid description of events and may reduce student confusion. In 2014, the pH demonstration videos were posted on the same server students used to access class materials and use of the videos was recommended to the class during lecture. Despite these steps, it was observed that many of the students claimed not to be aware of the videos and did not respond to a survey about them, suggesting the online videos were not widely used. Thus, the act of simply making videos available may not result in their use by the students. It is likely that the use of incentives to hold students accountable for their use of the online videos, such as the awarding of class points, would have improved the use and efficacy of the online videos. In the absence of such incentives, the data in this study suggest that using short video demonstrations to supplement in-class activities may be an appropriate alternative. However, because of a low survey response rate for the year in which online videos were used and the lack of incentives to promote student use of the online videos, no definitive conclusions can be drawn from this study about the relative effectiveness of in-class delivery vs incentivized online delivery.

While demonstrations of acid-base concepts similar to those reported here have been published, no data have been presented on the effectiveness of these demonstrations on student learning.^6,7 Beyond the effects stemming from the choice of delivery method, evidence has been advanced that the effectiveness of in-class demonstrations can be greatly enhanced by pausing at critical points in the demonstration to have students predict results, discuss outcomes, and identify misconceptions.^8-10 The ability to pause video demonstrations at any point makes them amenable to the addition of questions that encourage critical thinking and metacognition by the students; thus, this approach may further improve the effectiveness of the demonstrations and would be a worthwhile subject of future work.

This study does have limitations. For example, there was no matched, concurrent control group. Year-to-year comparisons leave open the possibility that differences in class competency or in teaching quality may have caused the variations in quiz scores. However, the class was taught by the same instructor each year and the class learning objectives, quiz questions, and assignments remained identical over the entire time period studied. The Microsoft PowerPoint (Redmond, WA) slides varied only slightly and in ways irrelevant to the concepts demonstrated in the videos. Further, the incoming student GPA data suggests that quiz scores should have remained low in 2015 and 2016 if class academic competency was the main determining factor in quiz performance (Table 1). GPA is commonly used by admissions offices as a marker of applicant academic ability and may provide a rationale for why student performance on the more challenging quiz questions trended downward as opposed to staying stable from year to year. However, there are limits to how much can be concluded from GPA, since it is a very broad marker compared to the specific knowledge required for an acid-base quiz. The entering GPA decreases should therefore be considered only one of many possible factors to explain declining quiz performance.

Another possible limitation is that, because the same quizzes were used each year, students may have had access to the correct answers from previous years’ students, which could contribute to higher quiz scores. However, quizzes were anonymous and only treated as learning exercises, students received no points for participating in them, and had no incentive for artificially boosting their scores. Another limitation is that, for reasons unknown, 2014 students showed a significant decrease in both demonstration-related and -unrelated questions, suggesting this class was academically different from the others in some way. Nonetheless, the demonstration-related questions still showed a decreasing trend that reversed upon use of the demonstration videos in class. Another limitation is that the survey response rate in 2014 was only 14% and, while the qualitative responses from this survey were sufficient to promote changes that resulted in significant improvements, the lack of student use of the online videos in 2014 is somewhat presumed. Finally, while the study does analyze the performance of six different pharmacy classes over six years, it was confined to a single classroom on a single campus. Thus, the results may not be completely generalizable to other pharmacy colleges.

CONCLUSION

This study supports the idea that classroom demonstrations can improve student learning. However, choosing an appropriate delivery method, as guided by class feedback, was required to see learning improvements.