A Novel Teaching Tool Combined With Active-Learning to Teach Antimicrobial Spectrum Activity

INTRODUCTION

Health professions students are expected to command a tremendous volume of facts and concepts, but long-term retention of this information is a substantial challenge.¹ Studies suggest retrieval practice, connection to prior knowledge, self-explanation, and organization into schema are associated with improved long-term retention.² These strategies require active engagement from learners as opposed to passive receipt of information. Incorporating these strategies into scheduled class sessions creates an active-learning environment that is associated with improved student performance in science courses generally and pharmacy courses specifically.^3,4

Learning antimicrobial spectrum of activity is particularly challenging because it is difficult to predict activity based on knowledge of drug mechanism and organism physiology (because one would have to know the physiology of hundreds of target organisms). For this reason, rote memorization is frequently employed. Such methods are associated with poor long-term recall and poor performance on transfer-type questions.⁵ A robust knowledge of antimicrobial spectrum of activity is important for pharmacy practice, as the role of pharmacists in antimicrobial stewardship has expanded dramatically in the last decade.^6,7

The educational innovation in this study was the introduction of an active-learning approach and incorporation of a novel cognitive schematic into a portion of an antimicrobial pharmacology course (about 50% of the antimicrobial-related content), with the goal of increasing the likelihood of students’ long-term retention of antimicrobial pharmacology knowledge, especially with regards to antimicrobial spectrum of activity.

DESIGN

At the University of California San Francisco School of Pharmacy, Antimicrobial and Oncologic Pharmacology is a two-unit course taught over 10 weeks in the fall quarter of the third professional year and followed by Therapeutics of Infectious Diseases and Oncology, a six-unit course taught over 10 weeks in winter quarter of the same year. This stepped approach requires students to retain foundational knowledge learned in the pharmacology course for months in order to apply the knowledge in the therapeutics course. The active-learning instructional method was used to teach two of the topics in the pharmacology course (Beta-lactam Antibacterials, Antifungals) in-class sessions across all three years. In 2015, a third session (Glycopeptides and Lipopeptides) was also delivered using the active-learning method. Other topics in the course were taught primarily using traditional-lecture approaches. The sessions relevant to this study are listed in Table 1 by year of delivery and in-class contact time. Of note, the amount of in-class time was not increased when sessions were transitioned from a traditional lecture method to an active-learning approach, both for the period under study and for the course historically.

Prior to the active-learning in-class sessions, students were provided with a course reader that discussed key characteristics of each drug or closely related class of drugs in a standard format. In 2015, self-assessment questions (with answers at the end of the relevant course reader section) were added to promote immediate concept review. Students were advised that they would be expected to review this in advance of the corresponding class sessions, but there were no graded preparatory assessments and attendance was voluntary. During the in-class session, brief summaries of key concepts were followed by student exercises and then a review of the correct answers. Students were encouraged to collaborate with nearby classmates on the exercises, but forming groups was not specified or required. In particular, students practiced learning spectrum of activity using specially designed worksheets. These worksheets (Figure 1) arranged the key organisms to be learned in specific patterns that incorporated both the general organism characteristics (Gram-positive vs Gram-negative) as well as their relative resistance to therapy in a radial approach. During the activity, students circle the organisms that are “covered” by the drug in question (Figures 2a, 2b). This can lead to a flower-like appearance; hence the student-coined term “flower diagrams.” In addition to providing an opportunity for practice, the flower diagrams allow for comparison across drugs to see how spectrum of activity builds, for example, across generations of cephalosporins, or the effect of adding a beta-lactamase inhibitor to an aminopenicillin. Correct answers with explanations for all in-class exercises were posted after the end of the class sessions.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Spectrum of Activity Worksheet Schema for Bacterial Infections.

• Download figure
• Open in new tab
• Download powerpoint

Figures 2a, 2b.

Examples of Completed Spectrum of Activity Worksheets (“Flower Diagrams”).

EVALUATION AND ASSESSMENT

The following quarter in the therapeutics course, approximately 10 weeks after the final active-learning session, students completed a low-stakes multiple-choice examination on key knowledge from antimicrobial pharmacology (the assessment took place in the next calendar year; dates listed are from when the content was delivered in the pharmacology course). This was administered in class via an individual online survey link using the Qualtrics platform (Qualtrics, Inc, Provo, UT). The examination consisted of 15 multiple-choice questions; 12 of which related to antimicrobial spectrum of activity. Each question had five possible responses from which all acceptable responses were to be selected (ie, no answers were mutually exclusive). This approach resulted in essentially 60 true-false items on the examination. The total score on this baseline assessment was strongly associated with the score on the subsequent therapeutics examination that covered bacterial infections, even after controlling for student overall grade point average (GPA) (adjusted coefficient .42, p=.003).

Each examination item was classified as being related to information taught in active-learning or traditional lecture sessions. Questions regarding glycopeptides and lipopeptides were classified as traditional-learning related in 2013 and 2014 and active-learning related in 2015. For 2013 and 2014, 44 of 60 items were related to active-learning sessions; in 2015, 50 of 60 items were related to active-learning sessions. The mean percentage of items answered correctly for each question type was compared using paired t tests to account for intra-subject correlation for normally distributed data or Wilcoxon tests for nonnormally distributed data. The standardized effect size and associated 95% confidence interval was calculated using Cohen’s d. Student evaluations of the instructor providing the active-learning sessions were reviewed and compared to those of all instructors in the course.

Across all years and students (n=360), the mean percentage of correct items was 74.2%. For items related to content taught using the active-learning approach, the mean percentage of correct answers was 75.3% vs 70.4% for the items testing content taught using traditional lecture methods (difference 4.9%, p<.0001) (Table 2). Differences between the content areas were significant for all three study years and ranged from 2.9% to 5.2%. The effect size across all years was 0.34 (95% confidence interval 0.19-0.49); effect sizes ranged from .24 to.45 across the three study years. For the content related to the glycopeptide lecture that changed from traditional to active-learning in 2016, the number of correct items increased from a mean of 4.4 out of 6 to 5 out of 6 after incorporation of the active-learning approach (p<.001).

Instructor ratings were high for the active-learning approach (mean scores for active learning instruction 4.5-4.8 out of 5, mean scores for traditional lecture instruction 4.1-4.4) and student comments about the active-learning approach and flower diagrams were overwhelmingly positive (representative comments noted in Table 3).

DISCUSSION

Incorporating an active-learning approach into an antimicrobial pharmacology course led to a significant improvement in long-term performance on an assessment related to antimicrobial spectrum of activity compared to content delivered in a traditional lecture format. Performance on this assessment was subsequently predictive of performance on related therapeutics assessments independent of student overall GPA, emphasizing that mastery of this foundational knowledge is important preparation for clinical coursework.

Although the mean absolute percentage difference in assessment scores between groups was a modest 4.9%, the mean effect size of .34 is comparable to the effect of active learning interventions in the meta-analysis by Freeman and colleagues, as well as other well-established interventions in educational research, including reduced class size and test-taking practice.^3,8,9 The costs of this approach were primarily the up-front investment of instructor time; the active-learning sessions required no more classroom time than that allotted for traditional lecture sessions. Compared to traditional lecture approaches, the active-learning approach requires more effort from students in terms of both prior preparation and active engagement in the classroom. Some studies have found student dissatisfaction with active-learning approaches, especially with the length of preparatory activities, lack of distinction between key and supplemental information, and insufficient instructor review of material prior to engagement in learning activities.^4,10,11 In this study, student evaluations of the active-learning sessions were highly favorable. The preparatory activity consisted of a course reader, with essential points highlighted and embedded concept review questions. These features may have allowed students to prepare more efficiently for the session. During the in-class sessions, activities were preceded by brief mini-lectures (approximately 5 minutes). Because students were not required to review the material in advance, this approach may have helped less-prepared students orient to the material that formed the basis of the active-learning activities.

Besides promoting engagement with the material, students may have benefitted from the use of the flower diagrams as a cognitive schema in which to embed their knowledge. The flower diagrams incorporate two key dimensions that divide the spectrum of activity of antibacterials: cellular envelope characteristics (Gram-positive and Gram-negative) and prevalence of resistance mechanisms to the drug. Organizing knowledge into schema, as opposed to rote memorization, is characteristic of experts versus novices.^12,13 The flower diagrams may also act as a cognitive scaffold and reduce the intrinsic cognitive load, by providing a fixed relationship of organisms to each other, allowing students to concentrate on the relationship between the drug and the organisms, and by sequencing of the drugs discussed, allowing students to focus on the differentiating characteristics between drugs rather than learning each in isolation.

Pharmacists play key roles in promoting the appropriate use of antimicrobial agents.^14,15 Justo and colleagues surveyed graduating pharmacy students at 12 US schools of pharmacy and found that while 84% overall considered their education in antimicrobial use beneficial, there was wide variability among students at the schools in perceived value to their education of performing antimicrobial stewardship-related activities.¹⁶ Further, the mean score on an 11-item knowledge assessment related to antimicrobial use was only 5.8 (52.7% correct). This was numerically similar to that of a cohort of medical students completing the same instrument (5.6 items correct), despite that the medical students viewed their education regarding antimicrobial use as much less valuable (58% viewed it favorably).¹⁷ Thus, there may be a disconnect between pharmacy students’ views of their education regarding antimicrobials and their ability to recall and apply that information. Greater incorporation of active-learning activities such as the one described in this study into the antimicrobial curriculum in pharmacy schools may be one avenue for improving the ability of pharmacy students to retain and apply this knowledge in practice.

This study has several limitations. The number of items on the assessment was skewed toward active learning-related content out of proportion to the number of contact hours for active-learning versus traditional lecture content. This is primarily related to the large number of agents and significant clinical importance of beta-lactam antimicrobials, which represented the majority of items. The assessment only tested spectrum-related antimicrobial content, not content related to pharmacokinetics or adverse effects. Whether the active-learning approach would lead to similar gains in these domains is not known. The assessment tested knowledge approximately 10 weeks from the last course session; whether a similar effect would be noted at even longer-term follow up (eg, by graduation) is not known. Although students were required to complete the assessment, no stakes were attached to the outcome, which raises questions regarding performance and motivation. However, there is no reason to suspect this would lead to a difference in performance on the items regarding active-learning and traditional lecture.

The spectrum tool was developed over several years by the author and required approximately 40 hours of time in development and refinement. The tool is available to use for free and distribution under a Creative Commons license at the PharmAcademy website: http://pharmacademy.org/item/schematic-teaching-worksheet-learning-antimicrobial-spectrum-activity. An interactive Web-based version, also for open use, is under development. Instructors in antimicrobial pharmacology and therapeutics can use the tool to augment or replace their current methods for teaching antimicrobial spectrum of activity. The active-learning exercises required approximately four hours of development to integrate into the existing coursework, and were conducted during existing class time.