Impact of Abbreviated Lecture with Interactive Mini-cases vs Traditional Lecture on Student Performance in the Large Classroom

INTRODUCTION

Over the past decade, pharmacy educators have transitioned from predominantly lecture-based to learner-based instruction using various strategies to engage students in the learning process. Engaging students, regardless of delivery method (ie, lecture, seminar, video, or online), should help students learn and apply knowledge while “in class.” Strategies employed to engage students are most commonly termed active-learning methods or approaches. Gleason et al, in their primer on active learning, emphasized that active learning is an approach to teaching that is learner-centered.¹ For this approach to be successful, students need to participate in the learning process, but teachers must first offer meaningful activities that encourage engagement and enhance learning of course material.^1,2

Active learning used in biomedical sciences programs other than pharmacy includes process-oriented guided inquiry learning, problem-based learning, case-based learning, team-based learning, small group work, think-pair share, concept tests, debates, learning games, and think-aloud activities.^3-15 Dentistry, medicine, occupational and physical therapy, physician assistant, and nursing are among biomedical science programs that use active-learning approaches documented and evaluated in the literature. Problem-based learning (PBL) has been used in some medical schools for more than 4 decades.^3,4 Undergraduate science professors, who primarily teach in large classrooms, have also described active-learning methods including PBL, process-oriented guided learning inquiry (POGL), role playing, poster presentations, equation demonstrations, unannounced quizzes, and reflections of learning.^16-22 Much of the literature on biomedical science programs other than pharmacy and in undergraduate science courses describe student engagement or perception of learning, or compare the impact of active learning vs lecture on student engagement or perception of learning. Fewer reports include comparisons of student performance.^7-10,18-21 In an article on an undergraduate biology class, Walker et al compared student performance of a “traditional” section (n=240) to an “active” section (n=263) in the large classroom during one semester.¹⁹ The traditional section included lectures, quizzes, and examinations; the active section included mini-lectures, group work, homework, quizzes, and examinations. There was a small but significant difference in overall student performance in the active section compared to the traditional section. Analysis showed that lower performing students benefited the most from active learning.¹⁹ In a 2-year medical school study on a pharmacology course, Li et al compared performance of students randomized to a traditional lecture group (the control) to performance of those randomized to a case-oriented, self-learning and review group (the study group).¹⁰ Each year, students in the study group performed significantly better than those in the control group on the mid-term, but not on the final examination, although final examination scores were slightly higher in the study group.¹⁰

In pharmacy literature, there are numerous articles about active-learning strategies used in the didactic setting, but fewer articles on evaluations of student learning that document differences in knowledge acquisition, abilities, or mastery of course material.^23-35 Active-learning approaches with student learning evaluations in the pharmacy didactic setting include PBL, POGL, team-based learning, patient simulation, case-based learning, flipped classroom, student response systems, and quick-thinks.^23-35 Of these, fewer reports compare student learning using active-learning approaches to student learning using a traditional lecture-based approach in required courses in the large classroom setting.^23-30 Some studies compare student grades or examination performance in a required large classroom course using an active-learning approach during 1 year with grades and performance from a previous year or years when lecture based approaches were used.^23,25-28,30 Lui et al studied student learning in one course offering by comparing quiz grades of students randomized to traditional lecture to quiz grades of those randomized to the same lecture with the addition of a student response system (SRS).²⁴ The SRS group scored higher on the quiz given immediately after lecture, but not on a quiz on the material given one month later. ²⁴ Lucas et al used a comprehensive examination during an advanced pharmacy practice experience (APPE) class to compare retention of content from prior pharmacotherapy courses, 1 of which had a lecture format and 2, an active-learning format.²⁹ The students’ overall examination scores were higher on content from the 2 active learning-based courses than from the lecture-based course. The authors admitted, however, that the results could have been influenced by the 5 APPEs students had completed by the time they took the examination.²⁹

The current Accreditation Council for Pharmacy Education (ACPE) Standards state that active learning should be incorporated throughout the pharmacy curriculum.³⁶ These Standards utilized the American Association of Colleges of Pharmacy 2004 report of the Center for Advancement of Pharmaceutical Education (CAPE), which included 3 domains of pharmacy practice: pharmaceutical care, systems management, and public health.³⁷ The 2013 CAPE report contains 4 broad domains, with suggested learning objectives for each: foundational knowledge, essentials for practice and care, approach to practice and care, and personal and professional development.³⁸ Maintaining motivation, attention, and interest during learning and work-related activities are learning objectives for personal and professional development,³⁸ which are all qualities of an engaged learner or practitioner. The CAPE 2013 report notes that integrated assessments are needed to ensure students retain, integrate, and apply knowledge, skills, and attitudes.³⁸

As with many colleges and schools of pharmacy, our required pharmacotherapy modules are taught in a large classroom setting. Instructors use different types and amounts of active-learning strategies, while others predominantly use a traditional lecture format. The literature on actual impact of a teaching and learning method on student learning in the large classroom setting, as measured by mastery of course objectives, is incomplete. For this study, we sought to evaluate the effect, if any, of 2 different teaching and learning methods on student mastery of learning objectives in a required pharmacotherapy module taught in a large classroom setting. Method 1 consisted of an abbreviated lecture with multiple interactive mini-cases. Individual performance was assessed with graded case questions. Method 2 consisted of a traditional lecture with 1 non-graded, inclass case discussion. A project goal was to efficiently incorporate more individually graded active-learning activities in the large lecture hall to increase student engagement and decrease lecture time. We hypothesized that the students’ mastery of learning objectives, as demonstrated by performance on course assessments, would be higher using method 1 than using method 2, regardless of the therapeutic topic.

DESIGN

The pharmacotherapy module chosen for this project was called Disorders of the Musculoskeletal System and Pain Management, a 4-credit hour, third-year required module taught in a block schedule format. The 5 faculty members in this team-taught course were from the departments of pharmacy practice and pharmaceutical sciences. The course met 18 hours each week for 4 weeks and multiple-choice, 2-hour examinations were given weekly. The classroom setting was a room that sloped from back to front, had fixed, non-movable rows of seating, 2 video screens in the middle of the room, and 1 larger video screen at the front of the room. When all enrolled students were present, there were few empty seats and no space for students to group together. Presentations were captured via audio and video and were available to all students after class, regardless of class attendance.

Two different teaching and learning methods were implemented for 2 years for 2 different therapeutic topics (Table 1). The 2 topics chosen were the therapeutics of osteoarthritis and the therapeutics of gout. The same faculty member taught both topics both years. Three and a half 50-minute class periods were allocated to each topic each year. The learning objectives for both topics were the same for both years and covered Bloom’s Taxonomy cognitive levels of knowledge, comprehension, application, analysis, synthesis, and evaluation.³⁹ For method 1, the abbreviated lecture format, 70% of class time was spent on guided discussion of multiple interactive mini-cases, after which individual performance was assessed via graded case questions, and 30% of class time was spent on lecture. With method 2, the traditional lecture format, 70% of class time was devoted to lecture while 30% of class time was spent on a lecturer-guided class discussion of 1 comprehensive case and there was no individual assessment via graded case questions. For both methods, an iRAT was given at the beginning and end of the topic presentation. In the literature, the iRAT is a readiness assurance tool used in team-based learning, but it is also applicable to individual learning.^40,41 In year 1, method 1 was used to teach the therapeutics of osteoarthritis and method 2 was used to teach the therapeutics of gout. In year 2 the methods were flipped in an attempt to eliminate differences in student performance based on therapeutic topic (Table 1).

The iRATs and mini-case-based individual assessments covered the course learning objectives and represented 1% of the total course grade. For each topic presentation, the learning objectives, assigned readings, PowerPoint presentations, including the mini-cases or, in the lecture format, 1 comprehensive case, were posted to the Moodle online software platform (Moodlerooms, Baltimore, MD) 72 hours prior to class. Students were instructed to review all material prior to class. The iRATs and the mini-case-based questions were not posted to Moodle and were only accessible to students while in class.

Multiple interactive mini-cases were the focus of method 1. An example mini-case for a patient with gout is included as Appendix A. During the topic presentation, 4 mini-cases were presented and discussed with the students. The Socratic method, a question and answer discussion based on inquiry and response used to encourage critical thinking and debate, was used for the mini-case discussions. Student mastery of topical learning objectives taught via the cases was assessed with graded case-related questions. Twelve graded, multiple-choice questions were given throughout class time as the cases were presented and discussed. Although students were able to discuss mini-cases and questions, each student individually submitted answers to questions using the classroom response application in Moodle. Students completed the 12-item mini-case assessment for osteoarthritis in year 1 and for gout in year 2 (Table 1). Only those students logged into Moodle in class at the time of each question were able to submit their answers. After each question, the overall class response was projected for class viewing, and the instructor and students discussed the results, distracters, and muddy points for immediate feedback. Access to each question was timed, so students were not able to change their responses. Bloom’s Taxonomy levels of comprehension, application, analysis, synthesis, and evaluation were assessed with the mini-case questions.³⁹

Student performance on the iRATs and on subsequent examination questions on each topic were used to measure mastery of learning objectives for both methods. Student performance on the mini-case-based assessment was used to measure mastery of learning objectives for method 1. For both methods, students were asked to complete iRAT version A when they arrived in class, before the topic presentation. They were asked to complete iRAT version B in class after the topic presentation. Each iRAT was a 5-item, multiple choice quiz and was administered through Moodle. The iRATs were only available to students logged into Moodle in class, were timed, and were not accessible or viewable before or after the timed assessment. The instructor projected the class iRAT results for class viewing and discussed them in class after administration to provide immediate feedback. Knowledge, comprehension, and application levels of Bloom’s Taxonomy of learning were covered in iRAT A, while iRAT B covered comprehension, application, analysis, and evaluation levels.³⁹ The iRAT A, at the beginning of class, was different from the iRAT B, at the end of class. The same versions of iRAT A and B were used for each topic in both years. Additionally, for method 1, students completed the 12-item mini-case based assessment for osteoarthritis in year one and for gout in year two.

In both years, students took a 14-question, multiple-choice examination covering the learning objectives a week after the topic presentations. Student performance on examination questions on the therapeutics of osteoarthritis and gout were recorded each year. Questions varied slightly from year 1 to year 2 for both topics, but each year’s examination sets covered the same course learning objectives for the therapeutics of osteoarthritis and gout and assessed comprehension, application, analysis, synthesis, and evaluation on Bloom’s Taxonomy.³⁹ In both years, 8 of the 14 questions were case-based.

The impact of the interactive mini-cases with abbreviated lecture (teaching and learning method 1) was assessed in several ways. First, student performance on the prepresentation iRAT (version A), was compared to their performance on the postpresentation iRAT (version B). This allowed for paired comparisons of student performance before and after each topic was presented using the different teaching and learning methods. In addition, because the study was performed over 2 years, comparisons could be made between each individual topic taught with method 1 or method 2. For example, students in year 1 were taught therapeutics of osteoarthritis with method 1 and students in year 2 were taught this topic with method 2. Student performance on iRAT versions A and B for each topic were then compared between year 1 and 2. In addition, for both years, student performance on examination questions for topics taught with method 1 and method 2 were compared. This eliminated confusion that can occur when comparing the effect of a teaching method on student performance utilizing 2 different topics.

Students evaluated the 2 teaching and learning methods by completing an anonymous, voluntary, 20-item evaluation online at the conclusion of the course each year. Students ranked each item on a 4-point Likert scale of strongly agree (SA), agree (A), disagree (D), and strongly disagree (SD). The online evaluation provided information on students’ feelings regarding each teaching and learning method as well as on preparation time before class for each method. For consistency, all data were presented as mean ± standard deviation and N (percent). Inclass grades (iRATs A and B and mini-case questions) and student evaluation results were not normally distributed and did not meet criteria for parametric statistical tests (ie, Student t test); therefore these data were compared using the Mann-Whitney U between year 1 and 2 and the Wilcoxon signed rank test within each year. Remaining continuous normally distributed data were compared using paired t tests or Student t tests within each year and between years 1 and 2, respectively. This project was approved by the Institutional Review Board and student consent was obtained.

EVALUATION AND ASSESSMENT

Class enrollment was 136 and 141 for the 2 years of this study. Basic demographic data was similar both years. The average student age was 27 and 60% of the class was female both years. Sixty-eight percent had a prior bachelor’s degree year one; sixty-four percent year two. For both years, 100% of enrolled students provided voluntary informed consent for participation in the study. After giving consent, students could still choose not to attend class or complete the inclass assessments. Of the 136 students enrolled in the course in year 1, 128 (94.1%) completed all of the pre/postpresentation iRATs and all of the inclass questions for the mini-cases. One hundred and twenty of 141 (85.1%) completed all of these assessments in year 2. All enrolled students took course examinations on osteoarthritis and gout in year 1; one student did not take the examinations in year 2.

Student performance on the iRATs, mini-case questions, and examination questions are presented in Table 2. Student performance on postpresentation iRAT (version B) was compared to performance on prepresentation iRAT (version A). The version B scores were significantly higher than the version A scores (p<0.001) when teaching and learning method 1 was used to teach both the therapeutics of osteoarthritis in year 1 and the therapeutics of gout in year 2 (p<0.001). With method 2, regardless of the topic, student performance was lower on the postpresentation iRAT than on the prepresentation iRAT. There was even a negative significant difference between iRAT versions B and A using teaching and learning method 2 for osteoarthritis in year 2; postpresentation scores were significantly lower than prepresentation scores (p<0.001). As shown in Table 2, student performance on the postpresentation iRAT was compared between year 1 and 2 to determine if there was a difference between methods by therapeutic topic. The iRAT version B scores for the therapeutics of osteoarthritis were significantly higher using method 1 compared to method 2 (p<0.001). There was no significant difference between method 1 and method 2 postpresentation iRAT scores regarding the therapeutics of gout.

Student performance on questions related to the therapeutics of osteoarthritis and therapeutics of gout on subsequent examinations were also compared using Student t test (Table 2). There was no significant difference in examination performance for osteoarthritis or gout with method 1 compared to method 2. However, when this assessment was limited to only those students completing all inclass assessments, there were positive differences in performance for method 1 vs method 2, but they were not significant (p=0.072 for osteoarthritis; p=0.062 for gout).

In addition to the iRATs and examinations, inclass individual performance on mini-case questions was assessed with method 1 (year 1 for osteoarthritis and year 2 for gout). There was no significant difference in performance by topic, osteoarthritis (n=131) versus gout(n=120) (p=0.053) . The mean grade on the mini-case questions was 83% for osteoarthritis and 85.2% for gout.

Results from each question of the survey were compared within each year using Wilcoxon signed rank test. In year 1, 125 of 136 (91.9%) enrolled students answered all items on the anonymous evaluation of the teaching and learning methods; 116 of 141 (82.3%) answered all items in year 2. The first 18 items (1-9 related to method 2 and 10-18 related to method 1) asked students to evaluate and compare the methods in terms of format and delivery, preparation needed before class, participation in class, value of activities in class, and student-perceived confidence in discussing pharmacotherapy, in critically evaluating a patient case, and in recommending appropriate therapy with appropriate patient counseling. In year 1, there was a significant difference on 3 of the 18 items when answers for methods 1 and 2 were compared. No other item comparisons were significant. Students responded that they felt more confident in their ability to critically evaluate a patient case and to recommend appropriate therapy after method 1 used for osteoarthritis compared to method 2 used for gout (p=0.028). Students ranked their class participation as higher with method 1 vs method 2 (p<0.001). However, students reported that they enjoyed the format of method 2 more than they enjoyed method 1 (p=0.007). In year 2, there was a significant difference on 2 of these same 3 items. No other item comparisons were significant. Year 2 students ranked their class participation as higher with method 1 for gout compared to method 2 for osteoarthritis (p<0.001), and students reported enjoying the format of the traditional lecture more than they enjoyed the abbreviated lecture (p=0.007). There was a positive difference in student confidence in their ability to critically evaluate a patient case and recommend appropriate therapy after method 1 was used for gout compared to method 2 for osteoarthritis, but it was not significant (p=0.075).

The final 2 items of the evaluation asked the students to compare the methods in terms of preference and perception of learning, respectively. In year one, 72% of respondents agreed or strongly agreed that they preferred method 1 for osteoarthritis to method 2 for gout, and 70.4% agreed or strongly agreed that they learned more in class with method 1 than with method 2. In year 2, 74.6% agreed or strongly agreed that they preferred method 1 for gout to method 2 for osteoarthritis, and 76.3% agreed or strongly agreed that they learned more with method 1 than with method 2. This occurred despite students responding that they enjoyed the format of the traditional lecture to the format of mini-cases with abbreviated lecture (p=0.007).

DISCUSSION

The data suggest students learned more while in class using the mini-cases, active-learning approach compared to a traditional lecture with 1 lecturer-guided case discussion. This occurred despite a similar level of self-reported preparation prior to class regardless of the teaching and learning method. Per the student evaluations, the percentage of students who read the required materials prior to class did not differ significantly according to the teaching and learning method either year.

Postpresentation iRAT scores were significantly higher than prepresentation iRAT scores (p<0.001) when method 1, the active-learning approach, was used to teach the therapeutics of osteoarthritis in year 1 and the therapeutics of gout in year 2. Postpresentation scores were not higher when traditional lecture was used. Thus, student mastery of learning objectives increased while in class within each year for method 1 vs method 2 based on paired student comparisons. Also, postpresentation iRAT scores for osteoarthritis after method 1 in year 1 were significantly higher (p<0.001) than iRAT scores after method 2 in year 2 (p<0.001). Thus, student mastery of learning objectives increased while in class between years for method 1 vs method 2 for the same therapeutic topic. These results add to the positive findings of previous studies on student learning with active-learning approaches in required courses in large classroom settings.^23-30

In this study, students ranked their class participation as higher with the abbreviated lecture compared to the traditional lecture both years, regardless of therapeutic topic. However, they reported that they “enjoyed” the traditional lecture more. When asked directly to compare the 2 methods, more than 70% of students in both years indicated that they preferred the abbreviated lecture with multiple mini-cases, which seems contradictory to their enjoying the traditional lecture more. More than 70% in both years also indicated they believed they learned more with abbreviated lecture. These findings are similar to those of other pharmacy educators who have reported that case-based learning was well received by pharmacy students at their institutions.^25,30,35 As this study assessed 2 years of students, student perceptions were not biased by the topic taught and more accurately reflected their feelings related to the teaching and learning method.

In both years of this study, there was no significant difference in student performance on examination questions the week following the presentations when methods were compared, despite documented improved learning on the iRATs after interactive mini-cases. As mentioned in the Introduction, Lui et al reported a similar finding when they evaluated the impact of student response system (SRS) on learning.²⁴ They reported a positive short-term effect for the student group with SRS compared to the student group without SRS, but no long-term effect. However, the SRS served a valid purpose by keeping the students engaged and providing immediate feedback.²⁴ In our project, the multiple mini-cases with graded questions also served a valid purpose by keeping the students engaged and providing immediate feedback in class, as demonstrated by performance on the iRATs, mini-case questions, and the student evaluations. As a result of this project, we now use the abbreviated lecture with interactive mini-cases for both the osteoarthritis and gout presentations. We also use this method for other course topics, including rheumatoid arthritis, systemic lupus erythematosis, and migraine headaches.

A limitation of our project was the lack of comparison between teaching and learning methods on long-term mastery of learning objectives because, for both years of this study, the end-of-year examination at our institution included only 4 total questions on the therapeutics of osteoarthritis and gout. Another project limitation was the lack of data on the extent and type of student work experience in the actual pharmacy setting prior to and during the course. These factors could have impacted student performance and student course evaluation. A potential limitation of this project was that it was conducted at our college of pharmacy’s single campus, so a comparison of student performance at multiple campuses of the same institution with matched school curricula, syllabi, grading systems, and learning objectives, was not possible. Previously published literature on the effectiveness of a teaching and learning method report the results from 1 institution as well, as a valid comparison of student performance at different institutions would only be feasible if the institutions had matched curricula, syllabi, grading system, and learning objectives; the problem of comparing 2 sets of students would still exist.^{7-10,18-21,23-35}

The primary objective of the project was to determine how different teaching and learning methods impacted student learning in a team-taught required pharmacotherapy module in a large classroom setting. A project goal was to efficiently incorporate more individually graded active-learning activities in the large classroom to increase student engagement and decrease lecture time. We hypothesized that active-learning activities would have a positive effect on student participation and learning, compared to the effect of a more traditional lecture format, and they did in class. Another strategy for active learning in large lecture halls involves adding small group break-outs; these are limited by faculty and physical resources. Team-based learning has been successful in engaging students and increasing learning,^{23,26,27,40,41} but with the existing classroom at our institution with fixed seating and no extra space in the room for student grouping, it would have been difficult to adopt such an approach. Other instructors using traditional fixed-seat lecture halls with relatively large student enrollment may wish to include iRATs and multiple interactive mini-cases with individual assessments as a component of their active-learning approach.

SUMMARY

In the large classroom setting, using multiple interactive mini-cases with graded questions had a positive impact compared to traditional lecture on immediate mastery of learning objectives and student performance, with increased class participation, regardless of therapeutic topic, but did not significantly improve performance on subsequent course examinations.