Learning Outcomes and Student Preferences with Flipped vs Lecture/Case Teaching Model in a Block Curriculum

INTRODUCTION

Faculty members in higher education face numerous challenges in designing instructional content and assessments, including the limits of brief class time, maintaining student engagement, promoting student accountability for their own learning, and ensuring that students achieve the appropriate depth of understanding of course material. Many potential solutions have been proposed to address these dilemmas for educators teaching within traditional course schedules, including classroom accountability assessments, required homework activities, and use of active-learning strategies in the classroom.^1,2 While it is desirable for students to complete assigned reading prior to class so that they show up with a baseline knowledge of the subject matter on which faculty members can build, it is often an unrealistic expectation.¹ Despite years of research on effective teaching and learning that supports the use of active-learning strategies in the classroom, many professional programs have been slow to shift away from large-group lectures and passive transmission of knowledge.^1,3,4 Research indicates that using active-learning strategies can stimulate higher-order thinking, problem solving, and critical analysis, which are all desirable learning outcomes.⁴ When used in professional programs, active learning may also help to address the challenges of content volume, rapid advances in practice, and development of the skills needed to work on a team.^5,6

One instructional approach that has the potential to address many of these concerns is the flipped classroom. The flipped classroom format refers to a model that promotes active learning by “flipping” the timing and location of class and homework activities. It is intended to allow for self-paced learning and to maximize class time for students to interact with learning materials in the presence of a faculty member, who is the content expert. In the traditional classroom, time for learning is the constant and the amount of learning per student varies, sometimes widely. The purpose of the flipped classroom is to allow the time allocated for learning to vary and for the learning outcomes to be similar across students. Prior to class, foundational knowledge is absorbed by the student in the form of readings, lecture videos, or podcasts, while in class, instructors focus on resolving misconceptions, nurturing problem-solving and critical-thinking skills, and facilitating team discussions.⁷ This method of information delivery is an appealing potential solution to multiple barriers in the classroom, such as ensuring learner preparation, accounting for different levels of prior exposure to material, and making time available during class hours for application activities. As professional programs have continued to seek and adopt instructional approaches that best support more complex learning outcomes, the flipped classroom model has gained popularity in health sciences programs but has produced mixed results.^8-10 Flipped classrooms in pharmacy programs are no exception. Many programs have examined student perceptions regarding the flipped format, as well as its impact on student performance in courses, from the basic sciences to the pharmacotherapy skills laboratory.^11-19

Among the many applications of the flipped classroom in pharmacy courses, there is wide variation in the specifics of instructional methodologies and the outcomes. Examples where entire courses were flipped, such as described by McLaughlin and colleagues for a three-credit basic pharmaceutics course,¹¹ Giuliano and colleagues for a two-credit drug information course,¹² or Ferreri and colleagues for a three-credit self-care course,¹³ all showed improvements in student assessment scores when using the flipped course format. Additionally, the majority of students in the pharmaceutics and self-care courses preferred using the flipped format.^11,13 While Giuliano and colleagues did not examine student preferences, they found no differences in course evaluations for their drug information course in its flipped format compared to evaluations of the lecture-based delivery in the prior year.¹² These similar findings were reported, despite the use of different pre-class material formats and in-class activities. However, when Camiel and colleagues flipped their three-credit self-care course using a highly structured team based learning (TBL) format in the classroom, student performance was merely maintained compared to the prior year using a lecture-based format, despite a generally positive response from students.¹⁴

Aside from the conventional comparison of the flipped model to the traditional lecture format, some faculty members have compared it to a mixed instruction model and others have examined flipping only a segment of their course. Both Koo and colleagues and Prescott and colleagues redesigned their courses from a format that included a variety of activities as well as lectures to a flipped format.^15,16 Both studies showed improvement in student performance overall and positive feedback on the flipped format; however, in Koo’s study the students still preferred the non-flipped format.^15,16 Wong and colleagues flipped only the topic of arrhythmias in an integrated course and found that mean examination scores were significantly higher for material on pharmacology and therapeutics of arrhythmias, but no different for the basic sciences material taught using the flipped method. Students in this study felt the flipped teaching method was useful for therapeutics concepts, but not for calculations or basic sciences.¹⁷ Similarly, Pierce and colleagues flipped only the renal module of their pharmacotherapeutics course and found that student performance on the final examination was significantly higher using the flipped format vs traditional lectures.¹⁸ However, when Bossaer and colleagues flipped only the oncology model of a pharmacotherapy course and compared it to a mixed instruction model, student performance was significantly worse.¹⁹ These studies show mostly positive experiences with the flipped classroom, either as an improvement over existing mixed-instruction models or as a small segment within a larger course.

The available literature regarding the flipped classroom largely supports its use either as an entire course format or within a course; however, these explorations have all taken place in traditionally structured curricula. While the findings of Khanova and colleagues indicate that requiring students to take multiple flipped courses simultaneously may impose an excessive burden on them,²⁰ the impact of the flipped teaching method on academic performance and student preference has not been sufficiently explored in the setting of a block curriculum, which potentially has very different demands on learner preparation and organization. Continued investigation and identification of the characteristics of effective flipped instruction will potentially lead to improved implementations of the model.

This study was designed to assess the impact of using a flipped classroom instructional approach on pharmacy students’ performance on formal assessments and instructional preferences compared to the existing class format in the first week of a four-week pharmacotherapy course in a block curriculum. It was hypothesized that student performance on topics taught using the flipped model would be higher compared to when those topics were taught using the traditional lecture/case format. Additionally, it was expected that students may be resistant to a new teaching and learning format, but that their perceptions of the flipped classroom would improve after experiencing it in a course.

METHODS

Gastrointestinal and Liver Pharmacotherapy is a required pharmacotherapy course for second-year (P2) students in a four-year Doctor of Pharmacy (PharmD) program. The course is part six of a 13-part pharmacotherapy series and is worth three credit hours. Topics in this team-taught course include conditions impacting the upper and lower gastrointestinal tract, liver diseases, and parenteral and enteral nutrition. The course is delivered during the month of January for a total of 18 six-hour days. Traditionally, each topic consisted of required homework reading from a required textbook, an in-class lecture session with audience response questions (for individual accountability for the content), a small group discussion of a case, submission of a team SOAP note or similar assignment, and a final discussion of the case (lecture/case format; Figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Comparison of Classroom Formats and Time Allotment. The flipped classroom format shifted previous lecture content to pre-class videos and principal class time was spent on application activities. Both instructional approaches included a component of student accountability for completing pre-class work and a case activity.

In 2014, one instructor delivered half of her topics in the flipped format (Gastroesophageal Reflux Disease [GERD] and Irritable Bowel Syndrome [IBS] on days two and four) and the other half in the lecture/case format (review of Anatomy and Physiology of the Gastrointestinal Tract [GI A&P] and Peptic Ulcer Disease [PUD on days one and three) during the first week of the course. The following year, the faculty member switched the formats for these topics, ie, content for GI A&P and PUD was presented in a flipped format and GERD and IBS were presented in lecture/case format. The instructor had six years of teaching experience at two different colleges of pharmacy and had attended multiple workshops and conferences that focused on active learning and the flipped classroom. The flipped classroom was implemented as a potential solution to students not completing required reading prior to class and not having sufficient time for application activities during class. The other three course instructors opted not to modify their teaching model for weeks 2-4 of the course.

The flipped format used in this study consisted of a series of required videos (approximately 60 minutes total per topic), with content divided into smaller segments for user-friendly viewing. Each video series was accompanied by a handout with specific learning objectives and a set of self-assessment questions. Videos were designed and produced using Camtasia Studio, v8.2 and v8.4.4 (TechSmith, Okemos, MI) by the topic lecturer and consisted of on-screen slides and an audio track of the instructor’s voice. Students accessed all course materials through the university’s learning management system (LMS), Blackboard Learn (Blackboard, Washington, DC). After watching the videos, students submitted responses to self-assessment questions for credit via the LMS. Students were released from class one hour earlier on the day before a flipped topic to facilitate appropriate preparation. Once in class, students participated in a discussion of key points and were given the opportunity to ask questions directly or via Google Doc to gain clarification prior to completing an individual assessment administered using audience response clickers. Students then met in teams to develop case-based examination questions on the topic, which included providing evidence-based justification for each correct and incorrect response. Finally, students participated in a team case activity and classroom discussion that mirrored the experience in the lecture/case format (Figure 1). The exception to this format was in covering the topic of GI A&P, which did not include a case when administered in either format because of the time constraints of the course schedule and the nature of the topic.

A pre-post course survey design was used. Students were surveyed using Qualtrics Online Survey Software (Qualtrics, Provo, UT) on the first day of class and again after the final examination, which was administered approximately one month later. Survey questions covered topics such as preferences regarding pre-class and class formats, time dedicated to class preparation, and overall experience with the flipped classroom method. The Likert-scale used for responses included only four options and had no neutral response. The survey questions were developed by the authors specifically for this study.

Students' prior knowledge of topics was assessed before course orientation using a short, unannounced, multiple-choice quiz (the quiz scores did not count toward students’ grades). This baseline knowledge assessment tool was used to verify that students in 2014 and 2015 had a comparable knowledge base prior to starting the course. Student performance on the formal quizzes, and on midterm and final examination questions on the four topics were evaluated for both years, and performance for the lecture/case and flipped version of each were compared. Assessment questions were in multiple-choice format and required students to either choose a single correct response or select all that applied. Identical questions were not used in both years as this was discouraged by the curriculum committee; however, the topic objectives being tested remained the same, as did the weight dedicated to each topic. The number of questions per topic ranged from five to 15. Questions that were dropped following any assessment were excluded from the analyses. This study was reviewed by the institutional review board of Western University of Health Sciences and deemed exempt.

Mean student performance using the flipped classroom approach and the lecture/case format were compared using an independent t test. Likert-scale results from student surveys were collapsed into agreement and disagreement designations and chi-square tests were performed on statements regarding student preferences and preparation behaviors regarding traditional courses and the flipped format. Finally, independent t tests were used to determine whether students who preferred the flipped classroom approach were more comfortable using various technologies than those who preferred the traditional teaching method.

RESULTS

In 2014, the course was delivered to 133 students. One hundred ten students completed the pre-course survey and 58 completed the post-course survey. In 2015, 131 students took the course; 110 completed the pre-survey and 80 completed the post-survey. Response rates for the two years combined were 83% for the pre-course survey and 52% for the post-course survey. Seventy-one percent of survey respondents were female, which is consistent with the overall class composition, and on the first day of class, students’ mean age was 27.8 years (SD=4.4).

There was no significant difference in final course scores (which included all topics covered in the course by all instructors) of 2014 and 2015 students (84.5% vs 83.6%, p=.17). When the four topics examined in this study were analyzed separately (Figure 2), student performance on formal assessment questions was higher for topics taught using the flipped classroom method compared to the lecture/case format (76.9% vs 71.8%, p<.001, Cohen’s d=.45). When analyses were conducted by topic, student performance was significantly higher using the flipped classroom vs lecture/case format for three of the four topics (Figure 2). When students’ overall performance (ie, formal assessments, homework (self-assessment), and clicker (readiness assessment) credit) was analyzed, the same three topics still favored the flipped format, while the fourth favored the lecture/case format (82.0% vs 89.6%, p<.001).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Student Performance on Formal Assessments between Teaching Methods. Each topic was taught using the flipped classroom model (open bars) one year and the lecture/case model (black bars) the following year or vice versa. GI A&P=Anatomy and Physiology of the Gastrointestinal Tract, GERD=Gastroesophageal Reflux Disease, PUD=Peptic Ulcer Disease, IBS=Irritable Bowel Syndrome.

Students preferred live classroom lectures over prerecorded lectures, both before and after participating in the course (pre-course survey 62% vs 38%; p<.001; post-course survey 62% vs 37%; p=.004), although this pattern was driven by the 2014 cohort. The preference of the 2015 cohort was divided evenly prior to the course and showed slightly higher preference for live lectures after the course. Despite this clear preference for live lectures in the classroom, when asked after the course about whether the flipped classroom format was more beneficial than the lecture/case format, survey respondents were split almost evenly (Table 1).

Although students did not always prepare for class in other courses, the majority reported preparing for class even when preparation was not required (Table 1). The subjects were split regarding preference for or against required preparation for a course (Table 1). Students’ estimated preparation time for prior courses that required preparation (based on information collected in the pre-course survey) and estimated preparation time for flipped topics in this course (information collected in the post-course survey) are reported in Figure 3. There was a trend toward more preparation time being reported for a flipped topic, but no statistical difference was detected. Student evaluation of the time needed to prepare for the flipped classroom is presented in Figure 4. Over half of respondents considered the amount of time needed to prepare for a flipped topic to be appropriate.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Student Self-Reported Daily Class Preparation Time for Prior Courses with Required Homework vs. Flipped Topics in Current Course. Students were asked via survey to estimate the mean preparation time they needed for prior courses that had pre-class requirements (black bars) and the preparation time they took for the flipped classroom topics in the present course (open bars).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

Student Reflection on Time Demands of Preparing for the Flipped Classroom. Upon completion of the course, students were asked via survey to categorize the time requirements for preparation of topics taught in the flipped classroom model as too little, appropriate, or too much time.

Pharmacy students who reported being comfortable using specified technologies were more likely to prefer the flipped classroom method than those who reported lower comfort with those technologies after completing the course. The specific technologies included in the survey were streaming videos (p=.005), technology in personal life (p<.001), technology in education (p<.001), and prerecorded lectures to prepare for class (p<.001). Self-reported comfort level with Blackboard (LMS) did not correlate in any way with preference for or against the flipped classroom (p=.32). Students were also asked to designate their overall experience with the flipped classroom format as positive, neutral, or negative at the end of the course (Figure 5). Most students indicated that the experience was either positive or neutral, and a lower percentage of students considered the experience to be negative by year two.

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

Self-Reported Overall Student Experience with Flipped Classroom. Upon completion of the course, students were asked via survey to classify their overall experience with the flipped classroom as either positive, negative, or neutral.

DISCUSSION

Although the present study took place over two years and involved two separate sets of students, they had similar familiarity with and comprehension of the topics involved in the study prior to the start of the course. The performance improvement seen for the more typical pharmacotherapeutic topics in this study when taught using the flipped format mirrored results seen in other studies that explored flipping therapeutics topics in a pharmacy program,^16-18 despite being a part of a block curriculum. The effect size varied by topic from small to very large, but the overall effect across all four topics included in the study was small to medium. The only topic that did not show improved student performance using the flipped format was a review topic for students and was more foundational rather than a therapeutics topic. This made in-class application activities a bit less straightforward and may account for the similar student performances on this topic between flipped and lecture/case format. This finding was also reflected in the work of Wong and colleagues who did not see a change in performance when using the flipped format for a basic sciences topic.¹⁷ For the more conventional pharmacotherapeutics topics, the format of a block curriculum does not seem to preclude use of a flipped classroom format to improve student performance on formal assessments.

Although nearly half of students perceived the flipped format as a more beneficial approach to learning by the end of the course, and approximately 78% of them reported a positive or neutral experience with the flipped classroom, they maintained a clear preference for experiencing in-class lectures over prerecorded ones. This support for the flipped classroom in theory while still not preferring it in application has been seen previously.²⁰ In the present study, this inclination may be for a number of reasons. For example, the additional preparation time mandated by a flipped format is magnified when used in a block curriculum. Because students are in the same class all day, every day during the week, the amount of time available to review and interact with videos outside of class is much more limited. Students in this study did not experience flipped topics back-to-back (they were purposefully spaced every other day), but still reported a significant homework burden that they had not felt in other courses. Students were provided with release time from class that was approximately equivalent to the duration of the required videos, but this was probably not sufficient based on best-practice recommendations for the flipped classroom. Free-text comments from the student surveys also uncovered a common frustration that students felt because they could not ask questions of the instructor while watching prerecorded lectures. Other factors that may have impacted student perception of the flipped classroom include their resistance to new technologies, the preference that many students had for no required preparation at all prior to class, and limited prior experience with the format. There is clearly not just one factor that steers student preference toward conventional in-class lectures.

This study had a number of limitations worthy of discussion. Although flipped and lecture/case topics were all taught by the same instructor, comparisons between the two teaching methods involved two different sets of students (2014 and 2015). Despite taking measures to show that student knowledge base was similar between the two groups prior to participating in the course, there may have been unforeseen confounding factors. Assessment questions used to test students were also not identical from year to year, which may have impacted student performance comparisons. Additionally, the faculty member who flipped the topics in this study had limited previous experience in teaching using this method and did not have the benefit of an instructional designer or production team to plan and create the videos and class activities used in this study, which may have resulted in a suboptimal execution of the flipped teaching method. With only one of four instructors in the course choosing to use the flipped classroom method, student perception of this instructional design method may have been affected and their ability to become comfortable with the new format may have been limited. Future use of this format may be better received by students if they are given more opportunities to experience it within a single course. Post-surveys regarding student perceptions and experience with the flipped classroom were administered after the final examination for the course, which was three weeks after the topics were covered. This required student reliance on recall to describe their experiences with the flipped classroom, which may have caused a decrease in the accuracy of the survey results. Students were asked only to estimate the amount of time they spent preparing for class initially, but were neither asked to examine the amount of time and depth of study needed to prepare for their formal assessments in each topic, nor asked to track this time accurately through journaling. Specifically, more active engagement in application activities during class should result in decreased study time for quizzes and examinations outside of class. However, without survey data to highlight this difference, students may not see the benefit on their own. Survey response rates were lower in the post-course surveys, particularly in 2014, which may have reflected a response bias.

This study did not address the potential differences in long-term retention of content learned using flipped vs lecture/case teaching methods, but a more recent study showed that students who learned content through traditional lecture in the classroom had better retention after 17 months than those who experienced a team-based learning classroom format.²¹ Whether this pattern would hold true in the setting of a block curriculum is unknown. Additionally, there is a growing body of work that indicates that use of the flipped classroom is more successful in fostering higher-order cognition elements of Bloom’s taxonomy (analyze, evaluate) than lower-order elements (recall).^22-24 Future studies involving the flipped classroom in PharmD programs should explore this idea to see if it may help account for the conflicting results seen in the current literature.

CONCLUSION

The flipped classroom model resulted in improved student performance on formal assessments for multiple pharmacotherapeutic topics over a traditional lecture/case model in a block curriculum. However, implementing the flipped classroom model may pose a challenge in terms of student acceptance, particularly if it is introduced in a limited capacity. Faculty at colleges and schools of pharmacy with concentrated instructional formats who are considering use of the flipped classroom should address the potential barriers highlighted in this study to improve the student and educator experience.