Student Perceptions of a Flipped Pharmacotherapy Course

INTRODUCTION

Blended learning, a pedagogical approach that integrates face-to-face and online learning experiences, is touted for its potential to improve the effectiveness and efficiency of education while also improving student and instructor satisfaction.^1,2 A growing body of literature illustrates successful implementation of blended instruction across different disciplines, including pharmacy education.^3-6 The design of these blended learning environments in higher education is largely based on constructivist and adult learning principles.⁷

Constructivism views knowledge as dynamically constructed by the learner, who integrates new information with prior knowledge while interacting with other participants and elements of the learning environment.^8,9 Constructivist pedagogy calls for meaningful activities that engage the students in the learning process through opportunities for reflection on ideas and problem solving, supported by access to information resources.^10-12 Similarly, principles of adult learning emphasize the importance of engagement, interest in problem solving, role of prior knowledge, as well as high level of self-direction and motivation.^13,14

The flipped classroom is a blended learning model increasingly popular in higher education. The flipped approach is characterized by packaging didactic content to students for self-directed study before class so class time can be dedicated to learning activities that promote deeper understanding and higher-order thinking, such as case-based learning.¹⁵ A primary rationale for the flipped model is that learning at the lower levels of Bloom’s Taxonomy (acquiring/remembering information and understanding the concepts) is something students can easily accomplish on their own, while learning to apply these foundations is best accomplished with the guidance of an instructor.¹⁶ This model has a positive impact on learning outcomes and student satisfaction in courses from a wide range of subjects in STEM (Science, Technology, Engineering and Math) disciplines and health professions.^17-21 There are a number of successful implementations of the flipped course model in pharmacy education as well.^22-24

The flipped model is student-centered, treating each student as an independent and self-directed learner. Self-directed learning outside the classroom and active engagement of students in higher orders of thinking and problem solving inside the classroom define the new doctor of pharmacy (PharmD) curriculum at the University of North Carolina (UNC) Eshelman School of Pharmacy.²⁵ While the new curriculum was not implemented until fall 2015, this approach to course redesign (ie, the flipped model) was incorporated in a number of courses in the curriculum before then.^22,24,26,27 The transition started with pharmaceutical science courses (eg, basic pharmaceutics), resulting in improved learning outcomes and student perceptions.²⁴ This success stimulated interest in flipping other courses in the curriculum, notably pharmacotherapy.^26,27

Pharmacotherapy courses integrate knowledge from prior course work in physiology, biochemistry, pharmaceutics, pharmacokinetics, and pharmacodynamics to help students develop the working knowledge of clinical pharmacology and pharmacotherapy that forms the basis of pharmacists’ professional responsibility. These courses can be difficult for learners, particularly disease states for which there are rapidly changing treatment guidelines or lack of guidelines and available medication. The dynamic nature of pharmacotherapy requires the ability to learn new information, integrate knowledge, formulate plans, use higher-order thinking skills, and work as a team to solve complex cases. This type of course lends itself to a flipped format, with class time devoted to application and analysis. Implementations of a flipped approach in pharmacotherapy courses on a limited scale (ie, select topics and/or class sessions) result in improved learning outcomes and an increased level of student engagement.^24,26

The purpose of this paper is to present findings on a full redesign of a pharmacotherapy course at the UNC Eshelman School of Pharmacy. Specifically, this study examined student perceptions of a flipped course to help inform guiding principles for an effective and sustainable blended learning approach to teaching pharmacotherapy. The following sections describe the course redesign strategy, assessment of the course, and lessons learned from the experience. These lessons may be useful to educators and course developers in selecting appropriate pedagogical strategies and in considering potential pitfalls.

DESIGN

In spring 2014, a 5-week psychiatry/neurology pharmacotherapy course at the school was redesigned using the flipped model. The pharmacotherapy course is a required course for pharmacy students in their second professional year of the PharmD program, and is one component of a 3-year pharmacotherapy sequence for students, beginning in their first year of the curriculum. It is a team-taught course, relying on the clinical and real-world patient care expertise of 10 instructors to cover the pharmacotherapy of 18 different disease states.

The redesign of the course from a traditional lecture-based format was guided by a desire to better meet the course learning objectives: (1) develop critical-thinking skills needed to analyze and resolve complex pharmacotherapeutic problems; (2) develop the ability to integrate information from several disciplines; and (3) apply evidence-based principles to make rational drug treatment decisions. Ultimately, the goal of the redesign was to promote deeper learning and problem solving to better prepare students for their future roles in patient care. The design strategy for this course was inspired by findings from a flipped pharmacotherapy seizure module at the school.²⁷ As described by McLaughlin and Rhoney, the 2 key components of the redesign included: (a) moving acquisition of foundational knowledge outside the classroom through the development of self-paced online learning modules for review prior to class; and (b) using class time for active and applied—primarily case-based—learning.²⁷

All online modules were developed as a collaborative effort between each of 10 clinical instructors for the course and the school’s educational technology team. Each module focused on a particular disease state and included a short (approximately 60 seconds) video introducing the instructor and the significance of the topic, a series of linked webpages containing comprehensive information about the topic and pop-up definitions, embedded interactive assessments termed “Quick Check” questions, and real-time discussion forums for students.

The modules varied slightly in length and took 40 hours on average to produce. The production of each module involved full-time effort by one of the school’s instructional designers, as well as a graphic designer. Instructor involvement in module design varied in terms of time commitment and included providing content, writing assessment questions, reviewing and revising modules created by the educational technology team, and recording the introductory video.

With the majority of foundational content off-loaded onto the modules, overall in-class hours were reduced by approximately 20% to balance the time required for students to complete out-of-class learning activities. Based on the consensus of course instructors, 5 of the 18 disease states were covered as independent study using online modules followed by homework assignments to assess learning. The factors in selecting modules for independent study included overall topic complexity and degree of content overlap with other topics.

There were no dedicated class sessions for these modules, but students had an opportunity to consult the subject matter expert during the scheduled review session offered before mid-term and final examinations. These review sessions were optional for students, but all instructors who taught the content covered on the examination were required to answer questions either in-person or via video conferencing technology and to offer additional explanation of unclear concepts. One of the 5 modules also was covered as an assignment in the pharmaceutics care laboratory course, which allowed for further discussion of subject matter with group facilitators. For the remaining disease states, online modules were intended to provide foundational learning to prepare students for the subsequent class session.

Course instructors were encouraged to focus the 50- or 90-minute class sessions on clinical applications and engagement of students in critical thinking on the subject matter, with only occasional mini-lectures, if needed, to clarify important concepts. The in-class learning activities were designed to center on case studies and incorporated a variety of active-learning strategies, such as audience-response (aka “clicker”) questions, peer discussions using “think-pair-share” technique, use of videos for symptomatic portrayals and review of diagnostic criteria, and interactive exercises to complete missing information in drug tables and formulate drug therapy assessments and recommendations.

It is worth noting that the students in this cohort had experienced a variety of blended and flipped course designs at the school prior to this pharmacotherapy course. Specifically, at least 6 of the courses at the school in the 2 years prior to the study had incorporated some form of blended or flipped learning for all or part of the course. The course director held a 10-minute informal discussion with students one week before the course to outline the new structure and explain the rationale for the approach.

EVALUATION AND ASSESSMENT

To understand the impact of the course redesign on student perceptions and attitudes toward a flipped classroom, data were collected to evaluate the specific design solution, namely the use of online text-based modules as the main format for didactic material. Precourse and postcourse survey instruments were developed to examine student perceptions of their learning experiences with the flipped classroom. These instruments were adapted with course specific modifications from the instruments used for similar research and course evaluation purposes by other instructors at the school.²⁴

The questions on the precourse survey were designed to establish a baseline of student attitudes and preferences related to course format and various learning activities. Six questions used a 4-point Likert scale (“strongly disagree,” “disagree,” “agree,” and “strongly agree”) to measure student perceptions of learning activities used in this course. The last question about course format preference had 2 choices for an answer: “coming to class and listening to the professor lecture” and “learning important foundational content prior to class and using class time for more applied learning.”

The postcourse survey repeated the same questions, but added 6 questions specific to student experiences in the redesigned course. It also included 2 open-ended questions on what they liked and did not like about the blended approach to learning. Both surveys were administered using Qualtrics software (Qualtrics, Provo, UT) by a nonteaching, nonprecepting postdoctoral research associate. All students in the class were invited to take part in the survey via e-mail, with an e-mail reminder sent to survey noncompleters via Qualtrics. The course director made an in-class announcement about the survey and its purpose before the start of the course (for precourse survey) and toward the end of the course (for postcourse survey). Participation in the survey was voluntary, and no incentives were provided for participation. This study was classified as exempt by the UNC Institutional Review Board.

A mixed-methods approach was used for the study. All statistical analyses were conducted in SPSS, v22 (IBM, Armonk, NY). Chi-square goodness-of-fit test was used to compare demographic characteristics of the study sample to the overall class cohort. Comparison of responses to the same precourse and postcourse survey questions was done using McNemar’s test after collapsing the responses into 2 categories: agree (for “strongly agree” and “agree” responses) and disagree (both “strongly disagree” and “disagree responses”). A p<0.05 was considered significant.

Responses to the 2 open-ended questions on the postcourse survey were analyzed qualitatively to identify main recurring themes by identifying the most salient points in each response and using the constant comparison method to cluster the semantically similar statements.²⁸ Responses for each of these questions were sorted into groups of similar-themed comments within the question set, and then between the questions for the overall themes.

Of the 171 students in the class cohort, 134 (78% response rate) completed both surveys. As seen in Table 1, the demographic characteristics of the study sample are not significantly different from the demographic characteristics of the class cohort as reported at admission, except for a slight difference in race/ethnicity. More students in the study sample (78%) identified as white/Caucasian compared with the class overall at admission (62%), while fewer identified as Asian (14% in the study sample vs 19% at admission), with only 3% selecting other, compared with 12% other/unknown at admission.

In the precourse survey, 72% of participants indicated that they preferred the traditional lecture course format over the flipped classroom model. However, 83% of participants also agreed or strongly agreed that “in-class activities focused on application and problem solving enhance my learning.” Only 26% of participants agreed or strongly agreed with “assigned readings from textbooks/articles greatly enhance my learning.”

Comparison of precourse and postcourse survey data (Table 2) found significant changes in student perceptions of 3 out of the 6 learning activities examined. After the course, a significant decrease was reported in students’ perception that learning was enhanced by lectures, by in-class activities focused on application and problem solving, and by in-class discussion of course concepts with peers.

When students were asked at the conclusion of the course whether they preferred coming to class to listen to a professor lecture or learning foundational content prior to class and using class time for applied learning, 83% indicated that they preferred the former, a significant increase from the 72% on the precourse survey (p<0.001). These results appear to contradict the decrease in positive perception of lecture reported above.

Postcourse survey responses provided additional insight into student perceptions of the course design. Namely, only 28% of participants agreed or strongly agreed that preclass modules enhanced learning. However, 59% agreed or strongly agreed that the assessment questions embedded within the preclass online modules enhanced learning. Of the students who reported attending the scheduled review sessions (79% of the study participants), 43% agreed or strongly agreed that the sessions enhanced learning. Similarly, 44% agreed or strongly agreed that in-class activities enhanced their learning, with only 19% of participants agreeing or strongly agreeing that overall course format enhanced learning.

Finally, the survey included a question asking students to indicate why they used the online modules, and it allowed students to select all answers that applied. Of the 132 students (99% of study participants) who provided responses to this question, 85% indicated they used the modules for concept reinforcement or to study for the examination, while 54% reported using them for class preparation.

In response to the 2 open-ended questions on the postcourse survey, 102 and 123 participants respectively, provided elaboration on the positive and negative aspects of the course. This does not include the small fraction of responses (14 for “What did you like?” and 6 for “What did you not like?” questions) that were nonelaborative one-word responses, such as “nothing” and “everything.” These were taken as expressions of overall dissatisfaction with the learning experience and excluded from further analysis. Some of the comments included in analysis mixed critique of course design with suggestions for future improvements. The main themes that emerged from student comments are summarized below.

Regarding the blended learning approach, comments indicating preference for traditional lecture, such as “I would much rather come to class and be lectured” were common, but many students stated they liked “the idea” of “learning concepts prior to class” and using class time for discussion and other forms of active learning. However, students’ comments distinguished between the idea and its implementation, offering critique of the latter. The main critical theme regarding the overall course design was the perception that “most everything was self-taught,” and there was a lack of guidance from professors.

Online modules were valued as a “comprehensive resource for the subject” that were helpful for review and also would be “excellent reference for the future.” They were seen as a good substitute for a textbook and a means of reducing note-taking. Checkpoint questions embedded in the modules often were mentioned as useful study tools. Nevertheless, respondents reported being frustrated with the module design. Specifically, students indicated the modules contained too much information and required too much time to read, making it difficult to complete modules before class and balance coursework from other classes. The following comment is representative: “With all of our other classes, I did not have time to read the large amount of material before each class. This made class time very confusing and unproductive.” Further, students seemed to need additional “guidance regarding what information [within the module] was essential and what was supplementary.” Finally, occasional typos and other errors in the module text contributed to students’ dissatisfaction.

Regarding in-class learning activities, students found the discussion of clinical cases enjoyable and useful. They liked “having case-based application during class time,” and learned “the most while actually being in class during case-based discussion facilitated by professor.” However, some students also felt unprepared to participate in case discussion, expressing a strong need for in-class review “before jumping right into cases,” and observing that “classes that went over key points at the beginning and then into cases were more beneficial than others.”

Covering some topics as independent study with no class session dedicated to them and only homework assignments to assess learning received substantial negative feedback. Students felt strongly that, for every disease state, “[some] class time should be spent explaining difficult concepts” prior to applying self-learned material to a high-stakes assessment like an examination. They indicated that only studying online modules without instructor-dedicated time to those topics provided insufficient preparation for future patient care.

Students’ suggestions for course improvement included reducing the length of online modules and reorganizing the material in a way that made it clear what is essential and supplemental. Other suggestions related to modules were to have professors develop study guides or drug tables based on the modules and to have video lectures as an alternative to text-based modules. Another recurring suggestion was to start the class session on each new topic with an instructor-led review of the key concepts and opportunity for students to ask questions, “then go over a case or two highlighting the main points.”

DISCUSSION

A growing number of educators are designing and implementing the flipped classroom as a means for improving student outcomes. However, because of the novelty of the flipped classroom as a blended learning approach, there is a notable lack of evidence-based best practices and guidelines for implementation.²⁹ The lessons learned from this study are valuable additions to the growing body of research about implementing blended learning.

As noted in the Introduction, the flipped model has been implemented in a number of courses at the school. However, the flipped classroom redesign reported on here differed from prior implementations in several respects. The previously reported successful implementations of the flipped model of a full-course were for basic science courses, not pharmacotherapy courses.^22,24 Furthermore, the basic science courses used video lectures or narrated animations, not text-based modules for the preclass learning. Finally, those courses had one primary instructor, with some guest lecturers, whereas the course described in this paper was team-taught, with 10 instructors having roughly an equal share of teaching responsibility.

Implementations of the flipped model in pharmacotherapy courses, including the use of online modules, have been done before at the school but on a much more limited scale—for class sessions within a traditionally taught course.^26,27

In this study, the precourse survey provided insight into the importance of understanding pre-existing attitudes and learning preferences of students enrolled in a flipped course. Before the course, the vast majority of students in the study sample (72%) expressed preference for traditional lecture over flipped classroom. Additionally, the responses to precourse and postcourse surveys indicated the majority of students in the class did not perceive reading as an activity that enhanced learning. Considering those preexisting preferences and that the use of mostly text-based online modules for all out-of-class learning was new to the students, it is not surprising that students struggled with the course format.

This case highlights the need for course flexibility and for faculty members to be prepared to make adjustments in response to student concerns, especially when the pedagogical approaches used are new to students. As students’ experience with the flipped classroom and other forms of blended learning grows, and as improvements in blended-learning approaches are informed by evolving best practices, it is reasonable to expect that attitudes will change. However, additional effort may be needed to help current students understand the new forms of learning, set appropriate expectations, and develop new study skills.

Our findings also highlight the importance of designing preclass learning materials with attention to amount of content and its structure. The critical perception of online modules expressed in the postcourse survey contrasts with successful implementation of a similar text-based online module within a pharmacotherapy module at the school.²⁷ This could be attributed to a significant difference in students’ time and effort required for studying a single online module within a course vs using text-based online modules for all 18 disease states in a full course.

The qualitative analysis of open-ended comments highlights student-perceived flaws of module design. For example, students were critical of excessive module length as well as the structure that did not differentiate between most important and supplemental information. The latter may have made skimming difficult, and the former appeared to result in many students choosing not to view modules before class and thus coming to class unprepared. General student reluctance to learn by reading was also likely a contributing factor to insufficient preclass learning. This was further enabled by lack of mechanism in the course design to hold students accountable for class preparation.

Additionally, some of the challenges students experienced can be attributed to known differences in cognitive strategies people use in learning from screen vs paper-based materials, including poorer time management and distractibility when learning from screen.^30,31 However, there were also positive comments about the interactive checkpoint questions embedded in the online modules, as well as the value of the modules as an information resource extending beyond the course timeframe. The former is consistent with growing research that the use of questions enhances interactivity and learning in online modules.³² Taken together, student feedback suggests that online modules can be a useful learning tool in a blended learning environment, but more attention should be given to properly designing and testing them before implementation.

Some students expressed a preference for video lecture format for preclass learning. This may be attributed to a high preference for lectures in this cohort of students, as well as their prior experiences with flipped learning. Video lecture has become a staple of flipped classroom design. In a recent review of 24 studies of flipped learning, text-based online modules were used only in 4 cases, all of which were in a “partially flipped” category.³³ The present study is unique in examining the use of primarily text-based modules for out-of-class learning for an entire pharmacotherapy course. This particular format of preclass learning was a change for the students, who noted issues with design. Students may perceive video-enhanced and text-only online modules equally if the latter are designed with careful “chunking” of information and less text.³⁴ We believe that with streamlined content and structure, online modules can be effective learning tools in a flipped classroom.

Findings from this study demonstrate the need for thoughtful design of in-class activities. The most salient theme that emerged from open-ended comments was the importance of instructor-led review of material learned at the start of class. The challenges with class preparation noted above make the instructor-led review even more crucial in ensuring students are prepared to engage in active learning during class. Within the context of constructivism, our analysis suggests that failure to bridge prelearning material to in-class activities, on top of inadequate class preparation by students, can limit students’ ability to construct knowledge during in-class learning activities, which may introduce frustration and generate dissatisfaction with the learning environment.

It is likely that, although the postcourse survey questions repeating precourse survey questions were worded in general terms, students responded to them in reference to the pharmacotherapy course experience rather than general beliefs about learning. This interpretation helps explain the seeming contradiction between increase in preference for traditional lecture-based format and significant decrease in agreement with the statement, “Lecture greatly enhances my learning.”

Some element of direct in-class instruction to reinforce key foundational concepts may be necessary to achieve the full benefit of applied learning, such as analysis of clinical cases. Students did have a high regard for the clinical expertise of instructors and valued the opportunity to learn from “[the] country’s most highly regarded experts.” This finding is in accordance with a report by Means et al that suggested blended learning is more effective when it is instructor-directed.² The importance of the instructor in the learning process is further supported by the negative feedback from students regarding topics covered as independent-study only.

The course design was based on the adult-learning principles and assumptions, including the view of students as independent, motivated, and self-directed learners. These assumptions need to be moderated by an understanding of student learning environment expectations, preferences, and prior experiences. This conclusion is similar to observations from Bradley et al’s study of active learning implementation in medical education, which found while students understood the importance of active learning and critical engagement with the subject matter, they still tended to focus on memorizing for the examinations.⁷

This may be in part a result of the strength of what Bereiter described as “schoolwork module” – the habit of approaching academic work as a job that students try to complete with minimal amount of effort.³⁵ In that mode, a student approaches learning difficulties not as challenges to be overcome, but as hard work to complain about. Schoolwork module develops early in one’s schooling and tends to be stable, interfering with learning motivation and development into an “intentional learner.”^35,36 The challenge is to design learning tools and experiences that get away from the test grade driven schoolwork module by emphasizing learning interaction with faculty members and peers, as well as rewarding ongoing active engagement in the learning process.

There are a number of factors that limit generalizability of the study findings. First, the course redesign, namely the production of the online modules, would not have been possible without the effort of the school’s instructional design and educational technology professionals. This approach to course development may not be feasible in settings lacking instructional design and technology support. Second, while team teaching is common, this course had a particularly large 10-member team consisting mostly of practicing clinical pharmacists. Students value the opportunity to learn from experts in the field, but frequent changes in instructors and teaching styles may have negatively affected students’ perception of the learning experience. In addition, this may have made it more difficult for the course director to ensure a seamless transition from instructor to instructor and manage a new approach to learning.

As noted in the Design section, the effort put forth by instructors into module production varied (chiefly because of time constraints related to their clinical practice responsibilities), and this likely contributed to inconsistencies between topics and class delivery. Finally, the school is undergoing curricular and pedagogical change, and this particular cohort of students has been exposed to a variety of course redesigns.^22,24,27 The cumulative effect of these experiences, all designed with genuine interest in improving student learning, likely impacted student perceptions of various teaching and learning approaches.

The high preference for traditional lecture expressed by students on the precourse survey suggests that prior experiences, not just course design, factored into their perceptions of learning in the redesigned pharmacotherapy course. This limitation was partially mitigated by using a mixed-method approach to investigation, and the triangulation between quantitative analysis of survey data and qualitative thematic analysis of open-text responses strengthens our confidence in the findings.

Despite these limitations, the findings are informative for teaching practice. The study offers guidelines for instructors or course developers considering implementation of some form of blended learning. We plan to modify the course based on the lessons learned from this iteration, incorporating suggestions from the students. Namely, the modules will be substantially edited for length and structure, and additional assessments will be added to hold students accountable for preparation. Faculty development efforts are under way, with focus on: identifying essential foundational content and condensing the preclass modules with focus on those key concepts; highlighting best practices from instructors who have successfully restructured their classroom teaching with emphasis on a bridge between foundational knowledge and application; and active learning as a means for stimulating a deeper sense of learning and how to effectively implement it. Careful attention will be given to consistency of student learning experience with all topics in the course as well as efforts to increase student motivation and foster their development as learners.

SUMMARY

Pharmacy education is evolving to better prepare students for the growing complexity and variety of pharmacy practice. Flipped classrooms and other blended-learning approaches align with the goals of developing students’ ability to solve problems and make patient care decisions based on the dynamic context of modern health care. Our study highlights practical challenges of implementing a flipped learning model. Student feedback informed potential redesign of the course and the development of a more effective model for teaching pharmacotherapy. Such feedback underscores the importance of designing courses that effectively balance time requirements for preclass and in-class activities and the crucial role of the instructor in bridging the gap between material learned during independent study and its application.