Combination of a Flipped Classroom Format and a Virtual Patient Case to Enhance Active Learning in a Required Therapeutics Course

INTRODUCTION

Educational outcomes and accreditation standards in pharmacy education urge faculty members to replace traditional lecture-based formats with active-learning strategies that promote higher cognitive learning.^1,2 While a lecture allows faculty members to disseminate information to a large number of students, it does not provide students with the opportunity to practice higher-order cognitive skills such as application, analysis, and synthesis.^3,4 Engaging students in active learning affords students the opportunity to practice higher-order cognitive skills, fosters critical-thinking and problem-solving skills, and leads to a better degree of understanding.^3,5,6 Moreover, active-learning strategies prompt students to explore their personal attitude towards learning, and foster the motivation to acquire more knowledge.⁶ In response to the recommendations set forth by accreditation and education councils, schools of pharmacy have sought to integrate innovative, active-learning strategies to promote inductive reasoning and self-directed learning.⁴

Two examples of active-learning strategies include the flipped classroom and virtual patients. The flipped classroom is a student-centered pedagogy in which students complete pre-class work to gain basic knowledge, and class time is dedicated to activities that promote application and mastery of this knowledge.⁷ Flipped classrooms have been successfully implemented at the kindergarten to 12^th grade educational level as well as within higher education, including health professionals education.⁷ Instructors at several different pharmacy schools have integrated flipped classrooms into their curricula.^8-14 Within flipped classrooms in pharmacy education, preclass work typically consists of readings (textbook, Web-based, or literature) or videos that students can complete or watch at a pace suitable to their own learning needs. Class time in this model is interactive, requiring students to develop their knowledge and skills through active-learning strategies, including small-group problem solving, debates, role-play, student presentations, small-group sharing, and case-based activities. Students have indicated that they value this classroom structure because it promotes engagement in the material and the development of higher-order cognitive skills like reasoning and problem-solving.^8,10,14

Virtual patients have been used effectively in the training of students in different healthcare disciplines, including medicine, dental medicine, nursing, and pharmacy.^15-18 Virtual patients simulate lifelike clinical scenarios in which the learner becomes the healthcare professional, making clinical decisions in a safe environment.¹⁹ This technology provides opportunities for students to practically apply course learning objectives, and it can be programmed to provide immediate feedback to the learner.^16,19 Implementation of virtual patients into curricula has been associated with achievement in knowledge, clinical reasoning, and skill outcomes, although qualitative research to date has not definitively demonstrated superior outcomes compared to other educational interventions.^16,20

Both flipped classrooms and virtual patients have been implemented into courses at the University of Pittsburgh School of Pharmacy to enhance the overall learning experience for students.^21-24 However, to our knowledge, combining these active-learning strategies had not been attempted. Our study assessed the design, implementation, and evaluation of a novel approach that integrated a virtual patient activity in a therapeutics course that already used a flipped classroom model of learning.

DESIGN

Gastroenterology and Nutrition is a required two-credit hour course for second-year doctor of pharmacy (PharmD) students at the University of Pittsburgh School of Pharmacy that is a part of the therapeutics learning sequence. Within this therapeutics course, students develop their knowledge and abilities in patient assessment and design of therapeutic regimens for effective and safe treatment of patients with common gastrointestinal diseases. Since 2010, preclass videos have been used to introduce foundational concepts critical to each course component. Each video lecture was prerecorded and downloaded to a website embedded in Blackboard (Blackboard, Inc, Washington, DC). Students were expected to view videos in preparation for class time. Depending on the topic, the length of each video lecture ranged from 10 to 60 minutes; longer videos were broken into shorter sections that were no longer than 17 minutes. During the once weekly 100-minute class, the instructor engaged students in active-learning activities that included working through case-based scenarios in either a small-group or large-group format. Small groups (n=19 during the time of this study) consisted of five to six students who were preassigned at the start of each semester; students were in the same group for all required pharmacy courses during the semester. Patient cases were typically provided in a static format, either written or as videos of patient interviews. One hundred nine students enrolled in Gastroenterology and Nutrition in the spring of 2013 served as a historical control group for our study.

In the spring of 2014, a virtual patient case dedicated to the complications of liver disease was integrated into the course in lieu of written (static) cases. One hundred nine students were enrolled in the course and participated in this active-learning strategy. In preparation for class, students watched a 60-minute video-recorded lecture that was broken into six shorter segments. During class time, students worked in their assigned groups to complete a virtual patient case. The virtual patient case was developed through vpSIM software (Decision Simulation LLC, Chadsford, PA) created by the Laboratory for Educational Technology at the University of Pittsburgh. Computer-simulated virtual patient cases developed through this platform use a branched-narrative model in which the learner must choose the best recommendation based on available options, and all interactions are strictly text-based. Learners were provided with positive or negative patient consequences throughout the simulation, along with in-module feedback that was unique to their decision.

Learning objectives developed for the virtual patient case were identical to those used in designing the video lecture (Table 1). Therefore, the virtual patient allowed students to practically apply knowledge gained during the lecture. Figure 1 depicts the progression of the patient case using the virtual patient simulation technology. All information was in accordance with the current guidelines published from the American Association for the Study of Liver Diseases (AASLD).^25-27

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Progression of the virtual patient case on the Complications of Liver Disease.

Periodic questions and challenge screens were embedded into the virtual patient simulation to allow students to self-assess their learning and gauge their progression toward mastering course learning objectives. Student performance on the assessments completed as part of the virtual patient case did not count toward the course grade, but were meant to help students and faculty assess skills. While students worked on the virtual patient case, faculty members were available to field questions from the groups and provide insight into how expert clinicians might view or reason through similar scenarios. After the completion of the virtual patient case, a structured debriefing involving the entire class took place. A faculty facilitator used this time to reiterate key concepts, address concerns or questions, and clarify confusing points in the material. After completion of the class time, students could access the virtual patient case in order to review this material independently.

Table 2 summarizes the differences in the class format in 2013 and 2014. Students in 2013 (control group) were given a low-stakes in-class pretest prior to completing the in-class written cases. The pretest consisted of four knowledge-based multiple-choice questions, based on materials presented in the preclass video. An open-book, low-stakes, postclass quiz consisting of questions intended to assess students’ performance according to Bloom’s taxonomy was posted to Blackboard after class, and students had one week to complete it. The purpose of the postclass quiz was to help students prepare for the examination.

Students in the 2014 class (virtual patient group) were given in-class pre- and posttests, which flanked completion of the virtual patient case to assess the effectiveness of this intervention. The six multiple-choice questions on the pretest and posttest were not identical but were similar in content and difficulty level. Both tests were designed from the same learning objectives used to design the video lecture and virtual patient case (Table 1). Similar difficulty was achieved through constructing the questions to match specific domains within Bloom’s taxonomy.³ All pre- and posttest questions had four answer choices (a, b, c, or d) including the correct answer. The level of learning assessed by each question was independently verified by four pharmacy faculty members. Similarly, the validity of the pretest, posttest, and virtual patient case was established by an expert pharmacy panel and through testing pharmacists and student pharmacists at various levels of training (ie, pharmacy students in their fourth professional year, postgraduate year 1 pharmacy residents, postgraduate year 2 pharmacy residents, and pharmacy faculty members). Scores on the pre- and posttests were not factored into the students’ grades within the course in 2014, but were used to assess the innovative teaching strategy.

The impact of the virtual patient case on students’ ability to master the learning objectives of the class was assessed by evaluating performance on 10 multiple-choice questions that were included on an examination administered five weeks after the class on the complications of liver disease. All examination questions had four answer choices (a, b, c, and d) including the correct answer. Performance on the examination in 2013 (historic control) and 2014 (virtual patient) were compared. The examination questions used in 2014 were identical to those in examinations given in 2013. The examination also included questions from five other gastroenterology and nutrition topics covered in the course. Students do not have access to examinations from previous years in this course, and since this particular examination is always scheduled during the final week of the spring term, only one or two students each year even make appointments to review their examination results with the course coordinator. While there is always a risk that recycled questions will be compromised by students, the course coordinators took several steps to ensure that this would not occur. First, each year, there were at least four different versions of the examination; in each version, the questions were presented in random order not only by subject content but also by the instructor who wrote them. In each version, the answer choices for each question were also presented in random order. Additionally, students were not permitted to have cell phones, calculators, or similar devices with them during the examination.

Student perception regarding the virtual patient case was assessed by two questions added to the end-of-term course evaluation. The first question asked students to rate on a five-point Likert scale the effectiveness of the virtual patient case in understanding and applying materials. The second question asked students to select the active-learning strategy they preferred (ie, virtual patient case, other case-based activities used within the course, or either learning method). Completing the course evaluation was not mandatory, but students received two extra-credit points (in a 300-point course) as an incentive for finishing it.

The investigational review board at the University of Pittsburgh approved this study under an exempt status. All quantitative data analyses were conducted in SPSS for Windows, version 21 (IBM Corp., Armonk, NY). All data were assessed for normality. Median pretest and posttest scores were compared using the Wilcoxon signed-rank test. Pretest and posttest questions were compared individually using McNemar’s tests. Median examination scores were compared between 2013 and 2014 with the Mann-Whitney U test. Individual examination questions from each year were compared with chi-square tests. Additionally, questions were divided into lower-level (knowledge and comprehension) and higher-level (application, analysis, synthesis) learning domains, based on Bloom’s taxonomy, and were compared pretest vs posttest, and 2013 examination vs 2014 examination with Wilcoxon signed-rank tests and the Mann Whitney U tests, respectively. To estimate internal consistency reliability, a Kuder Richardson reliability coefficient (KR20) was calculated for the examinations. Additionally, point-biserial correlations were calculated for each question on the examinations to determine item discrimination (see Appendix 1 for these results). Significance was established at p<.05. Survey responses were reported using descriptive statistics.

EVALUATION AND ASSESSMENT

Students had significantly higher median posttest scores compared to the pretest (33% vs 50%, p=.01) (Table 3). However, there was no significant difference between median pre- and posttest scores when divided into lower-level (p=.87) and higher-level (p=.12) learning objectives. Students had significantly higher median examination scores in the virtual patient group compared to the historical control group (70% vs. 80%, p=.025, effect size .15) (Table 4). There was no significant difference between the two groups when comparing median scores on questions assessing lower-level learning (75% vs 75%, p=1.0). However, students in the virtual patient group scored significantly higher on questions assessing higher-level learning compared to the control group (67% vs 83%, p=.003, effect size .20). Twenty-seven students (24.7%) used the virtual patient case on their own time, following class and prior to the examination, presumably as a study aid.

Perspectives of the virtual patient case were collected from 108 students (Table 5), and 67.6% agreed or strongly agreed that the virtual patient case helped them understand and apply course information on the complications of liver disease. When asked about preferred method of class time, 69.4% of students indicated that they either liked the virtual patient case better than the static case-based learning activities used in other classes in the course (32.4%) or thought both methods were equally effective (37%) in helping them understand the material. Overall, 82 students left open-ended comments for this course, and 27 (32.9%) of those comments dealt with the virtual patient case. Out of the 27 students who commented on the virtual patient, 20 (74%) responded favorably to the learning activity. Positive comments centered on the interactive nature of the case and how the case complemented the debriefing session. All seven of the negative comments regarding the virtual case were in regards to the length of the case in proportion to the debriefing sessions that followed.

DISCUSSION

We were able to successfully design and integrate a virtual patient case into a therapeutics course in which a flipped classroom format previously had been implemented. The complex, layered structure of the virtual patient case allowed students to see the progression of cirrhosis and the associated complications of chronic liver disease. The virtual patient technology engaged the students as evidenced by the in-depth conversation that occurred during the debriefing following the virtual patient case. Faculty members observed that students offered thoughtful questions that spanned across the pharmacy curricula and demonstrated independent exploration of the material.

Overall, students achieved higher scores on the posttest administered immediately after working through the virtual patient, compared to the pretest. However, both pre- and posttest scores were not high. This was probably because the pre- and posttests were composed predominantly of questions that assessed higher-level learning; these questions were significantly more difficult compared to other in-class quizzes in this course. The preclass quiz administered before other classes in this course contained knowledge-based learning questions, and were designed to merely evaluate student completion of the required preclass activities. In contrast, the pre- and posttests for the module on the complications of liver diseases were case-based and designed to assess students’ ability to apply, analyze, and synthesize knowledge.

Students scored significantly lower on the posttest question that covered content on SBP treatment. Despite expert review by a pharmacy faculty panel and testing the questions across learners at various stages, the low scores ultimately were attributed to question composition. We concluded that, in retrospect, the content for this particular posttest item was not adequately covered in the virtual patient case, which may have affected some students’ scores.

Students completing the virtual patient case had higher scores on the examination questions compared to students in the previous year. Most notably, students completing the virtual patient exercise scored significantly higher on questions designed to assess higher-level learning compared to the control year. Of course, students in 2014 had the advantage of having additional practice with higher-level multiple-choice questions, being exposed to them in both the pretest and posttest flanking completion of the virtual patient case. Of note, student performance on questions covering the other four gastroenterology and nutrition topics assessed on the examination was not different across the four years in which the same examination questions were used (unreported results).

While there was a statistical difference in the overall examination scores, the low effect size might suggest the addition of the virtual patient case had a low effect. Alternatively, it is possible that the small number (n=10) of total examination questions contributed to the calculated low effect. Of note, the median examination scores for questions assessing higher-level learning were 67% (grade of D+) in the control group and 83% (grade of B) in the virtual patient group.

The continuous feedback that is possible with computer-simulated virtual patient technology likely contributed to students’ improved examination performance more than the technology itself did. For instance, students were provided with positive or negative patient consequences throughout the virtual patient case along with in-module feedback that were unique to their decision. The purpose of this structure was to help students identify their strengths and weaknesses so that they could target areas in which they needed improvement; additionally, faculty members could easily identify where students were struggling. It would be impossible for students to receive this type of timely, formative assessment with a static, traditional paper-based case unless each group of students had their own facilitator. In our historical control group, in what was a time-consuming task, one faculty member hand-graded the traditional paper cases and provided students with written feedback. However, this feedback was routinely provided to students at least a week after they completed the case, and the feedback was likely inconsistent across the groups.

There were several strengths of this study worth highlighting. First, almost all students within the class participated in the course evaluation. This high response rate allowed for an accurate reflection of class opinion on the combined learning strategy. Students were positive about the experience when reflecting and reporting on their preferred teaching methods. Despite the fact that the virtual patient exercise was integrated midway through the course, after the students were comfortable with more traditional approaches to case-based learning, 75 students (69%) stated they would not be opposed to the use of virtual patient cases in the classroom to further their learning of clinical concepts.

Another strength was the comparison of examination questions with a historical control. Often, there is limited data or updated guidelines, both of which can limit the implementation of a historic control group in educational research; however, neither of these potential challenges impacted this study. Additionally, the validity of the pretest, posttest, and virtual patient case was established through an expert pharmacy panel and by testing pharmacists and student pharmacists at various levels of training.

This project had several limitations. First, since student groups’ progression through the virtual patient case was self-paced, introduction of this method required flexibility in the classroom to account for groups finishing at various times. It also created the need for instructors to develop an awareness of how to coach students to be mindful of their time to complete the case without feeling rushed. Second, the impact of outside learning (in addition to the preclass video and virtual patient case) was not assessed. Thirdly, survey questions assessing the impact of the virtual patient case on study habits prior to the examination would have been helpful. Observationally, a number of students did access the virtual patient case again prior to the examination; however, this was a minority of the class. A final limitation to this project was the amount of resources and time required to create the preclass videos and the virtual patient case. Approximately 40 hours of time were dedicated to the design and development of this combined active-learning class. This included case design, video lecture composition and recording, and virtual patient construction. Nevertheless, the initial resource-intensive needs of this classroom structure will be offset by future use of the same video lecture and virtual patient cases, as only minor changes will be needed to reflect updated clinical material. Additionally, since the course videos and virtual patient cases can be easily updated, instructors may recruit pharmacy residents or graduate students to help evolve these activities over time. Implementation of virtual patient cases as the in-class active-learning strategy for other classes within this therapeutics course is planned.

SUMMARY

The combination of a preclass video lecture and an in-class virtual patient case was successfully developed and incorporated into a required therapeutics course at the University of Pittsburgh School of Pharmacy. This combined pedagogical strategy allowed for a customizable, personalized learning experience, while promoting higher-level learning. The availability of online video lectures allows students to individualize their learning pace and affords time to acquire on-demand knowledge of the foundational course material. Virtual patient technology provides a safe and comprehensive experience that engages students to assess patients, create treatment plans, and exercise problem-solving skills. Students embraced this technology as an effective way to understand and apply the materials from the video lecture. Furthermore, the instructors believe that this method promoted more student engagement and in-depth classroom discussion compared to that in the traditional, static case-based activity used in the previous year. The combination of preclass video lectures and a virtual patient case is a novel instructional design that can be successfully employed at other institutions and is well received by learners.