Abstract
Objective. To integrate a blended-learning model into a two-course patient assessment sequence in a doctor of pharmacy (PharmD) program and to assess the academic performance and perceptions of enrolled students.
Design. A blended-learning model consisting of a flipped classroom format was integrated into a patient assessment (PA) course sequence. Course grades of students in the blended-learning (intervention) and traditional-classroom (control) groups were compared. A survey was administered to assess student perceptions.
Assessment. The mean numeric grades of students in the intervention group were higher than those of students in the traditional group (PA1 course: 92.2±3.1 vs 90.0±4.3; and PA2 course: 90.3±4.9 vs 85.8±4.2). Eighty-six percent of the students in the intervention group agreed that the instructional methodologies used in this course facilitated understanding of the material.
Conclusion. The blended-learning model was associated with improved academic performance and was well-received by students.
INTRODUCTION
Patient assessment, defined as the “evaluation of patient function and dysfunction through the systematic gathering of objective (physical assessment and laboratory data interpretation) and subjective (patient interview) data important to the provision of care,” is viewed by accrediting agencies as an essential component of a contemporary, high-quality pharmacy education.1-3 Patient assessment skills are an integral aspect of pharmacy practice, and will become even more vital as the profession transitions to focus more on medication therapy management. To effectively prepare students for high-level practice, this topic must be incorporated at an appropriate breadth and depth in the required doctor of pharmacy (PharmD) curricula.1-3 As of 2007, 96% of pharmacy schools in the United States taught patient assessment skills.4 Across these schools, patient assessment was either taught through standalone courses or integrated into parts of other courses. Eighty-six percent used a laboratory approach to teach this material, while 70% used lectures. Unfortunately, student attention and retention of information presented via lectures have been shown to decline steadily after the first 10 minutes.5-7 This issue can be mitigated by integrating active-learning strategies such as problem-based learning (PBL), team-based learning (TBL), and case-based learning into the classroom.8
A flipped-classroom, in which students gain necessary knowledge before class (eg, through recorded lectures) and apply that knowledge under the guidance of an instructor during class (eg, through active learning), can be used to maximize student-teacher interaction in the classroom.9 This contrasts with the traditional classroom model where instruction is provided during class time and student-teacher interaction is typically minimal. The use of a flipped classroom format that integrates TBL and case-based learning for teaching patient assessment skills to pharmacy students has not been previously described. The objective of this study was threefold: to introduce a blended-learning educational model that consisted of a flipped-classroom format that integrated TBL and case-based learning to teach patient assessment skills in a PharmD program; to assess the academic performance of students enrolled in this model as compared to that of students enrolled in the traditional classroom iteration of the course; and to assess student perceptions of using this model to teach patient assessment skills.
DESIGN
The first-professional year course sequence in patient assessment was composed of two one-credit-hour courses, Patient Assessment 1 and 2 (PA1 and PA2). This sequence was scheduled as a practical laboratory and allotted two weekly three-hour time slots. In the traditional classroom iteration of the course sequence, there were 30 to 60 students in each class. Students were taught using the following methods: lecture (40%), a skills laboratory (30%), case-based learning (25%), simulation (2.5%), and independent learning (2.5%). During the 2014-2015 academic year, the course was redesigned to incorporate a blended-learning model using a flipped-classroom approach. Teaching faculty members dedicated approximately eight hours per week to developing course content, recording online videos, creating TBL activities, etc.
Within this model, students were educated using: independent study through a series of 10-15 minute online videos (eg, prerecorded, online, voice-over PowerPoint presentations) prior to class (40%); case-based learning (30%); a skills laboratory (15%); TBL, consisting of an individual readiness assurance test (IRAT) and a group readiness assurance test (GRAT) (10%); and, discussion (5%) (Figure 1). A list of the topics discussed in the course sequence and the active-learning strategies used for each topic is provided in Table 1. Online videos were designed to be no longer than 10-15 minutes for two purposes: to divide the preclass work into manageable segments that could be completed at the learner’s convenience; and, to create a video archive that was easily searchable for students throughout the semester (ie, prior to class, prior to the final examination, etc).5-7,10,11 Camtasia Studio, Version 8.4 (TechSmith Okemos, Michigan) was used by instructors to record the online videos. In total, 43 faculty-produced videos and five external videos were used in the course sequence. The average length of each faculty-produced video and the average total length of the faculty-produced videos per class were 12.0 and 22.4 minutes, respectively.
Active-learning Strategies Used to Teach Patient Assessment Topics in a Blended-learning Model
Comparison of the PA1 and PA2 Traditional Classroom (Figure 1A) and Blended-learning (Figure 1B) Models. Abbreviations: IRAT = Individual Readiness Assurance Test; GRAT = Group Readiness Assurance Test; HR = heart rate; RR = respiratory rate; BP = blood pressure; PA1 = patient assessment 1; PA2 = patient assessment 2.
In addition to lecture content, each video contained at least one self-assessment question or case intended to encourage reflection on the part of the learner during what otherwise would be a passive activity. The first hour of the scheduled three-hour slot for each class was typically reserved for students to complete the online videos. In class, one instructor facilitated TBL and case-based learning classes. During TBL activities, instructors facilitated class through administration of the TBL and through a post-IRAT/GRAT group discussion focusing on key points of the TBL. During case-based learning activities, instructors circulated throughout the classroom (with the goal of touching base with each group at least once) and facilitated wrap-up group discussions focusing on the key points of each case. In the clinical skills laboratory, students worked in pairs under the guidance of an instructor or adjunct instructor. To enable one-on-one instruction, four pharmacy residents (adjunct instructors) assisted with the skills sessions.
First-year pharmacy (P1) students at the University at Buffalo School of Pharmacy and Pharmaceutical Sciences (UB SPPS) who were enrolled in PA1 and/or PA2 during the 2014-2015 academic year and second-year pharmacy (P2) students at UB SPPS who were enrolled in PA1 and/or PA2 during the 2013-2014 academic year were eligible for inclusion in the study. This population was chosen because they were either enrolled in the blended-learning iteration of the PA1 or PA2 courses (P1 students) at the time of the study (intervention group) or were enrolled in the traditional classroom iteration of the PA1 or PA2 courses (P2 students) in the prior year (control group).
To assess academic performance, an analysis of course grades was performed following completion of the 2014-2015 academic year. Final numeric and letter grades and academic performance on the final written examination and clinical skills examination were compared between the intervention and control groups for each course. The final written examinations consisted of 20 short-answer questions from a previously validated question bank, were unchanged between the two offerings of the course sequence (the final examination is not returned to students), and were similar in question rigor. The PA1 clinical skills examination consisted of vital sign collection (ie, heart rate, respiratory rate, and blood pressure). Collection of vital signs was graded using an internally validated five-point rubric (for heart rate and respiratory rate) and 10-point rubric (for blood pressure); students were assessed on both technique and accuracy. The PA2 clinical skills examination consisted of identification of basic medical conditions from a case and image, identification of seizure type from a case and video, identification of an abnormal heart rhythm from an electrocardiogram strip, and performance of basic physical examination techniques (eg, diabetic foot examination, cranial nerve examination, etc). Physical examination techniques were graded using an internally validated 10-point rubric. Student and group performance on TBL and case-based learning in the intervention group was also collected.
To assess student perceptions of this model, an 18-question survey instrument for the control group and a 35-question survey instrument for the intervention group were developed. The survey instruments were pretested by a focus group consisting of six third-year pharmacy (P3) students who were familiar with the course sequence. The focus group was asked to assess the clarity of questions and the overall flow and length of the survey instruments, and whether they believed the instruments addressed the study’s survey-related objectives. Minor revisions were made to the survey instruments following the pre-testing phase. The paper survey instrument was distributed to all students enrolled or previously enrolled in the patient assessment sequence at the end of both the spring (December 2014) and fall semesters (May 2015), respectively. The intervention group was surveyed at the end of each course (to mitigate recall bias) and the control group was surveyed one year after they completed each course. Participation in the survey was voluntary and anonymous for all students.
Data obtained via the de-identified course gradebook and via the survey instrument were numerically coded and entered into an Excel spreadsheet. The survey response rate and distribution of responses was determined in order to generalize the survey findings. Data were analyzed using Excel and MYSTAT 12, Version 12.02.00 (SYSTAT Software, Inc., San Jose, CA). Survey responses were analyzed by comparing the degree of agreement between groups using the Fisher’s exact test. Due to the small number of students receiving low letter grades, letter grades were compared by grouping those at the lower end of the grading scale to achieve expected values of at least 5 to allow an R by C contingency table analysis. Numerical grades were compared within groups using the paired t test and between groups using the Mann-Whitney U test. Statistical significance was determined at the level of p<.05.
ASSESSMENT AND EVALUATION
This project was deemed exempt by the University at Buffalo Institutional Review Board. In total, 132 and 126 students were enrolled in the blended-learning and traditional classroom iterations of the PA1 course, and 131 and 122 students were enrolled in the blended-learning and traditional classroom iterations of the PA2 course. Internal data indicated that there was overlap in the mid-50% PCAT scores at the time of admission between the groups: control group scores = 335-410 vs intervention group scores = 324-408. The mean GPA at the time of admission was also similar: control group GPA = 3.5 vs intervention group GPA = 3.4. Comparison of between-group academic performance assessing the percent of students earning a grade of A or A- in the other 13 first-professional year courses revealed the following: 9 of 13 were statistically similar, 0 of 13 favored the intervention group, and 4 of 13 favored the control group (p<.05). The overall response rate for the PA1 and PA2 course surveys was 89.5% and 87.9%, respectively. Based on the survey response rate, the demographics of the survey respondents can be assumed to be reflective of the study population as a whole. Complete demographic data for those students who returned a completed or partially completed survey are presented in Table 2.
Demographics of Patient Assessment 1 Course and Patient Assessment 2 Course
Students in the intervention group attained a higher final mean numeric course grade compared with students in the control group: PA1, 92.2% vs 90.0% (p<.001); and PA2, 90.3% vs 85.8% (p<.001) (Table 3). There was a significant difference in the letter grade distribution for both PA1 and PA2 favoring the intervention group (p<.001), and a higher percentage of students in the intervention group (vs those in the control group) earned a grade of A or A- in the PA1 course: 79.2% vs 65.1% (p<.05); and the PA2 course: 64.1% vs 18.9% (p<.001) (Table 3). The PA1 and PA2 intervention groups performed better than the control group on the final examination (p<.001) (Table 4). On the PA1 clinical skills examination, students in the intervention group performed better than students in the control group (p<.01); however, no difference in scores was observed on the PA2 clinical skills examination (Table 4). Within the PA1 and PA2 clinical skills examinations, students enrolled in the intervention group performed better on blood pressure (BP) assessment: PA1: 94.4% vs 89.7% (p<.01) and PA2: 94.1% vs 90.8% (p<.01); however, performance on heart rate (p=.059) and respiratory rate (p=.96) assessment, which was only included as part of the PA1 examination, was similar (Table 4). Students in the intervention group performed well on in-class active-learning activities: GRAT scores exceeded 95%, and as expected, were higher than IRAT scores (p<.001); and student groups scored better than 90% on case-based learning assignments (Table 5).
Academic Performance Between the Blended-learning and Traditional Classroom Models
Examination Scores Between the Blended-learning and Traditional Classroom Models, Mean (SD)
Active-learning Scores Within the Blended-learning Model, Mean (SD)
Students in both the control and intervention groups provided an overall course rating for PA1 and PA2 of above average (median score on a five-point Likert scale=4). Ninety-three percent of students in the intervention group of the PA1 course, compared with 77% of students in the control group of PA1 agreed or strongly agreed that the instructional methodologies used in this course facilitated understanding of the material (p<.05). Within PA2, 80% and 81% of students in the intervention and control groups were in agreement with this statement, a difference that was not significant. At the conclusion of the course, student confidence in their ability to apply the knowledge and skills developed in PA1 and PA2 was similar between groups (median Likert score=4).
The amount of preclass work was rated as being “just about right” by approximately 70% of students in the intervention and control groups for both the PA1 and PA2 courses, with the majority of the remainder of each class indicating the workload was “too much.” Analytics for video usage by the intervention group indicated that the mean time students spent viewing videos was 8.5 minutes per video and 16.0 minutes per class. Despite the similar amount of preclass work assigned to each of the groups, the reported completion rate of preclass work varied considerably. In the control group for the PA1 and PA2 courses, 80% of students reported completing the assigned readings prior to coming to class less or equal to 80% of the time. In contrast, 95% of students in the intervention group of the PA1 course and 86% percent of students in the intervention group of the PA2 course reported that they viewed the online videos prior to coming to class more than 80% of the time (p<.001, intervention vs control group). Furthermore, 47% of students in the PA1 course and 33% of students in the PA2 course indicated “very often” or “always” rewatching the online videos later in the semester.
These data were supported by the video usage analytics, which indicated a mean of 156 views per video, equating to 1.2 views per student. These data align with the perceived value of the assignments: a total of 86% and 90% of students in the PA1 and PA2 intervention groups, respectively, compared with 37% and 42% of students in the PA1 and PA2 control groups, respectively, agreed or strongly agreed that they needed to prepare for class in order to be successful in this course (p<.001). The majority of students in the intervention group were in agreement that the online videos contributed to their learning, indicating that they preferred viewing online videos prior to class over attending a traditional in-class lecture (Table 6). Furthermore, the PA1 intervention group rated the contribution of online videos to their learning higher than the PA1 control group rated in-class lectures (median score of 5 vs 4, p=.01). However, there was no difference seen between students’ scores in the PA2 course (median score of 4 for both traditional and intervention groups). Despite this, the majority of students in the intervention group from both the PA1 and PA2 courses were in agreement that learning key foundational content prior to coming to class prepared them for TBL, case-based learning, and practical laboratory activities; and enhanced their learning of course material during class (Table 6).
Student Perceptions of the Blended-learning Model
Both case-based learning and TBL were well-received in the intervention group. More than 80% of students in the intervention group from the PA1 and PA2 courses agreed or strongly agreed that the use of TBL and case-based learning contributed to their learning. The degree of agreement pertaining to the impact of case-based learning on learning was similar between the intervention and control groups. More than 80% of students in the intervention group from the PA1 and PA2 courses agreed or strongly agreed that the use of TBL and case-based learning effectively facilitated discussion of concepts during class.
DISCUSSION
Active-learning is a teaching approach that actively engages students in the learning process while also stimulating higher-order thinking, problem solving, and critical analysis. Active-learning can facilitate a more thorough understanding of material and may help students better retain what they have learned.8 The Accreditation Council for Pharmacy Education (ACPE) 2011 accreditation standards stipulate that faculty members employ active-learning strategies in their teaching, and the ACPE 2016 accreditation standards cite that curricular expectations must emphasize active-learning pedagogy.1,2 Colleges and schools of pharmacy must therefore embrace active-learning and identify ways to implement these teaching approaches within their respective PharmD program, including but not limited to courses that focus on patient assessment. Fortunately, the case-centric nature of medicine and the interprofessional way in which healthcare is administered lends itself well to case-based instruction using a team approach. In this paper, we have described a blended-learning (educational) model consisting of a flipped-classroom format that integrated TBL and case-based learning to teach patient assessment skills that can be implemented by PharmD programs across the academy.
Integration of a flipped-classroom (blended-learning model) may be accomplished through a variety of means. One way to do this involves transitioning the traditional in-class, unidirectional lecture containing foundational content to online delivery of a single or multiple prerecorded 10- to 15-minute videos, that can be viewed by learners on their own schedule, at their own pace, and as many times as needed in order to master the content prior to class. A meta-analysis published by the US Department of Education in 2010 revealed that online learning can be enhanced by giving learners control of their interactions with media and prompting learner reflection (eg, through the design of learning modules that trigger learner activity or reflection and self-monitoring of understanding).12 In our blended-learning model, online videos were designed to be no longer than 10 to 15 minutes, with each online video containing at least one self-assessment question or case intended to encourage reflection on the part of the learner during what could otherwise have been a passive activity.12 This facilitated students applying basic content before coming to class.
The flipped-classroom approach has been successfully introduced into higher education, specifically into graduate, medical, nursing, and pharmacy education.11-20 In pharmacy education, this novel teaching method has been inculcated into courses in pharmacotherapeutics, self-care, and pharmaceutics.17-21 In our Patient Assessment course sequence, this model was well received overall: students perceived that the online videos contributed to their learning, prepared them for TBL and case-based learning, and that viewing the videos was necessary for them to contribute to in-class group activities and succeed in the clinical skills laboratory. Although students were in agreement that the videos prepared them for the clinical skills laboratory in the PA1 course, there was less agreement that this was the case for the clinical skills laboratory in the PA2 course. This difference in perception may relate to the higher-level physical examination techniques taught in the second course of this sequence. The addition of a brief 10-minute demonstration of physical examination technique(s) at the start of each clinical skills laboratory session may help to mitigate this issue.
In previously published reports, instructors have used prerecorded videos of lectures while others have relied on preclass readings.17-21 Data from our survey, which revealed that few students completed required readings while the majority viewed the online videos, suggests that the latter format is more attractive to students and therefore may offer an advantage over preclass readings. However, although the majority of students did indicate preference for online videos over traditional lecture, the level of disagreement with this survey item was numerically higher than that for other survey items. This likely relates to time constraints with concomitant courses and examinations, which is supported by data indicating one-quarter to one-third of students felt the course workload was “too much.” Furthermore, the reason for the higher preclass assignment completion rate may have been that students enrolled in the blended-learning model were held accountable for completion of preclass activities better than were those enrolled in the traditional classroom model (eg, through TBL, periodic quizzes, etc).
The 2010 meta-analysis cited above reported that older students who were engaged in online education performed modestly better than those who were engaged in traditional face-to-face instruction; however, instruction that combined online and face-to-face teaching methodologies (ie, blended-learning) was found to be even more advantageous.12 Furthermore, the flipped-classroom has been found to improve academic performance, and has been favorably received by students when online videos are used as the preclass preparatory modality.17-20 Our study revealed similar results, with students enrolled in the blended-learning model scoring better on the final examination and on aspects of the clinical skills examination, and achieving a higher letter and numeric course grade. Interestingly, one-third to one-half of students indicated rewatching the online videos “very often” or “always” at some point later in the semester. This may have helped them prepare for the final examination, including but not limited to revisiting topics covered early in the semester and by improving their understanding of more difficult topics covered throughout the semester. Item analysis suggested that the rigor of the final examination and clinical skills examination were similar for the intervention and control groups, and the course demographics were similar, together suggesting it was the educational model (as opposed to outside factors) that impacted academic performance.
Internal data indicated that academically, the intervention group was at least similar (and possibly inferior) to the control group. There was overlap in the mid-50% PCAT scores at the time of admission and the mean GPA at the time of admission was also similar. Furthermore, comparison of between-group academic performance in the other first-professional year courses revealed that the intervention group performed the same in nine other first-year courses, worse in four other first-year courses, and better in none of the other first-year courses. Together, these data further supports the above-stated conclusion that it was the educational model (as opposed to outside factors) that impacted academic performance.
The implementation of a flipped-classroom approach freed up classroom time for integration of active-learning strategies (eg, TBL, PBL, case-based learning, etc). Emphasis in the classroom was therefore on application and learning rather than on rote memorization of facts. The active-learning instructional methods that were integrated in this model included TBL and case-based learning. TBL, which holds students accountable for their preclass preparation through individual and group quizzes, thereby setting the stage for productive in-class active learning, has been shown to be a successful teaching and learning strategy in pharmacy education.8,22-24
In each course, students performed better on GRATs than IRATs, suggesting peer-to-peer teaching occurred. Case-based learning, which is a more general active-learning strategy, uses real-life and/or simulated cases to facilitate application of foundational knowledge.8 Case-based learning was used in both iterations of the course and the cases themselves were similar in rigor; however, in the blended-learning model, instructors were able to dedicate more in-class time to the cases and expand the number of cases covered. In our study, student-groups scored well on case-based learning assignments. The survey indicated that both TBL and case-based learning were well received by students. A goal of the blended-learning model was to stimulate higher-level thinking. Students in the blended-learning model were confident in their ability to apply the knowledge and skills developed in the course, which suggests this goal was met; however, no difference was seen between those enrolled in the traditional and blended learning models, suggesting the model itself may not have been a contributing factor.
Faculty members and students have expressed concerns regarding the increased time commitment that is associated with the flipped-classroom, blended-learning model.18,19,21 Therefore, careful consideration and monitoring of both faculty and student workload is of the utmost importance when introducing a blended-learning model, as described herein. When designing our course sequence, we reserved the first hour of the 3-hour time slot for students to complete the online videos. This was done to preemptively address anticipated student concerns regarding the increase in preclass workload, as has been raised by other investigators who have introduced the flipped-classroom into their teaching.19,21 Video usage analytics indicated that the mean view duration was 16.0 minutes per class, suggesting they were able to complete the preclass work in the time allocated. The time spent by students watching the videos was substantially less than the average video length of 22.4 minutes, suggesting they did not complete the videos, viewed the videos in separate sittings, or viewed the videos at an accelerated speed. The results indicate that most students found the course workload acceptable. Although the strategy of blocking off time for students to complete the preclass videos may be conducive to a laboratory-based course (as described here), it may not be a viable option for courses that are allotted less classroom time.
In addition to student time, an increase in faculty time has been reported during the implementation phase of the flipped-classroom, blended-learning model; however, the amount of time required during subsequent years has been reported to be similar to the time spent preparing for a traditional lecture.18 Patient assessment lends itself well to the flipped-classroom because the content is relatively static, minimizing workload in subsequent years. As our model moves into its second year of existence, this appears to be the case with our course sequence. During the implementation phase (2014-2015), teaching faculty members dedicated approximately eight hours per week to developing course content, recording online videos, creating TBL activities, etc. However, during the 2015-2016 academic year, and in theory, during subsequent years, we estimate that approximately 1 hour per week of faculty preparation time will be dedicated to this course. In our model, transitioning classroom didactic content online has resulted in a reduction of the time that faculty members spend in the classroom from an average of 5.1 hours per week to 3.9 hours per week. Based on these data, we project the breakeven point for faculty time commitment will occur during the 2019-2020 academic year (ie, in year 6 of the project), after which time the faculty members teaching within this team-taught course sequence will recoup approximately 18 hours over the course of a 15-week semester.
The major limitation of this study was the risk for recall bias among survey respondents, particularly with those students who were enrolled in the traditional-classroom course model because the survey was administered to this group one year after they completed the course. Second, the control group was from a second-year class, while the intervention group was a first-year class having their first experience in pharmacy school, which could have influenced the survey-results in an undetermined way. Third, it is inherently difficulty to determine if the improvement in student performance with the blended-learning model was the result of the transition of didactic content online, the integration of more active-learning, or a combination of the two. Last, although the graded final examination was not returned to the control group, that group may have communicated information to the intervention group regarding what to expect on the final examination, thereby influencing that group’s performance on the examination. Despite these limitations, implementation of this model was associated with improved academic performance by and favorable reviews from students. Future research should focus on long-term retention of learned content with the blended-learning model.
SUMMARY
The introduction of a blended-learning educational model composed of a flipped-classroom format that integrated TBL and case-based learning to teach patient assessment skills to first-year pharmacy students was associated with improved academic performance and was well-received by enrolled students. Other colleges and schools of pharmacy should work to integrate active learning strategies, such as those described here, into their curricula.
ACKNOWLEDGMENTS
The authors thank the following faculty members for their contributions to this course sequence: Carolyn Hempel, clinical assistant professor, UB SPPS; Calvin Meaney, clinical assistant professor, UB SPPS; Nicholas Norgard, clinical assistant professor, UB SPPS; Joshua Sawyer, clinical assistant professor, UB SPPS; Kelly Sustakoski, academic coordinator, UB SPPS; Robert Wahler, clinical assistant professor, UB SPPS; Mark Wrobel, clinical assistant professor, UB SPPS.
- Received January 28, 2016.
- Accepted July 8, 2016.
- © 2016 American Association of Colleges of Pharmacy
REFERENCES
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.