Abstract
Objective. To assess second year Doctor of Pharmacy students’ academic performance in and perceptions of a heart failure (HF) virtual patient simulation used in a required pharmacotherapy course.
Methods. A heart failure virtual patient simulation was created to augment heart failure pharmacotherapy course material at the University at Buffalo School of Pharmacy and Pharmaceutical Sciences in the fall of 2019. This was a retrospective, pre-post observational cohort study. The primary objective was to compare student performance on heart failure pharmacotherapy examination questions in a cohort of students who completed a virtual patient simulation in 2019 compared to a control cohort who completed a paper-based case activity in 2018. Student perceptions of the simulation experience were assessed via electronic survey.
Results. Students completed either the virtual patient simulation (n=122) or a paper-based case activity (n=123). Overall, the proportion of correctly answered heart failure pharmacotherapy examination questions was 83.3% in the virtual simulation group compared to 79.2% in the paper-based case group. Survey results indicated that students would prefer that the virtual patient simulation be incorporated in the pharmacotherapy curriculum.
Conclusion. Use of a heart failure virtual patient simulation was associated with improved examination performance and was well received by students.
INTRODUCTION
In health care, a simulation-based learning experience is defined as “an array of structured activities that represent actual or potential situations in education and practice. These activities allow participants to develop or enhance their knowledge, skills, and attitudes, or to analyze and respond to realistic situations in a simulated environment.” 1 Simulation has been used in pharmacy education and across the health care disciplines using various modalities, including live human simulations, human patient simulators that have realistic human physiologic responses, game-based simulations, and computer-based simulations, including virtual patients. 2-5 Virtual patients are computer programs that simulate clinical scenarios in an authentic manner, allowing the learner to serve as the health care professional and make clinical decisions in a realistic yet innocuous environment. 6 These simulations can promote problem-solving, encourage self-directed learning, and enhance clinical skills. 3, 7, 8
One disease area that may lend itself well to simulation in a learning environment is heart failure, which is a commonly encountered disease state projected to affect nearly eight million people in the United States by 2030. 9 Because pharmacists have been shown optimize heart failure treatment for patients by increasing medication adherence, decreasing readmission rates, preventing adverse drug events, and decreasing the costs of care, this disease state is ideal for using in virtual patient simulations for pharmacy students. 10 At the University at Buffalo School of Pharmacy and Pharmaceutical Sciences (UB SPPS), pharmacotherapeutic heart failure content is taught in the fall of the second year of the Doctor of Pharmacy (PharmD) program. Because this content is covered fairly early in the curriculum, a virtual patient simulation was created to provide immediate and individualized feedback to better facilitate learning. The purpose of this study was to evaluate the utility and student perceptions of this technology.
METHODS
This was a retrospective study that evaluated second year PharmD students’ performance on heart failure examination questions after they took part in a heart failure virtual patient simulation in a required three-credit-hour pharmacotherapy course. Four didactic lectures were devoted to heart failure content delivered by the same instructor in 2018 (control cohort, no virtual patient simulation) and 2019 (intervention cohort, with virtual patient simulation) using the same content and learning objectives. The learning objectives were as follows: define heart failure with reduced ejection fraction (HFrEF) and preserved ejection fraction (HFpEF); explain the pathophysiology of heart failure and how it relates to drug therapy targets; recommend an optimal treatment regimen for a patient with HFrEF to decrease morbidity and mortality, including discontinuation and titration of therapies; recommend optimal treatment across the continuum of heart failure care (ie, a patient at risk of developing heart failure, a patient with chronic heart failure, and a patient with acutely decompensated heart failure; and monitor the safety and efficacy of recommended heart failure therapies. Lectures were delivered live in a lecture hall setting but were recorded and posted on the course site for students wishing to review the lecture material again. Table 1 compares the relevant classroom information between the control and intervention cohorts. The simulation addressed all heart failure learning objectives but was specifically designed to enhance the fourth learning objective, recommend optimal treatment across the continuum of heart failure care.
Comparison of Heart Failure Content in a Second Year Pharmacotherapeutics Course
As a supplement to the lecture material, the instructor assigned clinical casework. In 2018 (control cohort), this comprised “static” paper-based cases (one day of patient subjective/objective information from hospital admission) with short-answer questions and an optional case debrief recitation. In 2019 (intervention cohort), this was replaced with an evolving, branched-linear virtual patient simulation. The simulation followed the clinical time course of a single heart failure patient through the continuum of heart failure care (initial myocardial infarction, ADHF emergency department presentation, and advanced heart failure hospital admission) with associated clinical questions. The simulation contained 13 multiple-choice questions with specific feedback provided to the student based on their response. If they answered incorrectly, students were routed back to the question to try again and were only allowed to progress in the simulation once each question was answered correctly. Students were allowed to go through the simulation as many times as desired. The simulation was created using Adobe Captivate, 2019 release (Adobe Systems, Inc), which had a one-time cost of $399 (individual educator license) and was uploaded as a Sharable Content Object Reference Model (SCORM) package in the course learning management system, Blackboard (Blackboard, Inc). Two students completing an elective advanced pharmacy practice experience (APPE) created the simulation over two weeks with a faculty mentor who spent about 20 hours reviewing and editing their work.
The primary objective was to determine whether the intervention resulted in improved student performance on heart failure-related questions in course examinations, specifically on an individual examination within the course assessing heart failure (section examination) and on the cumulative final examination (final examination). Thus, we compared the proportion of heart failure questions that students answered correctly in 2018 (control) to the proportion of heart failure questions that students answered correctly in 2019 (intervention). Question performance was also assessed across Bloom’s taxonomy levels to evaluate differences in lower-order (knowledge/comprehension) vs higher-order (application/synthesis) questions as assigned a priori by the primary investigator (heart failure instructor) and confirmed by a second instructor in the course. 11 Examination questions were kept the same each year, but schedule differences resulted in four additional heart failure questions in 2019. See Table 1 for the question breakdown between years.
The secondary objective was to assess student perceptions of the simulation from a survey administered after the heart failure section examination. The survey, created in SurveyMonkey (Momentive Global Inc), contained questions pertaining to student perception of the simulation experience using a five-point Likert scale and an open-ended free-text question asking for any additional comments, particularly strengths and weaknesses of the simulation experience. Survey completion was voluntary and anonymous and did not impact student examinations or final course grades.
Differences in the proportion of heart failure examination questions answered correctly between the years were compared using the chi-square test; comparisons were also made across Bloom’s taxonomy levels. Descriptive statistics were used to summarize the student perception survey data, and qualitative content analysis was performed by the primary investigator to assess themes in the comments section of the survey. A value of p < .05 was set as significant. All statistical analyses were completed using Prism, version 8.0 (GraphPad Software, LLC). The institutional review board (IRB) deemed this project exempt from review.
RESULTS
There were 123 students enrolled in the course in 2018 and 122 in 2019. All students completed either the paper-based heart failure case or the virtual patient simulation. Examination performance is detailed in Table 2. The proportion of correctly answered heart failure pharmacotherapy questions (section examination and final examination questions) was 79% in 2018 compared to 83% in 2019, p=.001.
Students’ Performance on Pharmacotherapeutic Heart Failure Examination Questions
Eighty-five students completed the simulation perception survey (70% completion rate). Overall, student perceptions of the simulation were positive, as evidenced by Likert-scale responses (Table 3). Thematic content analysis of positive comments indicated student preference for this type of learning, with students asking for more simulation opportunities (n=9; eg, “more topics utilizing simulation technology,” “more questions on the simulation,” “more cases made into simulations for additional practice”). The other predominant positive theme from survey comments surrounded “feedback” (n=14; eg, “given immediately,” “given for right and wrong answers,” “opportunity to reattempt questions after feedback”). Content analysis revealed negative comments pertaining to technical issues (pop-up blockers, n=6; internet browser used, n=4).
Resultsa From the Survey on Students’ Perceptions of the Virtual Patient Simulation Experience
DISCUSSION
In this study, we evaluated the use of a heart failure virtual patient simulation in a required pharmacotherapy course taken in the second year. Use of the simulation was associated with improved performance on heart failure pharmacotherapy examination questions, and student perceptions indicated that students prefer this style of learning. Examination performance was improved on both lower- and higher-order questions in the simulation cohort, though this improvement did not meet statistical significance on the higher-order questions. Performance on the heart failure questions on the final examination was improved in 2019, but not significantly, which may be because this examination predominately included higher-order heart failure questions (n = 5/7, 71%), which was similar to the proportion of higher-order heart failure questions on the final examination in 2018 (n = 4/5, 80%). An application-rich activity should theoretically improve performance on higher-order questions, which virtual simulation technology has previously been shown to improve, 12 but it is unclear whether virtual patient simulations result in superior outcomes compared to nonvirtual interventions across health professions education. 3 It is possible that our investigation did not find a significant difference in performance on the final exam and on the higher-order questions between the control and intervention groups because of the low number of questions per subgroup, or it could have been because the control course in 2018 also offered an application-rich (though nonvirtual) learning activity via the paper cases that also enhanced student learning.
The time required to create the simulation might be viewed as an impediment to implementation. While the simulation did take approximately 20 hours to create in its inaugural year, that resulted in a product that was ready for repeated use. It saved faculty time compared to traditional paper-based activities, as completion and performance were automatically recorded in the course grade book. Also, additional teaching time was not required to review the case, as the feedback was provided immediately to the student at the time of clinical decision-making. Paper-based activities do not allow for immediate feedback, and this benefit of immediate feedback was a frequent theme of the comments in the survey.
The use of computer-based simulations depicting some heart failure content in pharmacy education have been evaluated previously. In a study by Douglass and colleagues, 10 computerized clinical patient simulations with multiple comorbidities (including HF) were created to improve drug therapy problem-solving. 13 Pre-post simulation assessments showed advanced clinical skills; however, performance on heart failure content was not significantly improved. Al-Dahir and colleagues used a virtual patient simulation (including some heart failure management) and demonstrated improved posttest scores, but these improvements were less than those seen via problem-based learning. 14 In another study, Bindoff and colleagues compared a virtual community pharmacy game to a paper-based alternative in which students had to recognize inappropriate beta blocker titration in an heart failure patient. 15 Student performance improved to a greater degree with the game compared to the paper alternative, similar to the findings of our investigation. Assessment of student perception in each study was favorable overall, as was the simulation in our investigation. Yet, because heart failure was not the sole focus of these other computer-based simulations, direct comparisons to our results are challenging. To our knowledge, we present the first evaluation of a virtual patient simulation specifically designed to enhance heart failure knowledge in pharmacy education.
There are limitations to our study. Year-to-year changes in instruction and assessment could have resulted in improved heart failure question performance; however, steps were taken to minimize this. Examinations were kept nearly the same from 2018 to 2019 and were administered using ExamSoft (ExamSoft Worldwide, LLC) in a secure, proctored environment. Questions and answer choices were randomized, and student review of the examination material occurred in an identical proctored environment (examination review). In the examination review, the course instructor reviewed the examination and answered student questions, but students were not allowed to have anything at their desks to prevent them from transcribing or recording content. The same instructor taught heart failure both years using the same content, but in the simulation cohort (2019), the instructor taught using a whiteboard/visualizer rather than PowerPoint for all didactic lectures, whereas this method was used for only one of four lectures in 2018. The instructor made this change due to student feedback from the 2018 course evaluations, which stated that students preferred this teaching style. Similarly, on course evaluations in 2019, many students commented on the effectiveness of this method compared to PowerPoint; thus, this teaching method may have contributed to the improved performance. The course averages were similar between the 2018 and 2019 cohorts (81% vs 82%, respectively), so it is less likely that the overall academic aptitude of students in the 2019 cohort played a role, but no other baseline comparisons were performed (eg, two- vs four-year experience before pharmacy school). The survey response rate (∼70%) bears mentioning. Respondents may have been higher course performers, which could have resulted in high Likert scores on the survey questions. Given the anonymous nature of the survey, student examination performance could not be tied back to survey responses to evaluate a response bias of this nature.
CONCLUSION
The use of a heart failure virtual patient simulation was associated with improved examination performance and was well received by student users. Schools might consider adopting a similar active-learning virtual experience to enhance their curricula, particularly in a remote-learning environment.
- Received March 22, 2021.
- Accepted September 8, 2021.
- © 2022 American Association of Colleges of Pharmacy
REFERENCES
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.