Virtual Versus Paper-based Cases in Reinforcing the Collect and Assess Elements of the Pharmacists’ Patient Care Process

INTRODUCTION

Doctor of pharmacy (PharmD) students are expected to interact with patients and other clinicians, gather detailed patient information needed to make appropriate assessments, and use clinical knowledge to make recommendations for patient care. Situational learning methods, such as the objective structured clinical examination (OSCE), place student pharmacists in simulated environments that afford them opportunities to employ assessment and decision-making skills in controlled scenarios.

Virtual patients involve computer-based programs that can be used in education to simulate real-life clinical scenarios.¹ According to the Association of American Medical Colleges (AAMC), the advantages of virtual patients include that they are easily accessible, are readily customizable, encompass multiple clinical aspects, and can be used to teach longitudinal and multidisciplinary care lessons.¹ The disadvantages of virtual patients include that they have limited fidelity, provide only limited physical interaction, and are more costly.¹

The use of virtual patients may promote active, patient-centered learning and improve student engagement and documentation.^2,3 A study by Benedict and colleagues evaluated the effectiveness of virtual patients cases in promoting self-directed learning for pharmacy students.⁴ For the cases in that study, students used a branched-narrative, decision-making model that students strongly supported and was just as effective as traditional paper-based cases.⁴

It is important to provide innovative, realistic, and engaging learning opportunities for students to process and implement the Pharmacists’ Patient Care Process (PPCP). The objectives of this study were to compare students’ performance on the collect and assess elements of the PPCP using virtual patients versus paper-based case scenarios, and to evaluate pharmacy students’ perceptions of each learning method.

METHODS

This randomized, crossover study was conducted in an ambulatory care elective course offered to third-year pharmacy (P3) students. The course structure required students to review one-hour long cases each week and submit a full subjective, objective, assessment, plan (SOAP) note. Each week the students would complete the same cases, however; one group would receive a paper-based patient case and the other a virtual patient case. Every week students were presented with a new case covering a different therapeutic topic.

Because the review and feedback provided for SOAP notes required an extensive amount of faculty members’ time, the course coordinators limited enrollment to 30 students. Once enrolled, students were randomized to complete either paper-based or virtual patient case scenarios in class for the first three weeks of the course, then they crossed over to complete case scenarios in the alternative format for the next three weeks.

Playposit (Playposit, Inc, Denver, CO), an e-learning platform, was used to develop the online interactive “branched pathway” virtual patient cases for this study. Initially, the intent was to have each virtual patient case structured to walk students through all the steps of the PPCP. However, after developing the cases in Playposit, we determined that assessing the plan, implement, and follow-up steps of the PPCP would not be practical using the virtual patient cases because of the need for multiple-choice questions to create branched pathways. Therefore, the comparisons in this study were limited to the collect and assess elements of the PPCP using virtual patient cases versus paper-based patient cases.

Students received a score for their performance on each of the collect and assess elements of the PPCP, in addition to a composite score for each case. At the end of the first three weeks, a survey was administered to students to evaluate their perceptions of the first patient case format they had used. The students were asked to rate their level of agreement with each survey item on a five-point Likert scale. The survey results were linked using a unique identifier for the purpose of conducting a matched-pair analysis after the conclusion of the study.

Following the first three weeks of the course, students crossed over and completed patient cases using the alternate format for the next three weeks. Students again received performance scores for each case. At the conclusion of this second three-week period they completed an additional survey to evaluate their perceptions of the patient case format they experienced.

Last, student focus groups were conducted at the conclusion of the study to obtain qualitative feedback regarding their perceptions of virtual versus paper-based patient cases in reinforcing the collect and assess elements of the PPCP and to solicit students’ suggestions for future improvement of the course. Each student focus group was moderated by an independent interviewer, who was trained by the study investigators on standardized interviewing techniques. The students’ responses were recorded, transcribed verbatim, and categorized into seven themes.

Student demographics were reported using frequency and percentages. Students’ performance scores on the collect and assess elements of the PPCP were reported using mean and standard deviation. A paired t test was used to compare students’ scores. Students’ agreement with the survey items was reported using median and interquartile range (IQR). Wilcoxon matched-pairs signed-rank test was used to compare students’ agreement with each survey item. For each of the focus group questions, thematic analysis was performed to inductively identify the main themes raised by participating students regarding their perceptions of paper-based and virtual patient cases, and to categorize them following the steps recognized for this type of analysis.⁵ This method allows themes to emerge from the data rather than imposing themes on the data before it is collected and analyzed.⁶ All statistical tests were two-sided and performed at a significance level of .05 using Stata, version 13.1 (StataCorp LLC, College Station, TX). This study was approved by the University of Texas at Tyler Institutional Review Board.

RESULTS

Twenty-seven students agreed to participate in the study. Thirteen students were randomized to first receive paper-based patient cases and then receive virtual patient cases and 14 were randomized to first receive virtual patient cases then receive paper-based patient cases. One of the 27 students did not complete the post-study survey. The demographics for study participants are presented in Table 1.

Of the 27 students enrolled in the study, four had technical difficulties (internet connection problems) with accessing the virtual patient cases and, as a result, their responses for the questions related to the collect element of the PPCP were not recorded. For the assess questions, performance was higher among students who completed paper-based patient cases (80.2%±21.7%) than among students who completed virtual patient cases (68.5%±21.4%), t=2.12, p=.04. For the collect questions (n=23), no significant difference in performance was found between students who completed paper-based patient cases (65.2%±37.6%) and those who completed virtual patient cases (62.0%±38.3%), t=.26, p=.80. When performance on the collect and assess questions was combined (n=23), no significant difference in performance was found between students who completed paper-based cases (78.7%±20.5%) and those who completed virtual patient cases (68.7%±24.3%), t=1.28, p=.21.

There were no significant differences between students’ perceptions of virtual patient cases and those of paper-based patient cases except on one item: “collecting necessary subjective and objective information about the patient.” For this item, the students favored using paper-based patient cases over virtual patient cases (Z=-2.38, p=.02). The survey items and students’ ratings are included in Table 2.

Of the 27 students in the study, 19 agreed to participate in three randomly assembled focus groups. Representative student quotes were selected and are presented in Table 3.

DISCUSSION

The students in this study achieved comparable scores on the collect and combined collect and assess questions of the PPCP regardless of whether they used virtual or paper-based patient cases. However, the students achieved significantly higher scores on the assess questions when completing paper-based cases than when completing virtual patient cases. Based on the focus group responses, paper-based patient cases seem to allow students to quickly identify relevant information and document patient responses without needing to listen to the patient as is imperative with virtual patient cases. Students commented during focus groups that paper-based cases were easier to use “because all the information is there.”

If students have difficulty identifying relevant patient information, their ability to appropriately assess the patient will be hindered. Although students expressed increased difficulty with the collect and asses elements of the assignment when using virtual patient cases, this should be explored further before changes are made as students will be required to actively listen and respond to patients in real time throughout their experiential rotations and in practice. Increased exposure to virtual patient cases may enhance students’ comfort level with this learning modality.

The results of this study align with those of Al-Dahir and colleagues when they evaluated the efficacy of faculty-led problem-based learning (PBL) versus virtual patient cases in training fourth-year pharmacy students.⁷ They found a significant increase in students’ test scores when both virtual patient cases and PBL were used, but the PBL group had higher post-experience test scores than the group that completed virtual patient cases, leading them to conclude that both were successful learning methods.⁷

Though the students in the focus groups stated that collecting information was harder with virtual patient cases, they expressed an appreciation for the learning method because it forced them to practice that skill. As one student stated, “In real-life you have to get the information out of the patient.” Student responses support that virtual patient cases provide students with an opportunity to practice empathic listening. The unanticipated finding of enhanced student self-reported empathy noted in this study is congruent with other investigations. For example, Curley and colleagues discuss how students reported that integration of virtual patient vignettes made it easier to develop empathy compared to reading about the patients.⁸ Furthermore, students’ responses indicated that virtual patient cases can improve critical thinking as one student stated, “Virtual cases promote critical thinking.” Students testified that “virtual cases are more fun and engaging...” Students also reported that “virtual cases force [you] to trust your knowledge” and “…would help [you] prepare for the OSCE.” Students suggested that clearer instructions should be given for completing virtual patient cases and that completing a combination of virtual and paper-based patient cases might be the ideal solution.

One of the limitations of this study was the impracticality of evaluating the plan, implement, and follow-up elements of the PPCP with virtual patient cases. Four students experienced internet connection issues and had trouble accessing the questions related to the collect element of the PPCP. Because the crossover design assumes "balance" in the two experiences being compared, it meant that all subjects should receive the same number of experiences. Though the four students did not have technical difficulties with the recording of collect responses for all virtual patient cases, they were removed from the comparisons involving questions related to the collect element because their missing data precluded the within-individual comparison that is inherent in the design. The missing data could result in bias of a magnitude that is difficult to predict.

Creating the virtual patient cases required a significant amount of faculty time (five to six hours per case). A study by Smith and colleagues employed a two-institution collaborative design where each school developed two to three interactive patient cases using two simulation programs, vpSim and DecisionSim (Kynectiv, Inc., Chadds Ford, PA), then shared them with the other institution.⁹ They reported that it took approximately 22 hours of faculty time for the initial development of the one case from their institution and only 1.2 hours to modify the shared case from the other institution.⁹ This collaborative approach may help lessen the demands and overcome the barriers of faculty time and resources required to create virtual patient cases.

The difficulty level of the questions included in the virtual patient cases using Playposit is limited. Because of the “branched pathways” method, specific videos can only be linked to other videos through multiple-choice questions. This limits the ability to provide more challenging questions, such as short answer questions or items that instruct the student to select all that apply. Finally, this study involved only a small number of students from one elective course at a single institution, which limits generalizability of study results.

CONCLUSION

The performance and perceptions of pharmacy students were similar in most areas when comparing virtual patient cases to paper-based patient cases as part of an ambulatory care elective course. However, students’ performance was better when paper-based patient cases were used for questions related to the assess element of the PPCP. Also, students perceived paper-based patient cases more favorably in terms of reinforcing the collect element of the PPCP. Future research in this area should involve larger number of students and cases, more advanced virtual patient case development technology that allows the assessment of all elements of the PPCP, evaluation of using virtual patient cases in a larger number of pharmacy schools, and longer periods of exposure to virtual patient cases.