Abstract
Objective. The purpose of this study was to determine whether the addition of standardized patients or the addition of interprofessional student teams with standardized patients to the use of a simulated electronic health record improved student knowledge retention and perceptions.
Methods. This was a prospective cohort study assessing three cohorts of first-year student pharmacists in pharmacy skills laboratory activities that occurred in 2018, 2019, and 2021. The primary objective of the study was to compare knowledge retention of the case material between groups at one month. Each year, an element of simulated experience was added onto the previous year’s case. In 2018, students completed the case using only the electronic health record web application. In 2019, the previous year’s experience was combined with an objective structured clinical examination (OSCE) with standardized patients. In 2021, the 2019 experience was supplemented with student physician assistants. Case scores and student perceptions were also compared between groups.
Results. Of the 260 potential participants, 238 students were included in the primary analysis. Results showed that with the addition of interprofessional team-based care and standardized patients, significant improvement was demonstrated in knowledge retention assessments at one month. Mean knowledge retention assessment scores for the 2018, 2019, and 2021 groups were 63.8%, 71.7%, and 76.1%, respectively. Significant improvement was also found in student perceptions.
Conclusion. Adding standardized patients and interprofessional team-based care to a pharmacy skills laboratory that uses a simulated electronic health record significantly improved student knowledge retention and perceptions.
- simulation
- objective structured clinical examination
- interprofessional education
- electronic health record
- knowledge retention
INTRODUCTION
Research indicates that simulated learning techniques increase students’ confidence, critical thinking skills, and clinical performance in health care education.1 These activities are used to improve student learning and familiarity with clinical experiences. Many techniques can be implemented to increase the realism of learning experiences, including innovative use of technology, integration of mock patient cases and standardized patient interactions, and low- and high-fidelity mannequins.
Most often, educational research has focused on evaluating a single simulation technique when determining the effects on students’ ability to retain knowledge. In terms of retaining information, the use of standardized patients and low-fidelity simulation mannequins can improve long-term knowledge retention in student pharmacists.2,3 Another technique that has demonstrated benefits on student learning and perception is the use of simulated electronic health records.4-6 However, to our knowledge, no study in pharmacy or interprofessional education exists that couples a simulated EHR with live SPs or interprofessional students to create a realistic, team-based care learning environment. Thus, the purpose of this study was to determine whether supplementing patient cases that use simulated electronic health records with SP interactions or with SP interactions plus interprofessional student teams improves students’ knowledge retention, case performance, and student satisfaction related to the topic of the case study. Here, the case study topic was endocrine pharmacotherapy, and outcomes were assessed in first-year (P1) Doctor of Pharmacy (PharmD) students.
METHODS
This study was a prospective cohort study assessing three cohorts of P1 student pharmacists, namely P1 students in pharmacy practice skills courses in spring 2018, spring 2019, and spring 2021. In fall 2017, students in the PharmD program began using a simulated EHR throughout three didactic years to enhance their learning and prepare them for clinical rotations7; the EHR is used most readily throughout pharmacy practice skills laboratories. The study was funded by an internal research grant, and the study protocol was approved by the Campbell University Institutional Review Board under expedited review.
This three-iteration activity was developed as part of the pharmacy practice skills laboratory section that coincides with the endocrine therapeutics module in the pharmacy curriculum, which falls in the spring semester of students’ first professional year. Two patient cases created by pharmacy faculty were built into the EHR: a male patient with type 2 diabetes presenting to urgent care with symptoms of a urinary tract infection and a female patient with type 2 diabetes and hypothyroidism presenting for medication management after finding out she is pregnant. Case questions were developed for each patient case, and students answered the questions and submitted them as a portion of their pharmacy practice skills grade for that module. The same two cases and case questions were used for all three cohorts, but they were applied differently each year. Only the first case was included for analysis in this study.
Students were included in the study if they were enrolled in the P1 skills course (2018, 2019, 2021) or the physician assistant program (2021) (Figure 1). Stay-at-home orders due to the COVID-19 pandemic required canceling and reformatting the interprofessional OSCE originally scheduled for spring 2020; thus, the P1 students enrolled during spring 2020 were not included. Two days prior to the laboratory, all cohorts of students were given access to the patient cases via the EHR. Case questions were provided to students the day of the laboratory. In the first iteration, spring 2018, student pharmacists completed the patient cases using only the information available to them in the EHR. They had a 1.5-hour laboratory period to work up their patients and answer the case questions.
Diagram of a study comparing student pharmacists’ knowledge retention utilizing electronic health records vs simultaneously completing simulated experiences.
Figure Legend: Day 2 – Following the activity briefing, student pharmacists from each year reviewed a patient case in the simulated EHR related to endocrine pharmacotherapy. Spring 2018 – P1 PharmD students reviewed the EHR case only and submitted answers to the case assignment for grading. Spring 2019 – P1 PharmD students reviewed the EHR case for 30 and interacted with a standardized patient, then submitted answers to the case assignment. Spring 2021 – P1 PharmD students were paired with first-year PA students to complete the same activities as 2019. Only PharmD students were required to submit the case questions for grading. Following each activity, a 1-month knowledge retention assessment was distributed to the PharmD students along with perception surveys regarding the activity.
Abbreviations: P1 PharmD=first-year Doctor of Pharmacy; EHR=electronic health record; OSCE=objective structured clinical examination; PA=physician assistant; SP=standardized patients; IPE=interprofessional education.
In the second iteration, during spring 2019, student pharmacists were paired and given 10 minutes to conduct an interview during a SP interaction. The students then had 15 minutes to develop a plan for the SP and 5 minutes to counsel the patient on the plan and any medication changes. Students then had one hour to complete the questions for the SP case (analyzed) as well as the second patient case for which they only had the information in the EHR.
The SPs in this second iteration acted as the patient described in the electronic health record. In preparation for their role, the SPs underwent a one-hour training session with a pharmacy faculty member immediately prior to the event in spring 2019 and 2021 to answer any questions they had about the case scenario and script, which they received one week prior to the interaction. All SPs received payment through the internal grant funding and had served as SPs through the college previously. The purpose of the SP interaction was to gather any additional information needed from the patient and communicate the plan based on the patient’s presentation.
Finally, in spring 2021, a third iteration added an interprofessional component. Student pharmacists were randomly paired with one first-year student physician assistant in 1:1 or 2:1 groups and completed the SP interaction as described in the paragraph above. The endocrine module for the physician assistant curriculum and the pharmacy curriculum coincide during the spring semester, so an interprofessional education activity during this module was a natural fit. In this cohort, the student physician assistants performed the physical examination and worked with the student pharmacists to develop a plan. After the SP interaction, the student pharmacists still had the remainder of the 1.5-hour laboratory session to complete the questions for the SP case and the case that was solely in the EHR.
In addition to SP training and interaction, the grant was also used to fund the EHR web application for the physician assistant program. One week prior to the event, EHR training was provided to the student physician assistants. As in previous years, the EHR information was posted two days before the laboratory, along with the list of student groups. Students were instructed to discuss the two patients with their partner(s) at their convenience before the laboratory.
One month after completion of the activity, all cohorts of student pharmacists completed a 25-question multiple-choice knowledge retention test that contained questions on the disease states reviewed during the patient case activities. The test asked about guidelines and medication treatment of the endocrine disorders presented in the case studies but did not include specific questions about the patients themselves. At the end of the knowledge retention test, there was a short six-item survey (rated on a five-point Likert scale) on student perceptions of the learning techniques used in the activity. Students completed this survey during their 50-minute prelaboratory class at the end of the semester. All student pharmacists who were present in the three cohorts participated in the activity and were invited to take the knowledge retention test. For students who took the knowledge retention test, their grade on the test did not contribute to their laboratory grade.
The primary objective of the study was to assess one-month knowledge retention test scores across all three cohorts. Secondary end points included patient case scores across the cohorts and results of the student perception surveys included at the end of the knowledge retention tests. Statistical analysis was conducted using JMP Pro 14 (SAS Institute Inc). A value of p<.05 was considered statistically significant. Continuous data were assessed using analysis of variance (ANOVA), and ordinal variables were assessed using the Kruskal-Wallis test.
RESULTS
Of 260 total students enrolled among the three P1 pharmacy student cohorts, 257 (98.8%) participated in the case component of the study, and 238 (91.5%) completed the one-month knowledge retention assessment and perceptions survey. The available baseline characteristics were similar between groups, with the only significant difference being in pharmacy work experience, with 76.0%, 57.0%, and 69.1% of participants in the spring 2018, 2019, and 2021 cohorts, respectively, having such experience (p=.04). No significant differences were found in final course grades, type of work experience, or age (Table 1).
Characteristics of Student Pharmacists Participating in Each Group
For the primary endpoint of one-month knowledge retention test scores, significant improvement occurred with the addition of SPs and student physician assistants (2021 cohort) as compared to SPs (2019 cohort) and the use of the electronic health record web application alone (2018 cohort). The spring 2021 cohort achieved a mean score of 76.1% vs 71.7% and 63.8% for the 2019 and 2018 cohorts, respectively, p<.01 (Table 2).
Results of One-Month Knowledge Retention Tests and Patient Case Scores
Improvements in case scores, although not statistically significant, were also noted, as the cohorts progressed with a mean of 81.1%, 84.4%, and 85.4% in the spring 2018, 2019, and 2021 cohorts, respectively (p=.05). In the survey to assess student perceptions of the activity, most student pharmacists either agreed or strongly agreed that the use of simulated learning activities positively benefited learning. Additionally, student perceptions significantly improved in each cohort (with the addition of standardized patients and with the addition of standardized patients plus interprofessional team-based care) as reflected by their responses to four of the six survey questions (Table 3). Although not compared statistically, the student physician assistants participating with the spring 2021 cohort also reported high levels of satisfaction (Table 3).
Student Perceptions of the Simulated Learning Experience by Year
DISCUSSION
In this prospective cohort study, we demonstrated benefit in student knowledge retention and perception with simulated learning activities that included standardized patients or included standardized patients plus interprofessional team-based care. For activities including these components, a significant improvement was found for one-month knowledge retention (p<.01), and a trend was found toward improvement in case scores (p=.05). Additionally, student perception surveys indicated that increasing simulation with standardized patients and adding an interprofessional component improved most survey items (Table 3). This study highlights that implementing multiple simulated learning techniques may facilitate student learning and retention of knowledge.
Several reasons exist that may explain the increase in knowledge retention at varying levels of simulated realism. The process of collecting and assessing information applied in the first iteration of this study was well known to the students in all cohorts, as our institution commonly uses her for case studies. The use of technology for simulated patient cases and EHRs has been shown to improve students’ satisfaction, perception of preparedness, and performance.8-10 In a study from Smith and colleagues, student pharmacists reported that using a simulated EHR improved their perception of preparedness for clinical practice (p<.01).8 In that study, however, there was no performance improvement when comparing advanced pharmacy practice experience grades. Likewise, Bernaitis and colleagues demonstrated increased student satisfaction and significant improvement in final examination scores with the use of simulated technology.9 Similar to the results of our investigation, Bernaitis and colleagues did not find a significant difference in scores on a short-term assessment (midsemester examination) but did find significant improvement on end-of-semester examination scores (p<.01).9 Their study examined final-year pharmacy students using computer-based oncology cases designed with DecisionSim.9 Our study reiterates the finding that simulation technology is beneficial throughout the pharmacy didactic curriculum, with positive impacts in first- through final-year students.
Similarly, the use of SPs has been shown to improve performance in skill-related courses and knowledge retention in student pharmacists. Prior studies have demonstrated the benefit of using SPs for first-year pharmacy students learning inhaled device and insulin injection techniques.2,3 Like our study, the addition of SPs not only improved students’ skills and counseling, but it also aided their knowledge retention of the topic at one month.3
Additionally, results published by Simko and colleagues showed a significant increase in nursing and pharmacy students’ perception of the importance of interdisciplinary roles and knowledge when they participated in an interprofessional education course.10 A study by Maerten-Rivera and colleagues analyzed pharmacy students’ and physician assistant students’ confidence in applying the Pharmacists’ Patient Care Process to a patient case and their perceptions toward interprofessional collaboration.11 This study showed a significant increase in students’ confidence around performing the Pharmacists’ Patient Care Process but did not show changes in attitude toward interprofessional collaboration. However, both physician assistant and pharmacy students reported high scores in interprofessional perceptions both before and after the interprofessional education activity.11
Lastly, a study by Mitzel and colleagues paired pharmacy and physician assistant students in their final didactic year to collaborate in an interprofessional OSCE that reviewed orders for an inpatient case.12 The study sought to assess the OSCE’s impact on perceptions regarding interprofessional education and to determine whether simulated patient outcomes improved with interprofessional collaboration. Indeed, the study demonstrated higher scores on correct patient recommendations with the interprofessional education group and increased perceptions of the benefit of interprofessional patient care.12
Despite the relative abundance of literature surrounding classroom technology, SPs, and interprofessional education, the combination of these elements, which in our study created a more realistic case scenario and required additional preparation for students, has rarely been studied. In our experience, for students to adequately prepare for their interaction with the SP, they needed to prepare questions in advance and be ready to provide follow-up questions if needed. Additionally, when student physician assistants were added as an additional realistic element, the expectations that the student pharmacists would be viewed as the medication experts on the interprofessional team may have influenced their preparation. Preparation, however, may have only been a portion of what accounted for improved knowledge retention, as additional preparation did not result in significant differences in case grades (Table 2). Thus, the activities themselves may have contributed to a more memorable learning experience and resulted in improved knowledge retention. While the use of simulated EHR, SPs, and interprofessional education have been demonstrated to improve learning and student satisfaction individually, to our knowledge, our study is the first in pharmacy education that combines these strategies.
Some limitations to this study exist. Duplicating this model for interprofessional team-based care across other didactic curriculums and in other therapeutic modules may be limited, as the timing of disease states taught will not always coincide across different professional programs. Additionally, there was a one-year gap in the study period (spring 2020) due to the implementation of remote and virtual learning models during the COVID-19 pandemic and, thus, the inability of students and SPs to meet in person. Finally, since we were unable to obtain some data related to academic performance, such as grade point averages and Pharmacy College Admission Test results, we cannot rule out academic success as a confounding variable. Future studies should assess the impact of this model on retention of knowledge and performance on clinical rotations.
CONCLUSION
The addition of SPs and interprofessional team-based care to a pharmacy skills laboratory using a simulated EHR web application significantly improved student knowledge retention. These additions also demonstrated a trend toward improving immediate performance on cases and led to improved perceptions of the simulated learning environment and interprofessional education. While it may not be feasible to integrate interdisciplinary learners regularly, this study supports the benefit gained by introducing team-based care early in the pharmacy curriculum.
ACKNOWLEDGEMENTS
This study was funded by the Campbell University Interprofessional Education Internal Research Grant. The authors would also like to acknowledge our colleagues in the Physician Assistant Program at Campbell University and Dr Kim Kelly for their numerous contributions to the OSCE event and grant process.
- Received August 18, 2021.
- Accepted November 30, 2021.
- © 2022 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.