Using Electronic Health Record Technology to Teach Inpatient Medication Order Verification to Pharmacy Students

INTRODUCTION

In an era of meaningful use technology, with 95% of critical access Medicare hospitals participating in 2016, exposure to electronic health record (EHR) technology is an essential component of graduating a practice-ready pharmacy student.^1-4 As part of the Health Information Technology for Economic and Clinical Health Act (HiTECH) of 2009, Medicare and Medicaid programs offer financial incentives to hospitals and providers participating in meaningful use technology. The overarching goal of meaningful use is to improve clinical outcomes by reducing medication errors through the use of EHR technology. The widespread adoption of EHR technology in health care since 2011 mandates that Doctor of Pharmacy (PharmD) students receive training in this area.^2,4 Unfortunately, many advanced pharmacy practice experiences (APPEs) limit student access to hospital-based EHR systems because of patient safety and privacy concerns.⁵ Although an encounter note written by a student may be entered into the system if a preceptor co-signature is included, unlicensed trainees are not permitted to verify medication orders. Because of this limitation, it is incumbent upon pharmacy schools to provide students with training in use of EHRs.

Studies describing the use of EHR technology in pharmacy curricula are limited and primarily focus on students collecting the necessary subjective and objective data to document a patient encounter in SOAP (subjective, objective, assessment, plan) note format.^6-13 In a survey of US schools and colleges of pharmacy, 37% of respondents were using an EHR system, with the majority using it for collecting patient information (86%) or writing SOAP notes (59%). However, that survey was limited by a low response rate of only 32% of pharmacy schools.¹⁴ In 2019, the American Association of Colleges of Pharmacy (AACP) Joint Task Force on Informatics published their recommended health informatics competencies, which endorses the use of EHR technology in pharmacy curricula.¹⁵ The EHR is a tool that could be used to facilitate achievement of one of the core entrustable professional activities (EPAs) for new pharmacy graduates, “Fulfill a medication order.”¹⁶ This technology may also be useful to meet the core domains for APPE readiness.^4,17

Our primary goal in selecting an EHR platform was to teach students how to process inpatient medication orders in a systematic way while simulating real-world technology. The primary objectives for this study were to measure student confidence in and assess performance when processing inpatient medication orders. The secondary objective was to determine student opinions regarding EHR technology.

METHODS

In the fall 2014 semester, the University of Maryland School of Pharmacy implemented EHR simulation in its health system-based skills laboratory course, Abilities Lab 3, to teach PharmD students how to conduct inpatient medication order verification. The course occurred in the first semester of the second professional (P2) year. This course has 50 medications assigned to it, which were linked to the curriculum in the concurrent therapeutics course. These medications served as the basis for the simulated medication orders. The course focused on health-system based skills, which was a critical factor in choosing an EHR platform.

Prior to the implementation of EHR technology, pharmacy students were given six medication orders on a paper-based simulated medication order form and told that the order set contained at least two prescribing errors. Specific error types were discussed with students during facilitated practice weeks, but no systematic process for error detection was provided. There were two practice weeks prior to the midterm practical examination, and students were permitted to use drug information resources during practice weeks and for the midterm practical examination. This assessment was worth 60 points, with each of the six medication orders worth 10 points each. For each order, answers were graded as “all or none”. While this experience provided students with practice in identifying prescribing errors in inpatient medication orders, they were not being prepared to use the EHR technology they would encounter during APPEs. After vetting several products, the platform EHR Go! (Archetype Innovations, LLC, Duluth, MN) was specifically chosen for three reasons: it was a web-based academic version of a hospital-based EHR that allowed students individual access to simulated patient data; it eliminated concerns about students accessing or compromising protected patient data while still providing them with an authentic interface; and it allowed instructors to customize orders to intentionally create medication errors.

Students were introduced to the EHR technology during an orientation session in the form of a scavenger hunt designed to familiarize students with various parts of a medical record. Course managers, who also served as weekly instructors, developed additional materials with step-by-step instructions for processing medication orders. A recorded session on best practices for inpatient medication order processing using medication safety principles and The Joint Commission (TJC) Standards was posted online for students to view prior to starting practice sessions.

Two practice laboratory sessions were held in which each student was individually assigned one unique patient with three inpatient orders to process in the patient’s profile. Similar to the paper version of this activity, medications for these orders came from the course medication list and were randomly assigned to students. For each order, the student was asked to determine if the order was acceptable as written and could be verified (“verify”) or if the order required an intervention and had to be “flagged” back to the provider. Potential reasons for flagging an order included: no allergy assessment had been performed or the patient had a documented allergy to a prescribed medication, the order contained a dose range or interval range, the wrong route of administration had been ordered, the wrong dose of the medication had been ordered, the wrong formulation had been ordered, the indication for administering the medication was missing or wrong, and the order set contained clinically relevant duplicate therapy. These medication errors were randomly distributed among orders. Students were instructed that an allergy assessment was required for every patient. If “no allergy assessment” had been documented in the patient’s EHR, they were required to ask a facilitator if the patient had any allergies. Medication orders with intentional errors triggered alerts from the built-in clinical decision support system in EHR Go! Alerts varied in applicability from logical to meaningless, and students were taught to analyze each alert to determine its merit. Student interpretation of an alert was assessed upon interaction with a facilitator. An assessment of inpatient order processing skills followed the two practice sessions.

Similar to the assignments during the two practice weeks, the midterm practical examination consisted of one assigned patient with three medication orders and up to three medication errors. Students had to decide to verify or flag each order. Students recorded their answers on the customized scantron form, which contained their name, the name of the assigned patient, the medication orders, the option to verify or flag the order, and the reason for flagging the order if applicable. The assessment was worth 90 points (out of a total of 250 points), with 30 points assigned to each of the three orders. Students received full credit for either correctly identifying that an individual order should be verified or correctly flagging an order and providing the correct rationale for doing so. Partial credit was given for identifying that an order should be flagged but selecting the wrong reason for flagging the order. Error types and medications used were similar between the paper version (2013) and the EHR version (2014) of this activity.

Pre- and post-implementation surveys were administered during scheduled class time to P2 pharmacy students enrolled in the course. Data from the pre- and post-implementation surveys were paired based on responses known only to the student (birth month and last four digits of their mobile phone number). Survey items were grouped into sections, including background information, confidence in order processing skills, and opinions regarding the use of EHR technology. Pre- and post-implementation surveys contained identical questions except that the post survey also asked about future uses for this technology and contained two additional open-ended questions soliciting constructive feedback regarding the EHR activities.

Confidence questions were evaluated on a five-point Likert scale with one indicating least confident and five indicating most confident. Opinions regarding EHR technology were assessed on a five-point Likert scale with one indicating strongly disagree and five indicating strongly agree. Open-ended questions were used to solicit student input on how the course could be improved.

For analysis purposes, Likert scale points one and two and points four and five were combined after a series of McNemar tests were used to conduct multiple post-hoc pairwise comparisons to determine if there were significant differences between Likert responses on survey items. The Wilcoxon signed rank sum test was used to measure the significance of median differences between the pre- and post-intervention test. Performance scores from the year prior to EHR implementation and the year of EHR implementation were analyzed by comparing the mean scores between years. Tukey test for multiple comparisons was used to determine statistical significance. Analyses were performed using SAS, version 9.4 (SAS Institute, Cary, NC) This study was granted exempt status from the University of Maryland, Baltimore Institutional Review Board.

RESULTS

The mean order verification score when normalized to 100% scale for 2013 students who completed the practical examination using a paper-based medication order form was 69 (n=120) and for 2014 students who completed the practical examination using the customized scantron was 83 (n=158). The difference between the means was significant (p<.05). Out of 158 students, 108 (68%) had pre- and post-intervention survey data that could be paired. Less than one quarter (24%) of students had hospital experience prior to the EHR activity. More than half (56%) of students reported no previous exposure to EHR technology. For the medication order processing questions, confidence levels doubled and, in some cases, tripled after EHR course implementation (Table 1). Results for all medication order processing questions were significant (p<.001). Student perception of the importance of EHR technology in pharmacy education was high to start with and did not change significantly after taking this course. However, significantly less students (82% and 66% pre and post survey respectively, p=.004) reported that the EHR activity would prepare them for IPPEs and APPEs (Table 2). The post-intervention survey indicated that approximately 60% of students would like to see the EHR continue to be used in ABL 3 and 40% of students would like to see the EHR used in other courses.

DISCUSSION

This study is unique in that it assessed the real-world skill of inpatient medication order verification. Implementing EHR technology allowed us to simulate a practice environment by assigning individual patients to each student and manipulating medication orders to create errors. This study demonstrated an improvement in students’ performance score when grades on the EHR activity were compared to those on the paper version completed by the previous cohort of students. Many studies have examined the use of the EHR on student performance related to the skills of documenting a patient encounter, performing medication reconciliation, and writing SOAP notes.^6-8,13 To date, only one study has assessed the skill of inpatient medication order verification. Metzger and colleagues described their simulation activity during an introductory pharmacy practice experience (IPPE).⁵ In that study, groups of 12-16 students were provided access to a single patient record to complete order verification and medication reconciliation within the training domain of a hospital EHR system. While use of a hospital-based EHR training system provided an authentic environment for students, lack of individual student access, large group sizes, and use of a single patient limited student exposure to verifying multiple order types.

The results of our study demonstrated that students felt more confident in their ability to process orders in each of the five areas queried: selecting the correct patient, identifying a clinically relevant drug interaction, identifying inappropriate duplicate therapy, detecting a drug allergy, and resolving range orders. Additionally, our results show that students did not perceive a benefit in using the EHR to help them prepare for practice experiences. Studies have reported that from 73% to 95% of students felt more prepared for APPEs after using an EHR.^9,10 However, other studies yielded conflicting results when a more rigorous statistical analysis was performed.^6,8 Traditionally, students obtain exposure to EHR technology during IPPEs and APPEs. Because the Abilities Lab 3 course occurs prior to students beginning these experiences, P2 students may lack perspective on the pervasiveness of EHR technology in the experiential setting. Furthermore, P2 students may not yet associate medication order verification with the ability to detect a medication order error or appreciate this skill as one of the unique contributions that a pharmacist brings to the health care team.

Potential limitations of this study include that the paper version of the activity to which the EHR version was compared lacked pre-laboratory materials and standardized error types making a true comparison of student performance in 2013 vs 2014 difficult. Furthermore, we did not track the number of times students viewed the pre-laboratory materials to determine how many students looked at these materials in preparation for the EHR activity. Although the medications and error types used in the tests administered in 2013 and 2014 were similar, they were not exactly the same given the dynamic nature of our skills-based courses. Students were permitted to use drug information resources in completing the paper version of this activity but not when completing the EHR version. We would expect the use of resources on the paper version to result in better performance, but our findings demonstrated that this was not the case. However, a baseline assessment of differences among the cohorts was not performed. An analysis of performance by error type(s) was not conducted but could help to inform future directions for this activity.

The evaluations reflected students’ desire to have additional practice weeks and more exposure to different types of medication orders. Based upon this feedback, future directions for this activity include increasing the number of weekly practice sessions to allow practice with more patients, additional medication orders per patient, and distribution of all medication errors types among students prior to the midterm practical assessment. Exposure to all medication error types during practice weeks will ensure that students have encountered medication orders similar to those on the assessment. Additionally, providing the list and assessing knowledge of course required medications at the beginning of the semester will encourage students to become familiar with these medications throughout skills-based laboratory and therapeutics related activities.

CONCLUSION

In our health system–based skills laboratory course we successfully implemented an EHR simulation platform to teach pharmacy students how to verify inpatient medication orders in a systematic way that mimicked real-world processes. This EHR platform provided a realistic look and feel and enhanced students’ order verification skills compared to our previous paper version. Exposure to EHR technology increased students’ confidence in their ability to process inpatient medication orders and improved their performance scores. However, after performing the medication order verification activity, fewer students felt that using EHR technology better prepared them for IPPEs and APPEs. These findings support the continued use of an EHR platform in skills-based activities to prepare students for future practice as the ability to use EHR technology will be required.