Abstract
Objective. To describe and assess the effectiveness of an ambulatory care introductory pharmacy practice experience (IPPE) module in medication safety monitoring that was conducted remotely for first- and second-year pharmacy students in collaboration with a Department of Veterans Affairs (VA) health care system.
Methods. A module in medication safety was developed and piloted as part of a required, one-unit IPPE in ambulatory care. The module encompassed three sessions of the 15-week course. Working remotely at the school under the direct supervision of VA clinical pharmacists, students accessed the electronic health records (EHRs) of veterans for whom methotrexate, sulfasalazine, or mineralocorticoid receptor antagonists had been prescribed as such patients require ongoing laboratory monitoring to screen for adverse drug effects. After reviewing the record, students ordered laboratory tests under the prescriber’s name, documented notes in the patient’s EHR, and sent laboratory reminder letters to patients. Students completed a brief survey at the beginning and end of the course. Differences in survey responses were compared using the Wilcoxon signed rank test.
Results. Fifty-eight students reviewed 148 patient records, including non-VA electronic records, and ordered laboratory tests for 79 patients. On the follow-up survey at course completion, students reported significant changes in their ability to monitor laboratory tests, use an actual (non-simulated) EHR to perform patient care, and explain the role of a clinical dashboard to conduct population health activities.
Conclusion. Remotely conducting an ambulatory care IPPE course in which pharmacy students used an actual EHR to participate in medication safety monitoring for ambulatory patients expanded the pharmacy school’s ability to provide crucial practice experience. Other pharmacy schools struggling with logistical challenges and limited resources should explore this approach to fulfilling the experiential education requirements for pharmacy students.
- introductory pharmacy practice experience
- medication safety monitoring
- ambulatory care
- remote access to electronic health records
INTRODUCTION
The Accreditation Council for Pharmacy Education (ACPE) Standards 2016 state that introductory pharmacy practice experiences (IPPEs) should expose students to contemporary, frequently encountered practice models and direct patient care activities.1 Barriers to offering IPPEs include a lack of time for precepting, burden of student orientation to the practice site, geographical distance between the school and practice site, identifying a mutually beneficial time for students to conduct IPPE activities, and student housing issues.2 A qualitative analysis of common concerns regarding pharmacy experiential education revealed that site capacity for student placements continues to be the foremost concern, along with the challenges of preceptor development and procedures for onboarding students.3
The role of pharmacists in the ambulatory care setting is evolving. Opportunities for significant growth of pharmacist-physician collaborations exist in the primary care setting.4 From 2013 to 2019, the total number of postgraduate year 2 pharmacy residency programs in ambulatory care increased from 65 to 175, highlighting the growth in this area.5 Proceedings from the American Society of Health-System Pharmacists 2014 Ambulatory Care Conference and Summit highlighted the importance of pharmacist participation as members of patient care teams for ambulatory patients, with responsibilities for patient and population outcomes.6 Introductory pharmacy practice experiences (IPPEs) that prepare students for advanced ambulatory care practice experiences and ultimately a career in ambulatory care practice are desirable; however, space and availability of preceptors at health system clinics may limit accessibility. Some pharmacy schools have integrated simulated practice experiences into their experiential curricula to introduce students to ambulatory care practice and help meet the number of training hours required by the ACPE.7 Limitations to simulated practice experiences include a maximum of 60 hours applicable to graduation requirements and limited direct patient care experience.
The Department of Veterans Affairs (VA) Clinical Pharmacy Practice Office designates approximately 10% of primary care clinical pharmacy specialists’ time each week for conducting population health activities.8 It is important to expose students to this essential component of clinical care. The VA Northern California Health Care System (NCHCS) uses population management dashboards to identify veterans who require clinical interventions.9 The medication safety dashboard facilitates laboratory monitoring of specific medications to screen for possible adverse drug reactions. The medications included on the dashboard are determined by pharmacists and physicians. When required laboratory tests results are needed, pharmacists can order the tests and notify patients. These activities do not need to be conducted at the health care facility. A search of the literature did not identify published manuscripts about IPPEs conducted remotely from a health care facility. This paper describes the partnership between a VA health care system and the Thomas J. Long School of Pharmacy whereby pharmacy students participated in medication safety monitoring at the university using remote access to electronic health records (EHRs).
METHODS
The Thomas J. Long School of Pharmacy offers an accelerated Doctor of Pharmacy (PharmD) program. Each semester is approximately 15 weeks in length. A collaboration between the VA Northern California Health Care System (VANCHCS) and the university was established to provide IPPEs that supported VA population management initiatives.
The VANCHCS is part of an integrated network of VA facilities that have developed clinical dashboards to support population management activities. The Medication Safety scorecard reports the number of patients receiving specific medications and when each patient has key laboratory monitoring parameters checked. Following discussion with VA pharmacists who oversee the medication safety dashboard, pharmacy school faculty made the focus of the IPPE medication safety module the laboratory monitoring of veterans prescribed methotrexate, sulfasalazine, and mineralocorticoid receptor antagonists. Methotrexate and sulfasalazine require close monitoring for bone marrow, liver, and kidney toxicity, while mineralocorticoid receptor antagonists, such as spironolactone, require close monitoring for hyperkalemia. The plan was for pharmacy students, under the direct supervision of VA clinical pharmacists, to participate in population health management for veterans within the VANCHCS.
A required one-unit pass/fail IPPE in ambulatory care was approved by the school’s curriculum committee and faculty. In order to accommodate a large class size, the school divided each class into roughly thirds and provided the ambulatory care experiential training over three semesters. The first required IPPE was offered to 58 first-year students in April 2018. Approximately 10 students were assigned to each of six weekly sections that were two hours in length. A medication safety module was developed as part of the course (Figure 1). The population health experiences were conducted at the university and precepted by VA clinical pharmacists. Some preceptors were full-time VA employees and some were university faculty hired by the VA for eight hours per week to perform student precepting for the population health program. Each session was precepted by a minimum of two preceptors.
Overview of the Introductory Pharmacy Practice Experience Curriculum Delineating the Modules and Sessions of the Ambulatory Care Experiences
The process for student on-boarding and course orientation is provided in Appendix 1. Students were granted remote access privileges to VA’s EHR and used personally owned laptops. Students were oriented to Joint Legacy Viewer (JLV), a browser-based clinical application that provides an integrated, read-only display of health data from the Department of Defense, VA, and its private sector partners. Students were taught how to review a patient medical record and JLV to determine whether laboratory tests were still needed. If recent laboratory results conducted outside of VANCHCS were found in JLV, students were shown how to enter results into the VA’s EHR. If no laboratory results were found, the students were shown how to enter laboratory orders for the prescriber to sign and enter a patient laboratory reminder letter into the EHR. Letters were printed remotely to a VANCHCS pharmacy where a technician mailed the notification letter to the patient. Student activity was documented in a progress note within the EHR. Each week during the three-week medication safety module, every student was assigned up to five patients. “How to” documents were available for students to download (Figure 2). Students performed their medical record review and drafted a progress note, then reviewed their work one-on-one with the preceptor before finalizing the plan, entering laboratory orders, or signing the progress note. Students were deemed competent when they demonstrated their ability to perform the activities listed in Appendix 1. Between the four-hour orientation during the first two weeks of the course and the three-week patient care module, each student earned 10 hours of medication safety IPPE hours.
Overview of Patient Assessment Workflow for Students Conducting Medication Safety Monitoring
The number of records reviewed and laboratory tests entered by students was tracked. For every patient assigned, the preceptor documented on a data collection sheet whether the laboratory test was no longer needed, the laboratory results were found in JLV, or the student ordered the laboratory tests and sent a laboratory reminder letter to the patient.
Students completed a brief assessment regarding their experience performing population health activities on the first day of the course (baseline) and again at the end. General items relating to the medication safety module included experience with laboratory monitoring, use of a clinical dashboard, and experience with a non-simulated EHR. Responses to questions used a Likert scale and were converted to a numeric value (eg, strongly disagree=1 through strongly agree=5). If a student did not answer a question either at baseline or post-IPPE, the student’s data was removed for that question. Changes in student responses from baseline were evaluated using the Wilcoxon signed rank test. Statistics were performed using SPSS for Windows, version 25.0 (IBM Corp, Armonk, NY). A p value <.05 was considered statistically significant. The VANCHCS Research Service reviewed this project and determined no research approvals were required.
RESULTS
Fifty-seven (98.2%) of 58 students completed the surveys. Comparing baseline and course-completion responses, students reported significant changes in their ability to explain the role of a clinical dashboard to conduct population health activities and their experience monitoring laboratory tests and using a non-simulated EHR (Table 1).
Pharmacy Students’ Responses to a Questionnaire Administered on the First and Final Day of a Medication Safety Introductory Practice Experience That Used Remote Access to Veterans Administration Health Records (N=58)
During the three-week medication safety experience, each student spent approximately six hours performing patient care activities. A total of 148 patient records were reviewed. Seventy-nine (53.4%) patients had a total of 160 laboratory tests ordered for provider co-signature. Laboratory tests included a comprehensive metabolic panel to assess for liver enzymes and serum creatinine, a complete blood count to assess white blood cells and platelet counts and a basic metabolic panel to assess serum potassium. Eleven (7.5%) patients had laboratory test results identified in JLV; results were documented in the VA’s EHR. This obviated the need for the laboratory test. The remaining 58 patients reviewed did not require an intervention. These patients were either recently contacted by another health care team member to obtain laboratory tests, had laboratory results pending or recently completed, or had not refilled their prescription in over four months and no longer required laboratory monitoring.
DISCUSSION
Site capacity is a significant challenge in experiential pharmacy education. In a survey of experiential sites, Gibson and colleagues found that 56.7% of experiential sites that responded had conducted between one and five IPPEs in the previous 12 months and approximately 6% had conducted over 25 IPPEs.2 While most of the sites had conducted five or fewer IPPEs in an entire year, our model using remote access enabled 58 students to complete a medication safety module as part of an ambulatory care IPPE during one semester. Over 60% of responders to the survey conducted by Gibson and colleagues stated that IPPE site capacity will stay the same until 2020, which may pose a future challenge as student pharmacist enrollment and demand for IPPE sites grow.2,10 Additionally, simulated IPPEs and those that consist primarily of shadowing a practitioner offer students limited direct patient care experiences.
Three primary challenges to increasing the quality or quantity of IPPE are time constraints, burden of orientation, and geographic proximity of the site to the college of pharmacy.2 The course design described in this manuscript provides an innovative solution to all three of those challenges. Furthermore, this model avoids common logistical issues such as inadequate space or resources at the clinical site. There is a paucity of published literature describing the possible utility of remote access to EHRs in pharmacy education. Previous literature in pharmacy, medical, and nursing education described the use of training EHR platforms with standardized or simulated patients.7,11-14 Our unique model uses remote access to a health system’s live EHR and real patient data. As such, it is essential to ensure that the school’s state board of pharmacy requirements for supervision are met. All patient care activities are reviewed by a pharmacist prior to documenting them in the EHR or ordering laboratory tests. The IPPE students gain experience performing required laboratory monitoring to ensure medication safety in ambulatory patients. Furthermore, IPPE students are able to enter patient care notes and send laboratory reminder letters while interfacing with a live EHR system. The pilot project involved the implementation of a three-week module in medication safety following a four-hour orientation that provided 10 non-simulated IPPE hours. Schools considering similar modules could increase the number of hours dedicated to performing medication safety activities to maximize the return on investment for time spent conducting student onboarding and orientation.
Standards 2016 defines the duration of IPPE as a minimum of 300 hours with at least 150 hours balanced between community and institutional health-system settings. Offering remote ambulatory care experiences like the one described here from the school should count toward institutional health-system setting hours.1 Other pharmacy schools can explore collaborative models with health-systems like the one described here to address site capacity or geographic limitations to existing ambulatory care IPPEs. However, implementing this approach is not without challenges. The school of pharmacy must commit to a partnership with the VA facility and provide time in the curriculum, appropriate classroom space to facilitate this type of work, and administrative and technical support (eg, adequate wi-fi bandwidth). Several services within the VANCHCS provided significant support, including primary care, human resources management, pharmacy, and chief of staff’s office. Onboarding processes must be carefully coordinated and timelines must be met. Hosting the IPPE on campus may cause students to think of the IPPE as another class and not a professional experience. To address this, it became essential for us to set expectations early, such as having students dress and behave professionally as if they were at the practice site. In-class instruction can be challenging if students rely only on their laptops. Laptop screens have limited space, making it difficult to switch between “how-to” documents and VA patient records. In-class demonstration using dual projection to screens in front of the class enabled students to view “how-to” documents with key instructions and a VA EHR simultaneously. Challenges with remote access included technology failures and students forgetting passwords. Finally, a commitment to quality improvement is needed. Prerecorded didactic coursework must be updated. Progress note templates should be reviewed for needed enhancements. An evaluation of health-system needs should be conducted so that students’ population health management activities reflect current health-system priorities for ambulatory care practice. In this model, VA pharmacists prioritized student activities. For medication safety, VA pharmacists facilitate laboratory monitoring, but the ultimate responsibility remains with the prescriber of the medication. Thus, outcomes associated with the number of laboratory tests completed and whether the results are normal or abnormal are not a part of this experience.
Expansion of pharmacy services in ambulatory care practice is essential, as is the need to prepare a cadre of ambulatory care pharmacy practitioners to fill this role. Pharmacy leaders offered essential strategies for meeting this need, three of which are addressed with the innovative approach to IPPE described here.15 One strategy is to implement virtual and mobile health care for ambulatory patients. Using remote access to EHRs to conduct population health activities supports virtual care by pharmacy students. A second strategy is to leverage predictive modeling to facilitate population health management. Veterans Affairs, Kaiser Permanente, and Indian Health Services effectively use analytics to facilitate population health activities, enabling pharmacists to target those patients most likely to benefit from interventions. Exposing students to clinical dashboards that facilitate population health management is valuable. Following the IPPE, students were better able to explain the role of a clinical dashboard to support patient care. A third strategy is to offer expanded training within primary and ambulatory care to students and residents such that students and residents contribute to the health care system at a level appropriate for their professional development. Exposing students to ambulatory care practice during their introductory practice experiences may help to meet future workforce needs.
CONCLUSION
A collaboration between a VA health care system and a school of pharmacy using a medication safety clinical dashboard and remote access to the EHR met many objectives. The VA offered pharmacy students an experiential learning opportunity without needing additional workspace and computers at the medical facility. Students were provided real-world introductory experiences with laboratory monitoring to ensure medication safety that involved navigating the VA’s EHR and documenting progress notes. This study showed that pharmacy students can perform patient care activities and earn IPPE hours without traveling from the school to a health care facility. The VA is a national health care system; thus, this model could be reproduced by other schools of pharmacy in various locations.
ACKNOWLEDGEMENTS
The authors thank Robert Malmstrom, PharmD, and Vanessa Vaupel, PharmD, for their invaluable support of student involvement with the medication safety initiative.
Appendix 1. Student Onboarding, Orientation and Competency Assessment Items

- Received April 23, 2019.
- Accepted July 14, 2020.
- © 2020 American Association of Colleges of Pharmacy