Abstract
Objectives. To implement and evaluate the effects of a simulated hospital pharmacy module using an electronic medical record on student confidence and abilities to perform hospital pharmacist duties.
Design. A module was developed that simulated typical hospital pharmacist tasks. Learning activities were modified based upon student feedback and instructor assessment.
Assessments. Ninety-seven percent of respondents reported full-time hospital internship experience and 72% had electronic medical record experience prior to completing the module. Mean scores on confidence with performing typical hospital pharmacist tasks significantly increased from the pre-module survey to the post-module survey from 1.5-2.9 (low comfort/confidence) to 2.0-3.4 (moderate comfort/confidence). Course assessments confirmed student achievement of covered competencies.
Conclusions. A simulated hospital pharmacy module improved pharmacy students’ hospital practice skills and their perceived comfort and confidence in completing the typical duties of a hospital pharmacist.
INTRODUCTION
A 2010 AACP report described the Core Performance Domains and Abilities that should be achieved by students prior to beginning advanced pharmacy practice experiences (APPEs).1 This report highlights 11 core performance domains, including patient safety, accurate dispensing of medications, basic patient assessment, identification and assessment of drug-related problems, and general communication abilities, and described several possible activities that might be used by colleges and schools of pharmacy to document student achievement of these competencies. Identification of activities that can be used to document this achievement is increasingly important as programs of pharmacy complete programmatic assessment mapping activities. Many of the competencies described can be taught within a pharmacy practice laboratory course.1
The need for strong preparation in hospital pharmacy practice has also been described by professional pharmacy organizations. A 2010 publication issued by a joint taskforce from the American Society of Health-System Pharmacists and ACPE concluded that graduates possessed strong clinical knowledge but were not universally prepared to enter practice or residency training in the hospital setting.2 The report identified a comprehensive list of competencies needed for hospital/health system pharmacy practice, including: describing relevant standards that hospitals use to ensure safe and appropriate medication use; performing medication reconciliation during transitions of care; documenting appropriate therapeutic recommendations related to medication therapy; and triaging multiple patient priorities in times of high activity and workload.
The hallmark of the 6-year doctor of pharmacy (PharmD) program at Northeastern University School of Pharmacy is student participation in full-time introductory pharmacy practice experiences (IPPE) through the university’s cooperative education program. All students are required to complete three 4-month long full-time cooperative education/IPPE experiences, one of which is required to be in an institutional pharmacy. Usually, students complete the required institutional experience in a hospital pharmacy department. Competencies developed for the institutional experience direct students to learn to fulfill medication orders; develop communication and problem-solving skills, professional behaviors, and professional ethics; and observe and appreciate hospital pharmacy operations.
As part of PharmD training, students must also be introduced to electronic health records or electronic medical records as such technology is increasingly used to improve the accessibility of patient information and the overall safety of healthcare delivery.3 To address several of these issues, and to expand students’ preparation for required APPEs in community pharmacy, hospital/institutional pharmacy, and ambulatory care, skills laboratory faculty members within the department of pharmacy practice wanted to provide a comprehensive laboratory experience where students could integrate the skills required for these practice settings. The Pharmaceutical Care Skills Laboratory course was redesigned to include activities organized into 3 modules: community, hospital, and ambulatory care.
At the same time the course was redesigned, the school of pharmacy needed to update the pharmacy system used in the pharmacy practice laboratory. In 2010, Medical Information Technology, Inc., donated the MEDITECH software package (Medical Information Technology, Inc., Westwood, Massachusetts) to the university for use in all health professions programs. As a result, the Bouvé College of Health Sciences began a plan to use the software in each of the programs (including in pharmacy, nursing, physical therapy, and physician assistant) and as a platform for interprofessional collaboration. In the school of pharmacy, the MEDITECH software was integrated into the Pharmaceutical Care Skills Laboratory course to use as prescription dispensing software and to create electronic medical records to present patient case information in the hospital pharmacy module.
The objective of this project was to develop and implement a simulated hospital pharmacy module in the Pharmaceutical Care Skills Laboratory course and to evaluate the effects of completing module activities on students’ confidence and ability to perform tasks typical of a pharmacist. We hypothesized that this module would ensure an opportunity for discussion of topics important to hospital practice such as the pharmacist's role in transition of care; increase students’ exposure to electronic hospital information management systems and medical records; and increase students’ confidence in their ability to complete activities typical of hospital pharmacy practice.
DESIGN
The Pharmaceutical Care Skills Laboratory course series was comprised of two 0.5-semester hour laboratory-only courses, taken by students in the third year (P3). The course was offered concurrently with an integrated therapeutics and pathophysiology course. Cases were designed to align with and reemphasize processes (eg, identifying drug therapy problems) and content (eg, treatment guidelines) from the concurrent classroom courses. The laboratory sessions were held bi-weekly for 3 hours during the fall and spring semesters. Each section of 25 students was coordinated by 1 of 2 faculty members and attended by 3 pharmacists from various practice settings. The laboratory coordinators designed and implemented curriculum, prepared activities and assessments, trained pharmacist staff members, and managed day-to-day laboratory and course issues. The pharmacists evaluated students and provided feedback on presentations and assignments. In the summer of 2010 the Pharmacy Care Skills Laboratory course was reorganized into 3 modules: community, hospital, and ambulatory care, and learning objectives were created for each module. Learning objectives for the hospital module are listed in Table 1.
Assessment of Learning Objectives Related to Hospital Pharmacy Activities
Incorporation of MEDITECH Software
The MEDITECH software, which was donated to the university at the same time the Pharmacy Care Skill Laboratory course was restructured, provided a fully functional, realistic electronic health record system with the capability to provide a patient’s demographic information, vital signs and laboratory data, medication information, progress notes, microbiology results, imaging and diagnostic reports, and graphic representations of recent data. Initially, 4 cases were created and entered into the MEDITECH system. Each represented a patient hospitalized for several days for a medical problem requiring laboratory testing and microbiology workup. For each case, different versions were created to represent various time points in the course of the hospital stay. For example, case “Emily Allstate” would contain all data from the first 24 hours of admission. Case “Emily Festival” would contain all data from the first 48 hours of admission. A different last name was used to reduce students’ ability to look ahead to future time points for answers and drug-related problems.
For the hospital module, 2 cases with 4 time points and 2 cases with 2 time points were used. All case information was entered prior to the start of each weekly laboratory session and students were told which patient name to access to view the updated record. Drug-therapy problems appropriate to students’ progression in pharmacotherapy courses were embedded within the cases to test students’ ability to identify and recommend solutions within the context of a longitudinal patient case.
While the MEDITECH software has a built-in medication administration record (MAR), the dates and times could not be manipulated to match those in the cases. Because of this, the instructors chose not to use the computerized MAR, and instead created one designed to look as if it were printed from a computerized electronic medical record and distributed this to students. For this module, students were provided with login credentials that did not allow them to modify any part of the cases to avoid alterations between laboratory sessions. No part of the cases could be modified from session to session, though cases were modified from year 1 to year 2 of the project. Students could use the MEDITECH software only in the laboratory.
Course Content
The hospital module was taught over 5 weeks. In week 1, students underwent orientation to hospital practice, then in weeks 2 through week 5, they were assigned to follow patients and perform other tasks. The module was designed to emphasize management of hospitalized patients over time. As such, student activities were progressive and organized throughout the module (Table 2).
Week-by-Week Hospital Module Activities
Prior to the week 1 orientation, students were assigned to read several background articles and then respond to discussion questions. During the first laboratory, coordinators facilitated a discussion about the pharmacist's role in transitions of care, medication reconciliation, presentation of patients to preceptors and team members, and providing discharge counseling. The session included several opportunities for students to practice medication reconciliation and to view a recorded example of a student presenting a patient to her preceptor during an APPE. The course coordinators also presented the process and instructions on how students would perform a final verification (“final check”) of prepared sterile products. Finally, a detailed orientation to use of the MEDITECH electronic medical record system was provided. At the end of the week 1 laboratory session, students completed a short quiz on concepts related to the readings, in-laboratory discussion, and upcoming laboratory activities.
In week 2, students were assigned to work on 1 of 2 patient cases as if the patient was newly admitted to their hospital team. The patient cases were similar and both required antibiotic therapy to treat a complicated infection. Students were assigned to: complete a medication reconciliation worksheet; compile relevant patient information in a pharmacist’s monitoring form; identify any drug therapy problems and provide recommendations for resolution using a paper form; and complete a “final check” exercise where they were asked to review prepared sterile products and materials (eg, marked syringe, empty vial) to verify the preparation was ready for distribution to a patient.
In week 3, students returned to their patient case and updated their monitoring form and identified any new drug-therapy problems and recommendations. In this session, students also presented their patient case to a laboratory instructor who asked follow-up questions, evaluated the content and style of their presentation, and provided feedback. To mimic the frequent APPE experience of having multiple students present to the same preceptor, students presented in groups of 2 (each student had a different case) and received general feedback as a group.
In week 4, students continued to update their monitoring form and identify drug-therapy problems and completed another “final check” exercise. To simulate the experience of following more than 1 admitted patient at a time, students were assigned 1 of 2 versions of a second patient case, both of which involved a newly admitted patient with deep vein thrombosis requiring anticoagulation therapy. The student was required to complete the medication reconciliation and monitoring forms.
In week 5, the last week of the module, the electronically simulated patient admitted in week 2 was ready for discharge, so the student completed a discharge worksheet asking them to consider the transition to home in which they recommended an appropriate discharge medication list and provided instructions to the patient. Students were encouraged to consider medications the patient was taking prior to admission as well as during the hospital stay. Students were also required to update the monitoring form for the second electronically simulated patient they were following and to present the second patient to a laboratory instructor.
EVALUATION AND ASSESSMENT
Although the module was first taught in 2010-2011, substantial changes were made to the case content and in-laboratory process after the first year based on lessons learned and on student feedback from formative and summative evaluations; therefore, only student assessment data from 2011-2012 were collected and evaluated for this study. One hundred thirty-five students completed the Pharmacy Care Laboratory Skills course during the 2011-2012 academic year. This project was approved by the Institutional Review Board at Northeastern University.
Student Surveys
To obtain student feedback on the success and impact of the module, all students who completed the course in the 2011-2012 academic year were asked to complete Web-based pre- and post-module survey instruments about their work experience and confidence using electronic medical records; performing a final check on prepared sterile products, and performing medication reconciliation, discharge counseling and patient presentations. The survey instrument included 23 questions with responses given using a 4-point Likert scale (1=strongly disagree to 4= strongly agree). Anonymous unique identifiers were used to link pre- and post-module survey results. The related-samples Wilcoxon signed rank test was used to compare pre- and post-module responses.
The response rate for the pre-module survey instrument was 97% (N=131). One hundred twenty-seven students (96.9% of pre-module survey respondents) reported having completed a hospital-based cooperative education/IPPE and 94 students (71.7% of pre-survey respondents) reported having used an electronic medical record to access patient information.
Results of the pre-module and post-module survey questions are described in Table 3. Ninety-seven percent and 81.5% of students completed the pre-module and post-module survey instruments, respectively. Results are reported for students who completed both survey instruments, N= 110. Despite students’ experience in hospital-based IPPEs, mean ranks on questions regarding typical hospital pharmacist tasks in the pre-survey ranged from 1.5 to 2.9, indicating low levels of comfort and confidence. Mean ranks increased in the post-survey for all questions, ranging from 2.0 to 3.4 (p<0.001 for all comparisons) indicating an increase to moderate levels of comfort and confidence with typical hospital pharmacist tasks.
Responses on Pre- and Post-Hospital Module Survey (N=110)
Assessment
Course assessments were used to evaluate students’ performance on laboratory activities and provided information about achievement of learning objectives. Student work was assessed by laboratory coordinators and instructors according to standardized grading keys. Learning objectives were assessed as described in Table 1, along with the number of times each assessment was used, the average pass rate per assessment, and the percent of students who did not pass any of the assessments for a particular learning objective. The pass rate for each assessment was determined after the assessment was completed based upon the difficulty of the item being assessed, the distribution of grades, and the professional judgment of the course coordinators. Course assessments confirmed student achievement of covered competencies.
Student Course Evaluations
Only a few comments were made specifically about the hospital module on the university’s course evaluation surveys, at which point students had not completed all of the module activities. One student indicated the course content was valuable, while 2 others stated the hospital module content was particularly challenging and sometimes difficult to complete in the allotted time. Two students also suggested that more instruction prior to the activities would have been helpful.
DISCUSSION
The hospital module in the Pharmacy Care Skills Laboratory course was designed to improve students’ comfort and confidence performing skills typical of a hospital pharmacist and to prepare them for the APPE year. We felt these skills were especially important to address because nearly a third of our graduates pursue hospital pharmacy careers.
Several publications have described other instructional activities using electronic medical records. A 2006 paper described the impact of electronic medical records on medical student and resident education.4 The authors highlighted a common feature of electronic medical records, notably a link to “point-of-care education” that allows users access to local educational resources and Internet-based information about a given medication, diagnosis, or laboratory test quickly, often while trying to perform patient-related tasks. While this feature may be useful to solve patient-related problems in the clinical or educational setting, in the 2006 paper, frequent access of educational materials was not associated with better performance on pre- and post-practice experience examinations. External links to drug information and other resources can be configured in MEDITECH as well. The students in this project did have access to online drug information software, but we did not capture data describing the extent to which the students used it in the hospital module.
Another paper described the process undertaken by faculty members in a nursing skills laboratory course as they evaluated several electronic medical record systems and subsequently implemented the NurseSquared software program (Elsevier, Maryland Heights, MO).5 The nursing faculty were specifically looking for electronic medical records software that was easily accessible (ie, Internet-based), could be used with high-fidelity patient simulators, and allowed for nursing students to practice documenting information within the electronic medical record. This electronic medical record system was specifically designed for use in nursing-student education, and while no formal evaluations were described in the article, the nursing program reported an overall positive experience with the system.
Lastly, a 2010 paper described the use of an electronic medical record in a Pharmaceutical Care Laboratory course.6 In this course, students accessed comprehensive patient information electronically and were assigned to formulate a treatment plan and document their recommendations in a SOAP-style note within the record. Course instructors evaluated notes and provided feedback. Seventy percent of students completed pre- and post-course survey instruments to evaluate the impact of using the electronic medical record. On the post-course survey, significantly more students agreed that the electronic medical record “would be useful in documenting when a pharmacist provides patient care services,” preferred using an electronic system versus a paper-based system, and felt that using an electronic medical record would benefit them in their P4 year. While our project similarly required students to use electronic medical record to find and assess patient case characteristics, they were not able to document care plans or interventions within the MEDITECH program. Because the cases were viewed by multiple students, any change to the case-related documentation would have been immediately viewable by all other students and permanently recorded in the case. Thus, students completed documentation and patient-related monitoring forms individually, using word processing software outside of MEDITECH.
The assessment data demonstrated student achievement of learning objectives. With the exception of the objective “Given a patient ready to be discharged from the hospital and knowledge of the patient’s home medication and hospital course, recommend reasonable discharge orders,” students had 2 or more opportunities to perform an activity assessing each objective and all of the students passed at least 1 of the assessments. Less than 7% of students did not pass the assessment focusing on the discharge orders. Review of student performance led course instructors to think that including more than 1 exposure to the activity or additional explanation of expectations to complete the activity might improve performance. More attention will be paid to this activity in the future. As students had already spent several weeks successfully providing counseling to patients in the community pharmacy module, we did not assess students’ ability to perform discharge counseling in this module; nor did we evaluate students’ ability to enter and verify hospital orders.
Nearly all of the students had previous internship experience in a hospital setting, and despite 70% of them reporting experience with using electronic medical records, their comfort and confidence using electronic medical records was low and improved with the MEDITECH experience. Additionally, students’ confidence in performing other tasks typical of hospital pharmacists significantly improved in the post-module survey compared to the pre-module survey, which indicated that the hospital module was a beneficial component of the laboratory. Though we are unable to report performance results separately for those students who did not have substantial hospital experience, this activity probably had a different impact on them.
Formal and informal student feedback indicated that the activities and content still may be too challenging for students at this level. Even though modifications to the case were made between year 1 and year 2, further modifications are needed for future cohorts to target a more appropriate level of difficulty. Also, despite availability of online materials, including background articles and example videos of patient presentations, some students did not feel they understood what they were being asked to do in the laboratory. In the future, we can further emphasize directions and potentially involve P4 APPE students in orientation to laboratory activities.
We encountered limitations associated with using MEDITECH in the classroom setting. Because MEDITECH is an actual health information system (ie, not designed for use solely in an educational environment), documentation appears in real time. Students are not able to practice documentation or order entry within the hospital patient case without modifying it, which in turn impacts use by subsequent students. The program does not allow educators to create and then copy patient charts. While we established a closed environment that contained only patient charts created for learning activities, each chart had to be constructed individually and all data had to be entered manually. The workload associated with creating multiple unique charts for each enrolled student required the use of a small number of cases with restricted student access. Also, the cases were too complex for students to enter themselves without extensive training (course instructors required approximately 20 hours training before using the system).
This assessment and evaluation strategy was comprehensive and we are able to report a high response rate to pre- and post- survey instruments. However, there were several limitations to this investigation. Rubrics and scoring sheets were completed on paper and only total grades were recorded for each student, which did not allow us to determine specific competency achievement for each objective when multiple objectives were assessed by one assignment. In the future, we plan to use Blackboard rubrics within the Blackboard Course Management software (Blackboard, Inc., Washington, DC), which we use to administer other courses. Blackboard rubrics produce itemized achievement reports for every row of the rubric and can be mapped to specific competencies. Finally, although there is great potential to use the MEDITECH program in interprofessional educational activities, this activity did not expose pharmacy students to other health care professions students within the college.
CONCLUSIONS
Despite substantial hospital internship experience, pharmacy students reported low initial comfort and confidence with typical hospital pharmacist activities; however, these were significantly improved upon completion of hospital module activities in a Pharmaceutical Care skills laboratory course. MEDITECH software is one of the most prevalent hospital information systems used in many areas within and outside Massachusetts. Early exposure to real, commercially available software may offer students additional advantages when starting their APPEs or looking for positions in hospital pharmacy departments. This project demonstrates that laboratory simulation can improve the comfort and skills required in hospital pharmacy practice.
ACKNOWLEDGEMENTS
The authors would like to acknowledge the generous contributions of the MEDITECH Company and thank the company staff and consultants for their assistance with program implementation and case development.
Footnotes
↵* At the time of the study, Dr. Guerra was a pharmacy student at Northeastern University. Her current affiliation is with the University Health System, San Antonio, Texas.
- Received August 16, 2012.
- Accepted October 29, 2012.
- © 2013 American Association of Colleges of Pharmacy
