Effectiveness of a Medication Reconciliation Simulation in an Introductory Pharmacy Practice Experience Course

INTRODUCTION

Medication reconciliation is the process of comparing medications a patient is currently taking or should be taking to existing medication orders and resolving any discrepancies that exist.¹ The importance of medication reconciliation as a component of patient care across health care settings is recognized by many national organizations, including the Joint Commission.¹ The need for effective medication reconciliation is further supported by the growing body of literature evaluating the frequency of and potential outcomes associated with medication discrepancies.^2-4 Toolkits such as the Medications at Transitions and Clinical Handoffs (MATCH) and the Multi-Center Medication Reconciliation Quality Improvement Study (MARQUIS) have been published to guide institutions through development, implementation, and evaluation of medication reconciliation programs.^5,6 A multidisciplinary team approach to medication reconciliation that includes physicians, pharmacists, and nurses is recommended by these guides. Pharmacists are uniquely suited to interview patients and identify medication discrepancies based on their knowledge and patient counseling skills.³ In two recent systematic reviews of hospital-based medication reconciliation practices, most successful interventions were found to include a high level of pharmacist involvement.^7,8

The ability of pharmacy students to obtain medication histories and identify and resolve medication discrepancies during introductory and advanced pharmacy practice experiences has been previously studied.^9-12 There is still limited information available about the type of training that will best prepare pharmacy students for their future roles in the medication reconciliation process. No studies have been published about the use of simulation to teach or assess pharmacy students’ combined medication history taking and medication reconciliation skills. Simulated medication reconciliation activities were incorporated in the hospital pharmacy module of a pharmaceutical skills course¹³ and an order verification activity in an institutional introductory pharmacy practice experience (IPPE).¹⁴ These simulations, however, did not include the opportunity for students to practice obtaining a medication history, a key component of the medication reconciliation process. Separately, studies evaluating techniques to teach medication history taking have been published, such as one that evaluated a game-like tool used to introduce variables such as patient personality and medication problems into a small-group role-playing activity about taking medication histories.¹⁵

Studies have been published about medication reconciliation instruction in medical education.^16-20 Lindquist and colleagues incorporated a medication reconciliation simulation in an established second-year course for medical students. In this activity, small groups of students attempted to prepare an accurate medication list for a patient (portrayed by an actor) after observing an initial interview conducted by a faculty member. Students then reviewed the patient’s medication bottles and partial medication list and completed brief calls and conversations with the patient, community pharmacist, and the patient’s primary care provider.¹⁶ A similar simulation activity was developed to teach medication reconciliation in a longitudinal elective at another college of medicine.¹⁷ These simulation activities were well accepted by medical students and improved their perceived skill confidence; however, their abilities to obtain a medication history and perform medication reconciliation after completing the simulation activities were not formally assessed.

At our institution, lectures and/or workshops about medication reconciliation were previously included in two IPPE courses during the second year. While these courses introduced the need for and process of medication reconciliation, the opportunity to apply these skills in a simulated medication reconciliation involving simulated patients, pharmacists, and providers was absent. Therefore, this project was undertaken to design, implement, and evaluate a simulated learning exercise about medication reconciliation in an IPPE course within the third year of a Doctor of Pharmacy (PharmD) program. The primary objective was to evaluate the effectiveness of the simulated learning exercise on students’ ability to perform medication reconciliation. Secondary objectives were to compare students’ perception of their ability to perform medication reconciliation before and after the simulated learning exercise and to describe the students’ perception of the educational intervention.

METHODS

During the 2015-2016 academic year, the medication reconciliation simulated learning exercise was incorporated in the IPPE III: Clinical course. This course is the final IPPE and emphasizes clinical practice through a combination of didactic and experiential learning opportunities. It is taught over three quarters in the third year with approximately a third of the class (about 70 students) enrolled in any given quarter.

The medication reconciliation simulated learning exercise was incorporated in the course each quarter prior to the students attending any clinical site visits. The learning exercise was based on previously published experiences in other health professional curricula and designed to fit a two-hour format.^16,17 The exercise began with a 30-minute review lecture by a pharmacy practice faculty member about how to perform medication reconciliation. This lecture was then followed by a 90-minute workshop where students worked in groups of three or four to perform a medication reconciliation. Personnel for the workshop included a simulated patient and two faculty members. This number of personnel was ideal to cover approximately five to six groups of three to four students. At the beginning of the workshop, the students were provided time (approximately 10 minutes) to individually review the patient’s electronic medical record. The class then observed the faculty member who had given the lecture earlier conduct a brief patient interview with the simulated patient. The interview set the stage for a clinical scenario involving a patient recently discharged from the hospital who was meeting with the pharmacist to review her medications before seeing her doctor in the clinic. However, the faculty member did not review any specific medications during the patient interview. The role of the patient was played by an actor from the university’s Clinical Skills and Simulation Center (CSC) who had experience as a standardized patient. Once the interview was completed, the student groups had approximately 40 minutes to prepare an accurate medication list, identify any discrepancies, and propose resolutions. To assist with this process, each student group was given a bag filled with the patient’s medication bottles and access to the patient’s electronic medical record. Anytime during the 40 minutes, each group was also able to interview the simulated patient or contact the patient’s community pharmacist (portrayed by a second pharmacy practice faculty member). Each team had up to six “lifelines” represented by six index cards. Each “lifeline” represented a three-minute conversation with the patient (three cards) or the community pharmacy (three cards). Information was only provided during these lifelines if the right questions were asked by the student groups. At the end of the workshop, each student group submitted their medication list and identified discrepancies with proposed resolutions. After all groups submitted their work, the faculty member led a 20-30 minute in-class discussion reviewing the patient’s accurate medication list. All medications and therapeutic topics in the patient scenario had been previously covered in the curriculum so that students could apply this knowledge as they identified the medication discrepancies and proposed appropriate resolutions.

Approximately one week after this educational intervention, students individually completed a medication reconciliation encounter with a standardized patient (SP) in the CSC. Students were allotted up to 10 minutes to review the patient's electronic medical record and then up to 15 minutes to interview the patient and review their medication bottles. Students then had up to 30 minutes to complete a post-encounter assignment documenting the patient's reconciled medication list, any discrepancies they identified, and their proposed resolutions. This SP encounter was a formative assessment where students received feedback on their performance and recommendations for improvement. This assessment counted as part of the students' score in the class; however, their score was based on whether they completed the activity, not how well it was completed. The SP encounter provided students the opportunity to individually practice the medication reconciliation process before going to their clinical sites. The same workshop and SP encounter cases were used each quarter.

A research plan was developed to evaluate the simulated learning exercise and approved as expedited by the university’s institutional review board. All third-year pharmacy students enrolled in the IPPE III: Clinical course during any quarter of the 2015-2016 academic year were eligible to participate in the study. The medication reconciliation simulated learning exercise and SP encounter were required for all students since they were a course requirement; however, students’ participation in the study (allowing the faculty to use their data for analysis) was voluntary. Students who enrolled and completed the study were awarded five extra credit points in the class, which accounted for less than 1% of the total course points.

In order to evaluate the simulated learning exercise described above, the students were randomized using an Excel spreadsheet into the following three groups: group A, 30-minute review lecture (same lecture as given to group B); group B, 30-minute review lecture plus a 90-minute workshop; or group C, no intervention. The simulated learning exercise previously described is what the students in group B received. Group A received only the lecture described above, and group C served as the control group to represent the course content that previous students received during their second year.

To assess the effectiveness of this educational intervention, all students completed the medication reconciliation simulation in the CSC. The post-encounter assignment, documenting the patient’s reconciled medication list and identified discrepancies with proposed resolutions, was deidentified so faculty members were blinded to the student’s name and group assignment. These de-identified items were then evaluated for accuracy by the investigators using a checklist (Appendix 1). The checklist was designed using a yes/no format and divided into two sections: reconciled medication list, and discrepancies and resolutions. This checklist was based on a form piloted in the IPPE III: Clinical course during the previous academic year to which minor edits were made based on user feedback before being used in this study. Students received a percent correct on each section, as well as an overall score.

Two survey instruments also were administered to all students who consented to be part of the research project. The pre-survey was administered during week 2 of the course before any of the medication reconciliation learning activities occurred. The post-intervention survey was administered electronically immediately after the students completed the SP encounter. Participation in both surveys was anonymous; however, students were tracked over time using non-personal identifiers to permit comparisons between the pre- and post-intervention surveys.

In both surveys, students were asked to indicate their level of confidence in completing 16 different activities related to medication reconciliation knowledge and skills (Appendix 2). These items were adapted from previously published quentionnares.^6,15 The confidence scale was a four-point scale including very confident, confident, unconfident, and very unconfident. On the preintervention survey, students also were asked demographic questions including age and gender, work experience in different pharmacy practice settings, and whether they performed medication reconciliation at their place of employment. On the post-intervention survey students were asked to provide feedback about the different medication reconciliation teaching strategies they experienced based on their group assignment. Students were asked to indicate their level of agreement to statements describing the different teaching strategies and SP encounter. There were also open-ended questions on the post-intervention survey for students to provide additional feedback about each of the learning experiences (lecture, workshop, or SP encounter) that were applicable to their assigned group.

So that all students ultimately had the opportunity to participate in the simulated learning exercise, students who were not originally in group B were scheduled to attend a make-up lecture and/or workshop after the SP encounter and post-intervention survey were completed. Timing of the intervention, data collection, and survey administration are summarized in Table 1.

To evaluate the primary objective, scores from the SP encounter checklists were compared between the three groups using ANOVA with additional post hoc comparisons. For the secondary objectives, survey results were analyzed using descriptive statistics and a mixed model MANOVA with post hoc comparisons to analyze the differences in pre- and post-intervention confidence levels between groups. The written comments from the post-intervention survey instrument were also reviewed and categorized into themes. Any p value <.05 was considered to be significant. All data were analyzed with IBM SPSS Statistics, version 22.0 (Armonk, NY).

RESULTS

This simulated learning exercise was implemented in all three quarters in the IPPE III: Clinical course. Over the academic year, 203 students were enrolled in the course and 183 students consented to participate in the study and allow access to their results for research purposes. Demographic information of the students is summarized in Table 2. Overall, 65.7% of students were female and had a mean age of 25.7 years (SD=3.3 years). The majority reported work experience primarily in the chain community setting; however, only a few students reported performing medication reconciliation as part of their employment responsibilities.

Scores on the CSC post-encounter assignment were significantly different between the three groups for both sections of the checklist and the overall score (p≤.01). Detailed results are provided in Table 3. Group B received the highest scores for the reconciled medication list, discrepancies and resolutions, and overall score (73.2%, 77.3%, and 74.5%, respectively). Post hoc analysis found a significant difference (p≤.02) in all three scores between groups B and C. A significant difference was also identified between groups A and B for the medication list (p=.02) and the overall score (p=.03).

Response rates were 99% (182/183) for the pre-survey and 82.5% (151/183) for the post-intervention survey; 78.6% (143/182) of pre- and post-intervention survey instruments were matched for comparison. In general, confidence in completing activities related to medication reconciliation knowledge and skills remained the same or increased, regardless of group assignment (Table 4). A significant increase (p<.05) was found in 11(68.8%) of the 16 criteria when the pre- and post-intervention confidence levels were compared for all respondents. In post hoc comparisons between the three groups, a significant difference (p<.05) was found in nine (56.3%) of 16 criteria. These criteria focused on obtaining detailed medication history information, identifying medication discrepancies, proposing resolutions, preparing a medication list, and performing medication reconciliation. For the majority of criteria where a significant difference was found between the three groups, trends suggest that the mean confidence levels increased for groups A and B but decreased for group C over time.

Student agreement with statements regarding the learning activities that were applicable to their randomized groups are summarized in Table 5. In general, more students in group B indicated “strongly agree” than in group A or group C when asked similar questions. Overall, the students in group B reported high levels of agreement (strongly agree or agree) to the statements: the workshop was a positive learning experience, the workshop improved my ability to perform medication reconciliation, and more workshops like this one should be included in the curriculum (97.7%, 100%, and 97.7%, respectively).

In reviewing written feedback from the post-intervention survey about the different learning activities experienced, students in group B who experienced the lecture and workshop had primarily positive comments about their experience. All students, regardless of group assignments, felt the SP encounter in the CSC was a helpful learning experience. However, more students in group B (n=4) felt prepared for the SP encounter, while more students in group C (n=10) felt unprepared.

DISCUSSION

The medication reconciliation simulated learning exercise was developed to prepare third-year pharmacy students to independently obtain a patient's medication history and then reconcile his or her medications, identify any discrepancies that exist, and propose appropriate resolutions. Based on scores from the SP encounter, the combined simulated learning exercise had a greater impact on student performance than lecture alone. This suggests that the previous curricular content related to medication reconciliation was insufficient to prepare students to independently complete this task and that the addition of a review lecture alone would not have significantly improved their performance. After the simulated learning exercise, students were still not able to compile a perfect medication list or identify all discrepancies and appropriate resolutions. Frequently missed items on the medication list were related to route of administration (omitted for some or all medications) and the specific dosing of nonprescription (over-the-counter) products (eg, nicotine replacement therapy).

While a variety of activities to teach medication reconciliation have been published, students' ability to individually demonstrate their skills as a result of the training has rarely been assessed.^16-20 Karpa and colleagues described a workshop during which medical students worked individually and then as a class to review a patient's chart, identify medication-related problems, and propose resolutions. After this one-hour workshop, students correctly identified 64% of problems, as compared to 39% before the course began (p<.001).¹⁹ Van Zuilen and colleagues assessed the ability of medical students to interview a patient (portrayed by a faculty member), identify potential medication concerns, and provide recommendations. The student groups identified, on average, 16.1 out of a possible18 medication concerns (range 13-18). Students’ individual competency was then assessed at the end of the year using an online assessment. Out of 14 possible points, students scored an average 10.2±1.9 points (range 2-14 points). Eight students (4.5%) did not pass and required remediation.¹⁸ Similar to our experience, additional hands-on training in the form of a workshop improved students’ abilities to perform medication reconciliation. However, as seen in the literature and in our experience, students’ skills still needed further improvement.

There are also opportunities for improvement in medication reconciliation performed in patient care settings. In one systematic analysis of hospital-based medication reconciliation practices, at least one unintentional discrepancy with the potential for harm was identified in a median 45% of patients (interquartile range 31%-56%).⁸ In Shiu and colleague’s study, 38.8% of patients had at least one unintentional medication discrepancy at 30 days after discharge. These discrepancies occurred despite the presence of an interprofessional medication reconciliation process that included pharmacists, nurses, staff physicians, medical residents, and medical students. However, the presence of a discrepancy was not associated with readmission, emergency department visits, or death 90 days after discharge (p>.05).²¹ These studies suggest the continued need to identify strategies to prepare students and practitioners to be effective at this important patient safety initiative.

In this study, students’ perceived confidence in completing activities related to medication reconciliation knowledge and skills increased regardless of group assignment. This may have been due, in part, to the SP encounter that all students completed. Several students across all groups commented that the SP encounter was a helpful or valuable learning experience. The perceived confidence ratings of students who participated in an active intervention (lecture or lecture and workshop) increased, while the confidence ratings of students in the control group decreased or remained unchanged. Overall, students did not identify themselves as very confident or very unconfident. There is room to further develop student confidence in performing medication reconciliation.

The simulated medication reconciliation activity was well received, especially the workshop component, as more than 97% of respondents who completed the workshop strongly agreed or agreed that it was a positive learning experience, improved their ability to perform medication reconciliation, and that more activities like it should be included in the curriculum. An interesting finding was that for items related to the 30-minute lecture, more students in group B consistently selected “strongly agree” as compared to students in group A, even though the lecture was the same experience for both groups. This difference highlights that incorporating an active-learning component such as a workshop can influence students’ perception of the value of a more traditional teaching approach, such as a lecture. Students considered the lecture to be more valuable when it was paired with a workshop where they could practice what was discussed in the lecture than when the lecture was standalone without the active component. The perceived value of the workshop was similarly reflected in students’ optional written comments. While those in group B (n=4) commented that they felt prepared or confident when completing the SP encounter, students in group A (n=2) and C (n=10) felt unprepared.

Based on the positive impact of the simulated medication reconciliation activity on students' skills and perceived confidence, the activity will continue to be required for all third-year pharmacy students as a component of the IPPE III: Clinical course. In addition, IPPE courses earlier in the curriculum have been revised to include more active workshop components in the medication reconciliation units. The activity could easily be adapted for any health care professionals or trainees involved in the medication reconciliation process. Depending on baseline knowledge about medication history-taking and the medication reconciliation process, the length of the lecture may need to be increased or pre-class readings provided as the 30-minute lecture in this activity was intended to serve only as a refresher on the topic. The case scenario itself also may be adjusted to include only medications previously taught (for students earlier in their curriculum) or to purposefully include high-risk medication or complex patient problems (for advanced students or practitioners). Inclusion of medications not previously discussed would also provide an opportunity to reinforce and assess drug information skills. The setting for the workshop and the SP encounter in the CSC could also be varied, given the importance of medication reconciliation at all transitions of care.

This study is not without limitations. Results collected were from only one year of students at a single institution. A portion of the students completed the study each quarter and, while the simulated learning exercise and SP encounter scenario remained the same to minimize variability, there is a chance that students may have shared information with students who took the course during a later quarter. Other potential confounding variables, such as students’ work history or previous experience with medication reconciliation, were not controlled for and may have affected the results. Furthermore, the students’ ability to perform medication reconciliation was assessed one week after the simulated learning exercise. The differences identified between the groups are reflective of short-term retention of the skills and knowledge needed to perform medication reconciliation. A more ideal assessment would control for potential confounding factors and be administered several weeks after the workshop. In addition, evaluating the impact of the intervention on medication reconciliation performed in the clinical setting would be extremely valuable as that is ultimately the goal of this simulated learning exercise. Finally, it is unclear how many hands-on learning opportunities are necessary to prepare students to perform medication reconciliation without errors. These concepts present several different areas for future evaluation.

CONCLUSION

This study describes a simulated learning exercise focused on teaching medication reconciliation skills in an IPPE course. Scores from the SP encounter support that the combined lecture and workshop significantly improved students' ability to perform medication reconciliation, including obtaining an accurate medication list, correctly identifying medication discrepancies, and proposing appropriate resolutions. The simulated learning exercise also was positively received by students. Overall, students’ perceived knowledge and confidence in performing medication reconciliation increased, regardless of group assignment; however, they were not very confident in their abilities. Simulated learning exercises should continue to be incorporated in pharmacy education, especially for pharmacy practice skills such as medication reconciliation.