Evaluation of a Unique Interprofessional Education Program Involving Medical and Pharmacy Students

INTRODUCTION

The Institute of Medicine recommends that health care education emphasize a team-based approach to patient-centered care, recognizing that health care is increasingly delivered by interprofessional teams.¹ To ensure graduates of health care professional programs are competent to practice in such environments, many accreditation bodies mandate that students have interprofessional education (IPE) experiences in their respective curricula.^2,3 According to the Centre for the Advancement of Interprofessional Education, IPE “occurs when two or more professionals learn with, from and about each other to improve collaboration and the quality of care.”⁴ The majority of published IPE studies in two systematic reviews published by Reeves and colleagues have reported positive outcomes, which may relate to learner reactions, perceptions, attitudes, knowledge, skills, behavioral change, organizational practice, and in some cases, even improved patient outcomes.^5,6

Many types of IPE activities have been described, including meet-and-greet opportunities, didactic instruction, joint courses, case studies, web-based discussions, community service learning, clinical placements, student-led clinics, interaction with simulated or standardized patients, team-based case conferences, in-home medication reviews, and joint Objective Structured Clinical Examinations.^7-18 Olson and Bialorcerkowski’s systematic review of various interprofessional activities suggested that the outcomes of these events may be dependent upon the context in which they are delivered.¹⁹ Specifically, the evidence in this review supports that, in general, IPE events appear to be effective. However, the authors note that the maturity (age) of the participants, or perceived power imbalances between professions may affect the success of a given IPE activity, and recommend that these and other factors be considered when planning, delivering and evaluating an event.¹⁹

Faculty members at the University of Waterloo’s School of Pharmacy, and the Waterloo Regional Campus of the McMaster University Michael G. DeGroote School of Medicine developed a mix of activities for an afternoon of IPE referred to as the Healthcare Interprofessional Education Day (HIPED). The schools are located in adjacent buildings on the University of Waterloo Health Sciences Campus in Kitchener, Ontario. Three different activities were held in both the medical and pharmacy schools: a patient-interview station, a reflective interprofessional communication discussion, and a prescribing station. The goal of the event was to build interprofessional competency in the areas of communication, collaboration and role-clarification for medical and pharmacy students. This study was designed to evaluate self-reported changes in these domains using a validated pre-post survey instrument.

METHODS

Pharmacy students are required to build a passport of IPE experiences over the four years of their pharmacy training to demonstrate attainment of the Canadian Interprofessional Health Collaborative Interprofessional Competencies.²⁰ The experiences consist of exposure activities (eg, in-class presentations by other health care professionals in the first year of the program), immersion activities (eg, participation in extracurricular activities such as the Geriatric Interprofessional Conference), and application activities (which occur during their clinical rotations). The Interprofessional Education Program (IEP) at the pharmacy school coordinates requests by students and instructors to have various activities designated as approved-IPE events.

The medical school has adopted a slightly different approach to IPE, under the direction of the Program for Interprofessional Practice, Education and Research (PIPER) at McMaster University. Prior to clerkship, students must participate in at least one IPE activity at the exposure level (eg, a lecture or tutorial) and one at the immersion level (eg, participate in a home visit as part of an interprofessional team). Instructors and students can apply to PIPER to have a specific event designated as a sanctioned IPE activity, like at the pharmacy school.

Although the two schools are on the same campus and located only 100 feet apart, designing IPE events has proven challenging for several reasons. The medical school is a satellite campus, and for accreditation purposes, must deliver the same curricular experiences at both campuses. Additionally, opportunities for students to attend lectures together are limited as the School of Medicine delivers its curriculum using problem-based learning. Finally, students in the School of Pharmacy have co-op placements interspersed throughout their training, which adds to the complexity of curricular alignment. One faculty member of both schools brought representatives from schools of pharmacy and medicine together for a series of meetings to determine how to develop an IPE event that could be embedded within the curricula of both schools. The result of these meetings was HIPED.

HIPED included 118 pharmacy students and 28 medical students. The pharmacy students were in the second year of their program, and the medical students were nearing the end of their first year. HIPED was a half-day event that was built into the curriculum of each school, and was designed to allow pharmacy and medical students to learn with, from, and about each other. The objectives for HIPED were for learners to be able to discuss how pharmacist and physician skills and knowledge complement and overlap with each other; describe ways in which physicians and pharmacists can work together to enhance care; and explain how to promote effective communication among members of an interprofessional team.

Students were randomly assigned to one of three large groups of 50 (41 pharmacy students and nine medical students), and then subdivided into smaller groups for each activity (ensuring at least one medical student per group). After a welcome and a brief outline for the day was presented, students rotated within their large groups through three different activity stations that emphasized communication and collaboration among practitioners, patient interviewing, and prescribing. Each activity was 55 minutes long, with 5 minutes allotted for travel between the stations.

In the first activity, three video vignettes were shown to the learners in groups of 16 (with three medical students per group). The videos depicted interactions between a pharmacist and a physician in three scenarios: a busy community pharmacy, a hospital ward setting with multiple distractions, and a primary care office setting. After each scene, students were engaged in a facilitated discussion led by a pharmacist, physician, and patient, highlighting the value of effective communication among health care professionals. The resources required for this activity included the six actors for the videos, a videographer, and the three facilitators. In the second activity, a pharmacist and a physician separately interviewed a standardized patient with type 2 diabetes mellitus (DM) and symptoms that may have been due to either a cold or related to the recent initiation of metformin while learners observed in groups of 12 (with 2-3 medical students per group). A patient with type 2 DM was chosen because both medical and pharmacy students had covered this topic by this point in their training.

After watching each professional interview the standardized patient, students were led through a facilitated discussion to draw attention to similarities and differences in patient assessment and history taking, and to identify opportunities for collaboration. The resources required for this station included two standardized patients, two pharmacists, two physicians, and one overall facilitator for a large group summary discussion at the end. In the third activity, students were placed in groups of eight (with 1-2 medical students) and were tasked to solve two paper cases. The first case involved the patient with type 2 DM from the second activity. A variety of probing questions were included so that students would discuss best practices in prescribing and scope of practice as it relates to selecting optimal pharmacotherapy. The second case highlighted legal requirements for prescribing narcotics and other controlled substances. This station required several rooms for groups of eight students to discuss the case, two faculty members who circulated between the rooms to answer any questions, and one facilitator to take up the answers to the case with the large group at the end of the activity.

After completing the three activities, all students met for a wrap-up session in a large lecture hall to reflect on key concepts from the event.

To evaluate the effectiveness of these activities, we administered the Interprofessional Collaborative Competency Attainment Survey (ICCAS) to students immediately after the event during the wrap-up session. This survey is a validated pre-post survey that measures changes in self-reported competencies of interprofessional care in IPE programs.²¹ The competencies measured include communication, collaboration, roles and responsibilities, collaborative patient-/family-centered approach, conflict management/resolution, and team functioning. The survey is administered immediately following an IPE event, and asks learners to rate how competent they felt prior to, and after the event for each competency. We chose this tool because it has been validated, is administered only once immediately following the event, which may increase survey response rates, and was recommended by an expert group.²² Once the wrap-up session was completed, a research assistant distributed a copy of the ICCAS to the students, and provided instructions on how to complete the survey.

Our primary objective was to compare the improvement in interprofessional competency at the level of each specific item for the entire cohort of students. Our secondary objectives included an attempt to determine which of the broader competency themes were most and least sensitive to the interprofessional intervention. We also wanted to determine whether there was a difference in effect sizes in any of the 20 domains between pharmacy and medical students.

We used a paired student’s t-test for the primary outcome, and determined the effect size using Cohen’s d with pooled standard deviations. Paired student’s t-tests were performed using SOFA Statistics version 1.4.6 (Paton-Simpson & Associates Ltd, Auckland, New Zealand). An a priori p value <.05 was considered significant. Cohen’s d was calculated using an online calculator. Cohen’s d was used to determine standardized differences between means, and values of >0.8 were interpreted to be consistent with a large effect size, 0.5 a moderate effect size, and 0.2 a small effect size.²³ Ethics approval for this study was obtained from the Office of Research Ethics at the University of Waterloo.

RESULTS

There were 133 surveys collected for a response rate of 91%. Three surveys were not filled in correctly and were not considered (either only the pre-items or the post-items were filled in).

As shown in Table 2, there was a significant improvement in all 20 domains. The intervention appeared to have the strongest effects in the category of collaboration, while the least robust effects were noted in the conflict management/resolution category.

An exploratory analysis was performed to compare the effects of the HIPED activities between pharmacy and medical students (Tables 3 and 4). We did not formally test for statistical differences between each group of learners because of the imbalance in the size of the groups, and the small number of medical students. There was a robust increase in all items for the pharmacy students. The items that were associated with the smallest effect sizes were “take into account the ideas of interprofessional team members” (Cohen’s d=0.79) and “address team conflict in a respectful manner” (0.73) under the theme of conflict management/resolution. There was also an increase in most (18 of 20) items for the medical students. Improvement was not significant for “include the patient/family in decision-making” and “address team conflict in a respectful manner.”

In general, the effect sizes appear to be smaller for the medical students than the pharmacy students, as were the pre-event self-ratings.

DISCUSSION

A consensus panel on Interprofessional Education recommended that a key goal of interprofessional efforts should include evaluation to ensure that the objectives of the education were met.²² Our evaluation of the HIPED event identified three key findings.

First, the HIPED event was successful, as it led to significant improvements in all 20 items measured by the ICCAS instrument. Many other IPE studies have also reported positive outcomes,^5,6 but given that the outcomes of events depend on the context,¹⁹ it was critical to evaluate whether our method of IPE delivery was effective. There are few studies employing the ICCAS tool since it is relatively new. The effect sizes that we report in our study compare favorably to the effect sizes that were seen in the initial validation of the ICCAS (effect size range 0.6 to 1.3 vs 0.4 to 0.6, respectively).²¹ Meaningful comparison between these two studies is limited by the fact that while our study population consisted of only medical and pharmacy students, the validation of ICCAS included a sample of both students and practitioners from more than 19 professions.

Second, the HIPED event seemed to have the largest effect (Cohen’s d>1.0) in various items under the themes of collaboration, roles and responsibilities, and collaborative patient-/family-centered approach. By inspection, the largest effect size demonstrated was under the “recognize how others’ skills and knowledge complement and overlap with my own” domain (Cohen’s d=1.3). This item also achieved the largest effect size among both pharmacy and medical students when analyzed separately. These findings are not surprising, as the learning objectives for each of the three activities tended to be similar to the items that showed the largest change from baseline.

Finally, the items that showed the smallest effect sizes (Cohen’s d<0.8) tended to be in the conflict management/resolution section, and in the domains that measured comfort with expressing ideas in a team setting, involving the patient/family in decision making, and being accountable for contributions to the interprofessional team. Possible reasons for this could include that none of the activities were designed to introduce the concept of conflict resolution in interprofessional teams. Additionally, both groups of students have curricula that emphasize learning in small groups; accordingly, the baseline scores in some of the domains that address interpersonal skills tended to be slightly higher than in some of the other domains that were more directly attributable to interprofessional practice. It is worth noting that even though these categories were not affected as much as the others, the HIPED activities still had at least a moderate effect size as the Cohen’s d was greater than 0.5 for all items.

Our plan was to perform a sub-group analysis to identify differences in the effect of the activities between pharmacy learners and medical learners. Findings generated by visual comparison of the results suggested that effect sizes were smaller among medical students than pharmacy students. Additionally, the baseline scores for medical students tended to be lower than for pharmacy students. These apparent differences may be true, as previous research has indicated that various student factors may influence the effect of IPE activities.^24,25 However, it is not possible to infer much from our data due to the imbalance between the group sizes and the small sample of medical learners which precluded any formal statistical testing.

There are several limitations to this study. First, there were significantly fewer medical students than pharmacy students. The reason for this is that medical students were from the Waterloo Regional Campus of the Michael G. DeGroote School of Medicine, a satellite campus of the medical school located at McMaster University Michael G. DeGroote School of Medicine. The Waterloo Regional Campus was chosen for the initial HIPED event because it is near the pharmacy school. Given the positive findings from this study, there are plans to invite students from the main campus and the other satellite campuses to future HIPEDs. Second, this study used a pre-post survey design, which limits the inferences that can be made from the results. However, the ICCAS is a validated instrument, and our survey response rate was excellent. Finally, it would be ideal to measure the impact that participation in HIPED has on the students in their future practices. Unfortunately, this was not possible since the students were still years away from graduation. We believe that measuring more proximate indicators is still valuable to help with developing future interprofessional events.

CONCLUSION

Planning and executing meaningful IPE requires investment of significant time and resources. The results of this study suggest that the HIPED activity was an effective IPE experience that met the objectives set by the organizers. The strong and consistent improvement in all ICCAS scores suggest a framework for pharmacy and medical school training to move from siloed educational experiences to synergistic learning opportunities and lends support to the decision to make it an annual event.