Impact of Pharmacy Student Observation Versus Active Participation in an Interprofessional Simulation

INTRODUCTION

Students in health professions degree programs are increasingly required to graduate with an entry-level competence in interprofessional education, collaboration, and teamwork. As such, interprofessional education (IPE) has become a major focus for degree programs. According to the World Health Organization, IPE occurs when, “students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes.”¹ Within pharmacy education, the Accreditation Council for Pharmacy Education (ACPE) Standards 2016 outline the expectations for professional Doctor of Pharmacy (PharmD) degree programs related to IPE.² These expectations are summarized in Standard 11 which states, “The curriculum prepares all students to provide entry-level, patient-centered care in a variety of practice settings as a contributing member of an interprofessional team.”² Key elements of IPE include opportunities for pharmacy students to practice and demonstrate competence in the areas of team dynamics, education, and practice.² Several barriers exist to implementing IPE including scheduling, logistics, and financial support.³ Utilizing a variety of different learning methods may be a strategy to help overcome some of these barriers, but little data exist on the effectiveness of each method.

Simulation has long been used as a part of discipline-specific curricula, and more data are emerging on the role of simulation in IPE. Among medical schools implementing IPE, simulation, games, and role-play accounted for 71% of the learning methods used.³ According to ACPE Standards 2016, simulation may be used as a learning method for some but not all of a pharmacy student’s training in interprofessional team dynamics, education, and practice.² Simulation may be subject to many of the same barriers that face the effective implementation of IPE, including scheduling, logistics, and financial support. Data, particularly within nursing education, have described best practices for simulation, including simulation-enhanced interprofessional education, and how different simulation strategies can be employed to optimize learning.^4-6 Conventionally, simulation creates an opportunity for the student learner to actively participate in a realistic scenario. The role of the student observer in simulation has been described as effective in changing knowledge and attitudes, and may aid in overcoming some of the barriers previously described.^5-8 These barriers include financial constraints which limit offering simulation experiences to large cohorts of students, logistical constraints in accommodating large cohorts of students over a finite period of time, and time constraints of the faculty members available to facilitate simulation and/or IPE activities.

To date, no data comparing the effect of active participation versus observation of an interprofessional simulation experience on pharmacy students’ self-reported competence toward interprofessional collaboration exist. Previously we reported on the positive effect of an interprofessional high-fidelity simulation experience involving pharmacy and nursing students.⁹ The purpose of this study was to compare the change in pharmacy students’ self-reported competence toward interprofessional collaboration between those that were active participants and those that were observers of an interprofessional simulation experience. This study contributes to the literature as it provides information on a strategy to expand the number of students positively affected by an interprofessional simulation experience, and may be of interest to degree programs that do not have the resources to allow for every student to be an active participant in an interprofessional simulation experience.

METHODS

This study was conducted during the fall 2017 semester. The University at Buffalo Institutional Review Board reviewed this study and determined it to be exempt.

The simulation experience took place in the Simulation Center at the University at Buffalo School of Nursing. The Center included two flexible clinical rooms, high-fidelity patient simulators, a control room, and two debriefing rooms. Second- and third-year (P2 and P3) pharmacy students and senior nursing students were scheduled for one required simulation session during the fall 2017 semester.

Each required simulation session was scheduled for 1.5 hours. Third-year pharmacy students and senior nursing students were divided into teams of four (two from each profession) and were the active participants in the simulation experience (herein referred to as active participants). Second-year pharmacy students were the observers. These classes of students (ie, P3 pharmacy students and senior nursing students) were selected to be the active participants because it was expected that they would have completed a similar number of experiential clinical hours. During the simulation session, the active participant teams completed two interprofessional simulation cases, each followed by a debriefing. Before entering the simulation, the active participant teams were given a synopsis of the scenario, which was a brief (three to five sentences) overview of the scenario. This same synopsis was provided to the observers so they had similar expectations as to what the scenario entailed. Active participant teams were then given an opportunity to review basic information regarding the scenario, participate/observe the scenario, and immediately afterwards, participate in a debriefing. Observer pharmacy students viewed each simulation via a live video in the debriefing room and actively participated in the debriefing. The debriefing was jointly led by faculty members from the School of Nursing and the School of Pharmacy. Faculty member who led the debriefing period were given a debriefing guide that contained a list of suggested, open-ended questions. Examples of questions included in the debriefing guide were: How did that feel? Can you briefly describe what happened during the simulation? What went well? What could have been done differently? Tell me about your decision-making process? What will you take away from this simulation and put into action the next time you are in a similar circumstance?

Two new interprofessional simulation case scenarios were designed jointly by School of Pharmacy and School of Nursing faculty for the fall 2017 semester. The setting for both scenarios was an inpatient/acute care setting. The first scenario involved transitions of care, with an older adult patient being discharged from the hospital after an episode of acute coronary syndrome. The second scenario involved a medication error that occurred with an older adult patient who received the incorrect dose of insulin and subsequently developed hypoglycemia. In both scenarios, challenges were embedded that required the students to work together to remedy.

All students received credit for participation in the simulation activity if they were present at their scheduled session and were active participants in the group debriefing that followed each simulation. Active participants were given brief feedback from faculty members on general performance measures for all students (eg, introduce self to patient upon entering room) as well as discipline-specific performance measures (eg, elevate head of bed for nursing students; perform focused medication history for pharmacy students). However, a formal grade was not assigned.

All pharmacy students were asked to complete a brief demographic survey and the Interprofessional Collaborative Competency Attainment Survey (ICCAS) after the entire session. Survey data were collected using a web-based survey system.¹⁰ The ICCAS is an instrument developed to self-assess interprofessional collaborative behaviors based on established interprofessional collaboration competencies.⁹ The reliability and validity of the instrument has been examined with participants from a variety of health profession programs with evidence in support of using the instrument for measuring students’ self-reported competency both prior to (retrospectively) and after an IPE intervention.^11,12 The ICCAS consists of 20 items that use a seven-point Likert-type scale (1=strongly disagree, 2=moderately disagree, 3=slightly disagree, 4=neutral, 5=slightly agree, 6=moderately agree, 7=strongly agree), with research suggesting that in addition to examining the individual items, computing an average overall score for each student is justifiable.¹¹ Descriptive statistics were used to summarize demographic information of survey respondents. Mean retrospective pretest and posttest ICCAS scores for each item and mean total ICCAS score for each class year were compared using paired sample t tests. The mean total retrospective pretest and posttest ICCAS scores were calculated as the mean score for the 20 items on the scale for each respondent. Use of the mean of the item responses, as opposed to the total summated score, makes the total score easier to interpret. To examine the difference in the change in mean total ICCAS score between observers and active participants, repeated measures analysis of variance (ANOVA) was completed, with participant type (ie, observer versus active participant) as a predictor variable. All statistical tests were conducted using Statistical Package for Social Science (SPSS), Version 25 software (IBM Corp., Armonk, NY).¹³ For all statistical tests, p values <.05 were considered statistically significant. In addition to examining the significance level, the partial-eta squared (η²) measure of effect size was reported. In this analysis, η² values greater than 0.01 and less than 0.06 were considered small; greater than 0.06 and less than 0.14 were considered medium; and greater than 0.14 were considered large.¹⁴

RESULTS

One hundred thirty-seven P2 pharmacy students (observers) and 131 P3 pharmacy students (active participants) were scheduled for the interprofessional simulation experience during the fall 2017 semester. Of those, 130 (95%) observers and 121 (92%) active participants completed the demographic survey and ICCAS, for an overall response rate of 94%.

The characteristics of the observer and active participant pharmacy students are summarized in Table 1. Generally, the differences between the groups could be explained by their level of experience within the PharmD program. Overall, the active participant pharmacy students were older, had completed a larger number of introductory pharmacy practice experience (IPPE) hours, and had more experience working with other healthcare professionals.

The results of the ICCAS instrument are displayed in Table 2. For each item on the ICCAS, there was a significant increase in scores from the retrospective pretest to the posttest for both observers and active participant pharmacy students.

Compared to observers, both the mean retrospective pretest (M=4.9, SD=1.0 vs M=5.2, SD=1.0; p=.02) and mean posttest (M=5.9, SD=0.7 vs M=6.2, SD=0.7; p=.001] total ICCAS scores were higher for the pharmacy students who were active participants. There was no difference in the mean change from retrospective pretest to posttest between observers and active participant pharmacy students (M=1.0, SD=0.9 vs M=1.0, SD=0.8; p=1.0).

DISCUSSION

The self-reported competence in interprofessional collaboration of both observers and active participant pharmacy students increased after participation in an interprofessional simulation experience. Both before and after participating in the simulation, P3 pharmacy students more strongly agreed that they had the ability to perform each of the 20 items on the ICCAS instrument compared to self-ratings of P2 pharmacy students who only observed. However, the magnitude of change in scores between the retrospective pretest and the posttest was similar among observers and participants. The results of this study indicate that both observation and active participation in an interprofessional simulation experience may increase students’ self-reported competence toward interprofessional collaboration.

Whether it be in response to accreditation standards, changes in the way healthcare is being delivered, or a combination of these and other factors, IPE has become a focus of health professions degree programs. Simulation has been a popular learning method employed in many IPE initiatives.³ Both IPE and simulation share some common barriers, including scheduling, logistics, and financial support. To overcome these barriers, programs need to be creative, but not at the expense of providing an impactful learning experience. Within pharmacy education, little data exists regarding the observer role within simulation, specifically as it relates to interprofessional competencies. Our study helps to address this gap in the literature by reporting the positive changes that occurred in pharmacy students who only observed an interprofessional simulation experience. This change was similar to that in those students who were active participants. These data support the inclusion of this additional IPE experience for our second-year pharmacy students. We did not incur any additional costs, including faculty time, as a result of including the observers into the simulation. The main barrier that may exist for programs interested in implementing a similar simulation exercise may be the capacity of the simulation center and/or debriefing rooms needed. In addition to the students who were active participants, we were able to accommodate four observer pharmacy students per debriefing room, which was a feasible number with our facilities.

The ICCAS is a validated instrument that is reliable in predicting meaningful outcomes with regard to students’ attitudes toward interprofessional competency attainment.¹² While many instruments exist, there were several factors that led us to use the ICCAS.¹⁵ First, we were seeking a self-assessment tool that was flexible to incorporate different levels of learners assuming different roles within the learning experience. For our simulation, we had some students acting as active participants and other as observers. This excluded the use of instruments that relied on external observers assessing student performance as about 50% of our study population were observers. Therefore, the self-assessment component was critical. Second, the administration of the ICCAS instrument, which has a retrospective pretest/posttest design, is convenient and that may have had a positive impact on response rate. While there are some strengths to the retrospective pretest-posttest approach, including a reduction in response shift bias, there are some limitations.¹⁶ This approach may introduce post-score inflation (ie, positive response bias) as a result of a respondents’ desire to reflect positively on their own abilities.¹⁷ Third, the ICCAS instrument may improve a learners’ self-awareness of interprofessional competencies which was relevant in exploring the role of the observer in an interprofessional simulation activity.¹⁵

The ICCAS instrument can be analyzed by each individual item, as well as the cumulative total score, to determine the impact the interprofessional activity had on students’ self-assessed competence.^11,12 In our study, scores for each item and the overall total score on the ICCAS instrument increased from the retrospective pretest to the posttest. The magnitude of effect for each of the individual items was large. These results indicate that the interprofessional simulation experience positively affected students’ self-reported behaviors related to interprofessional collaborative practice. While these were significant changes, the educational significance of these findings should be considered. On the retrospective pretest, the second-year students (observers) scores reflected that overall slightly agreed that they were able to demonstrate the interprofessional collaborative behaviors and P3 students (active participants) moderately agreed with this statement. Thus, large increases in self-reported competence were unlikely, given students’ already positive self-assessment on the retrospective pretest. On both the retrospective pretest and the posttest, the pharmacy students who were active participants reported a higher level of agreement with their ability to perform each of the 20 items on the ICCAS instrument compared to the pharmacy students who were observers. Some of the differences noted in the baseline characteristics of the study groups, specifically number of IPPE hours completed and number of students that had experience working directly with other health professionals, may explain why the active participants had a higher level of agreement with being able to demonstrate the interprofessional competencies described on the ICCAS instrument.

This is the first study to describe changes in pharmacy students’ self-assessed competence toward interprofessional collaboration when engaging in an interprofessional simulation experience as either an observer or active participant. The role of the observer in simulation has been explored to a greater degree in other health profession programs, including medicine and nursing.^5-7 O’Regan and colleagues conducted a systematic review which included nine studies that reported learning outcomes or factors important for the engagement of the observer in simulation.⁵ Five studies in the review reported learning outcome attainment in observers as greater than or equal to those of active participants.⁵ The authors also reported that observer learning and satisfaction were closely associated with the use of observer tools (such as a checklist or script of the scenario), as well as observer engagement in the debriefing.⁵ Normal and colleagues investigated the role of the observer with and without an observation guide during nursing student simulations.⁸ No significant improvement in knowledge, self-confidence, or collaboration was noted between those nursing students using an observation guide; however, students using the guide were more satisfied with the simulation experience.⁸ Our study did not use a formal observer tool or guide; however, the observers were briefed on the context of the scenario. Additionally, observers in our study were directly involved in the debriefing. Faculty facilitators were instructed to make sure all students participated and to tailor specific questions to the observers (eg, “what did you notice the team doing well?”).

Observation may also be effective at enhancing learning in interprofessional simulation. Reime and colleagues studied medical and nursing students self-reported learning outcome attainment during an interprofessional simulation exercise.¹⁸ Regardless of role, self-reported learning outcome attainment was high (mean 3.9 to 4.3 on a five-point Likert scale); however, scores were significantly higher in the active participant group for three of the six learning outcomes.¹⁸ Of note, observers in this study were also given a checklist as a guide. Overall, students were satisfied with the learning experience but did prefer active participation to build confidence in their professional roles.¹⁸ Our study was similar in that we included student self-assessment of their abilities to perform competencies related to interprofessional collaboration. We reported a significant improvement from retrospective pretest to posttest in all learners (observers and active participants) with no difference in the magnitude of change. Observers in our study were not provided a formal observation tool or guide. We did not formally assess learner satisfaction, which could have revealed important information about student preferences. Given our limited resources, including faculty time and funding to support simulation, these data support the role of the observer in interprofessional simulation as a strategy that can enhance student learning. It is important for educators to still consider the need for students to demonstrate their knowledge and skills, which can occur through active participation in activities, including interprofessional simulations. While observation may have a positive impact on student learning, as students progress toward interprofessional competency evaluation, they will need opportunities to actively demonstrate their knowledge and skills.

The strengths of this study include that it was the first to investigate pharmacy students’ self-assessed competence toward interprofessional collaboration when engaging in an interprofessional simulation experience as either an observer or active participant. We used a data collection tool (ie, ICCAS) that has been validated.^11,12 Nursing and pharmacy faculty members were jointly involved in the creation of the simulation scenarios to ensure that the simulated interactions were authentic and representative of each profession.

There are several limitations to this study. First, as in all simulations, students are aware that the scenarios are simulated rather than real. Student teams are instructed to treat the situations as real; however, there is no way to know whether students approached the simulation the same as they would have a real-life case. Second, it is important to acknowledge the limitations associated with observation and self-assessment. A learner may self-assess that they are able to perform a skill after observing that skill being demonstrated; however, when asked to actually perform it, they may not be able to do so.¹⁹ In our study, we did not assess the observer’s ability to demonstrate interprofessional skills after observing the simulation and therefore were not able to assess this effect. However, this would be an excellent focus for future research. Active participation, or allowing the student the opportunity to practice their skills, is important prior to any capstone assessment or competency attainment evaluation. Observation as a part of simulation may be most appropriate when the goals of the learning activity are to improve self-awareness and identify areas for improvement. Third, the debriefing periods were not standardized. Faculty members who led the debriefing period were given a debriefing guide that contained a list of suggested, open-ended questions; however, they were not required to ask a certain number or type of questions from this guide. The only requirement was that all learners needed to participate in the debriefing. We relied on volunteer faculty members from both the school of nursing and school of pharmacy. Faculty members at both schools had experience with leading debriefings for other simulation activities embedded into each curriculum. However, the students’ experience and the potential effectiveness of the simulation exercise could have been influenced by the faculty members leading the debriefing. Last, we did not assess learner satisfaction, which could have revealed important information about student preferences.

Future research of the role of the observer in interprofessional simulation should include observers from other professions. The use of an observer tool (eg, a checklist) should also be studied to further explore its role in enhancing learning and satisfaction. Additionally, future research of assessment of learning outcomes should include other measures of student performance, such as faculty or peer assessment, in addition to student self-report.

CONCLUSION

In light of some of the challenges health professions programs face with implementing effective IPE, incorporating observers into interprofessional simulation experiences has the potential to positively impact students’ self-reported competence toward interprofessional collaboration. Future research into the observer role could provide further guidance to programs looking to expand the impact of their IPE offerings without the need for significant additional resources.