Closing the Classroom Theory to Practice Gap by Simulating a Psychiatric Pharmacy Practice Experience

INTRODUCTION

Beginning learners confront scenarios in a controlled classroom setting, applying rules that require very little judgment. Within the academic context, students learn and apply knowledge and skills to simulated scenarios imagined by instructors. This forms schemas, or sets of rules and theories, that students come equipped with when entering advanced pharmacy practice experiences (APPEs).^1,2 However, real-life contexts are unpredictable and present new challenges without clear answers, even to established clinicians.^2,3 Learning within the clinical setting context is how health professional students close the theory to practice gap.^2-4 Finding ways for Doctor of Pharmacy (PharmD) students to incorporate prior didactic knowledge into their established schemas when confronted with complicated, and sometimes contradictory real-world situations requires guided reflective learning.^3,4 The role of the clinical preceptor is to provide this necessary guided reflective learning by both modeling it and coaching the learner on how to practice it.^1-4 This is why APPEs, in which students receive guidance and feedback from preceptors, are essential for training future pharmacists. The goal of pharmacy education programs is to create knowledgeable, skilled, and adaptable pharmacists who are able and willing to continue learning. We report on an educational design project and evaluation in which we attempted to address two curricular goals: first, to bridge the gap between didactic learning and APPEs, and second, to explore a way to expand and deepen the psychiatric pharmacotherapy experiences of pharmacy students. The project examined the feasibility and effectiveness of combining whole-task and guided reflection educational design principles with cloud-based learning technologies to simulate aspects of the clinical psychiatric APPE in the classroom to begin to close the theory to practice gap.

METHODS

A faculty course director, along with self-selected second-year pharmacy students, designed an advanced psychiatric pharmacotherapy elective using a peer-led, team-teaching design framework. Peer-led team learning is an established educational model that has its origins in science education and is built on the constructivist theoretical frameworks of cooperative and collaborative learning.^5-11 This educational approach is an example of active learning, which argues that meaningful learning is defined by student engagement with complex, authentic problems as well as social interaction with peers and others. In this course design, whole-task, competency-based learning (specifically the four component instructional design model 4C/ID), using collaborative learning and first principles of instruction strategies, were integrated within an elective advanced psychiatric pharmacotherapy course for third-year pharmacy students.^12-16 For additional description of how the four components of the 4C/ID model were defined in the course design, refer to Table 1.

Other than the case scenario and suggested citations for initial readings, no learning materials were provided to students. The complex case scenarios were created by a postgraduate year 2 (PGY 2) psychiatric pharmacy resident or submitted by visiting experts. Visiting experts were given an opportunity to review the resident-created case for their week and provide edits and guiding questions. The case scenarios were designed to be too difficult for the pharmacy students to manage, intentionally challenging them to extend themselves beyond their existing knowledge and past experiences. Students were expected to apply all of their didactic knowledge from the core curriculum, particularly the psychiatric and neurological content in the core curriculum from the previous semester (year 3, fall semester), and use evidence-based literature evaluation skills for novel situations found in the complex case scenarios.

The 15-week advanced psychiatric pharmacotherapy elective course had at least one visiting expert present to facilitate the class discussion, either in person or connected by Cisco Meeting Server (Cisco Systems, Inc, San Jose, CA), a video conferencing service. Visiting experts were psychiatric clinical pharmacists with practice experience in the areas of focus for the course each week. Student teams of two were required to sign up for two cases during the semester. The list of topics for spring 2017 (the semester evaluated for this project) and the adapted list for spring 2018 are provided in Table 2. The work-up template emphasized the importance of evidence-supported recommendations and the use of multiple information sources. The case work-up template is provided in Table 3 and the grading rubric in Table 4 (Appendix 1 provides the guidance given to graders). The case was not visible to the presenting team or class until one week prior to the next class session. The timing was intentional to provide students practice with patient work-up on a time schedule and to allow for equal preparation time for each student team.

The first case was worked up by the entire class and facilitated by the course directors and visiting expert to help students understand the level of detail and evidence expected. During subsequent cases, the student team entered the work-up into a shared Google Doc that was made accessible online to the entire class and visiting expert one day ahead of the class session. Students who were not presenting that week were required to submit three personal learning objectives prior to the class discussion and complete a reflection assignment with one next-step goal after the class session. In their weekly pre-class assignments, students were asked to list three things they hoped to learn more about related to the clinical topic of the case for that week. Students were reminded that they would receive a grade for this assignment so they were expected to submit thoughtful personal learning goals. An unacceptable response, “I want to learn about Parkinson’s Disease,” was provided as an example of a goal that would receive a score of zero. The post-assignment prompts had two prompts. The first asked students to list the primary lesson learned from the case workup and discussion. The second asked students to list areas of personal knowledge deficit that the case workup and discussion revealed and their plan for how to gain that necessary knowledge. The purpose of the personalized pre- and post-class learning objectives, reflections, and next-step goals were to engage non-presenting students in the topic each week and to foster the metacognitive habit of examining their own knowledge and progression. In addition to weekly assigned points, personalized pre- and post-class learning objectives also received a response and critique from the course director.

Each class session began with the student team briefly providing an overview of the case along with their treatment plan and rationale. Course participants had already reviewed the case and student team work-up, so an extensive presentation was not necessary. This allowed for most of the class session to be devoted to clinical discussion and debate. The short overview was followed by the student team responding to the guided questions provided by the visiting expert of the week. These questions were similar to the kinds of questions preceptors would ask an APPE student. While the student team presented their case work-up, non-presenting students added thoughts, comments, and questions into the projected shared Google Doc, providing a sophisticated participation role for the non-presenting students that was visible to the entire class and instructional team. The visiting expert was able to view this real-time activity and interject during the presentation with comments, questions, and corrections. This model also resulted in a detailed archive of the discussion for students to use beyond the course.

Video conference learning technologies created a bridge between classroom and clinical practice by making it possible for psychiatric pharmacists to join the discussions, either from their practice site or in person, and provide real-world feedback regarding clinical recommendations. The learning technologies also allowed for student participants across two campus locations, Twin Cities and Duluth. More specifically, Cisco Meeting Server was used to allow classroom participants on both campus locations to enter a virtual meeting space with individual outside participants where everyone could both see and hear each other and view a single computer presentation.

The educational experience was intentionally focused on the application of current knowledge, demonstration of self-directed learning using evidence-based literature search skills, and building self-awareness regarding personal clinical competency development and life-long learning habits. The learning goals from the course syllabus are to apply psychiatric pharmacotherapy and general medicine pharmacotherapy content knowledge to complex challenging case scenarios, producing a comprehensive medication management plan, demonstrate ability to use effective and thorough literature search skills and evidence-based interpretation and application to address novel patient situations, and identify clinical knowledge and skill limitations, including strategies for how and when to seek help or refer to specialists. This study included what Patton refers to as a “triangulation of data sources and analytical perspectives to increase the accuracy and credibility of finding.”¹⁷ The project examined the feasibility and effectiveness of combining whole-task and guided-reflection educational design principles with cloud-based learning technologies to simulate aspects of the psychiatric clinical rotation experience in the classroom to begin to close the gap between theory and practice. Multiple sources of quantitative and qualitative data were collected for the purpose of increasing the understanding of these project goals.^17-19 The quantitative data source included student-ranked course entrance and evaluation responses, while the qualitative data types consisted of open-ended responses on the course evaluation and students’ pre- and post-experience personal learning objectives and feedback received from visiting experts regarding student performance and progression.

Quantitative data were aggregated and summarized. Qualitative data analysis included content analysis using the Classic Analysis Strategy.²⁰ Within themes, the two investigators independently reviewed the student comments using the classic analysis strategy, a constant comparison-like approach, and compared for internal consistency.²¹

Entrustable professional activities (EPAs) is a strategy adopted by the University of Minnesota College of Pharmacy in 2015 as a way to define and assess student progression toward the goal of being a practice-ready graduate.^22-24 Because the University of Minnesota College of Pharmacy APPE evaluation uses EPA statements, and this course was designed to simulate an APPE experience, students were asked to self-rate their level of entrustability at the beginning and end of the course. This served the dual purpose of familiarizing students with the EPA concept and measuring changes in students’ perceptions of their own level of ability through the lens of EPA units. The students were given a brief tutorial about EPAs before completing the pre-experience survey. Students were also provided a unique code to enter to allow for pairing of their pre- and post-experience survey responses.

RESULTS

Twelve (10 female and two male) third-year pharmacy students from two campuses participated in and successfully completed this one-credit elective advanced psychiatric pharmacotherapy course. Most students were between the ages of 23 and 26 years old, with one student reporting her age as 42 years. Eleven board-certified psychiatric pharmacists served as visiting experts (some participating for multiple weeks) and provided preceptor-like feedback to the case presentations. The only compensation provided to them was a parking voucher if they chose to participate in person at one of the campuses. All students earned a grade of A for the course. Scores given by the visiting experts for the student team work-ups ranged from 35 to 40, with 40 points being the maximum possible. A score of 40 did not represent complete accuracy or thoroughness, but rather that the student met the expectations for performance of a work-up on an APPE. The visiting experts expressed satisfaction with students’ work-ups and the class discussions. Some of the experts attended additional class sessions out of interest. About half of the visiting experts attended more than one student team work-up and four were present at most (13 or more), participating across the semester from early to late weeks. Post-course conversations with the participating visiting experts revealed universal agreement that student skills improved over the semester as evidenced by the clinical decisions and justifications presented by the student teams, as well as the live questions and comments posted by non-presenting students.

Students were asked to provide information regarding their interest in psychiatric specialty practice at the beginning and end of the course. All students stated they were likely to pursue specialization in psychiatric pharmacy or in another area with significant focus on behavioral health, with three stating they were extremely likely at the beginning of the course and four stating they were extremely likely at the end of the course. Most students had limited prior experience or exposure to the psychiatric specialty practice setting, and nearly all found the core psychiatric pharmacotherapy content in the curriculum interesting but insufficient. Many students reported the specialty area appealed to them because of the obvious need for a pharmacist on the care team; however, there was some concern that other clinical areas were able to manage without the consistent presence of a pharmacist team member.

The open-ended course evaluation comments were combined with the pre- and post-experience learning objective/reflection assignments as the dataset for the content analysis. Two independent evaluators reviewed all responses for emerging themes. After several passes over the responses, four emerging themes were identified and used for coding.

The first theme identified was students’ recognition of their need for and absence of clinical judgement. Overwhelmingly, students reflected on the importance of clinical judgment and their sudden awareness of how little they had. Students assumed that their many academic courses, internships, and other experiences provided them with fairly sophisticated judgment. This course revealed to them how unprepared they were to address clinical situations for which there were not consistent guideline recommendations and existing primary literature support. The dialogue with the visiting experts was a significant feature in the student realization of what clinical judgment truly means and entails.

The second theme, which was somewhat related to the first theme, was a realization that “evidence-based” was different and more extensive than previously defined. Prior to the course, students stated that being able to refer to the current guidelines and perhaps a couple of primary literature sources that supported the guideline recommendations qualified as evidence-based. The extremely complex psychiatric cases (which included patients with multiple other difficult pharmacotherapy needs) shifted their perspective on what is required to support complex recommendations. They also stated that they had a better appreciation for how these situations are addressed in clinical practice.

The third theme was the realization that patient-centered care means something different in psychiatric practice. Students commented that while patient-centered care is always important, this practice principle takes on extra significance within the psychiatric practice setting. Patients with psychiatric conditions often have extensive medication experiences, and knowing what has and has not worked for the patient in the past and for what reasons is essential for informing current recommendations. Learning and respecting a patient’s past experience is also essential to the success of therapeutic partnerships. One student quoted one of the visiting experts, “Adverse events aren’t theoretical to patients and psych meds can expose the true burden of adverse events. We need to respect this and not dismiss it as a statistic without real human impact.”

The last theme identified was students’ excitement about starting their APPEs. Students strongly expressed an excitement to begin APPEs. The elective course gave students a new appreciation for the advanced problem-solving required in real clinical practice. Many stated that their previous classroom experiences were much simpler and presented a false (and sometimes uninspiring) picture of what it meant to practice pharmacy. Students also expressed new excitement about the opportunities for pharmacists within the psychiatric specialty setting because the complexity ensures the need for a pharmacist on the team. Within this same theme, many students hoped they would discover through their APPEs that other practice settings were as rich a practice environment for pharmacists as psychiatry. Participating students expressed a desire for this same course design (specifically, very complex cases with visiting experts) to be used in all pharmacotherapy courses.

DISCUSSION

This educational project provides a feasible and effective example of combining whole-task and guided reflection educational design principles with cloud-based learning technologies to simulate a psychiatric clinical rotation experience in the classroom designed to begin to close the gap between theory and practice.

This educational project also demonstrates a way to expand and deepen psychiatric clinical knowledge in a way that does not require additional content development or create a heavy grading burden for course directors. Incorporating active-learning design and clinician participation into what was formerly a didactic portion of the curriculum is no longer a new or controversial strategy. It is well established in the literature that engagement in complex learning experiences combined with guided expert feedback is important for competency development in health professions students. However, the challenges associated with using this approach, including the logistics and increased instructor workload, often lead to schools following a simpler course design.

The use of Cisco Meeting Server made integration of the clinical expert voice into the course possible by removing the need for the visiting expert to be physically present in the classroom. The use of cloud-based Google Docs (Google, Mountain View, CA) for the work-up discussion allowed students who were not presenting to post thoughts and questions to both the presenting student team and the visiting expert without having to interrupt or even speak up, resulting in a more inclusive discussion and debate. Through this process, the students and experts created an educational artifact that students could access as a resource after the conclusion of the course. The student engagement requirement was embraced and valued by the participants and led to significant knowledge gains and increased understanding of and appreciation for pharmacy practice. The design also modified the instructor workload requirements adding to the sustainability of this approach.

All students completed both the pre- and post-experience EPA rating survey. Although the students were given a brief informational tutorial about EPAs before the pre-experience survey was administered, after examining the initial survey entries, we determined that it may not have been sufficient to fully orient students to the concept and make the rating tool a useful progression measurement for this course design. Despite describing Level 5 entrustability as the expectation for a PGY2 resident rather than a third-year pharmacy student, some students rated their level of entrustability as a Level 5 for several EPAs even before completing the experience. Students’ self-ratings of EPAs also did not align with course evaluation and open-ended comments from students regarding their self-assessment of clinical skills and progression, and the overestimation of performance level using the EPAs may be an example of students reverting to a norm-referenced ranking of themselves as third-year pharmacy student rather than a criterion-referenced ranking of their performance against a set of standards or criteria of a practicing pharmacist.²⁵ Some students either lost or forgot the codes provided to them for the EPA pre- and post-experience surveys, which prevented us from pairing pre- and post-experience self-ratings as we had initially planned. While the actual EPA level may not have internal value, the pre- and post-experience EPA levels did suggest an increase in students’ confidence during the course (Table 5).

The addition of more psychiatric pharmacotherapy experiences into the pharmacy curricula is an established need.²⁶ The vast topic areas, often covered in a single semester course of combined psychiatric and neurologic disorders, have resulted in small amounts of time devoted to each topic and many topics not covered at all.²⁶ This common curricular deficit is especially important given the significant unmet psychiatric needs in the United States.^27,28 Pharmacists are an important but underutilized resource to address mental health needs. Community pharmacists want to provide service for patients with mental illness, but do not feel confident to do so.²⁹ Additionally, mental health needs are present in every practice setting, further emphasizing the urgency to find ways to expand the psychiatric content and experiences in the pharmacy curricula. Many barriers have been identified, including a lack of psychiatric pharmacists on faculty.³⁰ This course design is one way to address the need for more psychiatric content and experience within pharmacy education programs and the lack of expertise in this area among faculty members. Using the design described in this project, psychiatric clinicians can be incorporated from health systems, opening up the possibility of drawing in expertise nationally.

The course design has proven to be a sustainable approach; the third iteration will be offered in spring 2019. All the visiting experts asked to participate in the course again. The second iteration of the course in spring 2018 resulted in all the visiting experts from spring 2017 returning to participate again, plus additional clinicians, including a geriatric pharmacy specialist. For the spring 2018 offering, a comprehensive medication therapy management (MTM) case was added as a capstone assessment. All students were required to complete the comprehensive MTM case work up. Because the University of Minnesota College of Pharmacy uses the AACP EPA statements³¹ as the APPE evaluation form, for the spring 2019 offering, the grading rubric will use five of AACP EPA³¹ statements, to further connect the course and APPE experience.

Only 12 students participated in this elective course in 2017 and only 11 in 2018. This elective course model could be scaled up to include a larger number of students. However, one explanation for the success of the course was that all students who elected to take it had specific interest in psychiatric clinical practice. This model could be used as a way to address the gap in didactic learning and APPEs in this area, ie, creating several tracks of electives within clinical areas would meet both the need to build pharmacy students’ practice readiness and expand and deepen their clinical knowledge. Hardy and Marshall describe another example of simulating APPEs in the classroom.³² Their successful design is a much more complex, three-credit required course that addresses a broad range of disease and medication therapy management issues and includes creation of a mock health care system and additional learning materials. Despite differences in the design complexity of their course and ours, both courses require students to apply and extend past knowledge and experience and both have resulted in increased students’ confidence prior to beginning APPEs. Standard 12 of the ACPE Standards 2016 requires that the didactic curriculum “inculcates habits of self-directed lifelong learning to prepare students for Advanced Pharmacy Practice Experience (APPE).”³³ Replacing lecture-based instruction with complex patient case simulations, as was done in our elective course design, has been shown to promote self-directed learning.³⁴ While the generalizability of the results of this project is limited by the small number of participants, the strategies described and evaluated are transferable to other pharmacy education contexts.

CONCLUSION

Pharmacy programs are challenged to find additional ways to improve student readiness for APPEs and expand psychiatric learning opportunities to meet the increasing mental health needs across clinical settings. The project we present provides a feasible and effective strategy to do both, without the requirement to create extensive new learning materials or significantly increase faculty workload.