Assessment of Emergency Preparedness Modules in Introductory Pharmacy Practice Experiences

INTRODUCTION

The roles of health care professionals in the United States expand in response to disasters. Although pharmacists have responded to emergencies and disasters for many years in one form or another, their role is not well defined.^1-4 During natural disasters, pharmacists may be expected to assist with medication access and emergency care for displaced victims, or provide immunizations and medications during disease outbreaks. During man-made disasters, pharmacists may function as first-line responders at the site of an event or second-line in the care of victims and responders in health-system or community settings. Specialized training is necessary to fulfill these roles.

Pharmacist roles, readiness, and gaps in preparedness are described in the literature, and several studies evaluating the training and preparedness of practicing pharmacists have been published in peer reviewed journals over the past 15 years.^2-5 Additionally, research has assessed the preparedness of nursing students to respond to disasters.⁶ Schools and colleges of pharmacy have developed programs that integrate faculty members, residents, and students into exercises organized by public health and community partners, as well as electives in emergency preparedness.⁷ However, formal training opportunities for student pharmacists in emergency preparedness are limited.

In 2009, the Georgia State Board of Pharmacy mandated that all pharmacists licensed by the state complete three hours of continuing education in emergency preparedness. Following this one-time continuing education requirement, it was the expectation of the Georgia Board of Pharmacy that pharmacy schools in the state would incorporate emergency preparedness training into the curriculum. Prior to the mandate, the University of Georgia (UGA) College of Pharmacy (COP) offered an elective course in basic emergency preparedness. However, comprehensive, intentional instruction for all student pharmacists needs to be developed and integrated into the doctor of pharmacy (PharmD) curriculum.

The UGA COP initially provided emergency preparedness training in fall 2011 and then incorporated training into introductory pharmacy practice experiences (IPPEs) through simulation for second-year students beginning in 2012. The college has a 4-year program (2 plus 2) where students spend the first two years on the main campus and then third and fourth year at satellite campuses around the state. For simulation planning purposes, it is best to select a time when students would be in the same location. In addition to preparing student pharmacists to assume roles in emergency preparedness, the simulations also help meet several of the Accreditation Council for Pharmacy Education (ACPE) requirements. The simulations encompass six of the pre-advanced pharmacy practice experience (APPE) domains as listed in Appendix A of Standards 2016, including patient safety, basic patient assessment, medication information, general communication abilities, patient counseling, and health and wellness (public health).⁸ The simulations also address topics in two domains of the Center for the Advancement of Pharmacy Education (CAPE) Educational Outcomes: essentials to practice (health and wellness, population-based care) and approach to practice and care (cultural sensitivity, communication).⁹

This paper describes an IPPE conducted at the UGA COP spring 2015 consisting of two emergency preparedness simulations that focused on mass triage and mass dispensing. The purpose of the study was to evaluate student performance and perceived competency of students serving in pharmacist roles in disaster simulations.

DESIGN

The simulations were first introduced to second-year student pharmacists in a skills laboratory in 2011. During development of the IPPE, a literature search identified a questionnaire used in nursing education, the Emergency Preparedness Information Questionnaire (EPIQ).¹⁰ Instructors received permission to use the survey to evaluate student pharmacists’ familiarity with eight emergency preparedness domains. Sixty-three students completed the EPIQ survey as a pre/postassessment. Responses from this survey indicated that student pharmacists needed additional training in triage/first aid, incident command, isolation and decontamination, clinical decision making, and communication. Based on these results, instructors modified the IPPE to address learning gaps.

Simulation was viewed as the best teaching strategy because response teams are required to conduct emergency preparedness exercises similar to these simulations. Therefore, the learning experience would be similar to professional experience. Prior to the simulations, students completed two online modules to provide foundational knowledge: (1) National Disaster Life Support Foundation’s SALT (sort, assess, life-saving interventions, treatment and/or transport) Mass Casualty Triage Training¹ and (2) Federal Emergency Management Agency’s IS-100 HCB: Introduction to the Incident Command System (ICS 100) for Healthcare/Hospitals.¹¹ Students were also assigned prereading from a section of the Basic Disaster Life Support Manual for additional background on triage in mass casualty events.¹ Since students had such limited exposure to emergency preparedness principles, both simulations were graded as pass/fail components. Data on student performance and student perceptions of learning were collected for both simulations. This study was approved by the University of Georgia Institutional Review Board.

The mass triage simulation was designed to meet the following learning outcomes: (1) Explain principles and demonstrate SALT triage methods for mass casualty events; (2) Identify and use IDME (immediate, delayed, minimal, expectant) categories to accurately sort victims; and (3) Determine transport order after sorting victims. During the simulation, students learned to triage standardized patients following a mass casualty event using the IDME method. Forty virtual victims were created for the computerized portion of the simulation. Each simulated victim included a photograph along with a description of the injury. Students had 30 seconds to evaluate each victim and categorize them as immediate, delayed, minimal, or expectant. Immediate victims are those with life-threatening injuries who will die without immediate intervention and treatment, such as myocardial infarction and closed head injuries. Delayed victims are those with significant injuries but whose morbidity/mortality will not significantly increase after controlling bleeding if they have to wait to receive treatment. Examples of delayed injuries include amputations, open fractures, eviscerated bowel, and gunshot wounds. Minimal victims have minor injuries and could most likely receive care at a secondary treatment facility or at the scene, such as lacerations, bruising, and burns. Expectant victims are those whose injuries are so severe that their likelihood of survival, even under the best of conditions, is small.

After completing the virtual triage exercise, students were instructed to rank a subset of 12 victims for transport to treatment facilities. Determining transport order helps students realize that even within a triage category there is still a need to rank the victims based on the severity of their injuries. The computerized portion featured victims with the same primary injury to see if students would categorize them differently based on victim appearance.

Ten victims from the computer exercise were selected for simulation in the live victim portion of the IPPE. While not all of the injuries were identical to the photographs in the computer section, the major injuries or health issues were the same. Faculty members, staff, and graduate students portrayed the 10 victims, and a moulage kit was used to apply mock injuries. Moulage kits contain make-up and other supplies that can be used to create a variety of injuries, such as bruises, cuts, and broken bones. The initial financial investment for moulage supplies was approximately $800 and $50 to $100 yearly to replenish supplies.

For this part of the simulation, different methods were used on each student group. For Group 1 (individual), the victims were placed in alcoves in the skills laboratory. Students were instructed to assess the victims individually and categorize them using the same triage method. Each student rotated to a new victim every 30 seconds. For Group 2 (mass casualty scenario), a car chase with shooting was simulated, with one car crashing through the front window of the pharmacy. Props were used to create a realistic event. For this group, the actors were scattered within the laboratory. Distractions like news reporters, radios, and dim lighting were used. Students were allowed to enter the skills laboratory in groups of six; however, each student worked independently and was given five minutes to find all the victims and categorize them.

After all students were finished with computer and live exercises, a debriefing was held to discuss responding to mass casualty events and to give feedback on the process of triaging victims. Feedback was also requested from nonstudent participants (actors and observers) to give input on student performance and potential improvements for the IPPE. Student performance of mass casualty triage was evaluated by examining student categorization of computerized victims, live victims (both individual and scenario groups), and how students ranked the subset of patients on the transport order. In addition, after the IPPE, students completed a postexercise questionnaire that included objective and open-ended questions.

The mass dispensing simulation was designed to meet the following learning outcomes: (1) Describe potential roles and responsibilities for pharmacists in mass dispensing; (2) Effectively perform an assigned role during a simulated mass dispensing exercise; (3) Explain the function of points of dispensing (POD). During this simulation, students were presented with the scenario of a pandemic influenza outbreak in which oseltamivir was to be dispensed to simulated patients who were potential candidates for prophylaxis or treatment of influenza. Supplies needed to set up the POD cost approximately $100 annually and included prescription vials, labels, and candy in place of medication. Multiple exercises were conducted to give students the opportunity to serve both as clinic workers and patients seeking medication. Students were divided into 3 groups based on class size. Each student rotated through 3 roles – simulated patient, pharmacist, and case reviewer.

Students, while in the patient role, were assigned standardized patient scenarios through the use of symptom cards, which provided instruction for answering triage questions. These cards included information on disease states, drug allergies, and current medications, as well as acting instructions, such as specific symptoms, not speaking English or being fearful. Each card was numbered, and the number was documented at the top of the treatment and consent form for tracking. Symptom cards were given to students at random, with gender being the only controlled factor. Assignments for the pharmacist POD roles were on the back of the symptom card. When each exercise began, students playing the simulated patients entered the POD and completed the demographic portion of the treatment and consent form using their symptom card. Student pharmacists assigned to various roles in the POD moved the simulated patients through the stations based on the symptom card and screening questions. All simulated patients were assumed to have been exposed to influenza and the majority received treatment or prophylaxis. Select symptom cards represented simulated patients who were not candidates for oseltamivir. Those patients were sent for referral or exited the POD based on screening.

Students working in the POD stations were assigned to clinical roles (screening, registration, dispensing, counseling) or logistical roles (routing, referral, incident command). Student pharmacist roles and responsibilities are listed in Table 1. Students were given this information prior to the simulation, educated just prior to the exercise, and instructed to be prepared to assume any clinical or logistical role in the POD, as noted on the back of their symptom cards.

Following the simulation, a debriefing was held to reinforce POD organization, pharmacist roles, as well as to obtain feedback from students and faculty members to make improvements to the IPPE. Additionally, the appropriate treatment or prophylaxis for each simulated patient was revealed. At the end of the debriefing, students submitted the completed treatment and consent forms which indicated how the simulated patients were screened and treated. The form also included five assessment statements (yes/no), which students completed during the debriefing (Table 2).

Each group was assessed for accuracy at the screening station, accuracy at the dispensing station, and responses to assessment statements. Logistical regression was used to compare accuracy among the three groups, based on the student pharmacist role. Logistical regression was selected as the data involved a binary dependent variable being compared among three groups of students.

EVALUATION AND ASSESSMENT

One hundred forty-four students completed the mass triage exercise. Instructors evaluated how students categorized computer simulated victims and ranked a subset of victims for transport to health care facilities. In addition, instructors evaluated how students utilized SALT methods for mass casualty triage and categorized live simulated victims (both individual and mass casualty scenario groups). Percent correct was used to assess student performance. When >70% of the students correctly selected the appropriate triage category, it was determined that the victim had been correctly triaged. Because two groups underwent the simulation simultaneously but in reverse, instructors looked for an order effect to see if completing the live victim portion first improved performance on triaging the virtual victims.

Results from the computer victims were similar between the two groups, as only 13 of 40 and 12 of 40 victims were categorized correctly for the groups who performed the computer exercise before or after live exercise, respectively. Students performed well in identifying immediate (2/3 or 3/3 for the two groups), minimal (7/11) and expectant (2/3) victims. However, they had significant problems identifying delayed victims. Students achieved a score of 70% or higher for correct categorization for only one of 23 delayed victims. The most common problem was categorizing delayed victims into the immediate triage category (19/23 victims). Two victims with the same primary injury were replicated within the virtual computer victim module. No differences were noted when the victim had a compound fracture of the leg. While students selected the wrong category for this patient (>90% immediate rather than delayed), they selected it consistently. However, differences were noted in how students triaged the victims with severe head wounds. Victims 10 (closed head wound) and 31 (open head wound) were female and victim 17 (open head wound) was male. Students consistently put the male victim in the expectant category (92%) whereas with the two females, the students split between the immediate (60%) and expectant categories (37%).

Table 3 demonstrates how students ranked the subset of computerized victims for transport. According to the postexercise survey (Table 4), students perceived they were not prepared to determine transport order (3.7 out of 5 with 5=most difficult). However, they only had three of 12 victims completely wrong in the transport order, six were correctly ranked, and the other three were placed in the next quartile (Table 3).

As observed with the computer simulation, students had difficulty selecting the appropriate triage category (2/10 correct). The two most common problems students encountered were placing too many victims in the immediate category (>5/10) and incorrectly categorizing victims into the expectant category based on the external appearance of the victims. For example, more than 95% of students categorized the victim with an exposed bowel as expectant, when in fact, this victim would be in the delayed category. In addition, there was no difference between the computer and live simulations for six of the 10 victims. Of the remaining four victims, students performed better with the computer simulation for two victims and live simulation for two victims. According to faculty and actor feedback, students often failed to assess the live victims fully because of distractions in the room (eg, noise, blood, other victims) and/or not knowing what questions to ask. The heart attack patient was correctly identified as an immediate patient in the computer simulations; however, the live victim was most often placed in the minimal category.

The postexercise survey also revealed that the SALT triage online training was important for students to prepare for the IPPE exercise (Table 4). However, many students commented that a review of SALT triage would be helpful before the IPPE (Table 5). The majority of students (47%) thought evaluation of the live victims was a better method for learning mass casualty triage than for computer victims (16%). However, a substantial number (37%) thought they were equally effective. Overall, students did not perceive this IPPE exercise prepared them for a mass casualty disaster (Table 4). However, approximately 70% of the students indicated they would like to learn more about opportunities for pharmacist to contribute in disaster/emergency preparedness.

One hundred forty-four students completed the mass dispensing exercise. Students were evaluated on their performance in the POD. Knowledge related to POD functions as well as pharmacist roles and responsibilities was assessed in a postexercise survey. Eight treatment and consent forms were excluded based on inadequate documentation, leaving 136 forms available for accuracy analysis.

The accuracy of screening and dispensing was examined by reviewing the simulated patient treatment and consent forms collected at the end of the exercise. A between-group comparison was performed using logistical regression because some students went through the POD as simulated patients, while others were first assigned pharmacist roles in the POD. It was predicted that students who first served as patients might perform better than those who were first assigned pharmacist roles, based on increased exposure to POD functions before performing assigned tasks.

Results for student accuracy in the screening and dispensing stations are shown in Table 6. The overall accuracies for the screening and dispensing stations were 89% and 90%, respectively. For screening, errors involved inability to determine if simulated patients were candidates for oseltamivir or if they should be referred because of health conditions, medications, allergies, and/or symptoms. Some patients were screened out for flu-like symptoms that were not truly indicative of influenza (ie, cough) and did not receive further evaluation or treatment. For dispensing, errors included patients who were candidates for therapy being incorrectly screened out and dispensing a prophylactic regimen when a treatment regimen was appropriate. While group 2 had the lowest accuracy for both screening and dispensing, potentially because of screening errors that resulted in dispensing errors, there were no significant differences between groups for screening or dispensing (all p values >0.05). The screening results were further evaluated based on whether or not the simulated patients were candidates for oseltamivir as shown in Table 6.

There were few negative responses to the five assessment statements. Students completed the first four self-assessment statements based on their experience and evaluation on the completed treatment and consent form. To answer the fifth statement, students received a key, which contained the appropriate treatment regimen for each simulated patient. All students stated they could explain the overall concept of mass dispensing, and 97.8% stated they could describe their role in the POD exercise and perform assigned duties. Additionally, 98.5% could provide accurate, intelligible education on the use of oseltamivir. In evaluating the treatment, 97.8% of students in simulated patient roles were able to determine how they were treated, and 92.6% stated they received the appropriate dosing regimen based on the symptom card key. However, additional faculty evaluation revealed that the appropriate regimens were dispensed to 91.9% of simulated patients.

DISCUSSION

While this simulation exposed student pharmacists to a nontraditional pharmacist role and stimulated further interest among them in emergency preparedness, student performance varied in both pedagogical approaches based on injury severity. While students generally performed well in identifying immediate, minimal, and expectant victims, they did poorly on categorizing delayed patients. The incorrect categorization of the delayed computer victims into the immediate category seemed to be strongly influenced by the amount of blood in the picture (18/23 victims). It appeared to be difficult for students to see a patient with traumatic injuries and place them in a delayed category for treatment. However, students and all health care professionals must realize that not everyone can be treated immediately or be first in line for transport. Some patients will have to wait to receive treatment even though they have major injuries. While students reported that they did not feel prepared to assign transport order for the victims, they did well on this component, indicating that while students struggled assigning a triage category, they did have a sense of what injuries were more severe than others as only three of the 12 victims were assigned to the wrong transport group.

Another notable error was the incorrect categorization of the heart attack victim in the live exercise. The live victim did not have any other apparent injuries, whereas the computer simulated victim had an injury to her arm as well, and this may have played a role in the incorrect categorization. In addition, students often failed to ask the live victim questions to determine her status, and when she volunteered information about her history of heart disease and other symptoms she was experiencing, they failed to recognize she was having a heart attack.

Students reported the most frustrating aspect of the computer exercise was the inability to gather additional information before making a decision about which triage category to use. However, in the live exercise, when students were given the opportunity to ask victims questions, their performance did not improve. The lack of improvement was most likely a result of students’ failure to ask victims health-related questions.

Student comments on the postexercise survey and faculty feedback will be used to improve the simulation. Improvements will include a brief review of mass casualty triage immediately prior to the exercise, a video of a simulated mass casualty event, and time for feedback after completion of each section.

Through this IPPE, student pharmacists demonstrated they could accurately screen patients to determine if they were candidates for prophylaxis or treatment with oseltamivir and accurately dispense this medication in a simulated pandemic experience. Students reported that after the exercise, they understood both their individual roles and the overall concept of mass dispensing. However, faculty members noted that at the beginning of each exercise that some students required additional instruction or clarification on their assigned role. Based on survey results and outcome of the exercises, the majority of students were able to function in the assigned role, but in the future more presimulation instruction will be dedicated to emphasizing roles and responsibilities.

Limitations should be taken into consideration when examining the results of the POD simulation. Both pharmacist roles and simulated patients were assigned to students at random, with gender being the only controlled factor for the patients. Each exercise had a different group of standardized patient scenarios, which varied in complexity. Although randomly assigned, the number of patients who were not candidates for oseltamivir varied between the groups (Table 6). Group 2 had the highest number simulated patients who were not candidates for oseltamivir, and this group also had the lowest accuracy at the screening station. A higher number of patients who were not candidates for oseltamivir increased the complexity and potentially made students prone to screen inappropriately. Because of the randomization scheme, all patients who went through the POD in group 3 were candidates for oseltamivir. In the future, instructors will make an intentional effort to equalize the complexity of patients in each group.

In addition, multiple students were assigned to the same role during the three exercises. While all roles of the POD should have functioned as a cohesive unit and influenced each other, it is possible that a limited number of students were responsible for majority of the screening errors that occurred and, thus, the accuracy may not be representative of the class as a whole. For the five assessment statements that students answered at the end of the POD exercise, some students expressed a concern that negative answers would reflect on their grade, even though the exercise was graded as pass/fail and responses to these questions had no influence on the grade.

Two of the most significant obstacles to implementing these simulations were time and number of facilitators required. Live victim scenarios required a substantial commitment by volunteers to serve as victims and facilitators to keep the victims in character to standardize the experience for the students. For the simulation described, a minimum of 13 people were needed, 10 to play victims and three to facilitate the experience. For the mass dispensing simulation, approximately 15-20 hours of preparation was needed to develop materials, randomly assign students, organize supplies, and set up the POD. In addition, five facilitators were used the day of the exercise to ensure proper flow and answer student questions. Depending on resources, some schools may struggle to recruit the number of people needed to conduct these exercises. Another potential obstacle is content expertise. The UGA COP is fortunate to have multiple faculty members with experience in emergency preparedness to develop and deliver the content, but this may not be true for other schools.

An implication of the study is the potential to increase the number of pharmacists prepared to respond to disasters by exposing students to emergency preparedness through didactic and experiential education. Pharmacists believe they should have a high level of involvement in emergency preparedness and response activities.¹² Moreover, pharmacists with previous training in emergency preparedness indicated that pharmacists should have an expanded role in the areas of surveillance, triage, and community planning.¹²

Both simulations undergo continuing review and revision based on faculty, actor, and student feedback. The simulations continued in the new UGA COP curriculum starting in fall 2015 based on student feedback and interest in emergency preparedness. With the curriculum revision, this IPPE will become an experience for first-year student pharmacists and be expanded by adding first aid to the first-year curriculum. Other exercises, including emergency formularies and therapeutic drug management, are being developed for second-year IPPEs to ensure vertical integration of this material in the curriculum.

SUMMARY

The emergency preparedness simulations provided a unique experience that exposed student pharmacists to potential roles of the pharmacist in emergency/disaster situations. While student performance varied on the simulations, students indicated that this IPPE expanded their understanding of pharmacist roles and stimulated interest in emergency preparedness. Future plans include developing an interprofessional emergency preparedness exercise incorporating the UGA Office of Emergency Preparedness, area hospitals, the local emergency management coordinator, and other UGA health professions students. An interprofessional simulation is expected to further prepare student pharmacists for roles in emergency preparedness and to better understand the roles for other health care professionals.