Impact of Standardized Simulated Patients on First-Year Pharmacy Students’ Knowledge Retention of Insulin Injection Technique and Counseling Skills

INTRODUCTION

The use of simulation or standardized patients are proven learning techniques which have been shown to increase student knowledge, confidence, clinical performance, and critical thinking skills within pharmacy education.¹ Such techniques have been shown to improve learning in various clinical areas, including inhaler and injection techniques, acute decompensated heart failure management, medication error recognition, blood pressure assessment, and advanced cardiac life support.^2-8 Standardized patient techniques have further demonstrated improved knowledge retention outcomes when conducted in combination with traditional coursework from one-week to three-months post-intervention.^3,5 In addition to previous studies, the Accreditation Council for Pharmacy Education (ACPE) has recognized the value of simulated learning in their 2016 Standards, and recommend the use of simulation experiences within interprofessional experiential education.⁹

The benefit of simulated learning has been well documented when applied late in a pharmacy curriculum. In one study conducted in the fourth year of the pharmacy curriculum, students in the experimental group were exposed to a six-week high-fidelity patient simulation series in which they worked with a care team to manage patients with disease states such as asthma and heart failure exacerbations.⁵ Following this intervention, it was shown that knowledge retention was significantly higher in the simulation group than the comparator at three months.⁵ The students undergoing simulated learning experiences also saw significant increases from pre- to post-simulation quiz scores across all clinical topics tested and reported improvement in confidence making clinical recommendations to providers.⁵ A second study conducted outside of the United States showed that final-year pharmacy students scored higher on inhaler-related counseling checklist exercises after undergoing standardized patient interactions.³ In this study, students were randomly assigned to receive a simulated counseling scenario with a standardized patient in addition to supervised hands-on learning and peer-to-peer education.³ The experimental group was compared to an active control group undergoing only supervised hands-on learning and peer-to-peer education. Student assessments on proper device technique were then tested one week later, showing improvements in counseling skills and confidence in favor of the simulated learning group.³

Unlike pharmacy literature, studies have demonstrated the positive impact of simulated learning and standardized patient interaction throughout early medical and dental school curricula.^10-13 It was demonstrated in the early 1990s that standardized patient interactions in medical school are effective throughout the curriculum.¹⁰ Standardized patients were utilized in early curriculum learning to improve interviewing, counseling, and basic physical examination skills allowing the students early application of knowledge to build upon.¹⁰ Another study in the first year of the medical curriculum noted that students’ confidence levels and self-assessment skills improved when students reviewed their standardized patient interactions.¹¹ Standardized patient interactions have even been used to replace inpatient interactions due to a reduction in inpatient beds in first-year medical students at a teaching hospital.¹² While there was no significant improvement in clinical performance in the standardized patient (SP) group, the scores were equal to those of the control group and there was a trend noted toward greater student satisfaction in the SP group.¹²

Despite clear evidence of effectiveness, there exists a paucity of published data for utilizing simulation within the first year of pharmacy education. While the previous studies demonstrated improvement in student knowledge, knowledge retention, and counseling skills following simulated learning, all were conducted later in the pharmacy curriculum.^2-8 Although pharmacy students throughout all years of the curriculum likely have similar approaches to learning, this method of simulated learning has not been proven beneficial in the literature in P1 students. Another key difference from previous studies is that P1 students are learning clinical skills for the first time, whereas P3 and P4 students are often familiar with the content being evaluated. To our knowledge, simulation has not been investigated among P1 students studying proper insulin injection technique. We therefore sought to determine if the robust efficacy evidence on this established technique would be observed early in the professional pharmacy curriculum through the addition of standardized patients to didactic coursework on P1 students’ knowledge and retention of proper insulin injection technique. We hypothesized that simulated patient interactions would improve the knowledge levels and counseling skills of P1 students with respect to proper insulin injection technique compared to traditional learning methods.

METHODS

The study was conducted during the Pharmaceutical Care Skills Lab (PHAR332) course during the Spring 2016 academic term at Campbell University College of Pharmacy & Health Sciences (CUCPHS) and was granted exempted status by the Campbell University Institutional Review Board. Every two weeks, P1 students at CUCPHS attend the skills lab course covering topics such as inhaler device technique, injection technique, glucometer usage and diabetes monitoring, blood pressure assessment, and anticoagulation management allowing them opportunities for hands-on application. The instructors consist of faculty members and pharmacy residents, with lectures supervised by the course directors who in turn assist in the application portion of the class.

The primary objective of this single-blind, single-center, cluster randomized study was to compare pre- and post-intervention test scores assessing insulin injection technique among students using traditional learning with (intervention) or without (control) simulated patients among CUCPHS P1 students previously exposed to traditional coursework. Traditional coursework is a two-hour session consisting of a lecture-style presentation followed by hands-on demonstration and peer-to-peer application of the skill being learned. Differences in counseling checklist and knowledge retention test scores between groups served as secondary objectives. All P1 students enrolled in the PHAR332 class who were at least 18 years old and provided written informed consent were eligible to participate in the study. Students who did not give consent remained within their randomized group and participated in all study portions, but data were not collected or reported. Participants were excluded from the study if they were absent from any of the baseline and/or post-intervention knowledge tests. Students were automatically subdivided into four groups as part of course enrollment. Using these pre-determined cohorts, we utilized cluster randomization to determine the two subgroups receiving stimulated, standardized patient interaction in addition to traditional coursework and those control groups receiving traditional coursework alone.

One week prior to the beginning of the study, students were provided with consent forms and a baseline characteristic questionnaire to determine their previous experience with pharmacy practice, insulin utilization, and personal or family history of diabetes. Students were provided with the option to forgo participation in the study and were additionally notified that no data collected for study purposes would be identifiable or affect their PHAR332 grade. At the beginning of each session, prior to the intervention, all students were given a knowledge test to assess their baseline understanding of insulin injection technique and counseling skills, which consisted of a 25-question multiple-choice test with questions provided from the American Public Health Association’s (APhA) The Pharmacist and Patient-Centered Diabetes Care Certificate Training Program along with previous, validated exam questions used in PHAR332.^14,15 Students were given a maximum of 15 minutes to complete the knowledge tests. Upon completion, tests were graded by P4 student volunteers and checked by the course director or instructor for accuracy. The students were then given another 25-question multiple-choice test at the end of each session to assess the change from baseline in scores between groups. The post-instruction test contained the same material as the pre-instruction test, but questions and answers were rearranged and rewritten.

The standardized interaction consisted of each student-pharmacist counseling the patient on correct insulin injection technique using an insulin vial and syringe. Student materials were consistent among all groups and included 0.5 mL insulin syringes (BD UltraFine), 10 mL saline vials (Hospira), Novo Nordisk FlexPen demonstration pens, 5 mm pen needles (BD), alcohol swabs, sharps containers, and injection pads. Students counseled the patient using the teach-back method with hands-on demonstrations until they felt the patient was sufficiently trained in properly using their insulin. The standardized patients were provided with training prior to each session and were given a script from which to follow. Incorporated into the standardized patient script were mistakes to facilitate follow-up questions to most accurately portray clinical dialogue. Although student pharmacists were permitted to reference clinical resources during the exercise, no real-time assistance was provided by the course moderators overseeing the interaction.

After didactic teaching and standardized patient interaction in the intervention group, all students counseled one of eight blinded faculty evaluators on the proper use of an insulin pen. The evaluators were provided with a 15-item counseling checklist to evaluate each component of appropriate insulin pen counseling. All items on the checklist were in yes/no format and evaluators were instructed to check “no” if the point was not explicitly covered to provide objectivity. The students were not allowed to use notes when counseling the evaluators. Finally, at one month (± 8 days) following the intervention, all students completed a knowledge retention test. The knowledge retention test was a 25-question multiple-choice test using the same questions as the pre-intervention test, but the answer choices and question order were randomly shuffled.

The primary endpoint was the comparative pre- and post-instruction test scores assessing insulin injection technique and counseling skills within each group. Pre- and post-test scores were calculated for each student and the change compared between groups. The secondary endpoints were post-instruction counseling checklist scores assessing insulin injection techniques and change between pre- and post-intervention written test scores at one month. All data were collected and analyzed in JMP-10 (SAS. Cary, NC). All raw scores were converted to percentages and the mean ± standard deviation values were used to describe normally distributed continuous data. Two-sample t-tests were used to compare means between groups of continuous data with normality of distribution determined using the Shapiro-Wilk test. For non-parametric data, a Wilcoxon rank sum test was used to validate or dispute the results of the two-sample t-test. Knowledge test, counseling checklist scores, and knowledge retention test results were all analyzed using a two-sample t-test. Baseline characteristics were also collected as percentages and Pearson’s chi-square tests were used to analyze the characteristics between groups. Only the select, pre-specified baseline characteristics relative to the learning content were collected. Pre-intervention test scores were also compared between groups to assess baseline knowledge levels. For all statistical analyses, a 95% confidence interval with a two-sided alpha of .05 was used.

RESULTS

All 103 P1 students were enrolled in the study, with 48 in the control group and 55 in the intervention group. No significant difference was shown between participants in both groups regarding preselected key baseline demographic characteristics that may affect knowledge (Table 1). Seventy-one percent of participants had some previous pharmacy experience in either a community or hospital setting, but only 9.7% had been exposed to the appropriate use of an insulin pen or syringe. Two students were lost to follow-up due to absences.

The intervention group (n=55) exhibited a trend toward increase in knowledge test scores from baseline compared to the control group (n=48) [21.5% vs 17.8% respectively, p=.077] (Table 2). There was no significant difference shown for the baseline pre-intervention knowledge test scores between the control and intervention groups (46% vs 44%, p=.39).

Secondary outcome analysis saw the intervention group with significantly higher counseling checklist scores relative to the control group (72.1 vs 63.5%; p=.0012). From the absolute increase in counseling checklist scores of 8.6%, students in the intervention group covered one or two additional patient counseling steps relative to the control group (Table 2). Due to time constraints, the knowledge retention test had to be given to all students on the same day. The intervention group had a significantly higher degree of knowledge retention than the control group (20.4% vs 14.4% respectively, p=.014). The overall knowledge retention test score was also higher in the intervention group than in the control group, but was not statistically significant (64.1% vs 60.4%).

DISCUSSION

This study is the first to our knowledge that evaluated the impact of standardized patient scenario in P1 students’ retention of insulin injection technique. Knowledge retention and counseling skills were significantly higher in the intervention group relative to traditional teaching modalities. The intervention group also demonstrated a small amount of immediate improvement in learning shown by the primary endpoint. Although the change in test scores from the beginning to end of the two-hour class sessions were not statistically significant, a trend was noted toward greater improvements in the students who underwent simulated learning (Table 2). Our hypothesis was therefore not fully supported due to a lack of significance in the primary endpoint. Although there was a trend toward greater improvement in knowledge levels, this lack of statistical significance is likely attributable to a limited sample size resulting in an underpowered study. Another cause may have been the large absolute difference in test scores observed within the control group, as evidenced by the comparative results of the student knowledge retention endpoint. With only a limited time between the baseline assessment and the post-lecture test, students retained a great deal of information gleaned from the traditional lecture.

The results of this study reflected those of previous studies which have shown improvements in knowledge retention and patient counseling skills among students following simulated learning applied later in the PharmD curriculum. A 2010 study demonstrated a 2.1% mean difference in knowledge retention scores in students who underwent simulation relative to those who did not at three months post-intervention.⁵ The P1 students in our study also demonstrated improved knowledge retention at a mean of 6.0% at one month. Additional studies have demonstrated improved counseling on pharmacy student inhaler technique commensurate with the findings outlined herein.³ Unlike these previous studies, our findings highlight the potential utility of early introduction to simulated learning within a professional pharmacy curriculum. The results of our study support implementing standardized patients and simulated learning early in the pharmacy curriculum to improve student counseling ability and knowledge retention of clinical skills. It has been well documented throughout the pharmacy literature that pharmacist counseling improves patient outcomes and early clinical application could provide additional benefit to pharmacy students. Early simulated learning will also better prepare P1 students for introductory pharmacy practice experiences. This teaching modality enables early-curriculum learners to continuously reinforce key concepts throughout the curriculum and provides opportunity for future research on the impact of simulated learning spanning the entire curriculum. Additional prospective trials are needed to assess the long-term effects of early curriculum simulated learning experiences.

There are limitations to this study. Due to logistics of the course design, there was no stratification for potential confounders including student ability or experience levels. Student GPA was not collected, with the only descriptor of baseline knowledge level being the pre-intervention knowledge test. Some students may also have reviewed the material following the initial assessment which could have influenced long-term retention of associated material. Class sessions were also limited to two hours and the knowledge retention test had to be given to all students on the same day. Fortunately, eight blinded evaluators and eight standardized patients were obtained for each session allowing completion within the two-hour session. Additionally, this study was limited to a one-month follow-up time so the long-term ramifications are unknown and provide opportunity for further research in assessing the sustained knowledge retention following early introduction of standardized patient interactions within a pharmacy curriculum.

CONCLUSION

When added to traditional coursework within a first-year doctor of pharmacy skills lab, simulated patient interactions improved student counseling skills and knowledge retention scores. Our results suggest the potential benefit on knowledge and retention rates of incorporating simulation-based activities into early pharmacy practice curriculums. Additional prospective trials are needed to appropriately quantify this potential benefit and assess the long-term effectiveness of early-curriculum simulated learning.