Abstract
Objective. To identify common components of inpatient general medicine advanced pharmacy practice experiences (APPEs) across sites, determine which components were important to standardize, and distinguish a benchmark of each component that would indicate standardization.
Methods. In-depth interviews about the student experience were conducted with two students and one or more preceptors at each of 12 inpatient general medicine sites in the greater Seattle metropolitan area. Data from transcribed interviews were analyzed, then collapsed into a single spreadsheet for comparison. Overall themes for component groups were identified and components within each theme tested for importance in overall quality of the student experience. Components important for quality of the student experience were deemed “key components” and component metrics were developed for benchmarking.
Results. Three themes were essential in the quality of the learning experience at inpatient general medicine sites: welcoming of the student by the preceptors, integration of the student into the site, and student interaction with patients and other health care providers. Key components to be standardized across sites were a structured orientation, written site-specific student guidelines and performance expectations, regular and meaningful student-preceptor interactions, identified performance competency standards, clear delineation of student patient care load and responsibilities, and daily interactions with patients and other health care providers.
Conclusion. Qualitative evidence from sites can be used to identify key components to standardize across practice sites offering the same experiential course. These key components aid in the welcoming of the student to the site, integration of the student into the workflow, and daily opportunities for the student to interact with patients and with health care providers.
INTRODUCTION
Student pharmacist learning in the practice setting is an important component of the student’s overall educational experience. The practice setting is where students apply knowledge, hone skills, and display and modify attitudes and behaviors. It is also where their practice competence is ultimately assessed. If the quality of the student learning experience at the site is low, then the student may not achieve competency in some areas and is at risk for failing subsequent performance assessments or not achieving practice readiness upon graduation.
The quality of the student experience at the practice site is one the top three concerns for experiential education programs.1,2 A specific quality concern is the consistency of the learning experience for students across practice sites Haase and colleagues outlined general quality recommendations for sites, preceptors, and experiential education programs as well as some specific quality components for core advanced pharmacy practice experiences (APPEs).3 For example, they recommended that at acute care inpatient sites, students should engage daily in patient care activities and participate in interprofessional patient care rounds. The Accreditation Council for Pharmacy Education (ACPE) incorporated many of the general recommendations from Haase and colleagues’ white paper into Standards 2016, but not specific recommendations for core APPEs.4 Instead, Standard 10.15 requires a quality assurance procedure for all pharmacy practice experiences that “standardizes key components of experiences across sites offering the same experiential course.” ACPE Guidance statement 10q only identifies that the quality of the educational experience needs to be “consistent.”5 It is thus up to the school or college of pharmacy to determine which elements of each type of practice-based experience are “key components” of that experience, which of these components can and should be standardized (“consistent”), and what measures will be used to determine if standardization has occurred at each site.
Standardizing experiences across sites that offer similar experiences is challenging. Each institution has different models for pharmacy workflow, staffing, and patient care load. Pharmacists on the same service can differ in expectations about the student role on their service. Institutions vary in their commitment to student learning and in staffing models for other health care professionals, which can affect the student’s interprofessional interactions. Standardizing key components across sites is further challenged when pharmacists at most or all of those sites are volunteer preceptors, necessitating negotiation in identifying how preceptors at that site can meet directives coming from the school.
This article describes a qualitative approach to identifying common elements in the student learning experience at most inpatient general medicine sites across a metropolitan area. The first objective was to identify which components contributing to the student learning experience were common across sites. The second objective was to determine which common components were “key” to student learning. The third objective was to determine key component metrics that might be used to assess standardization. The final objective was to identify any notable practices that enhanced the student’s learning experience.
METHODS
This study used qualitative methods to gain a robust understanding of the pharmacy student experience at all inpatient general medicine sites in the greater Seattle metropolitan area where at least two students were placed in the 2015-2016 academic year. Qualitative data were primarily derived from in-depth interviews of two APPE students and at least one preceptor from each site. For triangulation, qualitative data obtained from a database of student site evaluations were used to confirm information obtained from the interviews. Quantitative data obtained from the database were used to clarify overall student perceptions of their interactions with patients and with non-pharmacist health care providers at each site.
Purposeful sampling was used to identify interview candidates who were current and past (within the prior six months) students and one or more preceptors at each site.6 Candidates were contacted by a staff member, who was not part of the research team, and asked to participate in the study. Signed informed consent to audiotape the interview was obtained prior to the start of each interview. The study protocol was reviewed by a University of Washington Human Subjects Division subcommittee and determined to qualify for exemption.
Semi-structured interviews were conducted by one of the investigators with a single student at each site during the final week of a 4-week APPE.7 At the same time, a semi-structured interview was conducted with that student’s preceptor or preceptors by the other investigator. To confirm the student’s description of his or her experience, a second student who had completed an APPE previously at the same site within the past six months was also interviewed by one of the investigators or by one of two students completing a pharmacy education APPE with the primary author. Both students completed a training module in conducting a semi-structured interview and observed an interview of a student study participant prior to conducting interviews independently. All interviews occurred between January 15 and April 30, 2016. The interview script for students and preceptors is provided in the Appendix.
De-identified data obtained from students’ site evaluations in the 2014-2015 and 2015-2016 academic years for each of the 12 sites were provided by a data broker who was not part of the research team. The data included qualitative comments made by students in response to a query about things done well and a query about areas needing improvement at the site, and quantitative data regarding the level of integration into the health care team (options: fully integrated, actively participating, passive listening) and accountability for patient outcomes (options: full, partial, none) perceived by students at each site.
All audio-recorded interviews were transcribed and de-identified by the interviewer. Each student transcript was independently reviewed by one of the investigators and by a student research assistant enrolled in a non-pharmacy degree program. These reviewers first compared transcript data between the two interviewed students at each site to identify common elements of the students’ descriptions of their experiences. The reviewers then built individual spreadsheets to characterize identified components that were common across the sites. For example, one spreadsheet was created to examine students’ typical daily schedules and another compared student patient care tasks. The data were compiled to create profiles of student experiences at each of the sites. The primary investigator compared profile elements to the information in the original student and preceptor transcripts to verify accuracy of reviewer interpretation.8 The primary investigator used ATLAS.ti version 1.0.41 (ATLAS.ti GmbH, Berlin, Germany), a qualitative data analysis software product, to thematically analyze data from students’ site evaluations, specifically student comments about what was done well and what improvements were needed at the site. Identified themes were compared with the accounts of the interviewed students to check credibility of those student accounts, a process called source-triangulation.9
Once profiles were deemed credible, reviewers then collapsed profiled activities and elements at sites into a single spreadsheet, allowing comparison of student general medicine experiences across sites. The primary investigator used thematic analysis to group related components into overarching themes and tentatively identify components within each theme that were important contributors to the quality of the student inpatient general medicine experience.10 A colleague who was not part of the research team but had substantial recent experience in the inpatient care setting discussed each component with the primary investigator, using the theoretical argument, “If this component didn’t occur, would the quality of the student’s learning experience be diminished or compromised?” The answer to this question determined which components were considered “key” components. The primary investigator then identified metrics for each key component that would serve as a benchmark to determine degree of standardization across the sites.
Baseline characteristics of interviewed students were compared to those of their classmates with Fisher’s exact test. Pearson correlation coefficients of precepting variables, site variable, and student variables were tested for significance against the hypothesis of no correlation. All statistical tests were done using R, version 3.3.1 (The R Project, Vienna, Austria).
RESULTS
Twelve students who had completed their general medicine APPE within the past six months and 12 students who were nearing the end of their general medicine APPE were interviewed about their experience. Demographics of student participants were not significantly different than those of their class and are shown in Table 1.
Baseline Characteristics of Interviewed Students and their Class Members
Fifteen preceptors for 12 general medicine sites at 11 different hospitals were interviewed. The preceptors averaged 12.4 years (range 3-40 years) of practice at their current site and 11 years (range 3-35 years) of precepting experience at their current or previous work site. Site demographics are shown in Figures 1 through 3.
Size of Study Hospitals (n=11) by Number of Beds.
Socioeconomic Trend of Patients at Study Hospitals (n=11).
Percentage of Patients Age 65 or Older Seen by Students at Study Sites (n=12).
Data analysis identified three broad themes affecting student learning across sites: how the student was welcomed at the site, how the student was integrated into practice at the site, and the extent of the student’s interactions with patients and with non-pharmacist health care providers. These broad themes were also found in the qualitative analysis of the student evaluations of the site. Components of each theme identified by interviewed students also appeared on multiple student comments in the site’s evaluations.
Four components were key to student welcome at the site. These components were the student orientation to the site, written materials provided to the student prior to or on the first day of the experience, initial clarity of the preceptors’ expectations of the student, and preceptor role modeling for the student early in the experience.
Students at nine of the 12 sites spoke of receiving an orientation on the first day of the experience. Orientations varied from being scripted and scheduled to being unscripted and informal.
Written materials given to the student on or before the first day of the APPE were supplied to the primary investigator by preceptors at three sites. Preceptors at other sites either did not have written materials or stated they were working on creating them. Written materials from one site identified student knowledge and skills needed for the core curriculum and required elements of assigned projects. Written materials at this site also contained checklists for preceptors to indicate student accomplishment of required skills. Each checklist outlined a performance standard and process for a specific skill. For example, students had to observe three vancomycin dosing adjustments made by the preceptor and then complete five dosing adjustments observed by the preceptor before having that skill checked off.
Students indicated at 10 of the 12 sites that preceptors identified and revisited their expectations during the APPE. When asked to name expectations, students generally identified activities, whereas preceptors usually described attitudes, knowledge, skills, and global or specific behaviors. Three sites had written expectations for students that were provided to the primary investigator. Even so, students at two of these sites in their site evaluations identified clearer expectations as an area for improvement.
Interviewed students and data from site evaluations identified the friendliness of the preceptor and recognition that the student was there to learn as factors that eased acclimation to the site. Both interviewed students and student comments on site evaluations noted the importance of initial role modeling by the preceptor, particularly for efficiently working up a patient and successfully interacting with the health care team. Elements hampering ease of acclimation included difficulty in navigating the electronic health record and unclear preceptor expectations, particularly when there were multiple preceptors. Both students at one site noted that fear of interrupting busy preceptors discouraged them from asking questions.
Student descriptions of a typical day at their APPE site revealed relative uniformity in experiences across sites and in the tasks students performed. At most sites, students worked on projects in addition to patient care tasks, but preceptors at two sites stated that they minimized projects for students because they wanted students to focus on patient care. The typical schedule and tasks are outlined in Table 2.
Typical Student Schedule and Activities for an Inpatient General Medicine APPE
Student integration into the practice site was affected by the precepting model used at the site. Data analysis identified three precepting models defined by the number of services where the student learned and the number of preceptors with whom the student interacted during the APPE. These models are differentiated in Table 3. Sites successfully using models 2 and 3 relied on calendars to organize the student’s experience. At one site where 5-7 pharmacists rotated through the student’s clinical service during the APPE, preceptors tracked student activities and skills, adjusting the day’s activities to what the student had not yet done. At each of the sites using model 3, students spent between three and 10 days on different services within the hospital, working with whichever pharmacist was scheduled for the service on that day. The experience at all three sites was designed to start the student on services with patients at the lowest level of acuity and advance to patients of increasingly higher acuity; at the end of the APPE, the student was seeing patients in the intensive care unit.
Precepting Models used by Study Sites
For precepting models in which students interacted with multiple preceptors, students reported having to adjust to each preceptor’s working style and expectations. Most students did not identify this as a problem. However, three students said they would not recommend the site to another student and the reason given was connected to the student’s perception of deficiencies in the quality of precepting. Each of these students was at a site with multiple preceptors.
In their daily interactions with preceptors, students reported either working near the preceptor on the floor throughout the day (six sites) or working apart from the preceptor (six sites), either on a different service, or in one case, a different hospital. Presenting patients to preceptors was a frequently mentioned reason for interacting; these sessions ranged from 20 minutes to 2 hours in duration. Another reason for interacting was topic discussions, which occurred at all sites. Topic discussions varied from being informal (spontaneously arising from a specific patient situation) to formal (scripted and scheduled). Most were led by the preceptor but at two sites, students were charged with preparing for and leading all topic discussions.
Preceptors were asked about how much time they lost from their daily tasks when precepting a student versus how much time was saved because of tasks the student performed, allowing that preceptor to focus on tasks he or she would otherwise be unable to do. Preceptors from all sites expended extra time and effort in training the student during the first two weeks of the experience. After the first two weeks, as shown in Figure 4, preceptors differed in their perception of the student’s ability to perform tasks that would save them time. Preceptors’ perceptions of time taken was inversely correlated with hospital bed size (r = -0.72; p<.01), but were not correlated with years of precepting experience, average precepting scores, or type of precepting model used at the site. A few preceptors noted that if the students were at the site for six weeks rather than four weeks, the student would be able to learn more and could also provide greater value to the site.
Preceptor (n=15) Perception of Student Time Burden After the First Two Weeks of an APPE.
Interviewed students reported providing care to a range of seven to 30 patients daily by the end of their experience; the most common patient load was 10 to 15 patients daily, as reported by students at six sites. There appeared to be two methods for assigning patients to students. At some sites the student was assigned every patient on the service, or patients on the service that the preceptor or student deemed interesting. At other sites, the students performed specific tasks, such as vancomycin or warfarin dosing, for some or all patients receiving those medications, regardless of service.
Students at four sites reported daily interactions with patients and described performing warfarin and enoxaparin teaching, medication reconciliation interviews, discharge counseling, and chronic obstructive pulmonary disease and heart failure medication teaching. Student interactions with patients at other sites were identified as occurring infrequently.
Analysis of data from all students over the prior two years indicated that a large proportion of students at two sites felt fully accountable to the medical team for the outcomes of their assigned patients. Students at these two sites were assigned to every patient on a specific medical service, usually caring for eight to 10 patients, and the student was the primary pharmacy care provider for the team. The percentage of students feeling fully accountable for patient outcomes at the other 10 sites ranged from 0% to 56%. Full student accountability for patient outcomes to the health care team was not associated with average precepting score or years of precepting experience, size of the hospital, or average student learning score. There was a weak but significant correlation between the percent of students feeling fully accountable to the health care team for their patient outcomes and precepting model used (r = -0.64; p=.02), with students in model 1 more frequently reporting full accountability and students in model 3 less frequently reporting being fully accountable.
Students described interactions with other health care providers occurring primarily during rounds, which occurred at 10 of the 12 sites. Outside of rounds, students reported almost no interactions with other health care providers at two sites where students left the patient care area after rounds, occasional interactions in two sites where they rotated through different services in the hospital, and daily interactions at the other eight sites, including the two sites where there were no rounds.
About 70% of all students reported feeling highly integrated into the health care team at two sites where the student was the primary pharmacist for the team, there was daily rounding, and there were multiple daily interactions with other providers. About 50% of students reported feeling fully integrated into the health care team at four sites with either daily rounds or multiple interactions with health care team members. Few students felt fully integrated into the team at the remaining six sites, where interviewed students reported passive listening during rounds or responding to questions from other health care providers only when the preceptor was unavailable.
Examining identified components of inpatient general medicine sites through the lens of how each contributed to the student’s ability to provide direct patient care produced seven key components that should be standardized across sites. These key components, as well as underlying subcomponents, illustrative student quotes, tools for preceptors to meet key components, and metrics for each key component are outlined in Table 4. The student site evaluation has been modified to reflect the new questions and the site visit script as well.
Key Components to Standardize Across Inpatient General Medicine APPEs
DISCUSSION
Interviewed students were reasonable representatives of their classmates in gender and nature of prior internship experience. There was a slightly higher percentage of interviewed students with little hospital experience compared to the whole class, so interviewed students may have had a slightly steeper learning curve in their lived experience compared to other members of their class.
Interviewed students completed general medicine experiences at hospitals with a variety of sizes, patient ages, and socioeconomic backgrounds. Prior to carrying out this investigation, these demographics had not been measured. Recording such site demographics allows measurement of the diversity of patients that students are seeing, important metrics for ACPE Standard 13.2, which requires students to be exposed to diverse patient populations with regard to age, gender, ethnicity, socioeconomic factors, and disease states.4
The purpose of the inpatient general medicine experience is for students to learn how to provide care to patients with a variety of acute medical conditions requiring hospitalization. Reviewing this purpose was an essential step in identifying key components of the experience and determining the three overarching themes in the student general medicine experience. While each theme contributed to the quality of student learning at the site, it is perhaps the student’s interactions with patients and with health care providers that most affects formation of the student’s professional identity as a health care provider.11-16
Key components for student welcome at a site included an orientation and written materials for the student outlining the schedule and expectations. A “notable practice” in welcoming the student was providing written performance criteria so that the student would be clear about the expected level of performance and all preceptors would use a standard measure for assessing the student’s performance. The difference in identified expectations of students expressed by students (activities) and by preceptors (knowledge, skills, and behaviors) highlights the need for preceptors to provide explicit written expectations to their students.
Key components for student integration into the site included daily interactions between preceptor and student, and student participation in patient care activities. A “notable practice” occurred when the preceptor incorporated the student into the practice enough so that capable students were taking on some of the preceptor’s workload by the end of the experience, freeing the preceptor to perform other duties. It is likely that this could occur more frequently with a 6-week APPE compared to the current 4-week blocks.
Key components for student interactions with patients and other health care providers included meaningful daily interactions with both patients and other health care team members. A “notable practice” occurred when the student was made the primary pharmacy care provider for a service, after adequate orientation and role modeling.
New tools and measures were developed to aid in orienting preceptors to the required components and measure the same metrics across sites. The wording used on the new measures was kept purposefully broad so the question could apply to any patient care site. The next steps will be to measure preceptor response to the new key components, work with preceptors on how the key components might look in their specific clinical service and precepting model, and create feedback on how successfully each site incorporates the key components.
Only one description of standardizing an APPE has been published. Investigators from Butler University and Purdue University attempted to standardize preparation for a general medicine APPE conducted at one of four health care system sites precepted by faculty employed by one of the colleges.17 Students completed the same three web-based learning modules prior to starting the APPE. The primary outcome measure was the mean difference in the students’ pre- and post-module knowledge assessment scores. The investigators were successful in standardizing preparation for a general medicine experience across several sites, however they did not standardize components within those practice experiences.
This study has several limitations. Student experiences at metropolitan-area sites in the Seattle area only were examined. Small hospitals in rural sites or hospitals in other metropolitan areas might have a different precepting model than the three that were described in this study. The structural aspects of our general medicine experiences were addressed but key behavioral, knowledge, or skill components were not. This investigation was also only the first step in standardizing key components, with the next step being to visit each of our inpatient general medicine preceptor groups and discuss with them how each key component might look in their specific model. There is no guarantee that preceptors will be willing to make any needed changes to become standardized, although the preceptors who participated in this project were intensely curious about what everyone else was doing. The process was rigorous and may seem overly elaborate for identifying key components to standardize across general medicine sites. This might be unnecessary at a school where most of the general medicine sites are precepted by school-based faculty members. But because all our general medicine sites are precepted by volunteer faculty, it was imperative to first inventory the student experience at our most commonly-used sites. Thus, not only were standardization decisions based on what was most important, but also on evidence of what was likely achievable at the sites.
CONCLUSION
A qualitative approach to distinguishing components of inpatient general medicine experiences common to many sites facilitated identification of those components that could potentially be standardized across sites. Key components in welcoming a student included providing an orientation and written materials that outline the student’s schedule and performance, knowledge, skill and behavior expectations. Key components in integrating a student into the site were regular interactions between student and preceptor and performing patient care tasks such as monitoring and dosing medications and clinical documentation of decision-making. Key components in daily interactions with patients and other health care professionals included patient interviewing and medication teaching and engaging in discussions about patient medication-related issues with other health care providers.
ACKNOWLEDGMENTS
The authors gratefully acknowledge the support of this project by a grant from the Elmer M. Plein Endowed Research Fund in Pharmacy. The authors also gratefully acknowledge the assistance of Sujay Chang and Jennifer Huynh, PharmD candidates, in conducting some of the student interviews; Stanley Weber, PharmD, for acting as data broker; Kieran O’Sullivan, BA candidate, for analyzing the study data; Donal O’Sullivan, PhD, for creating the graphic figures and for assisting in the statistical analysis of the data; and Jennifer Chang, PharmD, for determining key components and benchmark metrics. Finally, the authors gratefully acknowledge the time and candor of the students and preceptors who participated in this study.
Appendix 1. Interview Scripts
Student Interview Questions
Background
1. Do you intern or work? In what kind of pharmacy setting?
2. How much hospital experience have you had? How many inpatient rotations did you have prior to the general medicine rotation?
Experiences
3. Tell me how the first week went for you.
4. Was it difficult acclimating to the environment?
5. Describe a typical day for you while on the general medicine rotation. (probe for specific amount of time spent on APPE-related tasks on-site and outside of site)
6. Describe your daily interactions with your preceptor.
7. Describe your daily interactions other health care providers.
8. Describe your daily interactions with patients. On average, how many patients do you care for each day?
9. Describe typical age ranges and medical conditions for patients you see/saw at this site. (probe if not much detail is offered)
10. Did you have the same preceptor the whole time? If you had multiple preceptors, how did their teaching styles differ?
11. Were expectations addressed and revisited throughout the rotation? What were they? Did you think the expectations were reasonable?
Reflection
12. Do you think you were adequately prepared for this rotation, in terms of both schooling and prior experiences? (probe for reflection on how prior inpatient rotations have helped prepare them, if applicable)
13. Did the therapeutics series and therapeutics skills labs help prepare you for this rotation? How so or why not? What other classes were particularly helpful in preparing you for this rotation?
14. Do you think you were adequately supported by your preceptor?
15. Were there any issues or confrontations with your preceptors or other members of the team? If so, please describe one example.
16. What suggestions do you have for preparing students for this rotation?
17. Would you recommend this rotation to an underclassman who was thinking about signing up for it?
18. Do you have any other comments you’d like to share?
Preceptor Interview Questions
Background
1. How long have you been working at this site?
2. How long have you been precepting students?
APPE Structure (This section will need to be directed to APPE coordinator if student sent with other preceptors on different days)
3. Tell me how you have structured this APPE for your students. (if not covered probe for orientation to the site, rough schedule of activities, method of evaluating student performance)
4. Describe the general demographic of patients (ages, socioeconomic status, ethnic origins, common medical conditions) a student would typically see during this experience.
5. What expectations do you have for students at your site?
6. Describe a typical day for you as a preceptor and pharmacist.
7. Describe how you make yourself available to the student. (if student is with other preceptors for time periods, describe how you learn about student performance from those other preceptors)
8. Describe activities or projects you have all students participate in during their APPE at your site.
9. Describe activities or projects you have some students participate in during their APPE at your site. (Probe for a description of a student project.)
Reflection
10. Describe how you act as a teacher. As a coach. As a learning facilitator. As a role model.
11. How well prepared do you typically find students from the University of Washington for the APPE you offer? (Probe for knowledge, skills, and attitudes)
12. How does your manager support you (or not) in hosting students at your site? This information will not be shared with your manager or management.
13. What is your perception about how much extra work a student “takes” on for you versus how much extra work a student “makes” for you?
14. Do you have any other comments you’d like to share?
- Received December 9, 2016.
- Accepted September 8, 2017.
- © 2018 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.