Abstract
Objective. To assess the impact of a comprehensive preceptor development program.
Design. A comprehensive preceptor development program was designed that included live and recorded online programming, a preceptor manual, a preceptor newsletter, live events (local and regional), and one-on-one practice site visits.
Assessment. Over 5,000 evaluations (1,900 pre-implementation and 3,160 post-implementation) of preceptor performance were completed by students. Students rated preceptors higher in items related to providing helpful midpoint and final evaluations after program implementation. Over 1,000 Web-based preceptor development activities were completed by preceptors from 2007 to 2011. Preceptors felt activities enhanced their current knowledge, skills, attitudes, and values, and more than 90% felt the core development activities would improve their current practice.
Conclusion. A comprehensive approach to preceptor development that offered a variety of development and training opportunities received positive evaluations from preceptors and resulted in improved student evaluations of preceptors. A comprehensive development program should be made available to preceptors to foster their continuing professional development.
- preceptor
- experiential education
- advanced pharmacy practice experience
- continuing professional development
INTRODUCTION
Pharmacy preceptors are teachers who facilitate practice-based learning for student pharmacists. With preceptors delivering an estimated 30% of the doctor of pharmacy (PharmD) curriculum, their training and development is an essential component of a successful experiential education program. The Accreditation Council for Pharmacy Education (ACPE) requires orientation, ongoing training, and development of preceptors.1
Preceptors are “adult learners” and have many different preferred learning styles. Though different theories and models of learning styles have evolved over the years, the fundamental tenet of most is the idea that individuals differ in their approach to learning tasks and their responses to those tasks.2 Therefore, successful preceptor development and training must include a constellation of educational activities and include resources to meet the diverse needs of all preceptors as part of their continuing professional development. For example, one preceptor may prefer an online program to view at his or her convenience, while another preceptor may prefer a live program that provides opportunities for interaction with the speaker and other preceptors. Preceptors practice in many different types of pharmacy settings and therefore may have different learning needs. For example, a preceptor in a community setting may need to learn how to integrate students into medication therapy management practices while another preceptor at an institutional practice site may need to learn how to start a journal club. A preceptor needs a unique set of skills and individualized education in different areas within this skill set. For example, one preceptor may need to learn about cultural competence, while another preceptor may need to learn about giving appropriate feedback or integrating students into clinical activities.
Preceptor development is cited in the literature as being an important component of pharmacy students’ clinical learning.3 In 2002, even before required by ACPE, 90% of colleges and schools of pharmacy offered programs for preceptor development.4 Assemi and colleagues reported that preceptors who had received training were more confident than preceptors who had not received training in clarifying expectations, evaluating a student's knowledge, and fostering critical thinking skills.5 However, there are no data to support whether this training and development would actually improve performance as measured by students’ rating on preceptor evaluations.
This paper describes an approach to foster initial and ongoing development of preceptors at one college of pharmacy. Additionally, this paper will examine evaluation and assessment data to determine the impact of preceptor development activities. The expected outcome of this comprehensive approach to preceptor development were that preceptors would enhance their knowledge, skills, attitudes, and values, and improve their practice site through preceptor development programs, and that student preceptor evaluation scores would improve.
DESIGN
The University of Iowa College of Pharmacy created a constellation of preceptor development programs to meet the needs of a diverse group of preceptors (Figure 1). The tools and activities that make up the college's comprehensive preceptor development program are described below.
The constellation of preceptor development and training programs at the University of Iowa.
The educational environment consists of 4 years of curriculum with an average class size of 108 students per class. The college has roughly 850 preceptors and uses over 300 pharmacy practice sites. Each student pharmacist completes 8 introductory pharmacy practice experiences (IPPEs) during the first 3 years and 8 advanced pharmacy practice experiences (APPEs) during the final year (Figure 2). The college coordinates over 1,700 individual practice experiences per year. The composition of the Professional Experience Program office includes: Director of Experiential Education (faculty), Director of Preceptor and New Site Development (faculty), Associate Director of APPE (staff), Assistant Director of IPPE (staff), and a Program Assistant (staff).
The layout of the professional practice experiences at the University of Iowa College of Pharmacy.
Web-Based Preceptor Development Tools
To decrease costs, faculty members from 2 colleges of pharmacy collaborated on the development and production of four 30-minute Web-based modules intended for initial preceptor development. The presentations were developed using Microsoft PowerPoint and Camtasia (TechSmith in Okemos, Michigan) and content was created followed continuing professional development principles. These 4 core modules were titled: “Introduction to The University of Iowa College of Pharmacy Professional Experience Program,” “The Role of the Preceptor,” “Developing Your Rotation,” and “Giving Experiential Students Constructive Feedback.” The content of all modules was peer-reviewed by experienced faculty members from the colleges of pharmacy.
The modules were made available online to preceptors in May 2007 to complete at their convenience. Promotion of the modules was done via e-mail and newsletter announcements and distribution of promotional magnets. New preceptors were required to complete the core modules before their practice could become a practice experience site. Preceptors received continuing pharmacy education (CPE) credit free of charge upon completion of the modules. The cost for processing the CPE credit was shared between the 2 colleges of pharmacy.6
Additional preceptor development modules were added to the Web site on an ongoing basis. The additional modules included content related to adult learning strategies, ethics, generational learning, continuing professional development, creating IPPEs, and mentoring. Topics for the online programs were generated through preceptor suggestions and development needs as seen by experiential faculty and staff members. Additional programs on clinical topics and a monthly journal club were also made available to preceptors. Clinical topics had practice-based teaching themes, for example, assisting students in migraine management. A monthly journal club was offered as a CPE learning activity for preceptors to complete with students at their site. This included a preceptor discussion guide to facilitate dialogue with students.
Costs for the preceptor CPE modules were shared between the college, another nearby college, and the state association. These 3 groups developed a national preceptor development network which collaborates for the development and dissemination of the Web-based modules and online journal club for preceptors. The estimated cost for each Web-based module was $5000. This cost was shared between schools at a tiered rate based on the number of preceptors at each college. The beneficial partnership brought the cost to $3 per preceptor at our college.
In addition to the online CPE modules, other training videos were created and disseminated when a new policy or procedure was implemented. For example, when the college changed the student evaluation process, an online training video was produced and published to teach existing preceptors the rationale for the change and the new procedure for evaluating student performance.
Printed Preceptor Development Tools
Various forms of printed documents were used for the development and training of preceptors, including a preceptor manual and newsletters. Preceptors were also directed to multiple written resources on clinical-based teaching (eg, the American Pharmacists Association Preceptor Resources Web page7 and the American Society of Health-System Pharmacists Preceptor Information Web page8). The preceptor manual was produced in 2008 to provide written guidance to preceptors. Created by faculty and staff members of the professional experience program, this written tool gave preceptors information on the college's educational philosophies, resources, policies, and curriculum. The preceptor manual was available online for preceptors and a hard copy was also provided to all the experiential practice sites in 2008. Updates to the preceptor manual were made on an annual basis.9
The college, which had published its preceptor newsletter sporadically since 2001, updated The Preceptor and began regular publication in 2007. The newsletter was sent to preceptors 2 to 3 times per year and contained information on benefits and tools available for preceptors, introductions of new staff members, and a list of reminders and important dates. Most issues contained an article written by a preceptor describing an innovative idea for practice-based teaching. The newsletter was created using Adobe Writer (Adobe, San Jose, California) and posted on the college's Web site. An e-mail containing a Web-link was sent to all preceptors each time an issue was published.10
Preceptor Development Events
The professional experience program hosted live events for preceptors. Six regional preceptor development programs and networking dinners were held across the state of Iowa. These events included a CPE program on setting expectations for and giving feedback to students and provided networking opportunities for local preceptors. Additionally, an annual event was held for preceptors in association with the college's career fair for graduating student pharmacists. This event included a meal, networking, and discussion of “hot topics” for preceptors. A third live event occurred annually in conjunction with the state association annual educational event. At the event, a portion called “preceptor pearls” was required by faculty members presenting on clinical topics. Additionally, at this educational event, a continuing education program on a specific practice-based teaching skill was held for preceptors. Topics have included developing and implementing IPPEs, giving feedback, and using reflection to enhance student learning activities. All events were promoted through e-mails, the newsletter, and print mailings to preceptors.
One-on-One Preceptor Development
Practice site visits ensure that student pharmacists are receiving quality practice experiences and increase communication between faculty members and preceptors; thus, site visits are an important aspect to ongoing preceptor development and training.11 Schools and colleges of pharmacy in District 5 of the National Association of Boards of Pharmacy developed a plan for quality assurance site visits in 2006. New sites and sites with numerous students were targeted for visits. These site visits followed the RAFT approach (Rapport-building, Assessment, Focused preceptor development, Thanks and trinkets) as described by Moser and colleagues.12 Site visits were completed year round by faculty and staff members in the experiential education office. The college also had faculty and staff members in the experiential program available for personal consultation for pharmacists wishing to develop their practice site into a learning environment.
The college used various strategies for quality improvement, including standard course evaluations, site evaluations, and the American Association of Colleges of Pharmacy Preceptor Survey and Graduating Student Survey. Also, student feedback and evaluation were solicited for every practice site and preceptor (findings presented below). Preceptors were able to access their evaluation data at any time, provided at least 2 students had completed an evaluation of their performance. This study was determined by the institution's investigational review board to be exempt from review.
EVALUATION AND ASSESSMENT
Two types of assessments were conducted: one to evaluate preceptor performance (ie, student evaluations of the preceptor) and the other to evaluate the teaching strategy (ie, the preceptors’ evaluation of the Web-based CPE programs).
Student Evaluations
A standardized preceptor evaluation was completed by students at the end of each APPE. Students were asked to what level they agreed with an evaluation item (6 = strongly agree, 5 = moderately agree, 4 = slightly agree, 3 = slightly disagree, 2 = moderately disagree, 1 = strongly disagree). Students were also asked to rate their preceptor overall (ie, 4 = excellent, 3 = good, 2 = fair, 1 = poor). Specific evaluation items related to feedback and evaluation of the student, orientation of the student to the site, organization of the experience, and overall impression of the experience. The evaluation data were assessed at 2 time periods: pre-implementation (ie, May 2003-May 2005) and post-implementation (ie, May 2009-May 2011) of the comprehensive preceptor development program. Evaluations completed during the 2 time periods were included in the analysis. No evaluations were excluded. A non-equivalent control group design was used for this analysis. T-tests and chi-squared tests were used to compare student evaluations on the items. A p value less than 0.05 was considered significant.
Students completed 1,818 APPEs during the pre-implementation period and 1,688 APPEs during the post-implementation period. Post-implementation, there was an increase of completed student evaluations of preceptors, ie, 3,160 evaluations compared to 1,900 pre-implementation. Table 1 outlines students’ evaluations of their preceptors, pre- and post-implementation. Items with high scores pre-implementation continued to have high scores post-implementation. Examples of these included the preceptor's interest in teaching and the preceptor serving as role model. When examining differences between time periods, preceptors were rated higher post-implementation than they were pre-implementation on items related to providing helpful midpoint and final evaluations. Figure 3 shows more preceptors were rated “good” and less preceptors were rated “fair” or “poor” after the comprehensive preceptor development program began (p < 0.05 for each).
Pharmacy Students’ Preceptor Evaluations Before and After Implementation of a Comprehensive Preceptor Development Programa
Comparison of overall student evaluations scores pre-implementation and post-implementation of a comprehensive preceptor development program.
Preceptors Evaluation of the CPE Program
After completion of the Web-based preceptor development modules, preceptors were asked to complete a standardized, anonymous evaluation of the program in order to obtain CPE credit. Preceptors were asked to respond to several questions, such as the overall quality of the program, using a Likert-scale (5 = excellent, 4 = above average, 3 = average, 2 = below average, 1 = poor). Preceptors were also asked to answer other questions with a response of true, false, or not applicable. Examples of questions in this series included whether the activity enhanced the current values, attitudes, skills, and knowledge of the preceptor and whether it would improve his/her practice. There were also 3 open-ended, reflective questions focused on the “take home” points the preceptor gained. Participants were preceptors from various colleges and schools of pharmacy, including The University of Iowa College of Pharmacy. Each activity was available on the Web for a period of 3 years. Evaluations submitted by preceptors completing the activities from 2007 to 2011 were included in the analysis. No evaluations were excluded.
Fifteen preceptor development activities were incorporated into the preceptor development Web-based CPE activities between June 2007 and July 2011, with an average of 4 activities added each year. Table 2 outlines the course evaluation data for the teaching-based teaching modules. Preceptors completed 1,066 Web-based modules from May 2007 to August 2011. The most frequently completed module was “Developing and Implementing an IPPE.” The preceptors positively rated the quality of all activities in the Web-based library. The overall quality of the program yielded mean scores ranging from 3.7 to 4.3 on a Likert scale of 1 to 5. “Mentoring: Helping Students Create a Career in Pharmacy” had the highest score of 4.3. The “Core Preceptor Development Modules,” a series of 4 activities required for new preceptors and optional for existing preceptors, was completed by 68 preceptors. The overall mean score for this activity was 4.3. More than 90% of preceptors who completed the core activity believed it enhanced their attitudes, knowledge, and skills. This positive perception was consistent for most of the other activities as well. In particular, knowledge gained was consistently rated high for all activities. Preceptors also responded positively when asked whether a program would improve their practice, which was a unique measure for continuing education activities. Ninety-four percent of preceptors completing the core modules believed participation in the CPE program would improve their practice.
Preceptors’ Evaluations of Web-Based Preceptor Development Modules
DISCUSSION
The college created an array of preceptor development and training programs to meet the diverse needs of its preceptors. McDuffie and colleagues reported high overall satisfaction in a consortium-based approach to preceptor development that offered Web-based preceptor training.13 Similar to their findings, we believe a collaborative approach decreases costs and improves overall quality of online preceptor development activities. In our evaluation, preceptors responded positively to online preceptor development programs. The preceptors perceived enhanced knowledge, skills, attitudes, and values, as well as improvement in their practice sites through completion of Web-based modules. This interesting outcome has been reported previously by Cerulli and Briceland who found that preceptors who completed a training program not only had increased confidence in teaching, but also in their ability to provide patient care.14
The expansion of preceptor development opportunities added new dimensions to the previous structure for preceptor training. In a study by Assemi and colleagues, over 70% of preceptors specified interest in attending live preceptor development training programs, which was greater than the approximately 50% who were interested in participating in Web-based courses.5 While preceptors may have a preference for learning in a live format, our study showed higher completion rates for the Web-based activities. Offering many formats for preceptor development allows preceptors to select activities specific to their learning preferences.
Completion rates for the online modules were highest in areas that were new to preceptors, such as IPPEs. All new preceptors were required to complete the core modules, and after doing so, nearly all stated it would improve their practice.
Student evaluations of preceptors improved in certain areas, specifically in providing helpful midpoint and final evaluations. Giving feedback and evaluation processes were a focus of multiple preceptor development activities, including the Web-based CPE, newsletter articles, and a chapter in the preceptor manual. Interestingly, there was no difference in the number of preceptors rated “excellent” by students, but there were differences in other ratings. This could indicate that the preceptors who received “fair” and “poor” ratings may have improved to “good” post-implementation, whereas preceptors rated as “excellent” maintained that rating.
Limitations to this study include the indirect nature of the data and the lack of a control group. Additionally, the change in the student evaluation scores could be due to history and time. Preceptors gain experience over time. Preceptor and student populations in the pre- and post-implementation groups were different and there can be considerable variability in student evaluation data. While ratings for some of the items on pre- and post-implementation evaluations were significantly different, whether this was a relevant difference may be questionable. The large sample size could have led to the significant differences seen.
The completion rate for student evaluations of preceptors in the post-implementation group was much higher than in the pre-implementation group. During this time, the college changed the mechanism for gathering evaluations from students. The new mechanism provided e-mail reminders to students, which may have increased the completion rate in the post-implementation group. Additionally, in the post-implementation period, students could evaluate all preceptors at a site, thus enhancing the evaluation data.
By offering a variety of preceptor development programs, barriers can be avoided. For example, those who lack Internet access could complete live or print forms of CPE, while those who are unable to attend live events could complete print or Web-based activities. The comprehensive approach to preceptor development minimizes barriers and maximizes preceptor opportunities.
When considering each activity individually, the participation may be small. However, when the entire constellation of activities is combined, the reach expands to meet the needs of the college. Even with this comprehensive approach to preceptor development, full participation is difficult to achieve. The college continues to work on ways to increase preceptor involvement in development activities. The college now requires documentation of preceptor development on the preceptor's faculty reappointment paperwork, which is reviewed and renewed every 3 years. The college plans to continue to use and develop new preceptor survey tools as a way to perform routine needs assessments. Additionally, quality assurance evaluations and data are monitored on a routine basis to identify preceptors and practice sites in need of development.
SUMMARY
Offering a variety of delivery methods for preceptor development and training programs provides preceptors with options for their self-directed learning. Preceptors reported that the online development activities enhanced their knowledge, skills, attitudes, and values, as wells as improved their practice. Additionally, improvements were seen in students’ evaluations of preceptors in areas related to midpoint and final evaluation of student performance. Ongoing assessment of preceptor development is important to meet the needs of students and the college as a whole.
ACKNOWLEDGEMENTS
The authors thank our colleagues at Drake University College of Pharmacy and Health Sciences, the Collaborative Education Institute, and the Iowa Pharmacy Association for the successful collaboration to develop and implement live and Web-based CPE activities. In addition, the authors thank Dr. Bernard Sorofman who contributed to the statistical analysis and Ms. Lisa DuBrava who assisted in graphic design.
- Received October 23, 2011.
- Accepted January 15, 2012.
- © 2012 American Association of Colleges of Pharmacy
