Preceptor Perceptions of Virtual Quality Assurance Experiential Site Visits

INTRODUCTION

With advanced pharmacy practice experiences (APPE) and introductory pharmacy practice experiences (IPPE) comprising approximately 30% of the curriculum in doctor of pharmacy programs, experiential education programs contribute significantly to the quality of pharmacy education.^1,2 The 2016 Accreditation Council for Pharmacy Education (ACPE) accreditation standards for the doctor of pharmacy degree program (Standards 2016) indicate that pharmacy schools are responsible for establishing a quality assurance procedure for all pharmacy practice experiences.³ Standards 2016 outlines goals of the quality assurance process, including to facilitate achievement of stated course expectations, standardize key components of experiences across all sites offering the same experiential course, and promote consistent assessment of student performance.³

Quality assurance site visits are a common component of the quality assurance procedures used in experiential education programs. However, pharmacy schools may be challenged by the increased focus on quality assurance and subsequent preceptor development needed as outlined in Standards 2016.⁴ Quality assurance site visits allow preceptors to establish effective lines of communication and strong relationships with experiential education team members, foster their overall relationship with the school, and receive feedback on ways to improve the learning experience at the experiential site.⁵ Preceptors also report site visits allow them to develop and refine the practice experience, resulting in improvements that benefit students.⁵ However, experiential education programs may find it increasingly difficult to perform regularly scheduled site visits due to recruitment of sites farther from campus, experiential education team workload, and budget constraints.^2,4,5

In fall 2016, 139 schools and colleges were accredited to offer the doctor of pharmacy degree (PharmD).⁶ Enrollment in PharmD programs increased 25.2% between 2007 and 2015, from 50,691 to 63,460 students.⁷ With the increased demand for experiential education placements, experiential education programs are seeking to attract, develop and retain qualified preceptors.² A 2011 survey identified site capacity as the most common concern facing experiential education programs, potentially requiring an expanded geographical range.^2,4,5 While demands on experiential education programs are increasing, concerns for adequate resources were also noted. Experiential education directors cited workload and funding as the second highest theme of concern for experiential education directors behind site capacity.⁴ Without sufficient human and financial resources, experiential education programs may be unable to thoroughly evaluate and support preceptors and sites at a level that ensures quality.⁴

More rigorous accreditation standards, shortage of experiential education sites, and time and budget constraints have placed pressure on experiential education programs as they attempt to identify, develop, and monitor APPE and IPPE quality.^2,4,5 As a result, some schools have found it increasingly difficult to consistently perform traditional face-to-face onsite quality assurance site visits.⁵ Exploration of alternative methods for performing quality assurance site visits that may reduce experiential financial and human resources is worthy of study but ensuring acceptance of new processes by preceptors is critical to effective outcomes. This study evaluates video conferencing as a means of supporting and developing existing preceptors and experiential sites. The purpose of this pilot study is to determine preceptor perceptions related to experiential quality assurance site visits through virtual communication. The objectives of this study are to determine to what extent there is statistically significant difference in preceptor perceptions of the value of experiential quality assurance site visits between visits conducted through virtual communication and visits conducted onsite, and to determine to what extent there is statistically significant difference in preceptor preference for the type of site visit they received.

METHODS

APPE quality assurance site visits conducted between August 2015 and January 2016 were included in this study. The study used a convenience sample of APPE sites located at least 200 miles from campus in order to minimize the chance that participants may have relationships other than the experiential affiliation with the experiential education program or team members. Sites were selected for visit type based on the ability to complete multiple site visits per day. Onsite visits were conducted when multiple sites were in the same geographic location while virtual visits were conducted for sites not geographically clustered with other APPE sites.

The same site visit data collection form was used for all visits to ensure consistency of topics covered across sites and visit types. The same members of the experiential education team conducted all of the visits. Virtual site visits were conducted using GoToMeeting version 7.1.8 (Citrix Systems, Inc., Santa Clara, CA). This software package allowed visual and audio real-time communication between the preceptors and the experiential education team members.

After completion of all site visits, participating preceptors were invited via email to participate in this study. A description of the study and a link to the survey instrument were provided. Participation in the survey was voluntary with no compensation or incentives provided for survey completion. Following the initial email, two additional email reminders were sent to all potential participants. Responses were anonymous and not tracked by participant. Completing the survey provided consent of the participant. The Drake University Institutional Review Board approved this research protocol.

The survey instrument was based on previous research analyzing preceptors’ perceived value of site visits.⁵ Agreement with seven survey statements addressing the first study objective was scored using a 5-point Likert-type scale, with higher scores indicating more agreement with the statement. Scores were averaged for each survey statement and aggregated. The Mann-Whitney U test was calculated using SPSS, v24 (IBM, Armonk, NY) to analyze differences in means of the individual statements and in aggregate. Participating preceptors also responded to two questions to address the second study objective regarding preferred method for future site visits. The Pearson Chi-Square test was calculated using SPSS, v24 to evaluate differences in self-reported preferences. The survey also included six demographic questions.

RESULTS

Between August 2015 and January 2016, 29 APPE quality assurance site visits (onsite N=17, virtual N=12) were conducted with 44 preceptors (onsite N=28, virtual N=16). Twenty-seven (onsite N=16, virtual N=11) of 44 preceptors responded to the survey, resulting in a response rate of 61.4% (onsite=57%, virtual=69%). All returned surveys were used in the analysis and contained no missing data. Table 1 summarizes aggregate demographics of respondents. The type of practice experiences provided by respondents was evenly distributed with hospital, acute care/general medicine, and specialty practice being the most common. Approximately 44% had been precepting Drake students for 5 years or fewer, and slightly over 50% precepted between one and three APPE students each year. Agreement with the seven survey statements assessing perceptions of site visit value was compared by delivery method. Table 2 displays the score averages for the survey statements, ranging from 4.2 to 4.6 (Mean=4.4, SD=.6) in the virtual site visit group compared to 4.25 to 4.69 (Mean=4.38, SD=.546) in the onsite site visit group. The results of the Mann-Whitney U test showed no statistically significant difference between the virtual site visits and onsite site visits (p=.482, alpha=0.05).

Preceptors reported their preferred method for future quality assurance site visits as displayed in Table 3. Preceptors were more inclined to prefer the type of visit they received (virtual=91%, onsite=50%) compared to the alternative method. However, preceptors who received onsite visits were more likely to indicate no preference (onsite=44%, virtual=0%) for the type of future visit. The results of the Pearson Chi-Square test indicated these differences were statistically significant for each method cohort (p<.001, alpha=0.05, Cramer’s V=0.896).

DISCUSSION

Results suggest preceptors view virtual site visits as an acceptable option for conducting quality assurance site visits. In addressing the first study objective, no significant difference was found between preceptor perceptions of the value of the site visit based on method of visit. When exploring the second study objective, preceptors who received virtual site visits highly preferred this method of delivery, suggesting virtual site visits may be as effective as onsite visits from a preceptor’s point of view. Interactions between site visitors and preceptors were similar when using either communication method. Preceptors participating in both visit methods were able to ask questions, clarify expectations, receive evaluation data and other information regarding their performance, and provide feedback for both the experiential education and didactic programs. Likewise, experiential education team members were able to read body language, interpret agreement, disagreement, or uncertainty during the conversation, and meet other people who interact with students other than preceptors.

Performing quality assurance site visits using virtual methods may alleviate some operational challenges for experiential education programs, such as travel expenses and time. These costs are particularly significant when the site is a substantial distance from the school or college or is not located near a cluster of sites to share the travel expenses among multiple site visits. Redistribution of these resources may allow for more frequent alternative contact with all experiential education sites.

The survey tool used in this study was adopted with permission from a previous survey used by experiential education faculty at the University of Oklahoma.⁵ Like these results, Burgett and colleagues found a high level of agreement with the survey statements, indicating value of site visits.⁵ Validation of this survey was based on its use to determine preceptors’ views on the value and optimal frequency of traditional onsite quality assurance visits.⁵ The survey did not pose specific questions related to the method of the site visit and was not originally intended to evaluate the specific benefits or limitations of performing virtual quality assurance site visits. Additional survey questions related to overall visit experience, reasons for preferring a specific visit method, and comfort with technology may provide important feedback on ways to optimize the benefits of virtual site visits.

Sites in this study were all established experiential education sites for the program, and regardless of method of delivery, the site visits were conducted for ongoing quality assurance. This study did not consider the use of alternative communication methods for site visits for the purpose of initial review and selection of experiential education sites. This study only examined the value of the site visit from the preceptor perspective. It is possible that positive preceptor perceptions may not equal quality of the site visit from the view of experiential programs. Further examination of ongoing site quality or improvement would determine if quality of the site visit varies with method of delivery.

There are other limitations to be considered when evaluating this study. A larger sample size may potentially show a difference between these delivery methods and minimize the impact of potential non-response bias. In addition, some sites that participated in virtual visits had not received an onsite visit for many years. The novelty of the site visit may have caused these preceptors to appreciate the site visit more than preceptors who receive site visits on a regular basis. Technical difficulties could arise when conducting virtual site visits. In one case, the preceptor’s computer lacked a web camera so only the audio functionality was used.

This study serves as the foundation for future areas of research related to alternative methods to enhance experiential site quality. Additional exploration of possible benefits for experiential programs includes potential savings of financial, human, and logistical resources. Research could also explore whether certain attributes of preceptor or practice demographics predict the effectiveness of certain visit methods.

CONCLUSION

Accreditation standards, experiential education site capacity, and time and budget constraints may pose challenges for experiential education teams performing onsite quality assurance site visits at every experiential site at the frequency desired. This study found no difference in preceptor perceptions of the value of the site visit based on method of site visit delivery. Preceptors who received virtual site visits preferred the same method for future visits. This study suggests virtual quality assurance site visits may allow experiential education programs to effectively monitor the quality of experiential education sites while potentially minimizing expenses and time requirements for both the site and experiential education program.