Faculty Perceptions of Entrustable Professional Activities to Determine Pharmacy Student Readiness for Advanced Practice Experiences

INTRODUCTION

The Accreditation Council for Pharmacy Education (ACPE) Standards 2016 emphasize the need for students to not only demonstrate practice readiness upon graduation, but also readiness to enter advanced pharmacy practice experiences (APPEs).¹ The standards require schools and colleges of pharmacy to define performance levels that students must achieve prior to the start of APPEs. Standard 25.8 states, “the Pre-APPE curriculum leads to a defined level of competence in professional knowledge, knowledge application, patient and population-based care, medication therapy management skills, and the attitudes important to success in the advanced experiential program. Competence in these areas is assessed prior to the first APPE.”¹ These knowledge, skills, and attitudes should be expected of all students on day one of their first APPE. Matriculation through the didactic curriculum alone does not meet the intent of the standards. Even with the specific expectations listed within the standards, literature describing methods to assess APPE readiness is sparse and no consistent approach to determine APPE readiness has emerged. Therefore, it may be necessary to identify specific criteria or assessments to serve mechanisms to ensure students are ready for APPEs.

The discussion of specific criteria was further expanded in the Standards 2016 Guidance, with a notation that schools and colleges of pharmacy should identify professional activities that graduates perform routinely.² In response, the American Association of Colleges of Pharmacy (AACP) Academic Affairs Standing Committee developed a set of core entrustable professional activities (EPAs) for the profession of pharmacy.^3,4 The EPAs are "units of professional practice, defined as tasks or responsibilities that trainees are entrusted to perform unsupervised once they have attained sufficient specific competence."⁴ While the Center for Advancement of Pharmaceutical Education (CAPE) 2013 Educational Outcomes⁵ and Standards 2016¹ encompass the individual components of EPAs, the explicit EPA statements articulate the professional activities of practicing pharmacists. Therefore, EPAs create a framework to define practice readiness.

If EPAs are an appropriate framework for assessing practice readiness, backward curriculum design justifies the consideration of EPAs as a framework for assessing APPE readiness.⁶ The EPA framework establishes both specific skills and tasks, as well as a scale for the expected level of entrustment. The levels of entrustment (Table 1) range from solely observing (Level I) to supervising more junior colleagues (Level V). Literature suggests that upon graduation, students should be capable of performing skills and tasks with reactive supervision (Level III) at minimum.³

By working from the desired product of the PharmD curriculum, ie, a practice-ready pharmacist, a determination should be made as to where students need to be prior to APPEs in order to be on track to achieve these competencies.⁷ To our knowledge, there has not been a nationwide assessment as to whether EPAs have been uniformly accepted, or whether modifications (ie, adding or deleting EPA statements) when adopting the Core EPAs are common. Anecdotally, evidence exists that there may be a lack of uniformity across different schools of pharmacy with regards to the process of EPA implementation and integration.

We recognized that there are two constructs that needed to be considered when evaluating the use of EPAs for APPE readiness: the level of importance of the EPA in determining APPE readiness and the level of entrustment that the student should possess. The purpose of this paper is to quantify faculty perceptions of EPA importance and the expected level of entrustment that should be achieved to determine readiness to enter APPEs.

METHODS

A link to an online survey developed in QuestionPro (QuestionPro, Inc, San Francisco, CA), was distributed to the Ferris State University College of Pharmacy (FSU) faculty, staff, and administrators via email in August 2017, with a corresponding announcement made during a monthly college-wide meeting. A follow-up reminder was sent one week after the initial survey request to encourage participation. The survey was open for two weeks.

To assist with orienting survey respondents to various terms within the survey, definitions were provided. The term “APPE-ready” was described as, “students being ready to enter advanced pharmacy practice experiences (Block A).” “Block A” is the first scheduled APPE following completion of the didactic curriculum. “Practice-ready” was described as, “being ready to provide entry-level direct patient care in a variety of healthcare settings.” Levels of entrustment were also defined (Table 1). Additionally, FSU has adopted the Core EPAs as published, with the addition of a 16th EPA related to development of leadership skills.⁴ Pharmacy schools are not required to use the Core EPAs verbatim, which offers the opportunity for schools to customize EPA offerings to their needs.⁸ For uniformity with others that have adopted the Core EPAs as proposed, the scope of this paper was limited to the 15 Core EPAs.

A five-question survey was developed that included 18 unique statements that were adapted from Riskin and colleagues.⁹ These statements were selected as key examples of supporting tasks of the Core EPAs. Additionally, they provided greater context for professionalism and communication, which are foundational to the Core EPAs. The first question instructed respondents to rate how important they believed each of the 18 core pharmacist task statements were in determining whether a student was ready to enter APPEs; in other words, the degree to which they believed each statement was relevant for determining APPE readiness. Respondents selected their perceived level of importance of each statement from a four-point Likert-type scale, although they could select a fifth option of “unable to rate,” and responses were not required. When applicable, each of the 18 statements were mapped to the appropriate EPA and resulted in the mapping of 12 EPAs because they are taught and summatively assessed in the pre-APPE curriculum; the remaining three EPAs (EPA numbers 5, 12, 13) are taught throughout P1-P4 years, but are only summatively assessed during APPEs (Table 2). As the survey was intended to measure the opinion, beliefs, and attitudes of the respondents, Likert-type scaling was deemed an appropriate item format.¹⁰

The second question used the same 18 statements and asked faculty members to rate the level of entrustment they expected a student to have before beginning APPEs. The response options were the five levels of entrustment (represented in a Likert-type scale), along with an additional option of “not applicable” (Table 1). The remaining three questions were directed at internal quality-improvement issues and lacked external applicability. Therefore, this study focused only on the results of the first two questions, ie, importance and entrustment.

The FSU uses a consistent threshold approach to present assessment and related data to the faculty. To be consistent with this approach for the benefit of faculty engagement with the data, consensus thresholds were established to describe the level of importance and level of entrustment (Table 3). For the purposes of summarizing the data collected for each statement, the top two Likert-type scale responses (very important and moderately important) were collapsed and the bottom two Likert-type scale responses (slightly important and not important) were collapsed to create a dichotomous view of the results. Collapsing the Likert-type scale responses into a dichotomous measure allowed for a simplified comparison of respondent beliefs about each statement as a measure of APPE readiness. Specifically, they either believed it was important or they believed it was of little or no importance. This method of collapsing the Likert-type scale responses according to directionality is supported for purposes of interpretation of results and is generally accepted to have little, if any, impact on the validity or reliability of the findings.¹¹ Threshold criteria described in Table 3 were then applied to these dichotomous results to arrive at the overall consensus among the respondents on the level of importance for each statement (strong consensus, moderate consensus, little consensus). Additionally, a rank order was produced by ordering the percentage score of “very important/moderately important” for each statement. Percentage score ties were ranked by giving a higher ranking to the statement with a higher number of “very important” ratings relative to “moderately important” ratings. The Ferris State University Institutional Review Board (IRB) determined that this research did not meet the federal definition of human subjects research and was therefore deemed exempt.

RESULTS

Forty-four faculty members were invited to participate in the survey, of which 28 elected to participate (63.6%). Although the survey was administered to all faculty members, staff members, and administrators, a low response rate from administrators (n=1) and staff members (n=0) prompted the decision to eliminate the one administrator’s response from the data set and solely report the faculty responses. The majority of respondents were PharmD/practice faculty members (78.6% of all respondents, n=22), followed by PhD/science faculty members (21.4% of all respondents, n=6). The majority of respondents (55.17%) reported teaching primarily in the third professional year of the program, followed by 27.6% in the first professional year, and 13.8% in the second professional year. In terms of educational training, 89.3% (n=25) of respondents were licensed pharmacists. Of these, 88% (n=22) had earned a PharmD and 12% (n=3) had earned a BS in pharmacy. Faculty respondents who precepted direct patient care APPEs practiced in the following areas: inpatient medicine (41.2%, n=7), ambulatory medicine (35.3%, n=6), both inpatient and ambulatory medicine (5.9%, n=1), and electives (17.6%, n=3). For each of the 18 statements describing core pharmacist tasks to determine APPE readiness, respondents were asked to rate the two major constructs of importance and entrustment. Respondents’ reported levels of importance and entrustment were collected and thresholds for consensus levels applied to summarize the overall perceptions of the faculty.

Respondents were instructed to rate how important they believed each of the 18 core pharmacist task statements were in determining if a student was ready to enter APPEs. A ranking of the 18 statements according to the degree of consensus is included in Table 4. Overall, a strong consensus was reached on 16 of the 18 statements. Of the 16 strong consensus statements, six statements received zero responses of slightly important/not important, indicating that 100% of faculty members were in agreement that these items were most important. A moderate consensus was reached on two of the statements.

Conversely, of the 18 statements assessed for level of entrustment to enter APPEs, no strong consensus was ascertained. A moderate consensus was reached on seven of the statements, while little consensus was reached on the remaining 11 statements. The overall consensus results for levels of entrustment reported for each statement by the respondents is summarized in Table 5, along with the most frequently selected response. Of the 18 statements, 10 resulted in a median rating of a Level 3 entrustment. Additionally, 15 of the 18 statements spanned at least four of the five entrustment levels.

DISCUSSION

This paper describes a preliminary assessment of faculty perceptions of both importance and level of entrustment of EPAs as a metric to assess APPE readiness. Professional behaviors, including participation and preparation, ranked highest among faculty members. This supports the importance of these foundational behaviors necessary to establish trust and perform EPAs.⁷ A strong consensus was observed regarding which EPAs are appropriate for assessing APPE readiness, signaling that selected EPAs could be a logical metric. Of note, faculty members felt that EPAs that directly applied to patient care were more important as a metric to assess APPE readiness, which supports the expectation of students providing patient care on their first day of APPEs. Overall, public health-focused EPAs were seen as less important by faculty respondents. These data parallel those reported in a validation study of EPAs for new pharmacy graduates.¹² We believe that APPEs may provide greater context for continued learning about public health concepts, therefore it may not be critical to achieving a strong level of entrustability for public health-focused EPAs as a criterion for APPE readiness.

Little consensus was observed on the level of entrustment that is appropriate for EPAs prior to APPEs as demonstrated by the vast majority of EPAs receiving responses in at least four levels of entrustment. This is illustrated by some faculty members responding that APPE students should be able to supervise others in selected EPAs, while others felt comfortable with students only observing the same tasks upon entering APPEs. This wide distribution was somewhat unexpected and sparked several questions. To address the issue of faculty expecting an entrustment Level IV or V prior to APPEs, it seemed appropriate to consider the end goal of APPE training from an EPA standpoint. Although the literature suggests a Level III entrustment for graduates upon graduation, the data provided in this study suggest that many faculty members are not in agreement.³ In other words, we are still left asking the question: “What level of entrustment should we expect of our students upon graduation from a PharmD program?”

Standards 2016 require that graduates of a PharmD program are practice ready without any further (eg, residency) training.¹ If practice-readiness is the expectation upon graduation, it may be unrealistic to expect supervision of new graduates to be available in all practice sites. Recognizing that approximately 57% of pharmacists practice in community pharmacy settings, new graduates often find themselves as the only provider on duty.¹³ Level IV may be the closest supervision that a new graduate may receive in many settings, particularly when one considers practice in different urban or rural geographies. Additionally, new graduates often have technicians or interns staffing pharmacies where they are working, necessitating that new graduates provide supervision and give meaningful feedback to others (Level V) as a standard job expectation. Finally, Level III has been adopted from the medical education model where postgraduate residencies are mandatory, thereby requiring additional supervised teaching and learning over several more years of training.¹⁴ In pharmacy education, without the capacity or requirement for all graduates to enter a residency program, the expectation of substantial supervised teaching and learning after completion of the PharmD degree is impractical. Therefore, we question the appropriateness of a Level III entrustment threshold as a sufficient minimum for practice-readiness for pharmacists.

Based on the aforementioned limited consensus of APPE-readiness thresholds, plus the frequent expectation of Level III entrustment prior to APPEs, it is not clear whether faculty members at our institution consider Level III entrustment to be a practice-readiness threshold. Therefore, our next step is to build consensus around the level of entrustability we believe is appropriate for practice-ready graduates. Once we determine a level of consensus of entrustability thresholds for new graduates, we must accordingly adjust our expectations for entrustability prior to entering APPEs. In short, the lack of consensus of entrustability thresholds prior to APPEs points to the foundational issue for the profession to clearly define the level of entrustability for a practice-ready graduate of a PharmD program. Following the establishment of a consensus on the level of entrustment, our institution intends to use the rank order list to delineate which specific EPAs should first be assessed in the determination of readiness to enter APPEs. Additionally, EPAs alone may not be sufficient to define APPE readiness. Other constructs, such as professionalism, may be worthy of consideration as part of a comprehensive assessment approach.

This study has limitations. First, we surveyed faculty members from a single institution, and published literature on EPAs was relatively new at the time the survey was conducted. While the number of respondents was small and responses were collected in 2017, this early assessment data serves as a foundation for other schools and colleges of pharmacy to conduct similar assessment efforts to build consensus around APPE-readiness thresholds leading to appropriate entrustability of practice-ready graduates. The results of this study highlight the need for the Academy to devote greater thought to the issue of what constitutes a practice-ready pharmacist.

The following questions remain that may drive future research: What level of entrustment do employers expect following graduation across different practice sectors and geographies? What level of entrustment can our students reasonably attain while in our schools and colleges of pharmacy? How much growth is achievable within an APPE year? If specific sectors of practice require greater than a Level III entrustability upon entry to practice, how do we reconcile the disconnect of only expecting a Level III upon graduation?

CONCLUSION

This paper quantifies faculty perceptions of the importance of EPAs and the expected level of entrustment that pharmacy students should achieve to determine readiness to enter APPEs. Strong consensus was observed regarding which EPAs are important in the assessment of APPE readiness. Therefore, EPAs offer a useful framework for assessing APPE readiness. However, little consensus was observed related to level of entrustability. Lacking consensus of a threshold of entrustability prior to APPEs suggests that the Level III entrustability threshold for practice readiness in the literature is not universally supported. Furthermore, the authors question if a Level III entrustability threshold for practice readiness is aligned with employers’ expectations of graduates to practice independently in certain settings without additional residency training. In order for EPAs to achieve the intended outcome of advancing both pharmacy education and the practice of pharmacy, the questions posed need to be addressed and consensus within the profession must be built.