Abstract
Objective. To devise a pharmacy-specific, expanded entrustable professional activities (EPA) entrustment-supervision scale that would frame preceptor ratings in a prospective and retrospective manner for use in experiential learning settings.
Methods. A series of focus group sessions were conducted to solicit expert opinion on how to develop and refine two entrustment-supervision scales. Purposive sampling was used to identify experts from different professional groups (physicians, pharmacy experiential administrators, and pharmacy practice faculty) who had extensive knowledge regarding EPAs and at least one publication related to EPAs. Panelists were invited to participate via email. Three focus sessions were conducted via videoconferencing between June and September 2019. The primary outcome was development of a pharmacy-specific EPA entrustment-supervision assessment tool. Secondary outcomes were individual entrustment-supervision statements across five levels of the entrustment-supervision scale.
Results. The focus group consisted of four pharmacy practice faculty, two experiential administrators, and one academic physician. Four concepts emerged from the focus group discussion: need for more granularity in entrustment-supervision scales; limitations due to differences in licensure requirements across the United States; present and ongoing use of expanded scales by schools and colleges of pharmacy; and uncertainty regarding how to rate entrustment-supervision when a student exhibits unprofessional behavior.
Conclusion. A pharmacy-specific, expanded EPA entrustment-supervision scale will be useful to support longitudinal assessment of learners in experiential settings where an EPA framework is utilized. Determining when to use a prospective versus retrospective perspective requires further evaluation.
INTRODUCTION
Competencies describe the knowledge, attitude, and skill that a learner needs to master to be able to perform a professional activity.1,2 Competency-based education is essential for health professions to assure the public that each professional is capable of high-quality care and to reinforce the importance of lifelong learning.3 The 2016 Accreditation Council for Pharmacy Education (ACPE) Standards supports the use of competency-based strategies in pharmacy education similar to transformations occurring in medical education.4⇓-6
Entrustable professional activities (EPAs) describe professional tasks that a clinician is expected to perform autonomously after acquiring the requisite knowledge and experience.7 A trainee’s ability to perform an EPA is typically determined by the supervisor’s assessment of the trainee’s level of trustworthiness.7 Entrustable professional activities can serve as a link between competencies and professional duties in practice and as a framework for faculty to use to conduct ad hoc assessments and summative evaluations of the level of supervision or entrustment needed for each learner in a clinical environment.1,8⇓⇓⇓⇓-13 Fifteen EPAs, categorized into six domains, were devised by the 2015-2016 Academic Affairs Committee of the American Association of Colleges of Pharmacy (AACP) for pharmacy education to describe the core EPAs for new pharmacy graduates.14⇓-16 These 15 core EPAs for new pharmacy graduates have the potential to improve the efficiency of evaluations, be more meaningful to learners, and better document pharmacy students’ growth over time and readiness for practice.12,17 However, there is a need to further develop an assessment strategy when using EPAs in pharmacy education.
Student performance based on EPAs is evaluated by the level of supervision required rather than the assignment of a score, percentage, or letter grade as is typically done when rating student performance in traditional academic coursework.8,9 The 2015-2016 AACP Academic Affairs Standing Committee recommends new pharmacy graduates should, at a minimum, be able to perform each of the core EPAs with reactive supervision.15 Thus, new pharmacy graduates should be able to perform these activities without direct supervision but may still require the supervisor to be readily available and double-check their work.14,18 This level of supervision has been referred to in the literature as level III (Table 1). There are substantial differences between supervision levels I, II, and III. However, due to these relatively large differences in the currently available entrustment-supervision scales, it is difficult to track incremental improvements in learner performance over time, particularly during the early stages of skill development.19 Thus, more granularity in these lower levels is needed.
Multiple entrustment-supervision scales are used within medical education. Two such scales that are widely recognized are the Chen supervision scale and the Ottawa Clinic Assessment Tool.13,19,20 However, these scales frame entrustment decisions from preceptors (ie, supervisors) in different ways (Table 1). The Chen scale asks a preceptor to express their assessment as a description of the proposed level of supervision of a learner in future work.19 In contrast, the Ottawa scale asks the preceptor to describe how much supervision the preceptor provided during the experience.20 No research has directly compared the two assessment approaches. The Association of American Medical College’s Core EPA Pilot noted in May 2017 that evidence regarding the validity of the Chen and Ottawa scales was either limited or absent in the context of undergraduate medical education.21 Thus, we do not know whether a prospective or retrospective approach is the most useful way to make entrustment and supervision decisions. Moreover, these two scales are not designed to be applied in pharmacy education and use language that is specific to medicine, such as the level of oversight required during a surgical procedure or placement of a central line.19,20 Our goal was to devise a pharmacy-specific, expanded EPA entrustment-supervision scale that prospectively frames preceptor ratings of the learner’s level of entrustment, similar to the Chen scale, and retrospectively frames entrustment ratings, similar to the Ottawa scale.
METHODS
A series of three focus group sessions were conducted to gather expert opinion to develop and refine pharmacy-specific level of supervision scales. Focus groups were the qualitative method selected to best understand the experiences of pharmacy educators, clinical practitioners, and educational scholars regarding the use of EPAs as an assessment strategy in the experiential settings as focus groups enable researchers to gather data in a semi-structured manner and allows participants to respond to each other’s ideas.22 Purposive sampling was used to identify and invite experts who had extensive knowledge regarding and experience using EPAs to participate in the focus group. The research team initially identified experts through published works or professional organizational work related to EPAs or experiential education. Researchers sought an interprofessional panel, including physicians, pharmacists, and experiential education administrators, to increase the content validity of interprofessional and pharmacy practice activities, and implementation administration feasibility. Experts were recruited via email invitation. The goal sample size was eight participants from different professional groups (physicians, pharmacy experiential administrators, and pharmacy practice faculty).
Panelists were invited to participate in three focus group sessions via Webex (Cisco) videoconferencing. The three focus sessions were held between June 2019 to September 2019. For inclusion in the analysis, focus sessions required at least one of each professional group (physician, pharmacy experiential administrators, and academic pharmacists) and 75% of the total group of participants to attend. The objective for each of the focus sessions and the actions taken by the research team following each session are described in Figure 1. The primary author facilitated each focus sessions using a set of semi-structured questions. Recordings of each session were used by the research team to make tool modifications. A financial incentive (a $50 gift card) was provided to each panelist who participated in all three focus sessions.
Objectives and development plan for three focus group sessions held to develop an entrustment-supervision assessment tool for pharmacy experiential education
The primary outcome from the focus groups was an EPA assessment tool. Secondary outcomes were individual entrustment-supervision statements across five levels of the entrustment-supervision scale. Focus group recordings were made available to the research team. Within one week after the virtual focus group session, the research team reviewed the recorded focus group discussion and met via videoconferencing for the post-focus group to discuss tool development. During this session, the research team discussed concepts that emerged during the focus group discussion and implemented those ideas into the assessment tool. Drafts of the tool were sent to the expert participants one week prior to the next scheduled focus group session (ie, focus sessions 2 and 3). This study was approved by the University of Illinois Chicago Institutional Review Board.
RESULTS
Seven experts were confirmed and participated in all three focus group sessions. The focus group consisted of four pharmacy practice faculty, two experiential administrators, and one academic physician. Each expert had extensive knowledge regarding EPAs, including at least one publication related to EPAs and competency-based education.
Concepts emerged in each of the focus sessions. In focus session one, the lack of granularity of each level in traditional entrustment-supervision scales was not useful for early learners and the limitations of EPAs as an assessment tool because of state laws that restrict students from performing certain tasks without direct supervision. Additionally, some pharmacy schools are already using prospectively framed supervision scales.19 Based on this data, the research team drafted a revised tool with five levels of entrustment with sublevels. During focus session two, the question of how to rate unprofessional behavior on an entrustment scale was discussed. Additional editorial feedback was received and adjustment of the tool was completed. In the final focus session, the expert focus group provided approval of the final tool. The final tool is comprised of two matching components, a prospective and a retrospective frame for asking the level of entrustment and supervision questions. Each component has five levels with additional sublevels. The pharmacy-specific tool can be found in Table 2.
Prospective and Retrospective Entrustment-Supervision Scales Developed Using Sequential Focus Groups
DISCUSSION
The core EPAs for pharmacy graduates enable more authentic implementation of competency-based education in pharmacy.15 This potential shift from time-based educational models to competency-based assessments holds the promise of improving the practice readiness of learners to independently perform the core work of the profession. However, EPA statements must be operationalized for assessment purposes, including a supervision scale to use in the experiential setting. Our goal was to develop a pharmacy-specific, expanded EPA entrustment-supervision scale that enables preceptors to rate a learner’s trustworthiness, or conversely a preceptor’s level of supervision, using either a prospective or a retrospective approach. These tools are prepared with language appropriate for pharmacy practice and add the granularity of previously unavailable sublevels within the entrustment-supervision scale. These sublevels are useful for pharmacy preceptors to provide learner-centered support and feedback while maintaining high-level patient care during introductory and advanced pharmacy practice experiences.
Determining the practice readiness of each of its graduates is a critical responsibility of every pharmacy school. Unlike medical education, where postgraduate training is required before physicians can independently care for patients, new pharmacy graduates are expected to perform the professional activities of a pharmacist across multiple care settings from the time their license is granted.15 However, the practice readiness of new pharmacy graduates has been questioned, particularly their ability to independently perform direct patient care activities.23 Entrustable professional activities can act as a summative evaluation for practice readiness, particularly in the experiential setting, by describing the committed level of autonomy at which a learner has to perform a specific professional duty.12,13 This developed entrustment-supervision scale can provide comparable assessments of pharmacy learners across various pharmacy practice settings by preceptors to better describe a learners’ ability to perform professional work or rate their practice readiness.24 However, this is challenging in states that restrict the ability of a student pharmacist to complete an EPA, particularly one in which a retrospective scale is used. The prospective scale allows preceptors in restrictive states to provide assessments of students’ potential in licensed practice. Utilizing EPAs in this manner recognizes students’ ability to complete professional tasks, which aligns with patient safety goals as well as motivates and supports new learners through a clear understanding of how their performance on activities compares to that of others in their profession.19
The shift to using EPAs in pharmacy education enables educators and students to contextualize instruction as a continuum of learning toward the competent performance of specific tasks and roles. Advantages of an EPA assessment structure include promotion of a learner-centered approach focused on the learner’s abilities, trustworthiness, and growth, particularly in the experiential setting.25 This flexibility provides an opportunity for preceptors to meet students where they are in their learning journey and instill in them the important concept of lifelong learning. Fostering these habits of inquiry to build a pharmacists’ professional identity is essential for their continued development across the span of their career.25 This developed entrustment-supervision scale provides a framework for pharmacy learners from novice to near graduation to best understand where they are on the spectrum of becoming an autonomous pharmacist in a practical setting and has the potential to continue to drive them throughout their careers toward attaining supervisory roles. Next steps could potentially focus on a narrower scope on skills by observing practice activities and using a checklist of foundational steps in each EPA.26,27
The work presented here has limitations. Although there is interest in EPAs in pharmacy education, the number of experts who have experience using EPAs in the experiential setting are limited. Thus, the panelists invited to participate in our focus groups may miss some practical components that would increase the utility of the scale. Further, with a small group of experts, not all practice specialties are represented, such as critical care, oncology, pediatrics, or transplant. Use and validation of this tool is needed to support its widespread adoption.
This work developed a pharmacy-specific, expanded EPA entrustment-supervision scale that can be used as an assessment tool in pharmacy education in experiential settings. The expanded nature of this scale has the potential to describe learner progress toward practice readiness in their pharmacy experiential education. Further work is needed to validate these entrustment-supervision scales and test whether there are differences in outcomes using a retrospective rather than a prospective approach.
ACKNOWLEDGMENTS
This work was supported by the Hans W. Vahlteich Research Award from the University of Illinois Chicago.
- Received December 22, 2020.
- Accepted May 21, 2021.
- © 2022 American Association of Colleges of Pharmacy
REFERENCE
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.