Abstract
Objective. To review the published literature describing how entrustable professional activities (EPAs) are currently used in health professions education with a focus on pharmacy education.
Findings. English-language searches in the databases CINAHL Plus and MEDLINE were conducted for January 2011 through March 2021, which identified 21 publications for inclusion in this review. The following themes were identified: frameworks for the development of EPAs, the implementation and evaluation of EPAs for workplace learning, and gaps in knowledge and future directions for EPAs.
Summary. Currently, no standardized approach exists for developing EPA frameworks for health disciplines. Also, the ways that EPAs are implemented and evaluated as an educational tool in practice settings are inconsistent. An opportunity exists to further establish the development, implementation, and evaluation of EPAs in all clinical practice settings. Much can be learned from other health professions to ensure successful translation of EPAs into pharmacy education.
- entrustable professional activities
- competency-based education
- pharmacy student
- medical education
- pharmacist
INTRODUCTION
In 2007, the concept of entrustable professional activities (EPAs) were introduced within medical education to address the gap between educational theory and clinical practice.1 Specifically, EPAs were introduced in response to increased awareness of educational knowledge about workplace-based learning.2 In general, EPAs are an observable method of assessment in the workplace and translate competencies into clinical practice.2,3 An EPA incorporates one or more core competencies that are observable and measurable and can be entrusted to the learner to perform at different levels of supervision as they gain competence.3,4 The intention is for the learner to progress to each milestone and reach a level where the task can be performed independently and the learner is able to supervise more junior staff when performing the task.3
Around the world, EPAs are gaining attention as a practical method to assess competencies in clinical practice.5,6 Implementation of EPAs has been demonstrated in several countries and health disciplines, but a standardized approach to the development of an EPA framework has not been established and has been identified as a priority for researchers.4,5 Ideally, an EPA framework should consist of a set of support documents or templates that provide guidance to researchers and educators when developing the EPA statements.7 Development of an EPA framework also requires that educational institutions and workplace requirements align.2 Encouragingly, a study by van Loon and colleagues reported that adaptive workplaces and quality-trained organizations have demonstrated successful development of an EPA framework.2 To further strengthen designed EPA frameworks, it has been suggested that future researchers use a published tool, such as the EQual rubric or the Quality of Entrustable Professional Activities (QUEPA) tool, to determine the quality and structure of EPAs.4,8,9
The concept of EPAs is relatively recent in pharmacy education; literature to date has focused on the development of EPA statements, mapping EPAs to program and accreditation requirements, validation of EPA assessment tools, and pharmacy preceptors’ and learners’ perceptions and experiences of using EPAs in practice.10-12 There is a clear need for further evaluation of EPAs that have been implemented in pharmacy practice settings, and such evaluations should consider feasibility, practicality, and whether this form of assessment is appropriate to support student learning. Therefore, the objective of this review is to describe how EPAs are currently used in health professions education with a focus on pharmacy education.
METHODS
English-language searches in the CINAHL Plus and MEDLINE databases were conducted for January 2011 through March 2021 using the following search terms alone or in combination: entrustable professional activities, pharmacist, competency-based education, medical education, and pharmacy education. A search in the ERIC database was also conducted, but no additional relevant articles were found. Articles were included from medicine, nursing, and pharmacy. Methodological quality was assessed using the Johanna Briggs Institute (JBI) checklist for systematic reviews, the JBI checklist for text and opinion, and the Critical Appraisal Skills Programme (CASP) Qualitative checklist.13-15 The proposed framework for narrative reviews by Ferrari was used to guide the preparation of the review.16
RESULTS
Twenty-one articles were identified for inclusion in the review. From these articles, three themes were identified: frameworks for the development of EPAs, the implementation and evaluation of EPAs for workplace learning, and gaps in knowledge and future directions for EPAs. The inclusion and exclusion of literature is described in Figure 1. Overall, 181 articles were excluded because they involved specialized medical fields and were not pertinent to the review question.
Flowchart of the study design used in a review of Entrustable Professional Activities in pharmacy education.
Frameworks for the Development of EPAs
Eight articles relating to the development of EPA frameworks were identified for inclusion (Table 1). Three articles were descriptive in nature and five were analytical studies.8,17-23 The majority of the articles regarding frameworks for the development of EPAs for the workplace were in the setting of medical education. The two validated tools identified to assist in the development of EPAs are the QUEPA tool and EQual rubric.8,9
Findings From a Review of the Literature on Frameworks for the Development of EPAs
Pittenger and colleagues described the process of developing an EPA framework that suits the pharmacy practice context while adhering to the principles outlined by ten Cate for medical education as closely as possible.17 Pittenger also highlighted that it is important for EPAs to align with competencies of the pharmacy profession and for preceptor development sessions to explain the purpose of EPAs and how to use the framework; this approach is also supported by other authors.17-19 Kanmaz and colleagues also described how an EPA task force successfully mapped all 15 core EPA statements developed by the American Association of Colleges of Pharmacy (AACP) for pharmacy graduates in 2017 to the five educational guidance documents that govern the Doctor of Pharmacy (PharmD) curriculum in the United States.21 In another study, Pittenger and colleagues developed a questionnaire based on the validated QUEPA tool looking at pharmacy students’ perceptions of EPAs, and they found that >94% of students agreed that 14 out of the 15 EPA statements published by the AACP were relevant to pharmacy practice.23 Haines and colleagues also used the QUEPA tool, surveying pharmacists instead of students.19 In this study, the authors found consistently high agreement (>75%) among experienced pharmacy practitioners that the 15 EPAs developed by the AACP were important to pharmacy practice and set a clear expectation of what pharmacists are expected to do regardless of practice setting.19 A significant amount of work has been completed around the development and validation of the 15 core EPAs for new pharmacy graduates, published by the AACP in 2017, but limited published evidence shows that these EPAs are integrated in the clinical setting to support workplace-based learning.10
Further guidance has been provided around the development of EPA frameworks, as researchers have developed and validated a scoring rubric with descriptive anchors to evaluate EPAs.8 Taylor and colleagues developed the EQual rubric as a reliable measure for the quality and structure of EPAs and provided guidance for directing revisions. The EQual rubric was developed for medical education, but it can also assist with validation studies for researchers developing EPAs in other health profession fields.8 Meyer and colleagues applied the EQual rubric questionnaire to 13 core EPAs developed by the Association of American Medical Colleges (AAMC) for entering medical residency.20 Nine of the 13 EPAs scored above the overall cutoff score for validation, indicating alignment with the key domains for EPA design.20 The EQual tool is useful for identifying which EPAs need to undergo further revisions to ensure they are ready for use by supervisors and learners in the workplace. Further, the authors stated that EPA development requires determined standards and, thus, a criterion-based evaluation is appropriate.8 This contrasts with the QUEPA tool, which uses a normative scale for each question without descriptive anchors.8
An exploratory qualitative study of trainers and trainees in the Dutch general practitioner training program explored and reported their thoughts, views, and expectations about an EPA framework.22 Advantages identified through the focus group included trainers’ and trainees’ feelings that the EPAs provided a broad and structured overview of training required for general practice and that the EPAs reflected situations and tasks commonly encountered in daily clinical practice. Disadvantages included trainers’ and trainees’ feelings that there were too many EPAs, that some EPAs were very general, making them unsuitable for formal assessment, and, as one respondent mentioned, that the “long list of EPAs may lead to a situation in which this is used as a checklist, rather than as a supportive educational tool.”22 Trainers and trainees both indicated that EPAs can support trainee learning in several aspects of the learning process, including understanding training requirements, providing feedback on current performance, and developing feed-forward strategies to further progress trainee skill development.22
Implementation and Evaluation of EPAs for Workplace Learning
Eleven articles pertaining to implementation and evaluation of EPAs to support workplace learning were identified for inclusion (Appendix 1). Four articles were descriptive in nature, and seven were analytical studies.2,10,12,24-31 Many articles detailed the challenges of implementing EPAs in the workplace.2,29,30 In the context of medical education, van Loon and colleagues suggested that successful implementation of EPAs means that both the learners (medical residents) and program directors should have a clear understanding of the resident’s progress in training, that workplace creativity should be present to fulfill any remaining needs for training, and that the training program should be flexible. They also suggested that for successful implementation of EPAs, the workplace must be flexible and willing to change, and faculty development programs must focus on EPAs.2 Jarrett and colleagues described the potential benefits of implementing the 15 core EPAs published by the AACP into practice for PharmD students.29 A challenge to implementation they considered was translating EPAs into a traditional grading system. The AACP recommends that PharmD students achieve entrustment level 3 (reactive supervision) for each EPA upon graduation.29 This sets the expectation for student performance upon graduation but offers little guidance about how students achieve milestones as they progress through their course of study.29 Thus, the authors suggested regular progress checkpoints for assessing student progression by working backward from graduation to the first year of study.29 They also considered the potential of EPA personalization to specific practice environments. For example, an EPA for establishing a new patient care service could be developed for a student pursuing pharmacy ownership.29
A cross-sectional survey conducted by Jarrett and colleagues evaluated implementing EPAs in a medical residency program.28 The survey of 503 program directors showed that 90.1% were aware of EPAs as an assessment for residents, 82.8% reported understanding the principles of EPAs as an assessment, but only 30.2% reported being extremely confident in using EPAs. The largest challenges identified were difficulty in integrating EPAs into the current assessment program and a lack of faculty development regarding EPAs.28
Rhodes and colleagues conducted a pharmacy student self-assessment and a preceptor-based assessment of students’ EPA levels during a two-month pharmacy practice experience.12 Students’ self-evaluations and preceptors’ evaluations of students at the midpoint and end of the experience reported a median EPA level of 3.0 and 4.0, respectively. This study demonstrated that EPA statements can be implemented into pharmacy education and can be used to assess a learner’s level of performance on core pharmacy practice activities.12 In a retrospective review, Sjoquist and colleagues investigated and evaluated open-ended feedback from preceptors to student pharmacists regarding their performance on EPAs during a pharmacy practice placement.10 The authors concluded that subjective, preceptor-provided feedback can assist in identifying the strengths and challenges that influence student pharmacists’ progression and can assist in understanding the learners’ abilities during a practice experience.10 A limitation in both these studies was the EPA statements were developed for their respective institutions for pharmacy students, such that they differed from those published by the AACP for pharmacy graduates.10,12
Currently, literature in pharmacy education has yet to fully explore the implementation experience and effectiveness of EPAs for workplace learning for pharmacy graduates.10 However, Westein and colleagues described the development, implementation, and evaluation of EPAs for community pharmacy trainees in the Netherlands.27 The authors highlighted the use of design approaches from the medical field to develop the EPAs, similar to methodologies reported in other pharmacy education studies.17,19 After EPAs were implemented in a community pharmacy cohort, focus groups from the cohort reported a heavy workload, insufficient insight into what is expected of both trainees and supervisors within the curriculum, and challenges with using the assessment tools and portfolio.27 The authors concluded that further evaluation is needed around whether the educational environment in community pharmacies is suitable to support and encourage trainees’ learning and performance.27
In an exploratory qualitative study, Lau and colleagues described the perceptions and experiences of using new EPAs in nursing students as well as hospital and university clinical instructors.25 The EPAs allowed flexible assessment methods, fostered students’ critical thinking, promoted team-based care, and encouraged peer teaching.25 However, EPA assessment accuracy was impacted by multiple competencies for the EPAs, subjective assessment criteria, lack of standardization using checklists, and staff availability.25 The authors concluded that implementing EPAs in nursing education is an evolving process, and further refining is needed to adapt the EPAs to improve the learning experience.25
Martin and colleagues conducted focus groups to understand how EPAs and entrustment scales influenced feedback and learning in the clinical setting for medical trainees.24 The authors concluded that EPAs need to be standardized to support learning but also comprehensive and flexible.24 They also found that while EPAs can support goal setting and learning for trainees, they pose a risk of becoming a checklist exercise.24 Finally, the authors also concluded that reducing emphasis on the entrustment score and focusing on constructive feedback and discussion for skill progression may provide a more supportive environment to assist learning.24
As a strategy to enhance implementation of EPAs, ten Cate and colleagues explored the use of an electronic portfolio (e-portfolio) model for assessment and entrustment.30 In their study, 36 students participated in a 12-week pilot of EPAs within the existing curriculum, using an e-portfolio specifically designed to support the EPA-based curriculum.30 This study has yet to evaluate the EPAs in practice, but the authors concluded that incorporating the EPAs and using e-portfolios requires substantial explanation and guidance.30
Two additional small studies evaluated the feasibility of EPAs introduced to the workplace for medical education.26,31 A program feasibility evaluation piloted two EPAs in an internal medicine residency program. Attendings felt that the EPAs were useful for learning and feedback, but barriers included time demands and challenges in informing all staff in the program about a new form of assessment. Interns felt that the work culture pushed them to prioritize immediate work duties over EPAs, and medical attendings struggled to find time together with the intern to observe the EPA and complete the EPA rubric.31 Baer and colleagues assessed the feasibility and satisfaction of using EPAs for physical medicine and rehabilitation training program residents.26 Attendings found EPAs feasible and generally reported satisfaction with EPAs as a way of providing feedback to residents. Feedback also suggested changes that could be made to improve future implementation, which included a more tailored entrustment scale for this practice area of medicine.26
Gaps in Knowledge and Future Directions for EPAs
Two articles relating to gaps in knowledge and future directions for EPAs were identified for inclusion. Table 2 summarizes the gaps in knowledge and future recommendations for EPAs. Although commentary in all articles included in this review discussed gaps in knowledge and the future directions and recommendations for EPAs in health professions education, a systematic review of EPAs in medical education and a scoping review for various health disciplines highlighted this theme as a main discussion point.4,32 O’Dowd and colleagues included 49 articles in a systematic review of EPAs in medical education, including 39 articles on development of EPAs and 12 relating to EPA implementation and/or evaluation (two studies are included in both categories).4 The authors highlighted that a strong focus exists on EPA development, but EPA implementation is also a critical part of the EPA process, and currently this area of research is lacking. The authors recommended that further data be collected on validity, feasibility, and utility of EPAs as an assessment tool for trainees.4
Identified Gaps in Knowledge and Future Recommendations for EPAs
Findings From a Review of the Literature on Implementation and Evaluation of EPAs for Workplace Learning
Shorey and colleagues conducted a scoping review for EPAs in health care education that included 80 articles.32 The article types included narrative articles (n=13), research reports (n=40), perspective papers (n=10), commentaries (n=11), an editorial (n=1), and summit reports (n=5). A thematic analysis revealed the following main themes: development of EPAs, evaluation of EPAs, entrustment factors, and future direction and recommendations for EPAs.32 The authors concluded that the recent literature principally reports the development and implementation of EPAs in various health disciplines. Supporting the conclusion reached by O’Dowd and colleagues, this review also highlighted the lack of quality evidence-based research on the effectiveness and validity of EPAs as an educational tool on a worldwide scale.4,32
DISCUSSION
The findings from this review add to the literature by describing how EPAs are currently used in medicine and nursing education with a focus on pharmacy education. This review highlights key considerations in the development of EPA frameworks, EPA implementation, EPA evaluation, and future directives for EPAs.
Varied methods of development and validation of EPAs were reported, with some commonality in the themes. Nursing and pharmacy health disciplines tend to follow the approach used by medical education in the development of EPAs but individualize the EPAs to their local setting and needs. The development of EPA frameworks for the workplace are often modeled on medical education, which link the educational context into the practice setting.33 Currently, there is no standardized approach to the development of EPAs. The use of standardized frameworks could improve the development process of EPAs, and this has been acknowledged as a future focus for researchers.4,32 A suggested template by O’Dowd and colleagues follows three steps: initial development, expansion, and validation.4
The articles reviewed for EPA framework development indicated that medical and pharmacy educators recognize the importance of mapping the development of EPAs to the university’s curriculum and any existing competency frameworks to translate educational theory into clinical practice.17,18,21 Kanmaz and colleagues described how an EPA task force successfully mapped all 15 core EPA statements developed by the AACP for pharmacy graduates in 2017 to the five educational guidance documents that govern the PharmD curriculum.21,34-37 This demonstrates the connection between curriculum design and assessment of student learning.21
During the EPA framework development stage, the QUEPA tool or EQual rubric can be applied to assess the quality of EPAs.4,8,9 Applying these validation tools when developing an EPA framework ensures the EPAs are of high quality and have an appropriate structure for use both in the workplace and as an educational tool that assists the learner. No articles included in this review favored or prioritized the use of either the EQual rubric or the QUEPA tool. Further, no other methods or tools for the development of EPAs were identified; therefore the most effective method of developing high-quality EPAs could not be established.
The systematic review by O’Dowd and colleagues acknowledged the absence of a standardized approach for developing an EPA framework and highlighted that this should be a priority area for future activity.4 To assist in successful implementation of EPAs, they recommended establishing a uniform approach to developing an EPA framework.4 Such a uniform approach can provide guidance for future researchers and increase the possibility of EPAs being a valid and effective learning tool for students in clinical practice.4
Following the EPA development process, implementation and evaluation are necessary to ensure the EPAs are suitable for workplace learning. No standardized approach or method to guide implementation and evaluation of EPAs has been identified in the literature.4,32 Currently, only a few studies have evaluated the feasibility of EPAs as an educational tool once introduced to the workplace.26,31 In this review, documented methods to evaluate EPAs in the workplace included focus groups and surveys.24,25,28 This review found that successful implementation of EPAs may be underpinned by a rigorous and methodical approach in the development stage. Many authors reported challenges with implementation; thus, until a set of standards to guide EPA development is established, high-quality evidence-based evaluation of EPA implementation may be limited. This is supported by the systematic review by O’Dowd, which discussed multiple approaches to implement and evaluate EPAs, suggesting that best practices for implementing EPAs are currently unknown and/or not agreed upon.4
The current review found that pharmacy educators are following the strategies used by medical educators in the development of EPAs, including focusing on developing EPA statements, linking EPA statements to accreditation requirements, and validating EPA tools.10 We recommend that a standardized approach to developing EPAs in health education be established and include the use of published frameworks. This could mean that the development phase of EPAs includes using the QUEPA tool or EQual rubric validation tool.8,9
International accreditation programs are adopting EPAs into their pharmacy education standards.36,38 This review provides additional guidance to accreditation bodies about ongoing development of EPAs and strategies for implementation and evaluation. Based on this review, we recommend that pharmacy educators involve key stakeholders in the development of EPAs. Having stakeholder involvement and feedback in the developmental process may assist in the implementation and acceptability of EPAs in the workplace.
Literature describing the implementation and evaluation of pharmacy-specific EPAs is scarce, reflecting the current situation in medical education.4 Based on our findings from this review, we conclude that it is crucial to focus on the implementation and evaluation of EPAs in addition to the development of EPAs. Future directions for EPAs must include evaluating the feasibility and utility of EPAs in the workplace to understand whether they are the most appropriate educative tool to support learner progression and skill development.
CONCLUSION
A standardized approach to the development of EPA frameworks for medical, nursing, and pharmacy disciplines has yet to be established. Also, literature on the implementation and evaluation of EPAs as an educational tool in the practice setting is inconsistent and lacking. An opportunity exists to further establish the development, implementation, and evaluation of EPAs for clinical education. EPAs are relatively new in pharmacy practice settings, and much can be learned from other health professions to ensure successful translation of EPAs into pharmacy education.
ACKNOWLEDGEMENTS
The authors acknowledge Mina Nichols-Boyd for assisting with the database review and Julian Lindsay and Michelle Vienet for their assistance in editing the manuscript.
- Received August 28, 2021.
- Accepted December 13, 2021.
- © 2023 American Association of Colleges of Pharmacy