Abstract
Entrustable professional activities (EPAs) are specific tasks that a professional is entrusted to perform autonomously and, together, they define the scope of a profession. There are specific attributes that a task must possess both structurally and conceptually to be classified as an EPA. A high-quality EPA must be an observable, measurable, and professional task that requires training to execute, is fit for entrustment, and can be performed independently. In 2017, the American Association of Colleges of Pharmacy (AACP) defined 15 core EPAs expected of a pharmacy learner upon graduation (Core EPAs). Despite acceptance and implementation by pharmacy schools across the country, the Core EPAs have not been evaluated using an objective assessment tool to ensure that they meet specific EPA quality standards outlined in the literature. This article describes existing objective assessment tools for EPA quality and highlights the importance of ensuring high-quality EPAs in pharmacy education, which would be an important step for the AACP Academic Affairs Committee to take to further develop EPAs for implementation.
INTRODUCTION
Entrustable professional activities (EPAs) are specific tasks that a professional is entrusted to perform autonomously, and, together, they define the scope of a profession.1 Entrustable professional activities were proposed as a means to operationalize competency-based education, which is a curricular model defining learner outcomes using competencies where competencies are the components of a professional synthesizing a compilation of knowledge, skills, and attitudes.1,2 Competencies describe the abilities of learners, while EPAs describe units of work. A learner’s EPA performance should be assessed using an entrustment-supervision scale that asks the evaluator to indicate the amount of entrustment and supervision they provided a learner performing an EPA.3 Also, EPAs should be mapped to the competencies most important to their execution, allowing the educator to infer proficiency in those competencies when the learner achieves complete entrustment for an EPA.4
There are specific attributes, outlined in the literature, that a task must possess both structurally and conceptually to be classified as an EPA.2 A high-quality EPA must be an observable, measurable, and professional task that requires training to execute, is fit for entrustment, and can be performed independently.2 EPAs should avoid descriptions of learners and focus only on the task. Usually, an EPA incorporates one or more competencies. Therefore, an EPA’s execution should require proficiency in multiple competency domains.
Pharmacy education uses a competency-based approach, whereby the Accreditation Council for Pharmacy Education (ACPE) Standards 2016 employ competency-based objectives to define learner outcomes.4 In 2017, the American Association of Colleges of Pharmacy (AACP) defined 15 core EPAs, outlined in Table 1, categorized into six pharmacy practice domains, that a pharmacy graduate must be able to perform without direct supervision upon graduation (Core EPAs).4 The Core EPAs have been linked to several educational outcome frameworks like the ACPE Standards 2016, the Center for the Advancement of Pharmacy Education (CAPE) Educational Outcomes 2013, and the Joint Commission of Pharmacist Practitioners (JCPP) Pharmacists’ Patient Care Process, among others.5,6 In their implementation roadmap, the 2016-2017 AACP Academic Affairs Committee recommended that graduates be able to perform the EPA with indirect supervision but encouraged pharmacy schools to set individual standards for achievement and supplement the Core EPAs with additional EPAs as necessary to meet the institution’s goals.5 Pharmacy schools across the country have integrated the EPAs into curricula to develop instruments to assess practice readiness, create tools to assess experiential student performance, and blueprint objective structured clinical examinations (OSCEs), among others.3,7-9 Notably, the 2021-2022 AACP Academic Affairs Committee is charged with the responsibility of revising the Core EPA statements.10
Core Entrustable Professional Activities for New Pharmacy Graduatesa
Despite their implementation, the Core EPAs have not been evaluated to ensure they meet the specific extant quality standards outlined by ten Cate.2 The objective of this review is to highlight the importance of high-quality EPAs in pharmacy education and describe existing objective assessment tools to determine EPA quality for implementation.
DISCUSSION
Importance of Quality Entrustable Professional Activities
Ensuring high-quality EPAs is essential given their patient safety roles across the pharmacy profession. Entrustable professional activities clarify assessment expectations by setting a standard for reference.5 For educators, EPAs outline professional abilities as a framework to document student progression within the program.3 For pharmacy schools, EPAs serve as a mechanism for curriculum evaluation and improvement. For students, EPAs provide a foundation for setting goals, measuring progress, and building self-reflection skills.4
EPAs also define a pharmacist’s value, scope of practice, and responsibilities to prospective students, regulatory policymakers, payers, interprofessional team members, patients, the public, and the media. Through reliable and high-quality EPAs, prospective students may better identify with what it means to be a pharmacist and begin to build their professional identity. Regulatory policymakers and payers may incorporate pharmacists into more clinical settings with knowledge obtained from EPAs. For the health care team, patients, the public, and the media, the pharmacist’s role can be better defined as the medication expert rather than solely one who dispenses medication.5 Pharmacy organizations can align their visions using EPAs, thereby strengthening their communication, collaboration, and professional advocacy ability. Finally, revisions to existing EPAs can communicate changes in pharmacy practice within the profession.5
With their multifaceted role in curricular design, assessment standards, and representation to stakeholders, it is critical that EPAs be of high quality to ensure that new graduates can perform the tasks of a pharmacist that contribute to patient safety and improve patient outcomes. New graduates may be unable to perform the actual duties of the profession if they have been assessed using poor-quality EPAs. Misrepresentations of pharmacy practice to stakeholders may also result, deemphasizing pharmacists’ abilities and importance in health care.
Quality Assessment Tools for Entrustable Professional Activities
Two assessment tools have been developed to assess EPA quality, the Quality of EPA (QUEPA) tool and the Queen’s EPA Quality (EQual) rubric.11,12 Characteristics of each are outlined in Table 2. To develop the QUEPA tool, researchers initially identified seven domains essential to EPA quality: focus, observable, clear intention, realistic, articulates trustworthiness, generalizable across rotations, and integrates multiple competencies. Three items were created for each domain using a five-point Likert scale. After pilot testing, clear intention and articulates trustworthiness were removed due to low interrater reliability, leaving five essential EPA domains. These domains were then reduced to four distinct “factors” in the final 15-item instrument: realistic and generalizable, observable, focused, and multiple competencies. The development team applied the QUEPA tool to a series of established EPAs and identified multiple sources of validity evidence under content, internal structure, and criterion validity.11
Quality Assessment Tools for Entrustable Professional Activities
There are challenges to the use of the QUEPA tool. The QUEPA tool has been criticized for excluding two of the original seven domains, namely clear intention and articulates trustworthiness—initially identified as vital to EPA quality—simply due to poor interrater reliability.13 Further, the application of the QUEPA tool to established EPAs by the developers themselves likely resulted in the high interrater reliability exhibited by the rest of the domains, because users did not need to be trained for implementation. Moreover, the QUEPA tool does not provide user training resources, thus application, analysis, and interpretation of results by users outside of the development team may be inaccurate. Finally, the QUEPA tool does not provide predefined cut scores for acceptability of an EPA’s quality, limiting its utility and interpretation.
To address these critiques, the EQual rubric was created. It identifies three essential domains of EPAs: discrete units of work, entrustable and essential tasks of the profession, and having a curricular role.12 The criterion-based EQual rubric uses five-point scales with specific, descriptive anchors. During development, the EQual rubric was applied to 31 EPAs under revision for medical residents by a team of clinicians and non-clinician support staff separate from the development team. The EQual rubric’s results were compared to independent revisions made to these medical EPAs to demonstrate validity.12
The EQual rubric offers numerous solutions to some of the QUEPA tool’s limitations. A user training video is available to orient the user to the instrument, producing more reliable, consistent results.14 Additionally, the authors defined overall and domain-specific cut scores, above which an EPA is deemed to have acceptable EPA quality.15 Still, generalizability limitations remain for EQual rubric application, as it was created using EPAs for postgraduate medical education. Some rubric items demonstrated low interrater reliability. However, developers attributed this not to the tool itself but rather to the idea that some EPA constructs are challenging to quantify and may be better assessed by means other than an EPA.12 There are no published studies directly comparing the QUEPA tool to the EQual rubric.
Both assessment tools have been used to improve the development and evaluation of EPAs in other specialties. The American Society for Apheresis Graduate Medical Education (ASAGME) used the QUEPA tool during the development of 28 activities related to the Accreditation Council for Graduate Medical Education milestones and competencies. The ASAGME defined their own acceptability cut scores by consensus given the lack of predefined cut scores under the QUEPA tool framework. The most common reason for unacceptability was being unobservable, followed by unrealistic, not generalizable, and/or did not address multiple competencies. The ASAGME used the results to revise their activities to ensure that their final distribution list included high-quality EPAs.16
The EQual rubric has been used in both nursing and medical education to assess EPA quality.15,17 For example, Meyer and colleagues used the EQual rubric to evaluate the 13 Core EPAs for Entering Residency developed by the Association of American Medical Colleges (AAMC) for undergraduate medical education.15 The EPA content experts applied the EQual rubric to each AAMC Core EPA and offered recommendations for revision. Four of the 13 EPAs did not meet overall acceptability criteria, three of which did not meet the cut score for at least two EPA domains. One EPA did not meet acceptability criteria under the discrete unit of work domain, despite having an overall score above the acceptable value. Overall, this study reinforced the importance of evaluating the quality of EPAs prior to implementing them into a curriculum.15 Given the successful delivery of the EQual rubric on numerous limitations and problems with the QUEPA tool, the EQual rubric is the superior EPA quality assessment method.
Evaluating the Quality of the Core Entrustable Professional Activities
Despite the availability of EPA quality assessment tools, there are no published studies evaluating the Core EPAs for pharmacy education using an objective assessment tool like the QUEPA tool or EQual rubric. However, Haines and colleagues used a QUEPA tool–based questionnaire to evaluate the face validity of the Core EPAs.18 In this study, pharmacist preceptors assessed each Core EPA for both pertinence to pharmacy practice and whether a pharmacist is expected to perform the activity. Each Core EPA demonstrated high levels of agreement regardless of respondent training level, institution, or geographic region. This level of agreement was likely a result of the extensive Core EPA development process, including input from several pharmacy organizations. Preceptors exhibited a similar level of agreement when asked whether the EPA is focused, observable, transferable to multiple settings, and integrates multiple competencies.18
Despite being frequently mentioned in the literature, face validity has been widely criticized for being misleading and having little meaning to scientific validity evidence.19 Establishing an EPA’s relevance does not necessarily imply that the EPA measures what it is intended to measure or at a high quality. Face validity does not add to the understanding of assessment data due to its potential association with marketing or educational politics.19 Validity frameworks use various sources to better establish evidence for validity.20 For example, Messick’s framework identifies sources under content, response process, internal structure, relationships with other variables, and consequences.20 One source of validity evidence for the Core EPAs could originate in evaluating their quality by using an assessment tool like the EQual rubric. No such literature currently exists. Quality assessment is a critical step to ensure evidence-based revisions of the Core EPA statements to ensure they meet the extant EPA standards: discrete tasks, observable, entrusted to someone in the profession, and guide curricular efforts. This should be considered as an important component of the work by the AACP Academic Affairs Committee to establish integration of workplace-based assessments within a competency-based educational model.
CONCLUSION
The QUEPA tool and the EQual rubric are objective assessment tools that have been used to assess the quality of EPAs in other health professions education. While there is no universal agreement regarding which tool is preferred, the EQual rubric provides a higher quality assessment of EPAs for valid development and reliable outcomes of pharmacists upon graduation. More importantly, quality assessment results should be used to direct EPA revisions and should be quickly considered by the Academy and specifically the AACP Academic Affairs Committee in the execution of their work developing frameworks for competency-based education in pharmacy. Ensuring high-quality EPAs within pharmacy education is an important mitigating factor and supportive of the ongoing pharmacy practice evolution.
- Received January 21, 2021.
- Accepted March 28, 2022.
- © 2023 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.