A Modified Delphi Involving Laboratory Faculty to Define Essential Skills for Pharmacy Graduates

INTRODUCTION

The Accreditation Council for Pharmacy Education (ACPE) Standards 2016 outline broad categories of clinical sciences as required elements of a professional pharmacy curriculum. These include health information retrieval and evaluation, medication dispensing, distribution and administration, professional communication, and patient assessment.¹ These ACPE standards also direct schools to assess student preparedness prior to progressing onto advanced pharmacy practice experiences (APPEs). The intent is to develop practice-ready graduates who will be successful in the progressive landscape of pharmacy practice.

Skills laboratories are a fundamental component of pharmacy curricula and are a place where students engage in activities that allow them to apply learned concepts and test their knowledge.^2-5 Skills laboratory faculty design and develop experiences based on ACPE standards to ensure that by the time pharmacy students graduate, they possess the basic knowledge, skills, and abilities to independently practice pharmacy. While the ACPE standards are useful, they do not clearly outline the essential skills that need to be taught and assessed in a skills laboratory setting in order to best prepare practice-ready graduates.

In 2015, a group of pharmacy skills laboratory faculty surveyed advanced pharmacy practice experience preceptors from across the United States to help determine essential skills needed by pharmacy graduates.⁶ Preceptors reported that the most essential skill was the ability to communicate both verbally and in writing. In addition, graduates should be able to evaluate patient and prescription information, including vital signs and laboratory values, and to prepare sterile and nonsterile compounds. The ability to engage in teamwork, work independently, and to exhibit professionalism are also essential.⁶

At the same time, entrustable professional activities (EPAs) for pharmacy graduates were being developed through the work of the 2015-2016 American Association of Colleges of Pharmacy (AACP) Academic Affairs Standing Committee.⁷ The committee developed six EPA domains and further defined them by crafting core statements and providing example supporting skills. Pharmacy graduates are expected to perform all EPA domains unsupervised prior to graduation.⁸ Example supporting skills can be used to assess student performance, and pharmacy programs have been encouraged to develop additional supporting skills unique to their program design.⁸

This research builds upon the work of Frenzel and colleagues and the 2015-2016 AACP Academic Affairs Standing Committee to further define essential skills for pharmacy graduates as determined by pharmacy skills laboratory faculty experts and to develop a list of laboratory-focused items that can be used to supplement existing EPA supporting skills.^6,8 These laboratory-focused skills can be used by any faculty to guide curricula development, develop milestone markers, and ensure students are practice ready. However, skills laboratory faculty may find them particularly useful when developing simulations or objective structured clinical examinations (OSCEs).

METHODS

The Delphi technique is a form of action research used to gather data from a group of experts.^9,10 It is a qualitative technique used to build consensus through the systematic use of repeated questionnaires.^9,10 In the health professions literature, the Delphi technique has been used to establish expert consensus regarding competencies specific to student learning, student leadership, and faculty development.^11-18

To be considered for the study, participants had to be at least one of the following: a faculty with primary teaching responsibilities in a pharmacy skills laboratory; a member of the American Association of Colleges of Pharmacy Laboratory Instructors Special Interest Group (SIG); an author of a peer-reviewed publication focused on pharmacy skills education; a recipient of an award in pharmacy skills education; a presenter of a mini or special session on pharmacy skills education at the annual AACP meeting; or a presenter of a webinar on behalf of the AACP Laboratory Instructors SIG webinar. Participants were identified as experts if they met four or more of the six predefined criteria. Development of criteria used for recruitment was guided by the literature.^14,15,19,20

A national call for participants was sent via email to all members of the AACP Laboratory Instructors SIG. The number of experts needed to facilitate a Delphi technique has not been established, but it is typically between 10 to 100 participants.⁹ Fifteen experts were identified and agreed to be a part of the study. Of the 15 experts, all participants were faculty with primary teaching responsibilities in a pharmacy skills laboratory and a member of the AACP Laboratory Instructors SIG; 11 (73.3%) had authored a peer-reviewed publication focused on pharmacy skills education; 7 (46.7%) had received an award in pharmacy skills education; 10 (66.7%) had presented a mini or special session focused on pharmacy skills education at the annual AACP meeting; and 6 (40%) had presented a webinar on behalf of the AACP Laboratory Instructors SIG. All participants met study criteria and represented 13 schools and colleges of pharmacy, with 6 of 15 (40%) participants from private schools. School locations were predominantly in the Midwest (60%) and Southeast (40%) regions of the United States. For all three rounds of data collection, participants were sent a link to the study questionnaire in an initial email followed by three email reminders sent approximately one week apart. The questionnaires were generated and scheduled using Qualtrics (Qualtrics Provo, UT). For all rounds, data from participants were collected using Qualtrics and analyzed using Microsoft Excel 2019 (Microsoft, Redmond, WA). This study was approved by the North Dakota State University Institutional Review Board. The study was conducted by a group of laboratory faculty from seven schools and colleges of pharmacy located in the West (n=1), Midwest (n=2), and Southeast (n=4) regions of the United States.

An a priori condition of three rounds of data collection was set by the research team. During each round of data collection, the researchers communicated with participants privately and on an individual basis to ensure anonymity and to capture information free from the influence of others.^10,18,20 Participants received feedback during each round of data collection until consensus was reached.⁹ In the first round, open-ended questions were used to gather input on what students need to know and do related to the four most common sectors of pharmacy practice at entry to practice.²¹ Participants were asked to use the American Pharmacist Association’s vision statement (“Pharmacists are essential for optimizing medication use and improving patient health,”) and mission statement (“To serve society as the profession responsible for the appropriate use of medications, devices, and services to achieve optimal therapeutic outcomes,”) for the pharmacy profession as guiding principles.²² Participants submitted essential skills for all four practice sectors using Qualtrics, and the data were downloaded into Excel. Content analysis was performed by four of the study authors to eliminate redundancies and to separate double-barreled responses into individual skills. Unique skills were kept as worded and included in the study directly. Finalized lists of essential skills were sent to the research group for review and comment and no additional changes were made.

In round 2, participants were asked to rate all skills generated from round 1 with a level of importance using a 10-point Likert scale (1=not important to 10=very important). Skills from round 2 were sent to participants with each item statistically summarized to allow participants to compare their responses to those of the collective group. A free-text field was also provided to allow participants to comment as to why a skill should be included or excluded from the final list.

In round 3, participants were again asked to rate all skills from rounds 1 and 2 with a level of importance using a 10-point Likert scale (1=not important to 10=very important). All skills were recirculated to allow the experts to think about their initial rating decisions to ensure all skills had an equal chance of achieving the highest importance rating and level of consensus.^20,23 No skills were eliminated between rounds 1, 2, or 3; however, in round 3, included behind each skill was the percent by which faculty experts agreed on the importance of the skill during round 2. Skills that reached 80% consensus or higher were deemed essential skills. Skills that did not meet 80% or above were removed from the finalized list of skills. The essential skills for each practice setting were then mapped to the EPA domains.⁸ Initial mapping of the essential skills to the EPA domains was completed by one member of the research team using AACP EPA supporting skills as a guide and Excel for data management and analysis. All members then reviewed the initial map, discussed it, and came to a consensus to establish a final mapping document.

RESULTS

During rounds one and two, one of the 15 participants did not respond (RR=93.3%). In each of these rounds, the nonrespondent was unique. In round three, three participants did not respond (RR=80%). In round three, the nonrespondents included those from rounds one and two as well as one additional nonrespondent.

In round one, participants generated 108 community pharmacy skill statements, 68 health-system pharmacy skill statements, 83 ambulatory care pharmacy skill statements, and 30 managed care pharmacy skill statements. In round 2, participants rated all skills generated during round 1 with a level of importance. These same skills were again sent to participants in round 3. Following the third round, participants reached consensus using mean level of importance for 70 community pharmacy skills, 45 health-system skills, 59 ambulatory care skills, and 12 managed care skills. A list of the essential skills as mapped to pharmacy practice setting and the EPA domains can be found in Table 1. The number of essential skills identified by EPA Domain for each pharmacy practice setting is graphically represented in Figure 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Number of Essential Skills Identified by Entrustable Professional Activity Domain

Abbreviations: PCP = patient care provider domain, ITM = interprofessional team member domain, PH = population health promoter domain, IM = information master domain, PM = practice manager domain, SD = self-developer domain

DISCUSSION

Expert pharmacy skills laboratory faculty identified and agreed upon a set of essential skills that newly graduated pharmacists should be able to demonstrate prior to entering practice in community pharmacy, health-system pharmacy, ambulatory care pharmacy, and managed care pharmacy. These laboratory-focused essential skills can be used to supplement the published example EPA supporting skills.¹ As personalization of the EPAs has been encouraged, these skills provide a desired degree of specificity for skills laboratory faculty, making them especially useful in the development of laboratory exercises and skill assessments.^8,24 Skills laboratory faculty are responsible for ensuring that students can apply learned knowledge as they develop skills and develop their professional identity in a safe environment with opportunities to practice, refine, and receive feedback.^24,25 Simulations or OSCEs are often used to develop these skills.²⁴ The development of a laboratory-focused list of essential skills mapped to EPA domains can be useful to skill laboratory faculty in the development of authentic and robust learning experiences used to guide student progression from level to level.

Most essential skills trended up in percent of agreement between rounds two and three; however, a limited number trended down, and two skills were removed completely after round three. Skills that trended down but remained on the final list centered around the following: consultation on diabetic medications or testing supplies, responding to basic prescription and nonprescription drug information questions, and documenting patient information. Study participants collectively agreed that these were essential skills but prioritized other skills over them. The two skills that did not make the final list were complete a drug utilization review (ambulatory care pharmacy) and plan and conduct a medication use evaluation (managed care pharmacy). This may be because these types of skills are important but difficult for students to practice in a simulated skills laboratory environment in advance of APPEs.

Given the large number of essential skills identified in this study, the differential in full consensus (100% agreement) and high consensus (>80% agreement) should be used to prioritize the design of learning and assessment opportunities by targeting skills that achieved full consensus prior to skills that did not achieve full consensus. These skills should be foundational to the learning experience of all pharmacy students. In a community pharmacy setting, the skills identified as foundational to the learning experience were: collect patient allergy information, review a patient profile for drug interactions, counsel a patient on a new medication, educate a patient on an inhaler, have a working knowledge of the top 200 to 300 drugs prescribed, know legal requirements for prescriptions, and act ethically and professionally, upholding the Health Insurance Portability and Accountability Act (HIPAA) and following all state and federal laws. The skills identified as foundational to the learning experience of pharmacy students in a health-system pharmacy setting were: complete a patient profile review to evaluate drug-related problems; review a patient profile for drug-drug interactions, drug-disease interactions, and duplicate therapy; complete a patient profile review to plan and implement resolutions to drug-related problems; communicate and collaborate with other health care providers; contact a prescriber, communicate the problem, and give appropriate recommendations; conduct medication education at discharge; use drug information resources; evaluate medication orders for errors and omissions; verify intravenous medication orders; verify medication orders; and act ethically and professionally, uphold HIPAA, and follow all laws. The skills identified as foundational to the learning experience of pharmacy students in an ambulatory care pharmacy setting, were as follows: to collect an accurate medication history; perform medication reconciliation; document a patient encounter; document and report recommendations; be a member of an interprofessional team; present a patient and plan to another provider; answer drug information questions from other health care providers; collaborate with prescribers and other health care providers; have a working knowledge of the top 200/300 drugs, possess a working knowledge of dosing for chronic medications; and use effective patient communication skills when interviewing, counseling, and educating patients. The skills identified as foundational to the learning experience of pharmacy students in a managed care pharmacy setting were: have a working knowledge of the top 200 to 300 prescribed drugs, know how and where to look up unknown information, plan and conduct a medication use evaluation, and possess problem-solving and critical-thinking skills.

Faculty can use these focused lists of skills to tailor their students’ learning experiences to a specific type of pharmacy practice. Faculty may also find it valuable to concentrate on developing skills found across multiple practice settings such as helping students to develop knowledge of the top 200/300 drugs, practice reviewing patient profiles and counseling on medications, and promote ethical and professional behaviors while upholding HIPAA and following all state and federal laws.

The results of this study support the work of Frenzel and colleagues, as communication, both verbal and written, was mentioned by expert laboratory faculty in almost half (30 out of 62, [48.4%]) of the essential skills. Seemingly, skills laboratory faculty and preceptors agree that in order to be practice ready, students must be able to communicate. Focusing on the development of communication skills is important to student success in the workplace and can be supported by practicing communication-focused essential skills in didactic and experiential settings.

This study is not without limitations. First, there was a small number of participants with three dropping out between Delphi rounds. The low number of participants may limit the generalizability of the results. Additionally, there is the potential that there was bias in the responses of participants depending on their demographics and pharmacy background. Although the methods did ensure that participants met inclusion criteria requiring them to have had experience in the skills laboratory setting, other demographic information that could have informed our results was not collected. Thus, there may be some skills that were underrepresented or overrepresented in the results depending on the participants pharmacy background. For example, our experts represent schools of pharmacy in both the Midwest and Southeast regions of the United States. Perhaps gaining a more generalizable sample of skills laboratory faculty from other regions would result in a differing consensus of which skills are of highest priority. Skills were mapped to the EPA domains; however, taking a broader look at how they also map to ACPE standards could offer a more complete picture of how the essentials skills identified in this study can prepare students for advanced pharmacy practice experiences.²⁶

Next steps could include reconciliation of the essential skills lists from both the preceptor⁶ and pharmacy skills laboratory faculty groups. This master list could be used as a resource by curriculum committees to prioritize learning opportunities and the allocation of resources for skills laboratories. Additional research could be performed to gain additional perspective regarding essential skills from the perspective of new graduates.

CONCLUSION

Using the Delphi technique, pharmacy skills laboratory experts identified essential skills that pharmacy graduates should have upon entry to practice within four of the most common pharmacy practice settings. These laboratory-focused essential skills can be used by all faculty, but especially by skills laboratory faculty, to guide curricula development and ensure pharmacy students are practice ready at graduation. The essential skills were mapped to EPA domains to aid pharmacy programs in showcasing how their students demonstrate they are practice ready.