Summary of the ACPE Consensus Conference on Advancing Quality in Pharmacy Education

Insights from Pre-Conference Surveys

Donna Wall, PharmD, Clinical Pharmacist, IU Health University Hospital, Indianapolis, and the ACPE Vice President and Board member, highlighted results from pre-conference surveys² that suggest areas of PharmD education that might require more attention. In her view, greater attention is required in the following areas: (1) more student experience with real patients, (2) understanding the economics of pharmacy practice, (3) medication management systems, (4) leadership and management, (5) critical thinking and clinical judgment, (6) interprofessional collaboration, and (7) developing respect for practitioners in all areas of pharmacy.

Wall said that the current length of PharmD education for entry to practice is appropriate; however, major reform is needed in introductory and advanced pharmacy practice experiences to focus more on competencies required in general practice. She stated that it would be helpful if this conference provides clarity on the following issues: (1) whether residency training is required for certain areas of practice, (2) whether specialization should be allowed in PharmD education, (3) if PharmD education were to allow specialization, whether specialty-specific licensure should be developed, and (4) whether dual-degree programs compromise the attainment of essential pharmacy knowledge.

Current and Future Competency Requirements of Pharmacists

Six pharmacists contributed to a panel discussion on current and future competency requirements of pharmacists, beginning with Carmen A. Catizone, MS, DPh, Executive Director of the National Association of Boards of Pharmacy, who noted that the minimum requirements for entry into pharmacy practice are reflected in the NAPLEX examination. He encouraged pharmacy educators and ACPE to maintain a focus on core competencies related to ensuring safe and appropriate use of medicines, since state boards of pharmacy have found gaps in practitioner performance in these areas. He emphasized the need for pharmacists to be more fully engaged in addressing the national crisis in abuse of prescription drugs.

Stephen Gray, PharmD, JD, an executive with Kaiser Permanente in California who is responsible for regulatory compliance and profession affairs, stated that his integrated care organization has observed competency deficiencies among new pharmacy graduates in compounding, prescription consultation, and product identification. In the future, pharmacists will be required to be competent to work on interprofessional patient care teams, deal effectively with new ethical issues, take responsibility for the outcomes of medication use, and apply the principles of population-based use of medicines. Gray suggested that the pre-pharmacy curriculum should be used to teach some essential competencies (eg, leadership, project management, negotiation).

Nimesh Jhaveri, BS Pharmacy, MBA, Executive Director of Pharmacy and Healthcare Experience, Walgreen Co., described his company’s redesign of stores to better position pharmacists to interact with patients about their medications. He said that this new practice model requires additional competencies in leadership, interpersonal communications, motivating a team, strategic thinking, consultative skills such as problem solving, and empathy.

Thomas E. Menighan, BS Pharmacy, MBA, ScD, Executive Vice President and Chief Executive Officer of the American Pharmacists Association, identified pharmacist competencies that are needed in emerging health care models. Abilities in the following areas were among those he cited: team-based skills (clinical expertise, developing collaborative relationships, accountability for patient outcomes), communication skills (motivational interviewing, coaching, delivering care using various modes of communication), use of quality metrics, facilitating access to medications, understanding the implications of health information technology, understanding population management for targeting at-risk patients, and demonstrating the value of pharmacist services. He suggested that it might not be necessary for all pharmacists to have physical assessment skills.

Daniel C. Robinson, PharmD, Dean, Western University of Health Sciences College of Pharmacy, noted that in most health professions the goal of entry-level education is to create practice-ready graduates (in contrast, medicine and podiatric medicine prepare residency-ready graduates). Further, an explicit goal of education in dental medicine, veterinary medicine, and pharmacy is to prepare graduates to provide care independently upon graduation. Robinson suggested that pharmacists educated under current PharmD standards (which went into effect in 2007) are better prepared for providing direct patient care than those who graduated from programs covered by previous standards. He advocated “responsibility-centered education” in which the curriculum is structured to build confidence and demonstrate competence in providing patient care (for example, by creating closer links between learning and application). Recently, a group of pharmacy educators identified behavioral competencies that new graduates must have for providing patient care.⁷ Robinson said that the profession cannot meet society’s need for medication-related health care by relying solely on residency training. He stated that PharmD education can boost its capacity to produce pharmacists to meet this need by improving the quality of advanced pharmacy practice experiences, providing responsibility-centered education, improving students’ patient assessment skills, and documenting student learning of critical competencies.

Lori Golterman, PharmD, National Director of Residency Programs and Education, Pharmacy Benefits Management Services, Department of Veterans Affairs (VA), identified skills needed by pharmacists on patient-aligned care teams in the VA: communication, physical assessment, mini-mental health assessment, advanced cardiac life support, patient safety, interprofessional collaboration, and an understanding of the economic aspects of clinical services. She advocated for introducing experiential education earlier in the curriculum, expanding the time devoted to experiential education, and reducing didactic education. The creativity, adaptability, and leadership needed by pharmacists to be successful on patient care teams suggests that pharmacy education should concertedly select for admission students who already have strengths in these areas.

Current Directions in Pharmacy Education

Lucinda Maine, PhD, Executive Vice President and Chief Executive Officer of the American Association of Colleges of Pharmacy (AACP), reviewed the establishment (in 1992) of the AACP Center for the Advancement of Pharmaceutical Education (CAPE) as a resource for colleges and schools to support transition to the PharmD degree as the single entry-level degree for pharmacy practice. Three pillars of pharmacy education—pharmaceutical care, management of medication-use systems, and public health—were codified in 2004, consistent with the core competencies for health professionals articulated by the Institute of Medicine in 2003. She indicated that the ACPE conference will influence the next revision of the CAPE competencies, which is expected to be available in draft form by mid-2013.

Preliminary thinking among those responsible for drafting new CAPE educational outcomes is that the current three pillars of pharmacy education are still valid but more attention should be focused on the affective domain of pharmacy practice (eg, communication, professionalism, critical thinking, and leadership), patient safety, and interprofessional health care.

The future is likely to bring a real shift toward interprofessional team-based care because both educators in the health professions and leaders of reform in health care delivery are working toward the same goal. Wider use of technology in education will make interprofessional educational experiences more accessible. A key factor is the growing recognition by practitioners, including physicians, that safe, effective, efficient care cannot be delivered in isolation.

Maine discussed the following four unresolved questions: (1) What is the true core in pharmacy education and what can be let go? (2) Is there agreement that the core of pharmacy education should focus on competencies needed in patient care? (3) In an era of disruptive innovation, what does quality assurance in health care and in education look like? (4) Can the next generation of standards for PharmD education both invite innovation and allow preparation for differentiated career pathways?

Best Practices in Measuring and Improving Student Learning

Trudy W. Banta, EdD, Professor of Higher Education and Senior Advisor to the Chancellor for Academic Planning and Evaluation, Indiana University-Purdue University Indianapolis, opened the conference’s focus on assessment by commenting on the pressures on accrediting bodies to be transparent with respect to compliance with their standards, to measure quality of educational programs, and to foster continuous improvement in student learning. Since most faculty members in higher education are not trained as teachers, educational institutions must have a professional development program that will assist the faculty in writing clear learning objectives, connecting learning outcomes to course assignments, and developing assessment tools that test higher-order intellectual skills. She discussed specific tools that can be used for assessing student learning and evaluating course outcomes.

The use of standardized tests to assess learning of generic skills (eg, critical thinking, written communication, analytic reasoning) is problematic because those tests mostly measure students’ prior learning (not “college effects” on learning). Further, it is not feasible to distinguish among the effects of teacher, school, student, and other factors on changes in standardized-test scores that may occur during the college years.

Dr. Banta advocated the following methods for assessing student learning: standardized tests in specialized fields (eg, licensure and certification tests), objective structured clinical examinations, performance in experiential education, electronic portfolios, external examiners, peer assessment, and self assessment. She also discussed use of faculty-developed measures such as primary-trait scoring (identifying traits necessary for success in assignment; composing a rubric with clear definition of each point; grading using the rubric).

Dr. Banta concluded by recommending the following resources on assessment: National Institute for Learning Outcomes Assessment (www.learningoutcomesassessment.org), New Leadership Alliance for Student Learning and Accountability (www.newleadershipalliance.org), Assessment Update (www.planning.iupui.edu/58.html), and Assessment Institute in Indianapolis (www.planning.iupui.edu/institute).

Insights from Survey Results and Conference Readings

Heidi M. Anderson, PhD, Vice President for Institutional Effectiveness, University of Kentucky, and immediate past president of ACPE Board, reviewed the history of assessment in higher education and pharmacy education. She noted that while pharmacy education is now focusing on assessing student learning, more work is needed in evaluating institutional effectiveness and in using data for curricular improvement. Barriers to progress include a lack of understanding of assessment (especially as related to institutional effectiveness), lack of personnel and financial resources devoted to assessment, and lack of a culture of assessment that is grounded in institutional mission and championed by institutional leaders.

Dr. Anderson discussed how to establish assessment objectives that are specific, measurable, achievable, realistic, and time-bound. She also reviewed a method of selecting a measurement tool that is appropriate to a particular desired outcome.

She summarized current methods of assessing curricular outcomes in PharmD education, including comprehensive and focused reviews of program information, annual monitoring metrics (NAPLEX scores, enrollment data, progression/graduation data), and the results of AACP standardized surveys (graduating students, faculty, preceptors, alumni).⁸ ACPE site reviewers take note of innovative practices in assessment, and that information is disseminated on AACP’s Web site. ACPE also monitors the capacity of the faculty and of experiential education sites. Although ACPE has expanded its efforts in assessing curricular outcomes, it recognizes that more needs to be done and is looking to this conference for direction in that regard.

Innovations in Educational Assessment and Accountability in Other Health Professions

Panelists representing educational accreditation in medicine, nursing, and physical therapy discussed innovations in their programs that might have applicability in pharmacy. Barbara Barzansky, PhD, Director, Division of Undergraduate Medical Education, American Medical Association (AMA), and AMA Secretary of the Liaison Committee on Medical Education (LCME), discussed the utility of regular program monitoring based on LCME’s experience. The 131 standards for undergraduate medical education are under continuous review by stakeholder groups from the academic community, which are surveyed on the clarity and importance of each standard. Accredited medical educational programs are reviewed fully every eight years and are monitored annually based on their response to questionnaires on compliance with the standards. LCME monitors data from a variety of perspectives and sources in assessing programs during and between regular reviews. Cross-sectional national data allow normative comparisons across schools. Other standardized instruments are also used for benchmarking school performance.

To illustrate the range of methods used in regular program monitoring, Dr. Barzansky discussed methods of assessing “professionalism and the learning environment,” observing students’ clinical skills, and use of national standardized subject tests in required disciplines (eg, surgery, internal medicine). Before a new accreditation standard is proposed, LCME typically studies the prevalence of a particular practice (for example, requiring interprofessional experiences).

Heidi Taylor, PhD, Associate Professor of Nursing, West Texas A&M University, and Co-Chair, Standards Committee, Commission on Collegiate Nursing Education (CCNE), discussed how CCNE evaluates programs with respect to interprofessional education. Most CCNE-accredited programs (baccalaureate and graduate programs) are in general academic institutions, not in academic health science centers; this has fostered innovative methods of pursuing interprofessional education.

CCNE accreditation standards are based on essential educational requirements outlined by the American Association of Colleges of Nursing. Those requirements cover interprofessional communication and collaboration for improving patient and population health outcomes. Interprofessional education competencies include, in the case of baccalaureate education, for example, ability to use interprofessional communication and collaboration to provide evidence-based care and ability to use teambuilding and collaborative strategies. Higher levels of expected abilities related to team leadership are required in master’s and doctoral education for advanced nursing practice.

CCNE does not have a standardized measure of competence in interprofessional practice, but accreditation teams look for a “preponderance of evidence” of compliance with interprofessional education standards, including classroom and clinical observation of student learning and performance.

Dr. Taylor drew attention to a recent comparative analysis of interprofessional education accreditation standards in 10 health professions.⁹

Mary Jane Harris, MS, Director of the Department of Accreditation, American Physical Therapy Association (APTA), supports the work of the Commission on Accreditation in Physical Therapy Education (CAPTE). CAPTE standards focus on the competencies that graduates must have, allowing flexibility in curricular content to create those competencies. Full-time supervised clinical experience makes up 44% of the contact hours in physical therapy education (which, on average, requires 7.6 semesters following baccalaureate education).

Most programs use the Clinical Performance Instrument (CPI), a validated instrument developed by APTA, to assess students at the mid point and at the end of clinical experiences. The CPI delineates six levels of competency (ranging from “beginning” to “entry-level” and “beyond entry-level”) with respect to the amount of supervision required; the quality, complexity, and consistency of performance; and efficiency (in terms of handling caseload).

CAPTE reviews an educational program’s use of individual student CPI assessments and its aggregate CPI data. CAPTE maintains a database on the more than 25,000 clinical experience sites in the U.S., and it collects student evaluations of the quality of their clinical experiences.

Innovations in Assessment in Pharmacy Education

Four panelists from pharmacy education discussed innovations their institutions have made in assessment. Diane E. Beck, PharmD, Clinical Professor and Associate Dean for Curricular Affairs and Accreditation, University of Florida College of Pharmacy (UFCOP), noted that UFCOP adopted a systems approach to program evaluation in 2011. Several strategies have been used to better understand the College as a complex system. The College mission statement served as the framework for the program evaluation plan, and the program evaluation plan was designed to assess eight mission areas. Each mission area was defined via a set of dichotomous statements, which enabled a more organized approach to interpreting large numbers of metrics. This approach yielded insight into what happened before and during a learning period. UFCOP’s systems approach also stimulated thinking about how the organization functions and about how to more effectively communicate the results of assessment.

Melissa S. Medina, EdD, Assistant Dean for Assessment and Evaluation and Presidential Professor, University of Oklahoma Health Sciences Center College of Pharmacy (OUCOP), described an integrated assessment method. Integrated knowledge and skills examinations are embedded into final examinations of the pharmacy practice courses in each semester of the first three professional years. An integrated exam (IE) committee for each professional year (1) identifies the most pertinent skills and knowledge-based content from each course, (2) develops measurable IE objectives addressing the pertinent content, (3) creates multiple-choice and performance-based IE questions derived from IE objectives, accounting for 10% of the final course grade, and (4) submits objectives and questions for review and revision to an oversight committee. Student performance on each question is evaluated, and objectives and questions are revised as needed for the next year’s iteration. OUCOP’s method of assessment has transformed student understanding of the depth of learning expected for pharmacy practice.

Sam C. Augustine, PharmD, Professor of Pharmacy Practice, School of Pharmacy and Health Professions, Creighton University, described a three-phase assessment process for determining if the pharmacy curriculum initiated in 2010 is meeting the educational outcomes and competencies established by the faculty. Tools for curricular assessment were developed for use at semester, annual, and 3- to 5-year intervals. The Pharmacy Curriculum Outcomes Assessment (PCOA) examination of the National Association of Boards of Pharmacy is one element in annual assessment. PCOA is a psychometrically sound, nationally standardized test. Students use their PCOA test results to evaluate their strengths and weaknesses and to compare their performance with a national sample. Creighton is exploring how to best apply the PCOA in assessing curricular performance, verifying parity between distance and campus pathways, and predicting successful program completion.

Kristin Kari Janke, PhD, Professor, College of Pharmacy, University of Minnesota, provided examples of the College’s assessment efforts in leadership, professionalism, and continuing professional development (CPD) (sometimes called “soft skills”). She advocated that each pharmacy program should develop assessment methods related to its unique areas of focus. Although many assessment instruments are available for professionalism, none of them is sufficiently sensitive to gauge changes over time. Seven colleges of pharmacy collaborated in developing a new validated instrument for measuring professionalism in pharmacy students. Skills in CPD are difficult to develop as new graduates transition from instructor-directed education to self-directed learning. Hence, assessment of these skills must be multifaceted, using existing instruments, new instruments, and multiple observers with repeated measures over time.