Pharmacy Student Performance in a Capstone Course Utilizing the Pharmacists’ Patient Care Process

INTRODUCTION

The evolving focus of pharmacy practice is on pharmacists providing patient-centered care in a multi-disciplinary environment. Pharmacists consistently improve patient care outcomes and are considered vital members of the patient care team in numerous practice settings and specialties.^1-6 As such, high value has been placed on the independent ability of a pharmacist to identify drug-related problems in patients with a range of medical comorbidities and disease severity.^1,2 The Center for the Advancement of Pharmacy Education (CAPE) Educational Outcomes and the Accreditation Council for Pharmacy Education (ACPE) Standards for the Doctor of Pharmacy degree have been updated to place emphasis on students graduating as “practice-ready” pharmacists in leading medication therapy management as a member of the healthcare team.^7,8 Thus, a need has been established for the Doctor of Pharmacy (PharmD) curriculum to incorporate training focused on patient-centered care delivery.

In response to these growing practice trends in pharmacy, the University of Georgia (UGA) College of Pharmacy planned a major multi-phase curriculum revision for the first, second- and third professional (P1, P2, and P3) year courses that were implemented from fall 2015 to spring 2018. During this process, the faculty identified the need for a capstone course in which students could apply the steps of the Pharmacists’ Patient Care Process (PPCP) and previously learned drug therapy knowledge to increasingly complex patient cases.^7,9 Capstone courses are often defined as culminating experiences that rely upon previously instructed material within a curriculum.^10-15 As the emphasis grows with regard to preparing students to be APPE-ready, capstone courses have been implemented at other colleges of pharmacy as an approach to ensure student knowledge and skill development.^11-15 A capstone course, such as the one described here, allows students to apply the knowledge and skills in complex patient cases they will encounter in APPEs and clinical practice.

The University of Georgia offers a four-year PharmD degree program wherein students spend the first two years of the curriculum on the main campus in Athens. During the P3 year, approximately one-third of the students complete course work on one of the extended campuses in Augusta, Albany, and Savannah. All courses during the P3 year are taught synchronously through distance-learning technology. This course supports the college’s competency statements and terminal and enabling objectives for the PharmD degree relating to integrating patient, disease, and drug data to achieve desired patient outcomes; using knowledge of the medical record to obtain pertinent information to develop a drug therapy plan; and using appropriate drug therapy resources and treatment guidelines to evaluate patient-specific data. Giving students the opportunity to collect data from a simulated electronic health record (SEHR) in complex patient cases, evaluate the data, identify drug-related problems, and develop a plan to resolve the drug-related problems in a capstone course aligns with the following Key Elements of ACPE’s “Standards 2016”: 1.1. Foundational Knowledge, 2.1. Patient-Centered Care, and 3.1. Problem Solving.⁷ The goal of the course was to further advance patient care skills by applying the steps of the PPCP to complex patient cases, identifying and prioritizing drug-related problems, applying practice guidelines and drug information resources, creating a patient-centered care plan, and verbally communicating drug therapy assessments and plans. To meet these goals, we designed a case-based capstone course grounded in the PPCP. Specific course objectives are listed in Table 1. The purpose of this study was to assess the impact of the course on students’ knowledge and skills development as well as their confidence.

METHODS

Integrated Patient Care was a two-credit hour required skills-based course that was team-taught synchronously across four campuses in the P3 year. One primary course coordinator and three site coordinators provided course management and oversight. Each two-hour class session was led by one clinical faculty member with an active clinical practice or, alternatively, a pharmacy resident who was trained in course facilitation and key discussion points. An online course management system was used to deliver course announcements, post-course content, links to the cases, and grades; submit assignments; and administer weekly quizzes. A commercially available electronic health record developed for educational purposes (EHRGO, Archetype Innovations, LLC, Duluth, MN) housed the patient cases used to simulate a practice environment. This course was initially developed as an elective and offered for one semester before it was implemented within the core curriculum. The design of the course was a collaboration between faculty members based on all four campuses and instructional designers based on the main campus.

This skills-based course was built upon the PPCP, which students learned and practiced during the first two years of the curriculum with less complex cases that focused only on one disease state or a small number of drug-related problems. The focus of this capstone course was to develop student skill in the patient evaluation process to identify and resolve drug-related problems, as defined by Strand and Morley.¹⁶ Specifically, the course emphasized student development of skills related to assessing and developing plans as if presenting to the preceptor before a patient appointment, rounds, or collaborating with a provider. This process is widely known as a patient “workup.”^17,18

The course consisted of two practice cases, followed by 11 comprehensive patient cases. Practice cases were designed to give students a chance to become familiar with the course delivery and expectations. The patient cases increased in acuity throughout the semester, with students starting out by assessing ambulatory-based cases and finishing up the course assessing higher-acuity or critical care cases. The patient cases chosen for the students to manage emphasized disease states that students would commonly encounter in their APPEs and which they had previously assessed in other courses. Each of the cases developed in the SEHR were based on an actual patient case from the respective faculty member’s practice site. The cases were incorporated in to the SEHR using a database of patients, demographic profiles, health care professionals, and health systems contained within the program. Cases were posted at least one week in advance of the assigned class period to give students adequate time to thoroughly evaluate the case using the PPCP.

To give students real-world experience with identifying drug-related problems, faculty members were asked to include the medication list as it would be encountered in practice (eg, the list might contain errors such as a drug-drug interaction, or an incorrect medication dose, route, or product), in addition to laboratory values, results of diagnostic tests, and progress notes. Case topics highlighted a variety of diseases, scenarios, and drug-related problems. Students were expected to use appropriate and relevant drug information resources and peer-reviewed treatment guidelines (previously studied) to look up information about the case and develop a plan for resolution. Developing drug therapy plans based on the primary literature was intentionally omitted from this course to emphasize the common skills and disease states students would use and encounter during APPEs.

Each week, students were expected to provide a comprehensive assessment and evaluation of the patient case prior to class. A PPCP worksheet (Table 3) was provided to students to document their findings and assessment based on the first three steps of the PPCP (ie, collect, assess, plan). The completed worksheet was used in class for large- and small-group discussion. To make students accountable for class preparation and provide them with practice opportunities to develop patient assessment skills, quizzes were administered at the beginning of each class. Faculty members were asked to develop quiz questions that covered specific information about the case that only students who had worked up the case prior to class would reasonably be able to answer, such as laboratory values, medications the patient was taking, and goals of therapy for the primary problem.

For the purposes of case discussion before and during class sessions, students were assigned to groups. However, students were individually responsible for knowing the case information. In class, students were given five minutes and additional time thereafter at the instructor’s discretion to discuss the case within their group and compare notes on the case. In-class activities were focused on guiding students in refining their abilities to rationally assess and prioritize drug-related problems, and develop optimal medication and monitoring plans through the use of discussions and debates. The faculty member analyzed drug-related problems and medical conditions in order of priority, while explaining how he or she arrived at a particular conclusion. The discussion generally involved interpretation and application of evidence-based medicine to relevant patient-specific information from the SEHR to support the acuity or importance of the problem and subsequently, development of an appropriate treatment plan.

Graded assessments consisted of weekly quizzes (formative assessment) based on the assigned patient case and two required practical examinations that covered new cases not previously discussed in class. Quiz grades were not recorded for the practice cases. Students were allowed to drop their four lowest quiz grades. The course included dropped quiz grades to account for student absences and technology issues, as well as to provide students additional practice identifying and assessing drug-related problems.

Practical examinations were skills-based assessments of a student’s ability to evaluate and assess a patient case. Examination cases were developed by one of the course coordinators and reviewed by the other three course coordinators as well as content experts. Each examination included four to five drug-related problems, such as incorrect dose of a chronic medication, untreated medical condition based on laboratory or other assessment provided in the case, or a clinically significant drug-drug interaction. The patient case on the first examination was ambulatory-based, while the second examination used a hospital-based case. Students were given 20 minutes to evaluate the patient case in the SEHR at a monitored computer workstation with online access for drug information resources and guidelines, and were provided blank paper to take notes and organize their presentation. Students were then given up to five minutes to verbally present their prioritized problem list, including rationale and plan with monitoring parameters. Students were not permitted to access “smart” technology from the time of arrival in class until examination completion, and were required to sign academic honesty agreements prior to taking the practical examination. Faculty members used a peer-reviewed rubric to objectively assign points for identification of drug-related problems, prioritization, assessments, and plans. Eight faculty members were required to administer the patient presentation portion of practical examinations, with an additional eight staff and graduate students needed to proctor and move students through the process on all four campuses.

A six-item assessment was administered at the beginning and end of the semester to evaluate student self-assessment of achievement of the course objectives (Appendix 1). A three-point adapted Dreyfus model (1=novice, 2=advanced beginner, 3=competent) with objective definitions assigned to each ranking was used for students to self-rate their skills and abilities.^19,20 Two other categories in the Dreyfus model, proficient and expert, were omitted from the assessment after considering literature citing pharmacy students’ inability to reach this level of expertise within a course or curriculum.^19,20

We evaluated student performance on graded assessments and student confidence in meeting course objectives at the beginning and end of the course. Student t tests were used to compare means, the Pearson correlation was used to identify associations between continuous variables, and nonparametric tests were used to compare ordinal data. Statistical analyses were completed using IBM SPSS Statistics for Windows, Version 24.0 (IBM Corp, Armonk, NY). The two practical examinations counted 30% each, the combined quiz scores counted 30%, and participation counted 10% of the course grade. The same faculty member taught the course simultaneously on all campuses each week, so identical assessments were administered at the same time across all campuses. Quizzes were administered through an online course management system during class and graded electronically. Individual student performance on each of the practical examinations was assessed in person by a faculty member and recorded in writing. Grading across all campuses was reviewed by the course coordinators for consistency. The UGA Institutional Review Board determined that our study was not human research and therefore exempt.

RESULTS

Overall, 134 students completed the course in 2018 (Albany, n=12; Athens, n=85; Augusta, n=26; Savannah, n=11). The average grade for all quizzes was 81% (mean=24.3 out of a possible 30 points, SD=2.2). The range of scores on any given quiz was 40% to 100%. There was no difference in quiz performance between students at the main and those at the extended campuses (independent samples Student t test, p>.05). A significant correlation was found between the overall average quiz score and the average score on the second practical examination at the end of the semester (Pearson correlation=.316, n=134; p<.01).

Overall, average student scores improved from the first to the second practical examination (mean 10.4 vs 12.7, respectively; p<.01). There was no significant difference in students’ performance between campuses on the first practical examination. However, students on the extended campuses did score significantly higher (1.1 point) than students on the main campus on the second practical examination (Table 1).

There were 124 out of 134 students enrolled in the course during spring 2018 who completed both the pre- and post-semester assessments. Responses were paired for each question and analyzed using the Wilcoxon signed-rank test, with an alpha level of 0.05 for significance. For each of the objectives, the students rated their abilities in applying the PPCP to complex patient cases significantly higher after participating in the course than at the beginning of the semester (Table 2).

DISCUSSION

The purpose of this capstone course was to advance student patient assessment and evaluation skills using the PPCP to evaluate and develop a preliminary assessment and plan based on the drug-related problems identified. This course was unique in that it focused on using the PPCP to develop patient work-up skills for patient cases in a SEHR. After developing and practicing these skills in a variety of patient cases with varying degrees of acuity and in different practice settings, we believe students will be better prepared for their APPEs. Student skills in this area progressed over the course of the semester as demonstrated by significant improvement in performance from the first to the second practical examination at the midpoint and end of the semester, respectively. To minimize the practice effect, which can occur when students recall previous case details, the same case was not used in the practical examinations,.^21,22 Although different cases were used, all cases contained the same amount of data for review, similar drug-related problems, and were comparable in difficulty. Additionally, both cases were structured to require students to demonstrate the same skills.

Though there were a number of students who indicated there was no change in their confidence between the first and final examinations, the cohort’s overall confidence in their ability to apply the steps of the PPCP also significantly improved over the course of the semester as evidenced by the number of students rating their abilities higher (eg, advanced beginner or competent level) at course end compared to the beginning, and there was a decrease in the number of students who still felt they were at the novice level for each course objective after practicing patient case work-up during the semester. Although the improvement in abilities was self-rated by the students, objective definitions were used to describe the Dreyfus model stages of development, including novice, advanced beginner, and competent, to support these findings as the students were able to match their performance with expectations at each level.

Overall, students performed well on formative assessments, ie, the quizzes. There was a significant correlation between quiz performance throughout the semester and performance at the end of the semester on the practical examination. Although the relationship was not strong, it may have reflected better performance by students who spent more time and effort evaluating the cases each week. Student performance on the practical examination, which required identifying and assessing drug-related problems, significantly improved from midpoint to the end of the semester. All students, regardless of location, exhibited significant improvements in performance on the practical examination administered at the end of the semester compared to that administered at the midpoint. The improvements in performance and abilities over the semester were likely the result of repeated practice in applying the PPCP and a systematic approach to the patient case. There was no difference in performance between students at the main and extended campuses on the first practical examination. On the second practical examination, students based on the extended campuses scored on average 1 point higher than students located on the main campus. It is unclear why this occurred in a course that was administered in the same way to all students, but may have reflected small group discussion of patient cases that occurred on the extended campuses that led to enhanced skills in this area, greater faculty mentorship because of lower student to faculty ratios, or differences in evaluator grading.

Similar to our findings, other colleges have documented improvement in student skills in capstone courses that used a case-based approach.^11-14 Hirsch and colleagues developed a four-credit-hour course to improve APPE readiness by preparing students for the pharmacy licensing examination and evaluation of complex patient cases.¹³ Students reviewed compounding, calculations, pharmacokinetics, and pharmacy law, in addition to patient case presentation and clinical documentation. Through self-assessment, students reported that the course helped prepare them for APPEs. Saseen and colleagues described a nine-credit hour course in which students evaluated and presented complex patient cases, participated in evidence-based medicine (EBM) journal clubs, applied EBM principles to patient scenarios, and completed patient care plans.¹¹ Evaluation of student performance showed significant improvement in these skills. At the end of the course, students reported improved confidence in their abilities, and faculty perceptions of students’ performance were also positive.¹¹ Conway and colleagues incorporated complex patient cases, drug information questions, and a public health presentation in their two-credit hour capstone course that assessed knowledge and documentation skills.¹² Students’ documentation skills significantly improved and students felt their writing skills improved.

Our course initially used an in-house electronic health record to simulate the act of reviewing and retrieving information on a computer screen in a format similar to what students might see in practice. For implementation into the core curriculum, the course transitioned to a commercially available SEHR. Reasons for this switch included the robustness of a commercial system that was difficult to replicate without significant resources, ease of updating case information year-over-year, and the degree of authenticity a commercially available product offers to the electronic health records that are used in practice. The SEHR has been used by several colleges of pharmacy, often with less complex patient case information or to teach one particular aspect or skill related to practice, such as order verification or transitions of care.^23-27 In all instances, the authors reported improved patient care skills and student satisfaction or confidence with the use of a SEHR.^23-27 Although the use of a SEHR in this course was not singled out for evaluation in our course, students’ patient-care skills did progress while using the SEHR to access patient information. The use of the SEHR in this course was comprehensive in nature, based on patient case complexity and the amount of data available (eg, patient demographic and payer information, detailed progress notes, full reports and results of diagnostic tests, and medication administration record information) beyond basic information such as the medication list, past medical history, vital signs, and selected laboratory values that were contained in each patient record. The inclusion of these data was intentional to make the learning authentic and provide students with practice in reviewing all of the types of information that may be available for a patient even if it is not relevant to the specific drug-related problems being assessed.

Overall, student evaluations of this new course were mixed. Anecdotally, in the beginning and throughout the the semester, the coordinators received numerous verbal comments and email communications from students expressing their enthusiasm for the course, excitement to practice in a well-designed simulated environment, and how practice “working up” patients in a SEHR would better prepare them for APPEs. End-of-course evaluations noted the real-world environment and patient cases, but also students’ preference for the course to be taught through the use of recitation groups as in the P1 and P2 years, and a perception that faculty members were not consistent in their approach to the patient cases. Some students noted their performance improved in the practical examinations once they started applying a systematic approach to the patient cases. A student preference for team-based learning over individual assessment was also noted. This is not surprising considering some students identified “lack of familiarity with the SEHR” as part of the reason for their poor performance. Students working in groups may have been less motivated to complete case workups individually and subsequently, to develop needed skills. Although no historical data or control groups were available for comparison, anecdotal student and preceptor feedback to faculty members after starting APPEs was overwhelmingly positive in terms of the impact the course had on their patient care skills. Based on student feedback, planned updates for the course include development of a video modeling how an expert (eg, pharmacy resident) would approach case evaluation, as well as an example of a patient case presentation during the practical examination, implementation of a patient monitoring form for students to use on a weekly basis and during the practical examination to evaluate patient cases, and providing a framework for faculty teaching in the course to facilitate case discussion in a consistent format.

High faculty resource demand was the biggest challenge in offering a capstone course such as this one, both in number of faculty members and in faculty time. A large number of faculty members were needed to provide a wide range of cases with common disease states that students would likely encounter on different clinical APPEs, develop meaningful quizzes over their case material, deliver the material each week, administer the practical examination, work individually with students requesting assistance, and coordinate the course across four campuses. Developing both course content and practical examinations that captured the complex nature of patient care while also providing concrete drug-related problems that both students could identify and resolve and faculty could objectively score required a team-based faculty approach. Each practical examination as well as the grading rubric for it necessitated numerous iterations and review by the four course coordinators, in addition to faculty content experts. After each practical examination, student rubric evaluations were peer-reviewed for consistency and discussed among the coordinators. In addition to the faculty involved in the course, the coordinators collaborated with two instructional designers within the college to develop teaching strategies for the course, organize course materials, especially in the SEHR, set-up in the online course management system, and evaluate student confidence regarding course objectives. In addition to the faculty evaluators, graduate students, pharmacy residents, and staff members were needed to administer the practical examinations.

There were several limitations related to our investigation. First, the data provided were taken from the first offering of this required course in our curriculum. As such, outcomes data and evaluation of APPE-ready students are limited. Additionally, the data reported are from one college of pharmacy. The students’ self-assessment of their achievement of the course objectives was based on their individual perceptions and was not objectively measured. We were unable to use a control group in our evaluation because the cases were developed for this particular course and no historical controls were available. In the future, evaluation of our APPE students could assist in determining whether our goals were fully achieved.

CONCLUSION

This capstone course, designed to develop pharmacy students’ patient work-up skills in a variety of patient cases housed in a SEHR, achieved several initial successes. The course was unique because it used the PPCP framework to teach patient work-up skills in an authentic environment. Student skills significantly improved throughout the semester, and student confidence in their abilities also significantly improved. The course design required a lot of faculty time and expertise to develop course materials and administer assessments. The students appreciated the simulated environment, real-world and practical nature of the course content and activities, and recognized the value of these skills for APPEs. Student feedback included adding small group discussion and individual feedback. This capstone course appears to be an essential component of our PharmD curriculum that prepares students for patient care skills needed for advancement in the curriculum and for patient care practice.