A Standardized Patient Counseling Rubric for a Pharmaceutical Care and Communications Course

INTRODUCTION

Communication skills are essential to the development of patient-centered care. The Accreditation Council on Pharmacy Education (ACPE) and the American Association of Colleges of Pharmacy Center for the Advancement of Pharmaceutical Care (CAPE) suggest accreditation standards for training pharmacy students to possess the skills necessary to effectively communicate in the healthcare environment.^1,2 ACPE’s pre-advanced pharmacy practice experience (pre-APPE) Core Knowledge Domain 8 specifically discusses patient counseling and states that students should be able to “provide effective health and medication information to patients and/or care givers and confirm patient and/or care giver understanding of the information being provided” prior to beginning APPEs.¹ Further, ACPE standards require that these skills be verified throughout the program. In addition, the State of Texas Administrative Code requires that pharmacists counsel the patient or patient’s agent with each new prescription dispensed or at the request of the patient or patient’s agent in order to optimize drug therapy.³ The ACPE and CAPE accreditation standards and the Texas Administrative Code each set out a specific set of components on which each patient should be counseled including, but not limited to, medication name, dosage, indication, proper storage, missed dose instructions, and adverse effects.^1-3 The curriculum at Texas Tech University Health Sciences Center School of Pharmacy attempts to foster effective communication skills centering on the provision of patient-centered care through participation in laboratory courses and clinical practice experiences.

During the second year (P2), medication counseling skills are introduced in the Pharmaceutical Care Laboratory course. This is a 2-credit-hour course taught in the fall primarily by Department of Pharmacy Practice faculty members. Each semester is 16 weeks long with fifteen 50-minute prelaboratory lectures, eleven 2-hour weekly laboratory sessions, 2 formal medication counseling assessment practicals graded by faculty members, a computer-based final on drug knowledge of the Top 200 drugs, and the communication and documentation of pharmaceutical care using a subjective, objective, assessment, plan (SOAP) note format.

Students’ retention of core knowledge and skills taught in the curriculum are assessed each January on the school’s annual assessment. The assessment is targeted at determining student readiness to practice based upon the abilities, skills, and knowledge all students are expected to have mastered prior to graduation. It includes both pen-and-paper examinations and objective structured clinical examinations (OSCEs). The assessment is linked to the courses comprising the school’s curriculum by domain-specific ability statements. These ability statements form the basis upon which determination of the individual student’s readiness to practice is made. Each year, student’s individual scores are compared to their previous scores as well as to overall class scores from previous years to ensure that students are progressing in their knowledge, understanding, and skill ability. The Angoff method was used to establish criterion-referenced standards for all P4 subtests.⁴ Each year, a table of specifications is developed that maps the pen-and-paper portion of the assessment to a broad sample of curricular content by domain. This table of specifications and the rubrics used during the OSCEs are disseminated to students 1 to 2 months prior to the assessment each year. OSCE medication counseling assessment scores are generated through simulated patient interactions using actors as patients and faculty members as graders.

Prior to 2011, medication counseling was assessed in multiple courses across the 4-year program and in the annual assessment without the use of a standard rubric. In 2010, second-year students’ scores on the medication counseling portion of the annual assessment declined. Specifically, students did not perform satisfactorily those elements highlighted by ACPE and the Texas Administrative Code.^1,3 This decline prompted a review and subsequent revisions to the course in which counseling is taught. These changes focused on training pharmacists to be more effective counselors. During this review and revision of the patient counseling course, the decision was made to design and implement a standardized rubric throughout all courses in the curriculum that would be used in the annual assessment as well.

The teaching team responsible for the revisions hypothesized that by redesigning the course, student counseling proficiency would exceed the previous year’s performance. This article describes the redesign of the pharmaceutical care laboratory course to increase the learning and retention of medication counseling skills, and an evaluation of student learning after completion of the course as part of the annual assessment.

DESIGN

Modifications were made to the course during the summer of 2010 in response to the curriculums review and student performance on the annual assessment. Those modifications included revisions of the course mission, objectives, and ability statements. The course mission was revised to place a greater focus on providing the doctoral students with the skills necessary to competently dispense medications and counsel patients according to state and federal law as well as promote best practices. The team focused on approaches in the redesign that would be limited in scope, as opposed to broad sweeping changes as they felt that changes at the introductory course level might yield a larger long-term benefit.⁵ The course was redesigned to include an increased number of faculty-observed practice medication counseling sessions, requiring perfect scores on the core counseling requirements specifically noted by ACPE and the Texas Administrative Code, and remediation of students not earning a perfect score on the required components.^1,3 In order to focus solely on communication skills, all patient assessment skills were removed from the course and taught in a separate course during the spring semester.

As course objectives were tailored to meet the new mission for the 2010-2011 academic year, the number of course objectives was reduced from 21 to 7. The course objectives relevant to patient counseling included: effectively communicate with patients, caregivers, and healthcare practitioners; demonstrate competent situation-specific patient medication counseling; and demonstrate professionalism. Given the design of the course, objectives were application-based as defined by Bloom’s taxonomy (ie, demonstrate competent situation-specific patient medication counseling).⁶

Because the course contained a variety of content and delivery methods, multiple forms of pedagogy and andragogy were used in 2009 and 2010, and were similar between the 2 years. Classroom-based lectures were mostly teacher-centered, directed learning with active learning integrated throughout, using cases and activities to enhance student involvement. Laboratory activities were mainly student-centered, authentic learning (learning in a setting that mimics the “real world,” ie, role-playing medication counseling). Laboratory sessions in both years used peer counseling and grading, facilitated and self-directed learning, and critical-thinking skills.

Another major modification to this course was the revision of the counseling assessment practicals. In 2010, the midterm and final counseling practicals were converted to high-risk assessments to enhance student performance. Students who received a failing grade at any point on either of the practicals were required to repeat the practical until they achieved minimum competency in order to satisfactorily complete the course. Minimal competency was defined as performing each of 12 required elements, which were developed from ACPE and Texas Administrative Code core counseling requirements (generic and trade names, use/indication, dosage form and route of administration, dose and administration schedule, specialized medication preparation and administration, proper storage, missed doses, expected duration of therapy and whether refills were available, self-monitoring (efficacy and/or safety), common and severe adverse effects, prevent or minimize adverse effect, and common interactions where applicable (include drug-drug, drug-food, and drug-disease).^1,3

EVALUATION AND ASSESSMENT

All analyses were performed using the SPSS 21.0 statistical package (SPSS, Inc, Chicago, IL). Appropriate descriptive statistics were used to summarize all data. Data from paired samples were analyzed using either the paired t test or McNemar statistic. Data from independent samples were analyzed by either independent samples t tests or by using the chi-square statistic. In the case of independent samples, the Fisher exact test was substituted for the chi-square statistic in cases where expected frequencies were observed to be less than 5 in any cell. The level of significance for all statistical analyses was maintained at a p≤0.05. This study was exempt from formal institutional review board review.

For fall 2009, no students failed the course and the students’ average scores for the midterm and final practical examinations were 93.8±4.2 and 92.3±4.7, respectively. There was an overall decline in student performance on the 2010 annual assessment with regard to patient counseling skills, particularly in the P2 year, which was comprised of the students who had just completed the course (Table 2). These findings resulted in the redesign of the course in preparation for the fall 2010 semester. The average grades for the midterm and final practical examinations during the first semester of the newly revised course for fall 2010 were 96.2±4.8 and 95.6±5.8, respectively (p=0.283).

During the fall 2010 semester, 51 of 127 students required remediation after the midpoint practical. Twenty-seven of those students satisfactorily completed the midterm on the second attempt. The remaining 24 students satisfactorily passed the practical after the third attempt (second remediation). The number of students requiring remediation after the final counseling practical was drastically reduced, with 10 students requiring remediation. Nine of the 10 students satisfactorily completed the final counseling practical after the second attempt while 1 student required 3 attempts to successfully complete it.

The annual assessment results for 2011 also improved compared with the previous year’s results (Table 2). Overall mean patient counseling OSCE scores were 26.6±3.1 and 23.0±3.0 for 2011 and 2010 assessments, respectively (p <0.001). Subdomain scores were similar for communication (9.1±1.1 vs 9.4±0.9, p=0.83, 2010 vs 2011), but significantly improved in 2011 for the skills domain ( 17.5 ± 2.6 vs 13.6±2.7, p ≤0.001).

Table 2 contains the annual assessment performance for students in 2010 and 2011 (reflecting the course as taught in fall 2009 and fall 2010, respectively). The items marked with an asterisk were the 12 required elements included in the standardized medication counseling rubric during fall 2010. There was no difference between the students’ performance in 2010 vs 2011 for 42% of the elements assessed. Of the remaining 58% (or 18 elements) there was a significant improvement in student performance from 2010 to 2011. In the elements where a difference was detected, the majority of the findings favored the 2011 student performance (15 items). Of the 3 items that favored student performance in 2009, only 1 item was specifically included in the newly revised standardized rubric (proper storage and disposal). The 2 remaining elements specifically relate to the Indian Health Services method of patient counseling, which was removed from the course in fall 2010.⁸

DISCUSSION

As Texas Administrative Code requires counseling on all new prescriptions, improving the ability of pharmacy students to appropriately and effectively communicate with patients about their medications is imperative.³ The intent of the high-risk assessments in the course was to ensure that the students focused more on their counseling skills both within and outside the classroom setting, including the annual assessment patient counseling OSCE. There were several factors that contributed to the improved annual assessment performance by students in 2011. Among those were the creation of a standardized, objective rubric; frequent assessment of student counseling performance by faculty members and peers; and revising the counseling assessment into a high-stakes assessment. The culmination of these events contributed to improved student performance both in the course and in the annual assessment over that in previous years. The improvement in annual assessment and course performance was likely multifactorial, influenced by redesigning the course, changing the counseling to a high-stakes assessment, and using a standardized objective assessment tool.

The redesign of the course did not require additional resources. The greatest efforts were in developing a comprehensive rubric to be used in all settings where patient counseling is assessed and in redistribution of faculty time and effort in the laboratory portion of the course. Because the course team changed the counseling to a high-stakes assessment, they felt that each student should receive personal and frequent feedback from peers and faculty members prior to the first assessment. This premise required changes in the laboratory activities, ensuring that student counseling was assessed weekly.

While the alteration of the course design and practical examination grading in the course during the fall of 2010 appears to have contributed to the improved scores in the 2011 annual assessment patient counseling OSCE, course revision was not without complications. After the first practical examination, 41% of students did not pass because they missed at least 1 of the 12 required counseling elements on the grading rubric and were required to remediate, some up to 2 additional times. However, the students showed a marked improvement in skills on the second practical administered later that semester, with fewer students (8%) requiring remediation.

Remediation needs were high during the fall 2010 course and an unexpected demand was placed on faculty time for completion of remediation. With frequent individualized assessment and feedback using the newly developed rubric, the course team thought that there would be little or no need for remediation throughout the semester. The course team underestimated the time and effort required to remediate students in a timely manner. This identified the importance of contingency plans and having a flexible course team to meet the needs of the students to support the course changes. The course revision also highlighted the need of the course team to be responsive and accommodating to student needs and concerns. Fortunately, the school possesses the ability to digitally record student-counseling sessions, so there was a way to quickly and completely respond to student concerns about the high-stakes nature of the assessments and their desire to have independent verification of each counseling session.

The standardized rubric developed for the course has continued to be used in all courses and experiential clerkships throughout the school’s curriculum where student counseling of medications is assessed. The development of the rubric used to assess medication counseling has led to standardization throughout the curriculum and reinforced the importance of patient counseling. It has also served to link the various courses and to emphasize the medication counseling components required for minimal competency based on the Texas Administration Code.³

SUMMARY

Redesign of the patient counseling course in 2010 improved students’ medication counseling abilities and readiness to practice. Quantitative and qualitative data analysis proved a more objective approach to course revision and, ultimately, student performance. Ongoing annual assessment is warranted to ensure that minor changes in delivery or assessment do not stray from the overall course objective or terminal outcomes of the school or university.