Skip to main content

Main menu

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Other Publications

User menu

Search

  • Advanced search
American Journal of Pharmaceutical Education
  • Other Publications
American Journal of Pharmaceutical Education

Advanced Search

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Follow AJPE on Twitter
  • LinkedIn
Research ArticleRESEARCH

Expert Consensus to Finalize a Universal Evaluator Rubric to Assess Pharmacy Students’ Patient Communication Skills

Susanne G. Barnett, Andrea L. Porter, Sheila M. Allen, Karen M.S. Bastianelli, Jennifer S. Chen, Marlowe Djuric Kachlic, Casey E. Gallimore, Kristen L. Goliak and Jeffrey C. Reist
American Journal of Pharmaceutical Education December 2020, 84 (12) 848016; DOI: https://doi.org/10.5688/ajpe848016
Susanne G. Barnett
aUniversity of Wisconsin-Madison School of Pharmacy, Madison, Wisconsin
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrea L. Porter
aUniversity of Wisconsin-Madison School of Pharmacy, Madison, Wisconsin
bEditorial Board Member, , Arlington, Virginia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sheila M. Allen
cUniversity of Illinois at Chicago College of Pharmacy, Chicago, Illinois
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Karen M.S. Bastianelli
dUniversity of Minnesota College of Pharmacy, Minneapolis, Minnesota
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jennifer S. Chen
dUniversity of Minnesota College of Pharmacy, Minneapolis, Minnesota
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Marlowe Djuric Kachlic
cUniversity of Illinois at Chicago College of Pharmacy, Chicago, Illinois
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Casey E. Gallimore
aUniversity of Wisconsin-Madison School of Pharmacy, Madison, Wisconsin
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kristen L. Goliak
cUniversity of Illinois at Chicago College of Pharmacy, Chicago, Illinois
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jeffrey C. Reist
eUniversity of Iowa College of Pharmacy, Iowa City, Iowa
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Objective. To use an expert consensus-building process to develop a rubric used by multiple evaluator types to assess Doctor of Pharmacy students’ patient communication skills.

Methods. Faculty and staff members from six schools and colleges of pharmacy collaborated on a multi-step expert consensus-building process to create the final version of a communication rubric. First, faculty and patient content experts evaluated each item in the rubric for its relevance, criticality, and global comprehensiveness using a five-point Likert scale (0=not at all, 4=to a high extent). Descriptive statistics were used to analyze the resulting data. Faculty members evaluated the results and came to a consensus on the second version of the rubric. A corresponding codebook was developed and refined through a two-phase process.

Results. The initial communication rubric was evaluated by 13 expert reviewers. Mean global comprehensiveness on the rubric was 3.83 for faculty experts and 3.5 for patient experts. After evaluating results from the expert consensus-building process, 14 items on the rubric did not change, five items were revised, three items were removed, and two items were added. The second version of the instrument included 20 items in six topic areas. A codebook was finalized to increase scoring consistency for the 20 communication items.

Conclusion. Overall, content experts concluded that the rubric had high global comprehensiveness. Collaboration involving faculty members from multiple schools of pharmacy resulted in a 20-item communication rubric and codebook that can be used to increase consistency in scoring student pharmacists’ patient communication skills.

Keywords
  • communication rubric
  • medication education
  • patient education
  • assessment tool
  • consensus building

INTRODUCTION

Effective patient care communication by pharmacists has been shown to improve medication adherence and reduce medication errors.1,2 As such, a variety of teaching strategies are used by colleges of pharmacy in teaching patient care communication skills.3,4 It is imperative that these communication skills are critically assessed and feedback is provided to improve and refine student performance.

Communication rubrics have been created within pharmacy education and are being used to evaluate skills in a multitude of scenarios, including in actual and simulated patient care experiences.5-8 Within the pharmacy profession, there currently is not a universally accepted communication rubric to assess professional communication skills for student pharmacists across a variety of evaluators. A standardized communication rubric that could be used in multiple settings (experiential, laboratory, simulation, etc) by multiple evaluator types (preceptors, faculty, residents, patients, and students) and has been demonstrated to be reliable, valid, and void of interrater variability was needed. A professional communication rubric could be used by pharmacy educators in establishing benchmarks to ensure all students graduating from a college of pharmacy are proficient in their communication skills and that other pharmacy students from other colleges and schools are being held to similar evaluation standards.

The aim of the current project was to develop a rubric using an expert consensus-building process that could be implemented by multiple users to assess student performance of patient communication.

METHODS

A multifaceted approach, as illustrated in Figure 1, was applied to develop the Professional Communication in a Patient Encounter Rubric (communication rubric) and corresponding codebook. The Big Ten Academic Alliance-Performance Based Assessment Collaborative (BTAA-PBAC) is a group of faculty members and instructors from nine schools, including eight Big Ten schools and colleges of pharmacy and the University of Illinois at Chicago (UIC) that teach skills-based laboratory courses. The mission of the BTAA-PBAC is to work towards evolving and shaping performance-based assessment practices for excellence in evidence-based pharmacy education. To begin the process, each BTAA-PBAC school submitted rubrics they had used in the evaluation of students during performance-based assessments. Communication-focused components of each rubric were extracted and compiled into one rubric. Members of the BTAA-PBAC who taught Doctor of Pharmacy (PharmD) students communication skills vetted the rubric. Each item on the rubric was analyzed, discussed, and refined until group consensus was reached. The resulting draft of the communication rubric was piloted by several schools, which led to additional minor changes being made.

Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.

Expert consensus building process used to evaluate a universal evaluator rubric to assess pharmacy students’ communication skills.

Key: BTAA-PBAC: Big Ten Academic Alliance – Performance-Based Assessment Committee, UW: University of Wisconsin – Madison, IA: University of Iowa, UIC: University of Illinois at Chicago, MN: University of Minnesota, RU: Rutgers University, OSU: The Ohio State University

Subsequently, a multi-step expert consensus-building process was undertaken to develop the finalized rubric. External content experts completed a survey to assess content validity. Content experts were defined as either faculty members or instructors teaching in a required pharmacy communications course at a Big Ten pharmacy school who were not involved in the development of the rubric (faculty experts) or as individuals employed as standardized patients (patient experts). Experts were given two scenarios where they played the role of the patient to provide context to the survey. They evaluated every item on the rubric for relevance (How well does this item relate to the purpose of the evaluation?) and criticality (How crucial is it that the item be evaluated?), using a four-point Likert scale (0=not at all, 4=to a high extent). Additional questions were asked regarding the global comprehensiveness of the rubric (assessed on the same Likert scale) and identification of important items that should be evaluated by a rater to assess student communication skills. Finally, reviewers responded to three short-answer questions regarding omitted items that should be considered for inclusion, organization of the rubric, and additional comments.

Quantitative data from the consensus-building process were analyzed using descriptive statistics. Themes were identified and reported for the three qualitative questions. Members of the BTAA-PBAC applied results of the expert consensus-building process to make final edits to the rubric (Table 2).

Next, three pharmacy faculty members from the University of Wisconsin-Madison (UW) School of Pharmacy developed a codebook to standardize the scoring of individual communication items. The codebook described the content and structure of the rubric, and provided guidance on the scoring of each item. A draft of the codebook was vetted by the nine-schools represented within the larger BTAA-PBAC group, and additional edits were made based on group consensus.

The communication rubric and codebook were piloted in two phases to evaluate student communication during a patient education session. In phase one, seven laboratory faculty members and one fourth-year student pharmacist representing one of three pharmacy schools (UW, University of Iowa [IA], and UIC) evaluated students during live skill performance at their home institutions. These individuals subsequently brought suggested edits to an in-person meeting of the BTAA-PBAC. Codebook revisions were discussed and implemented after group consensus was achieved. Phase two pilot testing was designed to increase consistency of item scoring through clear interpretation of the codebook. Two simulated patient education sessions were video-recorded and made available to faculty members at six BTAA-PBAC schools (UW, IA, UIC, University of Minnesota [MN], Rutgers University [RU], and The Ohio State University [OSU]). Phase two faculty evaluators used the rubric and codebook to score student communication in each of the videos. Rubric item scoring was compared between faculty evaluators, and discrepancies were identified. Faculty members and instructors from the BTAA-PBAC schools then reconvened to evaluate the results of the phase two pilot and came to consensus on a final version of the codebook (Appendix 1).

This project was approved by the institutional review board (IRB) at each of the six schools that participated in piloting the communication rubric and codebook. Students participating in video recording agreed to participation and signed informed consent documents as required by IRB.

RESULTS

The initial communication rubric was evaluated by 13 content experts, including six faculty members and one graduate student (faculty experts) from seven pharmacy schools and six standardized patients from UW (patient experts). All available data were included in the analysis; however, not all experts completed every survey item. The numerical relevance and criticality ratings by faculty members and patient experts for each of the rubric items are included in Table 1. Mean global comprehensiveness of the communication rubric was 3.83 (n=6) and 3.5 (n=6) for faculty and patient experts, respectively. Common themes to short-answer question responses included suggestions to add omitted items for patient follow-up, a summary of the interaction, and the use of open-ended questions. Both groups of experts also encouraged the evaluation of only one item per line and the development of detailed descriptions for each of the items to assist in consistent use of the rubric. One prominent theme that emerged from the patient expert feedback was that the communication rubric was comprehensive and well organized.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1.

Expert Review by Faculty and Patient Experts of a Universal Evaluator Rubric to Assess Pharmacy Students’ Communication Skills

After the BTAA-PBAC evaluated the results from the consensus-building process and discussed among group members, 14 rubric items did not change. Five items were revised: in item 11, “spoke clearly” was changed to “spoke clearly and confidently”; in item 12, “used words that patient could understand” was changed to “used patient friendly language”; in item 18, “respected patient’s time” was changed to “utilized time efficiently”; in item 19, “achieved mutual understanding and if applicable agreement with plan” was changed to “achieved mutual agreement with plan”; and in item 21, “concluded encounter smoothly” was changed to “provides closure to encounter”. Three items were removed: item 2, “addressed patient by name”; item 15, “listened to patient responses”; and item 16: “appeared confident in abilities.” Two items were added: “listened to and engaged with patient” and “used teach-back.” The second version of the instrument included 20 items that covered six topic areas (Table 2).

Based on feedback from the content experts, a comprehensive codebook was developed and piloted to provide scoring guidance on the 20 communication items evaluated to enable rater consistency. Throughout the three phases of internal codebook consensus building (Figure 1), scoring guidance for all items was refined and finalized, including the addition of descriptive examples for items and how each should be scored.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 2.

Professional Communication in a Patient Encounter Rubric (Post-revisions Implemented during Expert Consensus Process)

DISCUSSION

The ability to communicate with patients is a vital component of health care and is emphasized in the Accreditation Council for Pharmacy Education (ACPE) Standards and the Pharmacists’ Patient Care Process (PPCP).9,10 Numerous learning activities designed to improve health care provider communication skills are described in the pharmacy education literature.4 Simulated and standardized patient encounters are common activities used to evaluate progression in student communication skills. Despite recognition of the importance that communication plays in patient care, there is a high degree of subjectivity in determining what exemplifies effective communication and in measuring the appropriate progression of pharmacy students’ communication skills. There is a deficit of standardized rubrics that can be reliably used to evaluate those skills, specifically in student pharmacists. Consequently, individual pharmacy educators often develop and use home-grown assessment tools. Members of the BTAA-PBAC reported having used rubrics that were revised over time without historical context, only used internally, or had not been validated. It is important to utilize a communication assessment tool that limits ambiguity, bias, subjectivity, and scoring inconsistency across assessors, activities, students, and institutions.

Validated tools for assessing communication skills exist, but few have been developed or studied for pharmacy education. The Patient-Centered Communication Tool (PaCT) is one validated tool for assessing student pharmacist communication with patients; however, two limitations of this rubric are validation within a single institution and validation between similar cohorts of students.8

Having faculty members from multiple institutions collaborate to develop a communication assessment instrument through an expert consensus-building process should help to minimize institutional bias and result in an instrument that provides more consistent feedback. The Professional Communication in a Patient Encounter Rubric is the product of such a collaboration and resulted in a shared vision of effective communication among stakeholders. The accompanying codebook provides a mechanism for improved consistency in scoring across multiple assessor types (students, faculty members, simulated patients, and preceptors) as noted by the mean rating on global comprehensiveness of the communication tool by faculty and patient experts.

The Professional Communication in a Patient Encounter Rubric was designed to assess student communication throughout a curriculum for all types of simulated and real patient encounters. Use of this rubric may allow faculty members to easily identify challenges in student mastery of communication and to consistently assess student communication over time; however, the assessment of clinical content will require a separate evaluation.

The process described in this manuscript required numerous meetings, communications, and revisions involving faculty members located across the country. Challenges encountered throughout this process were developing shared terminology with consistent interpretation and BTAA-PBAC membership turnover. A strong desire by all members to proceed with the research and strong collegiate support were necessary for the success of this project. Further research by the BTAA-PBAC will establish interrater reliability and validation of the rubric and codebook across multiple rater groups.

CONCLUSION

Patient communication skills must be taught and assessed throughout the PharmD curriculum. The BTAA-PBAC applied a multifaceted approach, including an expert consensus-building process, to develop and evaluate the Professional Communication in a Patient Encounter Rubric and corresponding codebook. While the rubric does not include assessment of clinical knowledge, this research demonstrates the importance and ability of multiple colleges and schools of pharmacy to collaborate on the development of a universally accepted communication rubric. External faculty and patient content experts provided valuable input necessary to finalize the rubric and the corresponding codebook.

Appendix 1. Codebook for Professional Communication in a Patient Encounter Rubric

Table
Appendix 1.

Codebook for Professional Communication in a Patient Encounter Rubric

  • Received February 4, 2020.
  • Accepted July 19, 2020.
  • © 2020 American Association of Colleges of Pharmacy

REFERENCES

  1. 1.↵
    1. De Young M
    . Research on the effects of pharmacist–patient communication in institutions and ambulatory care sites, 1969-1994. Am J Health Syst Pharm. 1996;53:1277-1291.
    OpenUrlAbstract
  2. 2.↵
    1. Davis NM,
    2. Cohen MR
    . Counseling reduces dispensing accidents. Am Pharm. 1992;NS32:22.
    OpenUrl
  3. 3.↵
    1. Kimberlin CL
    . Communicating with patients: skills assessment in US Colleges of Pharmacy. Am J Pharm Educ. 2006;70(3):67.
    OpenUrlPubMed
  4. 4.↵
    1. Wallman A,
    2. Vaudan C,
    3. Sporrong S
    . Communications training in pharmacy education, 1995-2010. Am J Pharm Educ. 2013;77(2):Article 36.
    OpenUrl
  5. 5.↵
    1. Grice G,
    2. Gattas N,
    3. Sailors J,
    4. et al
    . Health literacy: use of the Four Habits Model to improve student pharmacists’ communication. Patient Education and Counseling. 2013;90:23-28.
    OpenUrlPubMed
  6. 6.↵
    1. Schwartzman E,
    2. Hsu D,
    3. Law A,
    4. Chung E
    . Assessment of patient communication skills during OSCE: examining effectiveness of a training program in minimizing inter-grader variability. Patient Education and Counseling. 2011; 83:472-477.
    OpenUrlPubMed
  7. 7.↵
    1. Horton N,
    2. Payne K,
    3. Jernigan M,
    4. et al
    . A standardized patient counseling rubric for a pharmaceutical care and communications course. Am J Pharm Educ. 2013;77(7):Article 152.
    OpenUrl
  8. 8.↵
    1. Grice GR,
    2. Gattas NM,
    3. Prosser T,
    4. et al
    . Design and validation of patient-centered communication tools (PaCT) to measure students’ communication skills. Am J Pharm Educ. 2017;81(8):Article 5927.
    OpenUrl
  9. 9.↵
    Accreditation Council for Pharmacy Education. Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree. (Standards 2016). https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf. Accessed November 23, 2020.
  10. 10.↵
    Joint Commission of Pharmacy Practitioners’ Patient Care Process. 2014. https://jcpp.net/wp-content/uploads/2016/03/PatientCareProcess-with-supporting-organizations.pdf. Accessed November 23, 2020.
PreviousNext
Back to top

In this issue

American Journal of Pharmaceutical Education
Vol. 84, Issue 12
1 Dec 2020
  • Table of Contents
  • Index by author
Print
Download PDF
Email Article

Thank you for your interest in spreading the word on American Journal of Pharmaceutical Education.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Expert Consensus to Finalize a Universal Evaluator Rubric to Assess Pharmacy Students’ Patient Communication Skills
(Your Name) has sent you a message from American Journal of Pharmaceutical Education
(Your Name) thought you would like to see the American Journal of Pharmaceutical Education web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
3 + 2 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Expert Consensus to Finalize a Universal Evaluator Rubric to Assess Pharmacy Students’ Patient Communication Skills
Susanne G. Barnett, Andrea L. Porter, Sheila M. Allen, Karen M.S. Bastianelli, Jennifer S. Chen, Marlowe Djuric Kachlic, Casey E. Gallimore, Kristen L. Goliak, Jeffrey C. Reist
American Journal of Pharmaceutical Education Dec 2020, 84 (12) 848016; DOI: 10.5688/ajpe848016

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Expert Consensus to Finalize a Universal Evaluator Rubric to Assess Pharmacy Students’ Patient Communication Skills
Susanne G. Barnett, Andrea L. Porter, Sheila M. Allen, Karen M.S. Bastianelli, Jennifer S. Chen, Marlowe Djuric Kachlic, Casey E. Gallimore, Kristen L. Goliak, Jeffrey C. Reist
American Journal of Pharmaceutical Education Dec 2020, 84 (12) 848016; DOI: 10.5688/ajpe848016
del.icio.us logo Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • Appendix 1. Codebook for Professional Communication in a Patient Encounter Rubric
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

Similar AJPE Articles

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Multi-institutional Analysis of Student and Program Variables as Predictors of Performance on the NAPLEX
  • Exploring Factors That Influence Student Engagement in Community-Engaged Learning Activities Within a Pharmacy Context
  • Self-Assessment of Therapeutic Decision-Making Skills in Pharmacy Students
Show more RESEARCH

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Keywords

  • communication rubric
  • medication education
  • patient education
  • assessment tool
  • consensus building

Home

  • AACP
  • AJPE

Articles

  • Current Issue
  • Early Release
  • Archive

Instructions

  • Author Instructions
  • Submission Process
  • Submit a Manuscript
  • Reviewer Instructions

About

  • AJPE
  • Editorial Team
  • Editorial Board
  • History
  • Contact

© 2022 American Journal of Pharmaceutical Education

Powered by HighWire