Expert Consensus Building and Creation of a Universal Evaluator Rubric to Assess Student Communication Skills

Abstract

Objective. To develop a rubric using an expert consensus building process for multiple evaluators to assess student performance of patient communication.

Methods. Faculty and staff from six colleges/schools of pharmacy collaborated on a multi-step expert consensus building process to create a final version of a communication rubric. First, faculty and patient content experts evaluated each item in the rubric for its relevance, criticality, and global comprehensiveness using a 5-point Likert scale (0=not at all, 4=to a high extent). Data was analyzed through descriptive statistics. Faculty members evaluated the results and came to consensus on the second version of the rubric. A corresponding codebook was developed and refined through a two-phase process.

Results. The initial communication rubric was evaluated by 13 expert reviewers. Mean global comprehensiveness on the rubric was 3.83 for faculty experts and 3.5 for patient experts. After evaluating results from the expert consensus building, 14 items on the rubric did not change, five items were revised, three items were removed, and two items were added. The second version of the instrument included 20 items in six topic areas. A codebook was finalized to increase scoring consistency of the 20-communication items.

Conclusion. Overall, content experts reported high global comprehensiveness of the rubric. Faculty collaborations from multiple schools of pharmacy resulted in a 20-item communication rubric and codebook that can be used to increase student scoring consistency.