Comparison of Active-Learning Strategies for Motivational Interviewing Skills, Knowledge, and Confidence in First-Year Pharmacy Students

INTRODUCTION

Effective communication between pharmacists and patients is a vital component of pharmaceutical care, helping to foster constructive therapeutic relationships. The quality of this communication is important in creating and maintaining strong patient-pharmacist relationships, which can lead to improved medication adherence, integration of lifestyle changes, acceptance of preventative measures, and ultimately positive health outcomes. All colleges and schools of pharmacy incorporate communication courses or units into their curriculum. However, there is still a need for continuity in addressing communication skills starting in the early years through the advanced pharmacy practice experiences (APPEs).¹ Assessment of student skills remains inconsistent, and there is an apparent need for a more standardized process.²

Motivational interviewing is an important example of a communication technique used in pharmacy practice. It is a directive, patient-centered counseling style that attempts to elicit behavior change by helping patients explore and resolve ambivalence and resistance to change.³ Motivational interviewing has demonstrated efficacy in various health-related aspects, including medication adherence, health screenings, substance use, smoking cessation, diet changes, and physical exercise.⁴ Teaching motivational interviewing through courses integrated into medical curriculums has been successful in enhancing student knowledge of and confidence in providing patient counseling⁵ and has helped students to use these methods more consistently in their patient encounters.⁶

The Accreditation Council for Pharmacy Education (ACPE) addresses principles of behavior modification as an essential element that must be incorporated into the curriculum for the development of pharmacist communication skills.⁷ In recent years, the American Association of Colleges of Pharmacy (AACP) has focused on motivational interviewing as a strategy for patient counseling by pharmacists, offering programming at their national meetings. Auburn University also offers motivational interviewing train-the-trainer workshops through the Motivational Interviewing Training Institute. They also describe the concept of scripting a virtual patient as an exercise to teach motivational interviewing to first-year students.⁸ An elective course in motivational interviewing that included lectures, peer role-play, video demonstration and discussion, observed out-of-class interviews, and student evaluation was developed through a collaboration between the school of pharmacy and the department of psychology at the University of Missouri-Kansas.⁹ Finally, a motivational interviewing course that used quizzes and peer role-play for assessment resulted in improvements in knowledge and skills over time.¹⁰ Despite examples in the literature, however, there is little evidence demonstrating which instructional or assessment methods are most appropriate, considering the inevitable realities of time and resource limitations.

Medical education is heavily outcome-based, whereas curricular and assessment decisions are driven by learning outcomes.¹¹ Assessment is an intrinsic component of outcome-based education because it can determine how students learn.¹² The ACPE-guided PharmD curriculum in the United States is also heavily driven by educational outcomes and competencies.^13,14 ACPE recognizes the importance of both communication and performance-based assessment in its Accreditation Standards and Guidelines. Areas of emphasis include communication skills and assessment outcomes. Standard 11 addresses the integration of teaching and learning methods and curricular assessments, stating the following: students should take ownership of their learning through “self-assessment of their acquisition of knowledge, skills, attitudes and values”; skill development should be supported through a variety of experiences, including simulations; instructors should use active-learning strategies; and programs should develop “innovative program pathways, courses, or teaching methods” that are tested for efficacy through assessment activities. Standards 12 and 13 emphasize the importance of communication and collaboration skills. Moreover, standard 15 calls for the incorporation of “periodic, psychometrically sound, comprehensive, knowledge-based, and performance-based formative and summative assessments.”⁷ The Center for Advancement of Pharmaceutical Education has defined educational outcomes focused on pharmaceutical care and other areas that require communication with other providers, patients, and the public.^15,16

Assessment of outcomes involving specific skills rather than proof of knowledge can be challenging. Performance-based assessment requires the use of pre-established criteria and continuous feedback by students and instructors during student demonstrations in real-world situations. This type of assessment can be used in a formative and summative manner to provide feedback on performance.¹⁷ Performance-based assessment can be accomplished by means of observation-based ratings, simulations, verbal challenges, presentations, and completion of projects and written papers.¹⁸ The use of standardized patients, actors, or actual patients trained to portray a patient and score student performance on a rating scale is evidenced to be an effective means of teaching communication skills across various healthcare professions.^19,20 This method has been widely used in both medical and pharmacy education.^1,18,21

Recognizing the importance of motivational interviewing in pharmacy education, the primary objective of this study is to evaluate and compare the impact of learning and assessment methods on student attainment of motivational interviewing skills. Secondary objectives include comparisons of these methods on student knowledge pertaining to motivational interviewing and student self-perceived confidence and attitudes. The results of this study can be used to inform course and curricular development pertaining to communication and motivational interviewing skills.

DESIGN

First-year pharmacy (P1) students received instruction on motivational interviewing principles in a large-class lecture and practiced their skills in 1 of 3 practice-laboratory sessions using written, peer role-play, or mock-patient activities. Students were then assessed in a skill-based practicum examination using standardized patients. Student ability was measured using a validated instrument, student knowledge through pretests and posttests, and confidence through survey tools.^5,22,23

This quasi-experimental study used a double-blinded controlled design including both qualitative and quantitative methods. The university's Institutional Review Board approved the research as exempt from full board review, as the risk to participants was none or minimal. Data analysis included only responses from students providing informed consent to participate in the study. During the spring semester of the first year, students enrolled in Pharmacy Practice II: Professional Communication and Patient Counseling received a traditional-style lecture on motivational interviewing. All students were assigned identical required readings and received the same lecture, delivered in a large-class lecture hall. The lecture included an activity during which students watched a video displaying a series of motivational interviewing scenarios and participated in a discussion of strategies and methods used in each. All students were required to participate in a practice-laboratory session and a practicum examination as part of the course requirements. Students were randomized using convenience sampling to the intervention by practice- laboratory session, which was determined by course section (Figure 1). The time between lecture and practice-laboratory session was 3 weeks. The interventions were conducted as formative assessments during the practice-laboratory session. Students in Group 1 were asked to use motivational interviewing skills to counsel a mock patient played by a pharmacist facilitator. Students in Group 2 were asked to use motivational interviewing skills to conduct a verbal dialogue with their peers using identical scenarios. Group 3 was asked to use motivational interviewing skills to write a dialog with a hypothetical patient based on a series of identical scenarios (Appendix 1). Four weeks after the practice laboratory, students participated in a practicum examination during which 1 station was used for summative assessment of motivational interviewing skills. Students demonstrated counseling of a standardized patient based on a smoking-cessation patient case (Appendix 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Study Design

Student learning and beliefs were assessed using a variety of methods, including a self-assessment quiz, a confidence and attitude survey, and a modified version of the Behavior Change Counseling Index (m-BECCI).^5,22,23 Each of these tools is described in detail herein, and copies may be requested from the corresponding author. Raters were paid pharmacists recruited from a pool of evaluators who had previously facilitated or participated in the practice laboratories and practica for the course. They were provided training materials on the m-BECCI several weeks prior to the practice-laboratory sessions and practicum examination and received a 1-hour training session before the start of each.

Student knowledge of behavior change and motivational interviewing concepts was measured using the self-assessment quiz, a 6-question quiz developed by Spollen and colleagues for a similar research study conducted among medical students.²³ Correct answers were added to create a total score ranging from 0 to 6. In the study, the knowledge test demonstrated moderately high reliability at pretest and lower reliability at posttest administration, a difference that may be attributable to the greater variation in scores at posttest. Students took the quiz postlecture to test their “baseline knowledge” and again after the practicum examination to test their improvement. Completion of this assessment was voluntary and was administered by means of Blackboard (Blackboard Inc., Washington DC).

Student confidence in motivational interviewing skills was assessed using the confidence-attitude survey, a 5-item questionnaire adapted by the investigators in a study by Poirer and colleagues.⁵ The items are scored on a Likert-type scale on which 5 = very confident, 4 = somewhat confident, 3 = undecided, 2 = somewhat not confident, and 1 = very not confident. Confidence raw scores were calculated by adding the total of the 5 items from the survey, with a minimum of 5 points and a maximum of 25 points. The survey instrument was administered post lecture and after the practice laboratory using Blackboard. The post-practicum instrument included 10 additional questions, 5 of which evaluated students’ attitudes about motivational interviewing and 5 of which evaluated aspects of the course, such as helpfulness of readings, lectures, and activities. Attitude questions were adapted from the Spollen and colleagues study.²³ Student performance of motivational interviewing skills during the practice laboratory and the practicum examination was assessed using the m-BECCI. The original BECCI was an 11-item checklist completed by trained raters to assess competence in behavior-change counseling skills. Each item is scored using a 5-point Likert scale on which 0 = not at all, 1 = minimally, 2 = to some extent, 3 = a good deal, and 4 = a great extent. Mean total scores, or “practitioner” BECCI scores, are calculated as the mean across all items. Previous research has demonstrated reliability as well as content and construct validity when the BECCI was used in simulated consultations to train healthcare providers. Interrater reliability, intrarater reliability, and sensitivity to detect change on the BECCI are acceptable.²² The content of the m-BECCI is similar to that of the original BECCI, except for 1 additional item added by the investigator to test student use of concepts taught in class. The elaboration of each item and scoring were adapted to the scenario and customized to standardize grading. There were 12 items with a maximum of 4 points each, for a maximum total score of 48, which was used instead of the mean to assess overall performance. Students received copies of the m-BECCI before the practice session and were encouraged to familiarize themselves with the evaluation method.

Data were collected using survey methods previously described and entered into SPSS 19.0 for Windows (SPSS, Inc., Chicago, IL). Wilcoxon signed rank test was used to analyze ordinal data, and a paired student t test was used to analyze continuous data. Comparisons among the 3 groups were analyzed using 1-way ANOVA and the Kruskal-Wallis test. Student responses were analyzed using qualitative methods. Performance scores from the m-BECCI were compared for all groups between the practice-laboratory session and practicum examination using the Wilcoxon signed rank test. Overall student improvement from baseline (postlecture) to the end (practicum examination) was compared using a paired t test. The mean raw scores for confidence were compared using Wilcoxon signed rank test to determine if there was an overall improvement in confidence from baseline (postlecture) to the end (postpracticum examination).

EVALUATION AND ASSESSMENT

Application of motivational interviewing techniques was assessed during the practice-laboratory session and practicum examination using the m-BECCI. While practice-laboratory session and practicum examination attendance was mandatory, the quizzes and surveys did not contribute to student grades. Students were not penalized or rewarded for their participation, but students who participated likely benefitted in terms of knowledge.

Of the 164 students enrolled in the course, 143 consented to be included in the data analysis, with 36 students in group 1, 65 students in group 2, and 63 students in group 3. The proportion of students completing consent forms was similar in each group, as were the proportions of students completing the m-BECCI, quiz, and survey tool.

All 143 students who provided consent completed the practice-laboratory session and the practicum examination assessments. A significant overall improvement in scores was seen for all students between the practice-laboratory session and the practicum examination (p ≤ 0.01), with a practice-laboratory session mean score of 30.4 and a practicum examination mean score of 42.7. Of the students who completed both m-BECCI assessments, 3 students had lower scores in the practicum examination than in the practice-laboratory session, 1 student had the same score, and 139 students improved. Scores were then compared for the practice-laboratory session and practicum examination sessions separately among the 3 groups to determine which practice method produced the highest performance scores. In the practice-laboratory session, the students who used the mock patient exercise performed significantly better than did either students who practiced using peer role-play or those who practiced using written dialogue (p = ≤ 0.01). While students who had previously practiced motivational interviewing using mock patients had a slightly higher average performance score than did the other groups on the practicum examination, the difference was not significant among the 3 groups.

Twenty-eight students voluntarily completed both postlecture and postpracticum examination quizzes. Overall, there was a trend toward improvement in mean scores between lecture and practicum examination (p = 0.051). Scores were then compared for the postpracticum examination sessions among the groups in order to determine whether one specific practice method led to greater improvements in knowledge. The mean change in knowledge scores in the mock patient group was significantly greater than the mean change in the written dialogue group (p ≤ 0.01).

There was a significant increase in confidence from baseline to postpracticum examination (p ≤ 0.01). The 61 students who completed both survey tools were included in this analysis, and of those, 2 had a lower confidence raw score, 5 remained the same, and 54 improved. There was a significant improvement in confidence scores overall (p ≤ 0.01) from baseline to postpractice-laboratory session. The 80 students who completed both survey tools were included in the analysis, and of those, 17 had a lower confidence raw score, 21 remained the same, and 42 improved. There were no significant differences among groups for either practice-laboratory session or practicum examination.

Sixty-five students answered the attitude section of the survey instrument: 11 from the mock-patient group, 27 from the peer role-play group, and 27 from the written-dialogue group. In general, students in all groups agreed that their interactions with the standardized patient would lead to positive patient interactions in the future. Additionally, students responded that they were likely to use motivational interviewing in future patient counseling encounters. When asked whether they prefer to use the biomedical model or motivational interviewing in their future patient encounters, 13 students were neutral or undecided, 4 students said they preferred the biomedical model, and 48 students said they preferred motivational interviewing. When asked which style they believed patients would prefer, 13 students were neutral or undecided, 2 students predicted preference to the biomedical model, and 50 students selected motivational interviewing. Feedback about motivational interviewing was generally positive, revealing that students recognized the power of motivational interviewing to elicit behavior change and the value of this method to patient-pharmacist relationships.

DISCUSSION

These results provide insight into which methods are most effective for teaching motivational interviewing to pharmacy students. They illustrate improvements over time among all groups in skills, knowledge, and confidence. Results for skill assessment show that the mock patient group performed significantly better after the practice laboratory than did the others. Secondary objectives, such as improvements in knowledge and confidence, trended toward greater improvements in the mock patient group but were not significant.

We expected that all students would improve in skills, knowledge, confidence, and attitudes over time. Several studies have found that a combination of lecture and various active-learning strategies, often as part of a required or elective course, have effectively improved these markers in students.^5,9,10 Perhaps incorporating written dialogues, peer role-play, and standardized patients into a course is the best way to teach motivational interviewing skills because these allow for a diverse array of experiences that can appeal to different types of learners. However, because few colleges or schools of pharmacy have motivational interviewing elective or mandatory courses at this time, it may be more realistic to teach these concepts in a communication course or as a unit in a different pharmacy practice course. Our study examines such an example and provides evidence for which practice method is most effective, given limited practice opportunities.

Prior to the practice laboratory activity, students received motivational interviewing readings and a lecture, participated in discussions, and watched a video of a brief interview. The practice laboratory was a nonthreatening environment in which students were encouraged to practice these skills and received feedback. Evaluation of student skills at the practice laboratory revealed that those who practiced with a mock patient performed significantly better than did those who practiced with each other or who wrote a dialogue. Although students in the mock patient group received the same preparatory training as did students in the other groups, they performed at a higher level with the mock patient. It would be ideal to provide students with multiple, repeated opportunities to practice and evaluate their performance. However, the time, effort, and expense may be prohibitive, especially among pharmacy colleges and schools with large enrollment numbers. Therefore, in a setting in which time and resources are limited and students only have 1 opportunity to practice motivational interviewing skills, an experience with a mock or standardized patient could allow for the most efficient progress. Furthermore, there were no significant differences among groups at the time of the practicum examination when all students interviewed a standardized patient. This finding is consistent with the idea that student skills are demonstrated at the highest level when they practice with “live” patients, regardless of previous practice experiences. One factor that may have contributed to significant improvement in skills among all the groups was the feedback that students received on their performance from professors and group facilitators, which gave them a chance to identify their strengths and weaknesses prior to the practicum examination.

The trends toward improvements in knowledge and confidence in the mock patient group from the time of the lecture to the practicum examination suggest that this method may have advantages over more traditional methods. The lack of significant differences in skills and confidence among all groups at the end of the practicum examination suggests that peer role-play and written dialogues are also effective ways of practicing motivational interviewing. However, this type of practice experience should be coupled with a summative evaluation using standardized patients to provide students with opportunities to demonstrate motivational interviewing skills at the highest level. Although using mock and standardized patients can be costly and demanding of human resources, our study shows that this kind of interaction best prepares students for motivational interviewing. It may only be feasible to use standardized patients in a summative assessment; therefore, using peer role-play, written dialogues, or possibly a combination of the 2 for practice would also be effective. Overall, student attitudes toward using motivational interviewing were positive among all groups, though this may be attributable to the standardized patient assessment that all students experienced during the practicum examination.

Despite positive outcomes, our study has several limitations. Because the self-assessment quiz and confidence attitude survey tools were not mandatory and did not contribute to student grades, there were no incentives to complete all 3 of these throughout the semester. This limited the knowledge and confidence data to those who completed all 3 quizzes and survey tools, thus excluding large number of students from this part of the analysis. Providing incentives for students to complete all survey tools and quizzes more consistently might have led to more available data. Additionally, the response rate for the confidence attitude survey was 45%; students voluntarily responding to these items may have had more positive attitudes and experiences toward motivational interviewing.

Another limitation of the study was the potential for inter-rater variability in student evaluation, which was managed by providing the evaluators with training materials ahead of time and conducting a training session immediately prior to the practicum examination. Other issues surrounding this limitation that could not be controlled for included evaluator fatigue, which may have caused variations in grading. As previously stated, the use of standardized patients is resource-demanding and may not be feasible for all colleges and schools of pharmacy. However, whenever possible, standardized patients should be trained actors and an external rater or videotaping could be used for grading in order to minimize standardized patient exhaustion. Modifications to the BECCI were made with the intent of creating an evaluation tool that was easier for evaluators to use. However, the customization and adjusted scoring may have created a less robust tool for skill assessment. When possible, the changes should be evaluated and the tool revalidated, or an already-validated tool more specific to pharmacy should be used.

Ultimately, we hope the results will be used to inform the development and implementation of new active-learning activities and assessment methods when motivational interviewing and communication skills are taught in the curriculum. This study provides a possible model for future research to further evaluate the effectiveness of methods for teaching motivational interviewing skills. Further research may seek to identify whether specific learner types preferentially benefit from a particular method of instruction and/or practice. Alternative methods for content delivery augmented with practice using the methods described here could also be explored. Because the use of standardized patients is financially burdensome, evaluation of the true cost-effectiveness of these methods may be prudent. Most importantly, an evaluation of the impact of these teaching methods on student retention of motivational interviewing skills and use of these behaviors in practice is imperative.

CONCLUSION

Students demonstrated their best performance in both formative and summative assessments when they interacted with mock or standardized patients. Student knowledge and confidence improved over time as they accumulated experience and practice through formative and summative assessments. These outcomes support the use of standardized patients for practice activities as well as formative and summative assessments of motivational interviewing in the curriculum.