Abstract
Objective. To describe the implementation of a teleconferencing-based patient counseling role play assessment in a newly developed pharmacy communications course, assess student performance on the role play and compare it with other assessments, and measure student perceptions of and satisfaction with the experience.
Methods. Eighteen cases were developed, each belonging to one of three interactions in community pharmacy: filling a new prescription, completing a refill, or handling a request for a nonprescription (ie, over-the-counter [OTC]) medication. A rubric was designed to evaluate each type of interaction. Student scores in various course assessments were tested and scores on the three cases were compared. Students also evaluated their experience and satisfaction of the assessment and effectiveness of the course.
Results. Seventy-nine students completed the assessment, achieving a mean score of 17.4/20 (87%). Student scores in the counseling role play were similar among case types and positively correlated with other traditional assessments in the course. Despite internet connectivity issues and heightened student anxiety, 71% of students agreed that the assessment did not affect the quality of their counseling, and 74% agreed that the experience was comparable to traditional assessments. The telehealth activity led to enhanced perceived student knowledge and confidence in interviewing and counseling patients.
Conclusion. Students attained high scores that were consistent with other assessments in the course. Students reported that the assessment did not affect the quality of their counseling and felt the setting and context were still authentic. The experience required dedicated preparation and organization but was successful in demonstrating overall student satisfaction and positive perceptions.
INTRODUCTION
Studies have recognized that patient counseling by community pharmacists could improve patients’ adherence and self-care behaviors and could decrease medical costs. 1-3 Counseling is a face-to-face interaction between the pharmacist and the patient or caregiver, but telephone, social media, and technology-based consultations have been used. 2-4 Because patient counseling requires good communication skills, pharmacists need to obtain, practice, and apply communication skills to provide pharmaceutical care.
Telehealth is defined as “the use of electronic information and telecommunications technologies to support and promote long-distance clinical health care, patient and professional health-related education, public health and health administration.” 5 Telehealth also refers to e-health and mobile health (mHealth) and represents the delivery of health services through means other than face-to-face. 6 Telehealth-related terms currently have no universally accepted definitions, so they are likely to change over the next few years; but, a proposed definition of e-health is “the intersection of medical informatics, public health and business, referring to health services and information delivered or enhanced through the Internet and related technologies.” 7 By contrast, mobile health most commonly refers to “using mobile communication devices, such as mobile phones, tablet computers and personal digital assistants (PDAs), and wearable devices such as smart watches, for health services, information, and data collection.” 8
Telehealth tools have proven to be effective in helping pharmacists provide care related to public health in various settings. 9-13 Using telehealth, pharmacists have provided services related to diabetes, hypertension, lipids, anticoagulation, geriatrics, HIV, and hepatitis C. 14-16 Advantages to telehealth include convenience, access to routine care, decreased costs, and reduced exposure to disease. 17 Yet, telehealth has not been implemented widely, possibly due to lack of knowledge and training for both professionals and patients, lack of face-to-face contact, health professionals’ perceptions of negative impact on patient care, costs associated with resourcing the technology, technical and internet problems encountered with use, confidentiality concerns, workload and organizational support, and the inability to seek compensation for these services. 6, 18 Nevertheless, restrictions on human contact imposed by precautionary measures due to the COVID-19 pandemic have greatly affected how health care is provided and have accelerated the adoption of telehealth. Much of these changes are believed to be permanent, and for health care professionals, including pharmacists, to be able to deliver services using telehealth, they need to be educated and trained. 19 Pharmacy organizations are calling for “appropriately trained and equipped pharmacists” to use telepharmacy (the use of telehealth tools in pharmacy). 20-22
This growth of telehealth has prompted health professions education to provide virtual or e-learning experiences for students, which have resulted in improved student knowledge, perceived confidence, and learning in a number of areas. 23-27 As technology becomes increasingly integrated into the classroom, and with the growth of health professions programs with multiple campuses, the use of livestreaming or video recordings to reach students on these campuses has increased, which addresses time and space limitations. 25, 28-31 Telehealth was successfully used in the field of interprofessional education (IPE), whereby students obtained medication histories, provided clinical recommendations, and reported improved self-perception of interprofessional competence and attitudes toward interprofessional collaboration. 32 Further, in a team-based learning (TBL) approach related to pharmacogenomics, a simulated telehealth collaboration was shown to reduce the logistical difficulties teams faced in integrating IPE. 33 Virtual platforms have also been used to conduct skill-based assessments, including formative and summative objective structured clinical examinations (OSCEs); in these contexts, students have been found to perform comparably to when the examinations are conducted in person. 34, 35 Several studies have been conducted to compare study behaviors and grades of students in on- and off-campus licensure examinations and have found no differences between the two in achievement of learning outcomes. 36-38
The Bachelor of Pharmacy degree program at Ajman University, United Arab Emirates (UAE), involves completing a four-year, 160-credit-hour program. The newly offered course Communication Skills in Pharmacy is a two-credit-hour course introduced in the second semester of the program and delivered in two dedicated lectures per week. The course aims to introduce communication skills and their application in patient care, patient counseling, and when giving advice on medication use. Assessments in the course consist of various activities (assignments, quizzes, exams, etc), one of which is a patient counseling role play. The pre-class assignments (5.0% weighting) aimed for students to practice gathering information about the medications covered in the course. The quiz (5.0% weighting) consisted of a multiple-choice test of information gathered in pre-class assignments. The midterm (20.0% weighting), a mix of multiple-choice and text-type questions, covered various communication areas. The group assignment (20.0% weighting) was a five-page, student-scripted scenario focusing on pharmacist assertive behavior. This assessment, which is weighted at 20% of a student’s grade, evaluates student performance as they counsel a simulated patient who is presenting a new prescription, a refill, or request for a nonprescription (over-the-counter [OTC]) drug. In the spring 2020 semester, the course was offered for the first time: From January until March 15, 2020, the course was offered in person, but after the breakout of the COVID-19 pandemic, Ajman University adopted a completely online teaching and assessment strategy using Moodle as a learning management system (LMS). Hence, the patient counseling role play assessment was conducted using the Zoom audiovisual teleconferencing platform (Zoom Video Communications, Inc). Therefore, this project aimed to describe the implementation of a virtual patient counseling role play assessment in a first-year bachelor of pharmacy communications course with the aim of comparing student performance in this and other course assessments and evaluating students’ experiences with their first role play assignment using teleconferencing technology.
METHODS
The availability of chronic disease management and extended services is limited in community pharmacies in the UAE; therefore, interactions with patients in community pharmacies are mostly related to dispensing. 39 Hence, the role play assessments were developed to include a total of 18 case scenarios, each belonging to one of three types of interactions that occur in community pharmacies: a request for a new prescription, a refill prescription, or an OTC medication. The course instructor teamed up with a research assistant and three community pharmacists to ensure the cases were correct and logical. In addition, to ascertain that the cases had similar difficulty, team members worked to individually assign each case a difficulty score from one to 10, considering the complexity of the patient and the amount of counseling required. If scores were discrepant, the cases were then refined until consensus was achieved. Students were divided into groups of 10, leading to eight sessions conducted on different days. Two different cases were used in each session to total 16 cases; the two remaining cases were kept for contingency. Each of the 16 cases was given a number (case 1 to case 16); to assign the cases to students, the instructor sequentially assigned case 1 to case 16 to the 79 students according to their sequence on the class roster, as obtained from the university’s Registration Department; after case 16 case was assigned, the same process was repeated until all students were assigned a case. Students were not made aware of their case type allocation.
For each of the three case types, an evaluation rubric was developed to suit the interaction. 40-42 The rubric included items with variable weighting, related to opening an interaction (four points), gathering information (four points), giving information (six points), closing an interaction (three points), and communication skills (three points) (Table 1). The items were graded based on the following scale: zero points if not addressed; half points if partially completed; full points if fully completed.
Evaluation Rubrics Used for Assessing Pharmacy Students’ Patient Counseling Role Playa
After completing the assessment, students evaluated the activity on a five-point Likert-type scale (1=strongly disagree; 5=strongly agree). Specifically, they evaluated the case content, their experience with the counseling role play assessment, and the logistics associated with it. They also rated their satisfaction with the assessment and the effectiveness of the course.
Leading up to the role play assessment, to help alleviate the anxiety and ambiguity surrounding a new assessment approach, several live sessions were led by the course instructor on the LMS to review students’ expectations, the evaluation rubrics, and how students would be distributed to the sessions. Students were required to participate in all course activities and assessments, but students were clearly told that participating in the surveys after the role play assessment was optional. In addition, a mock role play session covering each interaction type was conducted and recorded on Moodle for convenient student access and review.
On the actual day of the assessment, students interacted on Zoom via high-speed internet using their own computer devices with a webcam. On each day of assessment, the sequence of events followed these steps: A session was started on the LMS to communicate with students outside of Zoom before and after the role play. Then, a Zoom session was initiated, and students were called in to join the session individually. When it was their turn, the student was admitted to the session, and he/she conducted their interaction with the simulated patient (the patient role was played by a trained research assistant). Each student was allowed a maximum of 10 minutes to complete their interaction. The session was digitally recorded on Zoom. When finished, the student was asked to leave and go back to the LMS to complete a post-assessment survey, which was linked to SurveyMonkey. The procedure was repeated for all eight assessment days until all students were evaluated. At the end of all sessions, the instructor conducted a group debriefing with the students in which they provided feedback on the optimal management of the cases and on student performance. The instructor reviewed recordings of all sessions (which were also available to the students upon request) and assigned grades by filling out the respective rubric for each student.
Data analysis was completed using Stata, version 15 (StataCorp, LLC). Continuous variables, such as students’ scores in various assessments, were presented as means and standard deviations (SD), while ordinal variables, such as post-assessment evaluations, were presented using counts and frequencies. Weighted means were also calculated for ordinal variables by assigning numerical values to response options and using their counts. Students’ scores in various assessments were tested using the Shapiro-Wilk test and were found not to follow a normal distribution. The Kruskal-Wallis H test was used to test differences in students’ scores in the counseling cases. The Spearman rank correlation coefficient was used to test correlations between students’ scores in the counseling role play and other assessments in the course. A p<.05 was considered statistically significant.
RESULTS
All 79 students completed the counseling role play assessment. Of the students, 36 (46%) achieved a B (receiving 80.0%-89.9% of the available points), 22 (28%) achieved C (receiving 70.0%-79.9% of the available points), and 15 (19%) achieved an A (≥ 90.0% of available points). Mean scores in all course assessments are shown in Table 2. No significant differences were found among students’ scores in the three case types (Χ2=4.86, p=.089). Notably, scores in the counseling role play were significantly associated with scores on patient counseling questions in the final examination (rs=0.26, p=.02), with the overall final examination scores (rs=0.40, p<.001), and with the midterm examination (rs=0.25, p=.024).
Students’ Assessment Scores in Communication Skills in the Pharmacy Course (N=79)
Fifty-four students (68%) completed the post-assessment survey. The majority agreed that the cases assessed specific knowledge areas in communications, but 50% of students agreed that it was more stressful than traditional written examinations (Table 3). Most students reported that the assessment was well administered, that they were well informed of what they needed to do, and that the time provided was adequate. Furthermore, 71% of students agreed or strongly agreed that use of a virtual assessment did not affect the quality of their counseling, 75% agreed or strongly agreed that the experience was comparable to traditional in-person assessments, and 71% felt the setting and context were still authentic (Table 3). Most students reported that they were satisfied with the counseling role play; two-thirds (66%) reported that they looked forward to doing more exercises virtually in the future (Table 3).
Students’ Experience and Satisfaction With In-Class and Online Learning and Counseling Role Play via Zoom (n=54)
DISCUSSION
To our knowledge, this is one of the first studies to describe the implementation of a patient counseling role play assessment using teleconferencing technology in a pharmacy curriculum. Our findings support available evidence that the use of e-learning in pharmacy education is effective. 25 Recently, patient counseling and communication skills were assessed via virtual platforms, whereby students and faculty considered the virtual assessment appropriate for evaluating students’ communication skills, but participants did not show a strong preference for future virtual skills-based assessments. 35 In our study, despite reporting anxiety associated with a new assessment, students were very satisfied and looked forward to undergoing similar assessments in the future. We think that the process adopted in developing the case scenarios and their evaluation rubrics, in addition to the student orientation and pilot session, were instrumental in making sure students were familiar with the new assessment and with the level of performance expected from them. Despite these positive outcomes on student learning and satisfaction, there is an overall paucity of research to demonstrate the benefits of pharmacists using telehealth to provide services. However, early evidence shows that telehealth may be an effective way to deliver pharmacists’ services and to improve patient care, such as by enhancing adherence to medications and achieving glycemic control in diabetes. 43-45
One challenge associated with e-learning may be maintaining the integrity of assessments. 46 Mechanisms for ensuring the integrity of assessments delivered online are being explored and evolve as our knowledge of the technology increases. 34 In this study, key integrity mechanisms included informing students that a variety of cases were developed (aiming to reduce intentional broadcasting of cases to students who had not yet taken the assessment), using more than one case per session, randomly assigning cases to students, limiting the examination administration time, and withholding feedback until all students had completed the assessment. Notably, the student scores of this assessment were similar and correlated well with other conventional assessments that took place in the course, indicating that the applied integrity mechanisms were sufficient.
Regarding the assessment platform, Zoom offered a feasible option to conduct the role play assessment given its various functionalities, ease of use, users’ familiarity with the technology, and its free-of-charge availability to the faculty and students. 47-49 As Zoom is already used by our university, no additional costs were incurred for use during the assessment. Clearly, other free or low-cost video/chat applications are available to academic departments if a teleconferencing system cannot be afforded.
Regarding the study’s limitations, although a rigorous process was used to develop and pilot the rubrics, they have not undergone psychometric testing and validation. Further, conducting the role play assessment this way required ample faculty and student preparation, smoothing of logistics, and high levels of organization. Also, several students needed to access additional devices with quality video and audio capabilities to undergo the assessment, and although internet connectivity problems were predominantly addressed before the actual assessment, some issues still arose for a minority of students.
Finally, the rapid spread of the COVID-19 outbreak imposed sudden changes in almost all aspects of modern life, including higher education. This rapid transition to distance e-learning has raised questions about the ability of faculty to efficiently use technologies in teaching and assessment. 50 This study demonstrated that the faculty can efficiently and effectively adapt to e-learning modalities with adequate preparation; students were very satisfied with their experience in the course.
CONCLUSION
Students attained high scores in all components of the communication skills and counseling virtual assessment. Student scores in the counseling role play were correlated with similar traditional assessments in the course. Students reported that the assessment did not affect the quality of their counseling, that the experience was comparable to traditional in-person assessments, and that the setting and context were authentic. Students reported improved knowledge, communication skills, and confidence in counseling patients. Despite some anxiety, most students evaluated their experience positively and looked forward to undergoing similar assessments conducted virtually in the future.
ACKNOWLEDGMENTS
The authors wish to thank community pharmacists Dalal Taweel, Asma Khadr, and Rami Ahmed, who participated in case development and validation and the pilot session.
- Received February 25, 2021.
- Accepted September 3, 2021.
- © 2022 American Association of Colleges of Pharmacy
REFERENCES
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.