Use of Adaptive Learning Technology to Promote Self-Directed Learning in a Pharmacists’ Patient Care Process Course

INTRODUCTION

Upon graduation, pharmacists should know how to maintain and grow their practice knowledge by independently seeking information through continuous professional development and self-directed learning (SDL). According to Hiemstra, SDL is, “any study form in which individuals have primary responsibility for planning, implementing, and evaluating effort.”¹ For SDL to be effective, learners should compare their comprehension of material to assessment scores and both of these with the desired learning outcomes.² However, learners often struggle to judge their own learning progress accurately.² Cues, such as questions or activities, can be given to help students more effectively compare what they think they know to what they actually know.² Because student pharmacists have generally learned via traditional lecture, they need help developing these SDL skills.³

The Accreditation Council for Pharmacy Education Standards 2016 recommends a curriculum “delivered via teaching/learning methods that promote student responsibility for self-directed learning,” which might be accomplished in part through the integration of adaptive learning technology (ALT) software.⁴ Adaptive learning technology allows for the real-time modification of learning materials to meet each student’s individual needs.⁵ In particular, ALT identifies a learner’s strengths and weaknesses and adapts the lesson to personalize learning.⁶ By meeting the various learning needs of students, ALT can help all students reach competency.⁶

There has been little ALT research in pharmacy education to date. One Caribbean school of pharmacy used an adaptive learning textbook in an anatomy and physiology undergraduate course.⁷ Overall, the tool was helpful, and students viewed its features, including quizzes, as positive.⁷ One US school of pharmacy used ALT for voluntary reviews on various topics for incoming first-year students, but ALT-using students had a significant knowledge increase in only one of the four subject areas compared to nonparticipating students.⁸ Adaptive learning technology may be beneficial to pharmacy education as it can address changing student learning needs, an ever-changing health care environment wherein lifelong learning is essential, and the need to personalize learning to ensure content mastery.

The primary objective of this study was to evaluate changes in SDL before and after the introduction of ALT. The secondary objective was to evaluate student pharmacists’ perceptions of and experiences with ALT in a new foundational pharmacy course for first professional year (P1) student pharmacists.

METHODS

This study was approved by the University of Mississippi Institutional Review Board and took place in the Pharmacists’ Patient Care Process (PPCP) course series. All first-year student pharmacists over the age of 18 years who were enrolled in the courses were eligible to participate.

The University of Mississippi offers a seven-year pharmacy program that includes three years of pre-pharmacy courses and four years of professional courses. Pharmacists’ Patient Care Process I and II is a required two-semester course series for P1 student pharmacists that meets for two hours once weekly. In fall 2018, the PPCP I course incorporated active learning into the traditional lecture-based delivery of content. Pre-class activities included reading article(s) and writing reflections on each week’s reading. In-class quizzes were given periodically to further encourage engagement with course material. Midterm and final course examinations and objective structured clinical examinations (OSCEs) were also given. The PPCP II course in spring 2019 was taught using a semi-flipped classroom structure in which students used adaptive learning technology, including learning modules and assessments, before class and completed activities during class time to apply content knowledge. Midterm and final examinations and a final OSCE were given in PPCP II.

In PPCP II, the ALT used was commercially developed. Course instructors submitted module learning objectives and content to course developers who then created ALT modules. Each module included learning objectives, content learning materials, and assessment questions or assignments, all of which related to the course learning objectives. Completion of pre-class ALT modules contributed to 25% of the course grade. To encourage students to interact with and master the material throughout the semester, half of this grade came from the on-time completion of modules and half came from the final score on each module.

This study employed a mixed-methods design, using surveys and focus groups. Student pharmacists were not given any incentive to complete surveys or participate in focus groups and were reminded that participation would not affect grades. Each survey included an adapted instrument to measure SDL, which was originally developed for adult learners returning to school.⁹ This previously validated instrument contained 26 items divided into 10 factors. For this study, only 21 of the items covering seven factors were assessed to better reflect course structure.⁹ The assessed factors included: assignment management, stress management, procrastination management, seminar (lecture) learning proficiency, comprehension competence, examination management, and time management.⁹ Responses were measured on a seven-point Likert scale ranging from strongly agree to strongly disagree. Appendix 1 outlines the adapted instrument items and associated factors. The baseline survey also contained demographic questions.

Focus groups were conducted to provide context to the survey results and gather data regarding student experiences with and perceptions of ALT and course delivery methods. The focus group question guide included questions about the delivery of the course and workload in the fall versus the spring semester, the ALT platform, and any changes that could be made to the course. The following are examples of questions included in the guide: “How often did you go back to refresh yourselves using ALT as part of your studying?” “Have you found that how you engage with course material from this semester to last semester changed at all, in terms of how you prepare for exams or assignments, or do you study in the same way that you have always studied?” “What's been the most significant change from how you interacted with the course material in the fall versus the spring?”

The surveys were distributed at three time points over the course of the 2018-2019 academic year. Using the course roster, survey links for each student were created in Qualtrics (Provo, UT). This made comparison of individual student responses over time possible, but allowed individual responses to remain anonymous after deidentification of data before analysis. The baseline survey was distributed at the beginning of the fall 2018 semester, and the midpoint and final surveys were given at the end of the fall 2018 and spring 2019 semesters, respectively. Students were given 10 minutes of class time to complete the surveys.

Four one-hour focus groups were conducted towards the end of the spring 2019 semester. All students in the course were invited to participate in the focus groups. Interested students were assigned to focus group sessions based on their availability. The focus groups were conducted by the course instructors only after all participants stated that they were comfortable with sharing their opinions about the course and the teaching methods used. Focus groups were transcribed using audio transcription software and cleaned and checked by members of the research team.

Survey results were analyzed using SPSS. Descriptive statistics were used to characterize the sample. Multilevel linear models were developed to evaluate the effects of time on each SDL factor. Self-directed learning scores from three time points were used for the analysis: beginning of the fall semester (t=0); end of the fall semester (t=1); and end of the spring semester (t=2). The t=0 represented scores at baseline, t=1 represented scores associated with traditional lecture, and t=2 represented scores associated with ALT. The SDL factor scores were compared at each time point to identify any potential changes that could be attributed to the transition to ALT in the spring.

Using qualitative content analysis, two research team members independently analyzed the focus group transcripts to identify themes. The first reading allowed them to familiarize themselves with the transcript content. In the second reading, the two research team members began individually identifying recurring themes and exemplifying quotations. During the third reading, they further refined themes and extracted quotations. After completion of these steps, the entire research team met to discuss the two team members’ individual findings and consolidate the number of themes. Consensus was achieved through discussion with the entire research team. The two team members then completed a final review of the transcripts to ensure that finalized themes still made sense within the context of the original transcripts.

RESULTS

Of the 106 P1s, 92 completed all three surveys for a response rate of approximately 87%. The average age of respondents was 23 (SD 1.3) years. Ninety-two (87%) were Mississippi residents, 91 (86%) were not first-generation college students, and 40 (38%) had experience at a different institution.

Linear and quadratic fixed effects from the multilevel models are displayed in Table 1. Quadratic models fit significantly better than linear models for the effect of time on assignment management (χ²=6.98, p=.008), seminar (lecture) learning proficiency (χ²=9.75, p=.002), and comprehension competence (χ²=23.23, p<.001). For other SDL factors, there were no significant differences between the quadratic and linear models. Thus, the simpler linear models were used to model the effect of time on these factors. The negative linear effect of time on assignment management (-0.52) and seminar proficiency (-0.22) suggested an increase in these SDL scores over time. However, the quadratic effect was positive for assignment management (0.21) and seminar proficiency (0.27), which indicated that as more time passed, the positive effect of time was lessened. Because comprehension competence is negatively worded, a similar effect was found with a positive linear term (1.41), indicating an increase in scores and negative quadratic term (-0.58), meaning the positive effect of time is lessened as more time passed. The positive linear coefficient for time management (0.15) showed a decrease in SDL scores over the academic year, and the negative linear coefficient for examination management (-0.14) showed an increase in SDL scores. Because stress management (0.15) and procrastination management (-0.17) are negatively worded, their positive and negative coefficients showed an increase and decrease in SDL scores, respectively.

The four focus groups contained 17 students, with four to five students in each group. Four themes were identified from the qualitative content analysis and are discussed below.

The first theme identified was student learning preferences. Students stated that ALT made it difficult to study for examinations. Students preferred to be provided with PowerPoint slides of essential information because slides could easily be adapted to meet individual study needs. Although students liked the learning activities during class, they did not like reading the pre-class information because, as one student from group 2 expressed preference for being told the information they need to know in class as opposed to learning it on their own before class. However, focus group participants did find answering assessment questions in ALT helpful for examination preparation and said they would like to be given more and different types of practice assessment questions.

The second theme identified was comparison of teaching methods across semesters. Students enjoyed having activities in class to help solidify learning and keep them engaged with the material, but they also wanted a mini-lecture to reinforce what was learned before class and convey clinical relevance from the “expert’s” perspective. “I think each semester was kind of an extreme, and I feel like a middle ground would be really good where you talk about it, like a really quick summary, hit the high points, and then have a lot of activities to reinforce it. And give us PowerPoints beforehand so we can familiarize ourselves so that the summary sticks.” A mix of teaching methods from both semesters would have optimized learning from one student’s perspective, where they would receive the content before class, summarize that learning, and reinforce it with activities during class.

Participants admitted to not spending very much time with the material in preparation for class in the spring semester. Some students would only answer the assessment questions because the ALT allowed for an unlimited number of attempts to encourage mastery of the content. Although students said they did not enjoy writing guided reflections for pre-class readings in the fall semester, they admitted that the writings helped them with retaining the information.

The third theme identified from the focus group discussions was value. Students enjoyed anecdotes and hearing instructors’ perspectives on the course material. In-class activities also gave material relevance by allowing students the chance to apply concepts. One student explained that it is important to them to hear why the content is important and how it is used in practice. The fourth theme identified during the focus group sessions was technology. Several technical issues occurred when using the ALT platform. For grades to synchronize appropriately, students had to access the ALT platform through the learning management system (Blackboard), and this process was not seamless. One student pointed out preference for being able to go straight to the ALT platform instead of accessing it through Blackboard.

DISCUSSION

In line with the primary objective of this study, several changes in student pharmacists’ self-directed learning scores were noted after implementation of adaptive learning technology (ALT) in a Pharmacists’ Patient Care Process Course. Analyses of the models suggested that students’ ability to manage stress and examinations improved over time. This finding was not surprising to us as first-year student pharmacists need time to adapt to the pharmacy school course load and assessments. Procrastination and time management worsened, suggesting that students may be facing burnout by the end of each semester. Also, procrastination may have worsened in some students because they preferred not to engage with ALT. An increase in the amount of assignments and coursework may have also factored into the effect on students’ time management.

Interestingly, the quadratic trends for comprehension competence improved at the end of the fall semester, but slightly worsened at the end of spring semester. This finding was supported by focus group findings that the pre-class reflection assignments in the fall semester helped students understand and retain information. The trends for assignment management indicated increased scores at the end of the fall semester, but a slight decrease from the end of fall semester to the end of spring semester. This was also supported by focus group findings that students preferred to be told relevant content rather than determine it themselves. There was essentially no change at the end of the fall semester and a decrease in scores at the end of the spring semester for seminar learning proficiency. This is slightly contradicted by focus group findings that students preferred in-class lectures, but it could be related to not having traditional lectures in the spring semester.

Overall, the focus group findings suggested that students preferred in-class learning compared to using ALT. The positives aspects of ALT included that it provided extra practice assessments and allowed time for more active learning in class. However, the negatives, which included easily bypassing the course material to respond to assessments, technological difficulties, and increased difficulty of studying the material later, outweighed the positives for students. This preference for traditional lecture was also found in second-year students in another study that examined didactic teaching methods vs pre-class video lectures in a team-based learning self-care pharmacotherapy course.¹⁰

While one goal of using ALT was to decrease cramming before the examination by encouraging ongoing engagement in course material, students’ own study habits proved difficult to overcome. During out-of-class time, students prioritized studying for more urgent deadlines and came back to this course material to study it more completely just before an examination. Thus, for ALT to work in a flipped classroom setting with early learners, the authors suggest that there be a mechanism in place to ensure greater accountability. This could mean allowing students only one or two opportunities to complete each assessment or adding more comprehensive in-class assessments that require greater preparation by students. Although this is not true SDL, it may help build a better foundation for first-year student pharmacists who may have never engaged in it previously.

Another reason the students may prefer traditional lecture over ALT is that they do not have the skills to decipher important information independently. More training and support in identifying relevant information from ALT (or any other modality) before immersing students in a flipped classroom could also help them to become more proficient self-directed learners. A study by Gleason and colleagues that varied implementation of nontraditional learning in a third-year course in a PharmD program found that the highest level of learning was associated with high levels of instructor involvement.¹¹ According to student focus group participants, instructors not only provide guidance on course material, but also give relevance to the material by relating it to pharmacy practice.

Some limitations of this study should be noted. First, the original validated SDL instrument was adapted to fit the nature of the course series in this study, nullifying the instrument’s original validation. The excluded factors were not applicable to the learning environment of the student pharmacists in this course series, as they pertained to courses delivered online. No changes in the other SDL scores measured were anticipated through this change. Second, while their were significant improvements in SDL scores over the academic year, this may not have reflected a meaningful change in educational outcomes. Third, a follow-up survey was not conducted in fall 2019 with these students. Thus, the long-term effects of ALT on SDL could not be ascertained. Fourth, course material may not have been fully optimized for ALT. As faculty members were developing an entirely new course, many lessons within the ALT were not linked to each other, making redirection to related modules difficult if students did not master content. Additionally, there may have been fewer assessment questions than optimal for ALT to be used to its full potential. Ideally, there would be a bank with questions of varying difficulty for each learning objective to meet each learner’s ability. Finally, this study was conducted in one cohort of students at one school of pharmacy and is not representative of all first-year student pharmacists.

CONCLUSION

Student pharmacists struggled with the integration of ALT into their previously established study routines. Nevertheless, they provided some helpful suggestions for future consideration when planning ALT and in general classroom organization. Students preferred traditional lecture-based learning with reinforcement of material through in-class activities. Students liked answering assessment questions using ALT but would have preferred the content to be delivered differently. Although differences were found for SDL factor scores and some trends became apparent, it was not possible to conclude from our findings that implementation of ALT improved SDL.