Relationship Between Pharmacy Students’ Use of Self-Regulated Learning Strategies and Course Outcomes

INTRODUCTION

Self-regulated learning (SRL) is a process whereby learners monitor and control their cognition, affective states, and behaviors to reach individual learning goals.¹ Differences in SRL often account for significant variance in achievement among learners.² For Doctor of Pharmacy (PharmD) students with heavy course loads, significant co-curricular activities, and work schedules, effective self-regulation is likely to be an important predictor of students’ use of instructional strategies and materials which, in turn, impacts learning. Learners are also confronted with an increasing number of distractions from social and other media, which may infringe upon a conducive learning environment and put a greater onus on the learner to self-regulate.³ In addition, when evaluating the quality and effectiveness of supplementary instructional materials, readings, and practice questions, SRL provides a useful lens to understand why some learners use or neglect instructional strategies, activities, and practice exercises proven to improve learning.

The importance of SRL has been recognized by the Accreditation Council for Pharmacy Education (ACPE) (the ACPE uses the term “self-directed learning” rather than SRL) during and after the formal PharmD curriculum.⁴ The modern pharmacy curriculum is also designed to foster lifelong learning, for which self-regulated learning is a necessary skill. One common and effective strategy to improve learning is structured practice. In online and blended learning environments, digital tools allow instructors to easily and quickly create practice regimens for learners to reinforce key terms, concepts, and problems. The use of these types of tools has been found to significantly improve learning.^5-8 At this point, however, the question is not whether the use of the tool as a learning strategy is effective, as the literature strongly supports this hypothesis, but rather why some students, often the ones who need to improve their competencies the most, neglect to use or engage with the tools and materials. Self-regulated learning provides a practical theoretical lens through which to evaluate this specific question.

There are three major constructs of SRL, which were previously validated in students enrolled in a science course at a high-school with low socioeconomic status: managing the learning environment and behavior, seeking and learning information, and maladaptive regulatory behavior.⁹ In this study, high-achieving students reported significantly better self-regulatory strategies in each of the three SRL constructs than did low achievers. Conceptually, students with poor self-regulation strategies in one or more of these areas may be less likely to participate in optional practice exercises as these metacognitive strategies require a structured, methodical approach to acquire and retain knowledge as opposed to traditional cognitive approaches to retain knowledge such as rehearsal.¹⁰ The purpose of this study was to determine the relationship between student-reported self-regulated learning through the use of supplementary material, and overall performance in a PharmD therapeutics course.

METHODS

This study was conducted in 2018 at a large Public University within a college of pharmacy classified as an R1 research, and an enrollment of over 1,000 students. The majority of the student population was female (59.0%). A majority of the student body was white (42%), followed by Asian (20%), Hispanic (22%), black (8%), Native American/Hawaiian (2%), and those not reporting ethnicity (6%). The mean (SD) age of the student body was 25 (4.1) years. The study was deemed exempt by the institutional review board. All third professional year PharmD students across three campuses were given access to an online practice quiz and invited to participate in an optional, modified version of the SRSI-SR⁹ during a required therpaeutics course.

In week 2 of the course, a 36-question set of practice exercises was designed by three subject matter experts and delivered to students electronically one week prior to an active-learning session. A single announcement was delivered by email to inform students of the availability of this material for the stated purpose “to help with studying” and “to focus on clinically applicable concepts.” The practice exercises were in quiz format, with content including only low-level Bloom’s taxonomy (knowledge) questions. They were formulated with a goal of student self-assessment of class preparedness, specifically an active-learning session on the topic of critical care that involved an individual readiness assessment test, a team readiness assessment test, and a case-based activity involving application of the Pharmacists’ Patient Care Process.¹¹

Students could complete the quiz as many times as they wanted using several different game-style environments, and the quiz was made available via the internet and assessable from both stationary and mobile devices. If a student used the supplementary practice exercises, this use was documented in an online log of trace data.

To operationalize the core components of the SRL strategies most associated with direct effects upon learning, Cleary created a 45-item Self-Regulation Strategy Inventory Self-Report (SRSI-SR) for three categories of self-regulatory strategies: managing environment and behavior, seeking and learning information, and maladaptive regulatory behavior. Because of the differences in population, age, school type (secondary vs professional), and instructional format (traditional face-to-face vs team-based blended learning) in Cleary’s study, we created a modified version of the SRSI-SR to address contemporary issues related to self-regulation (eg, personal use of social media), and to reflect the blended-learning environment in higher education (eg, student interaction with online lecture videos).

To our knowledge, the SRSI-SR had not been validated in higher education; therefore, we administered a modified version of this inventory that included 44 questions, 16 of which were customized questions to assess contemporary self-regulation strategies, such as students’ use of social media in managing their learning environment. The questionnaire was based on a seven-point Likert scale ranging from never to always, and items were reverse-scored as needed so that higher scores reflected better self-reported, self-regulation strategies. Students who agreed to participate completed the survey at the end of the course and there was no time limit, though most students finished within 15 minutes.

For purpose of analyses, the custom SRSI-SR was validated using principle component analysis and factor loadings, and the survey response was divided into three factor-categories with Cronbach alpha of 0.79, 0.84, and 0.80, for managing study behaviors, management of environmental conditions, and maladaptive behaviors, respectively (Appendix 1).¹² Factor 1 included questions about managing study behaviors (eg, “I try to identify the format of upcoming tests”), factor 2 assessed management of environmental conditions associated with studying (eg, “I try to study in a quiet place”), and factor 3 evaluated maladaptive behavior (eg, “I lose important worksheets, formulas, lists, dittos, or other materials”). We then calculated the total score for each factor domain, with maximum scores of 56, 49, and 56 in factors 1, 2, and 3, respectively.

Normally distributed continuous baseline characteristics are reported with mean and standard deviation (SD), and skewed data are described using median and interquartile range (IQR). Distribution of continuous variables was determined by visual inspection of normal quantile plots. Categorical baseline characteristics are reported with numbers and proportions. Actual quiz scores, overall course performance, and scores in specific factor domains were analyzed by use of the practice quiz with two-sample t tests, and effect sizes are indicated by the Cohen’s d statistic.¹³ The association of overall course grade with score in each factor domain was analyzed using simple linear regression. All statistical analyses were performed using JMP, version 14.0 (SAS Institute Inc, Cary, NC), with a two-sided p value less than .05 considered to be statistically significant.

RESULTS

One-hundred fifty-eight students (79%) reported using the supplementary practice exercise and 207 students (98%) completed the SRL inventory. The majority (66%) of students who completed the inventory were female. In terms of demographics, 49% of respondents were white, 20% were Asian, 19% were Hispanic, 5% were African American, and 7% did not report any ethnicityor race.

Learners who indicated using the supplemental exercises performed significantly better on the in-class quiz than those who did not use the exercises (8.2 vs 7.6 out of 10; t=2.38, p=.02) with a small to moderate effect size (d=0.43). The mean (SD) overall course grade was 87.3 (5) and was significantly higher for students who reported using the practice quiz (88.0/100 vs 85.3/100; t=2.99, p=.003) with a moderate effect size (d=0.57).

Learners who used the supplementary exercises scored significantly better in terms of managing study behaviors (factor 1 score 40.2 vs 36.8; t=2.86, p=.005, d=0.47), managing environmental conditions (factor 2 score 31.2 vs 27.7; t=2.73, p=.007, d=0.48), and maladaptive regulatory behaviors (factor 3 score 46.3 vs 43.3; t=2.79, p=.006, d=0.47) than learners who did not (Figure 1). In other words, higher-performing students self-reported more effective SRL strategy practices. Mean (SD) scores were 40 (7.0) for factor 1, managing study behaviors; 30 (7.4) for Factor 2, managing environmental conditions: and 46 (6.4) for factor 3: maladaptive regulatory behaviors. Overall course grade was not significantly associated with factor 1 (r²=0.02; p=.07) or factor 2 (r²=0.01; p=0.94), but was significantly associated with factor 3 (r²=0.09; p<.001) (Figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1a-c.

Student-reported use of an optional practice quiz and score in managing study behaviors (Figure 1a), environment (Figure 1b), and maladaptive regulatory behaviors (Figure 1c) for 197 students who completed the SRL and reported answer for use of the practice quiz.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Association of Doctor of Pharmacy students’ overall course grade with their score in managing study behaviors (Figure 2a), environment (Figure 2b), and maladaptive regulatory behaviors (Figure 2c) (n=207).

DISCUSSION

Although use of optional supplementary material by students enrolled in a professional program may seem intuitive to faculty members, use of a practice quiz was reported by less than 8 out of 10 students in our study. Given the faculty time and resources involved in developing optional learning content, it is important that these activities are effective in improving performance. As expected, self-reported use of the supplementary quiz was associated with higher in-class quiz scores, better overall course grades, and fewer maladaptive study behaviors. Numerous other studies havedemonstrated similar positive relationships between use of supplementary (optional) material and overall course performance.^14,15 Further, one study found a linear relationship between the number of optional resources accessed by students and their overall course performance, with a 16% difference between those who used none versus those who used all of the available resources.¹⁶

Although use of a supplementary practice quiz in our study represents a positive self-regulatory behavior (eg, “I quiz myself to see how much I am learning during studying”), the overall relationship between course performance and the modified SR-SI was weak. This is to be expected as the majority of students enrolled in a PharmD program can be considered high-achieving academic students who have likely adopted many effective self-regulatory strategies. Factor 3, maladaptive study behaviors, however, was a significant predictor of course performance, which suggests that the lower quartile of performing students would benefit from reflecting upon and changing maladaptive behaviors that undermine their performance in the PharmD program. This finding should encourage faculty members, administrators, and staff members who support and coach students who are struggling in the PharmD environment by focusing advising efforts on remediation of maladaptive learning behaviors rather than strategies associated with positive study behaviors and management of environmental conditions. In addition, when instructors have invested time and effort in designing supplemental activities, it may be useful to emphasize to students the importance and potential learning gains of supplemental practice rather than to assume all students will use available resources when necessary.

Two items in domain 3 (maladaptive study behavior) addressed the use of lecture videos and slides. Interestingly, approximately 20% of students reported that they at least sometimes downloaded the lecture slides without watching the lecture, and watched the video lectures but did not download the lecture slides. Patanwala and colleagues reported a similar trend in low use of video lectures that decreased over time in the course and was positively correlated with administration of a quiz during the active-learning session.¹⁷ Based on our findings, we have implemented several strategies to improve student engagement, including use of shorter recorded lectures (eg, 15 minutes), and innovative delivery methods such as interviewing a patient and using virtual reality (VR) clinical scenarios (eg, 360° VR, www.roundme.com).

Because self-regulated learning is imperative to successful completion of advance practice pharmacy experiences (APPEs), future directions include use of our validated instrument to determine the association of self-regulated learning with cognitive and professionalism performance during these experiences. Prior knowledge of students potentially at risk (ie, those with low scores in one or more self-regulated strategy domains) may represent an important remediation opportunity prior to students beginning APPEs. Further, this tool could potentially be used in documenting the requirement that pharmacy students should be APPE and team-ready prior to beginning clinical rotations.⁴ We administer the modified SRSI-SI at the end of the third professional year; however, we have considered administering it earlier in the curriculum for future iterations based on the findings reported here. Others have reported methods to assess APPE-readiness. In one case, student performance in a live, simulated assessment program that they were required to complete prior to beginning their APPEs modestly correlated with subsequent APPE performance, as half of students with failed competence in one or more domains experienced difficulties during APPEs.¹⁸

Strengths of our study include a relatively large sample size, generalizability across multiple campuses, high response rate, diversity of participants, and use of a validated instrument. We also had a high response but the rate was similar to that seen in other recent in-class surveys.^19,20 Limitations include those inherent to student self-reporting, including lack of temporal association in findings as the survey was only administered once and near the end of the semester. Students may be more motivated and focused at the beginning of the semester and less likely to demonstrate poor self-regulatory behaviors than they are later in the semester. Further, it is conceivable that some students knowingly engage in maladaptive self-regulatory learning strategies as part of a greater strategic decision in terms of allocation of limited time and resources.

CONCLUSION

In this study, use of instructor-created supplementary activities was positively associated with pharmacy student performance in a complex disease management course. In addition, maladaptive learning behaviors appear to be a leading predictor of performance in the context of a PharmD curriculum. In light of these findings, educators must recognize that no matter how well an activity, lesson, or formative assessment may be designed, student use of these supplemental activities may be mediated by SRL processes. In addition, academic advisers with limited time and resources should focus coaching and advising efforts on the maladaptive behaviors that appear to result in lower levels of learning performance. Finally, the impact of social media on learning in current curricular models and potential benefit of incorporating into innovative course design should be explored in future research.