Comparison of Rewatching Class Recordings versus Retrieval Practice as Post-Lecture Learning Strategies

INTRODUCTION

Lecture capture has become a common practice at higher education institutions. These recorded lectures are used in many ways, such as providing background information prior to class or providing additional information after class. Perhaps more common is the use of recorded lectures to supplement student learning by providing a means to review the class lecture and associated discussion. When surveyed, students have indicated they found value in using recorded lectures and ranked rewatching videos as the fifth highest activity they used to prepare for a final examination.¹ However, despite widespread use of lecture capture, educators understand very little about how rewatching lectures impacts students’ learning strategies during and after class. The primary purpose of this study was to determine the effectiveness of students restudying a lecture from a class recording and compare that to the effectiveness of using an alternate strategy, retrieval practice.

For most students, rereading is a common learning strategy.^2,3 However, research has found that rereading is not as effective as other learning strategies.⁴ Rereading increases students’ familiarity with content, which gives a false sense of security in terms of learning. For example, a student who rereads a textbook for an upcoming examination may feel overconfident about their knowledge of the content because they feel more familiarized with it; however, this may result in early termination of study and/or poor examination performance.⁵ A student rewatching a lecture that he or she already attended may have a similar effect to rereading, ie, it increases the student’s familiarity with the material and provides a false sense of confidence. This effect may prevent students from using other effective learning strategies and potentially creates a barrier for students to learn more efficiently and accurately. To date, most of the research on recorded lectures has focused on how they are used and not on their effectiveness as a learning strategy.^1,6 Our group conducted preliminary research and found that acute performance on an examination improved more from rewatching a lecture than by using no learning strategy at all.⁷ Regardless, it is still unclear whether rewatching a recorded lecture is an effective learning strategy for long-term retention of information, specifically compared to other study methods. Further, research has not been completed that compares rewatching recorded lectures to evidence-based learning strategies that have been found to be effective, such as retrieval practice.

Retrieval practice, otherwise called the “testing effect,” has been repeatedly found to be better at improving long-term retention compared to rereading.^8-12 In a study by Roediger and Karpicke, the effectiveness of rereading as a learning strategy was compared to students completing a free-recall test.⁹ Learning was assessed at five minutes, two days, and one-week later. Although students in the rereading group performed better on the first short-term test (five minutes), those in the free-recall group had higher scores on the long-term retention tests (two days and one week). That is, rereading helped students perform well acutely but did not promote long-term retention. These results have been consistently documented in the literature.¹² In most studies on the testing effect, the time-on-task factor is held relatively constant, suggesting that, if given equal time, retrieving information from memory (or taking a test) improves memory consolidation more than rereading does.

This study aimed to determine whether rewatching a lecture that students had already attended would improve their long-term retention of the content more than using an alternative strategy, ie, retrieval practice; whether there is a difference in student perceptions of effectiveness or difficulty of rewatching a lecture versus retrieval practice; and which learning strategy is more efficient in terms of time invested and test performance.” We believe our findings may help inform best practices in the use of recorded lectures by students and faculty.

METHODS

In this study, we used retrieval practice as an alternative learning strategy to rewatching lectures. We included time on task as an additional factor in determining the best learning strategy. Because students may have trouble testing themselves without instructor-provided material, we asked students to respond to the learning objectives for the lecture provided to them in class to help them retrieve information, which is a form of cued recall. Additionally, because the testing effect may be more pronounced with longer retention intervals, we examined the effect after a short-term retention interval (immediately after studying) and a long-term retention interval (one week after studying). The one-week interval is considered a long-term retention measure as forgetting occurs very quickly after learning and then plateaus. The one-week interval captures retention during this plateau period.^13,14 We estimated that an a priori sample size of 52 was needed to detect a moderate effect (d=0.4) with α=0.05 and β=0.2 (1-.8) (G*Power) using a within subjects’ design.

Study participants were recruited from a core course within the first year of the Doctor of Pharmacy program at the Eshelman School of Pharmacy. Prior to the course, participants completed a survey regarding their use of recorded lectures and other study habits. This included an assessment of motivation (Achievement Goal Questionnaire-Revised),¹⁵ as motivation could potentially impact implementation of study strategies. Participants were asked to complete the survey again at the end of the semester, with additional questions added about their views on effective learning strategies.

The study design is outlined in Figure 1. As part of the regular course schedule, students attended two 80-minute lectures given two days apart that previewed two different topics (topic A and topic B). The lectures were delivered approximately three weeks prior to a low-stakes quiz that was part of a readiness assurance process for team-based learning. These 80-minute, preview lectures were given well in advance of the quiz to minimize the effects of any outside studying of these topics that students did during the study period. The lectures started with the faculty member outlining the learning objectives. The lectures were largely expository with no active-learning techniques used (eg, classroom assessment techniques, example problems, collaborative learning techniques, etc). The lecturer used an electronic whiteboard (but no PowerPoint slides) throughout the lecture to illustrate diagrams and share their notes. The lecture ended with a review of the learning objectives. These sessions were recorded using the classroom capture system Media Site (SonicFoundry, Madison, WI).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Study Flow Chart

Within one week after the lectures, participants returned to the classroom where they either restudied from the recorded lecture (restudy group) or engaged in retrieval practice (test group). The two conditions, restudy and testing, were counterbalanced. That is, if a student restudied lecture topic A, they did retrieval practice on lecture topic B and vice versa. In addition, the order of the learning strategy used was randomized, with half the class restudying first and the other half using retrieval first. Participants were given 40 minutes to use one learning strategy, eg, restudy one lecture, and 40 minutes to use the other learning strategy, eg, retrieval practice, on the other lecture. For the restudy learning strategy, participants were given the link to the recorded lecture and instructed to behave as they normally would when reviewing class recordings (eg, take notes, watch at accelerated speed, rewind, pause, etc). For the retrieval practice strategy, students were given several open-ended questions to answer regarding the learning objectives. Participants recorded clock time throughout the session to capture time on task. For the re-studying condition, participants were allowed to take notes.

Immediately after completing each learning strategy, participants were asked a series of questions regarding the effectiveness of that technique, how boring or difficult it was. They also were asked to predict their grade if they were quizzed on the topic. These prompts have been previously used to assess student perceptions of restudy and retrieval practice.¹⁶ For the restudying condition, students recorded at what speed they rewatched the video (eg, 1x, 1.5x, 2.0x).

After all participants completed both tasks, half completed a separate 10-question quiz on each of the two lecture topics (20 questions total) that consisted of six multiple-choice items and four fill-in-the-blank items aimed at the knowledge/comprehension level of Bloom’s Cognitive Taxonomy.¹⁷ For blinding purposes, the other half of the students completed a 20-question quiz on material unrelated to the two topics but relevant to prior course material. Participants received no feedback on their performance on the quiz.

One week later, all participants completed an unannounced 10-question quiz on each topic (20 questions total) during the readiness assurance process for another course topic. This delayed test was the primary outcome measure as it assessed students’ longer-term retention.

The efficiency or “cost-effectiveness” of each study method was analyzed by using time on task as the independent variable and delayed performance as the dependent variable. An effect to cost ratio was calculated using a geometric mean and 95% confidence interval. A geometric mean was used because the ratio data had a positive skew (>2.0) and thus was normalized when log-transformed. This was expected as ratio data cannot have negative values.

This study had a within subjects’ design in that all students completed both interventions. A paired t test was used to detect differences between restudy and test group for the quiz at one-week (long-term retention) and the quiz immediately after study, and to compare changes over time for students who completed both the immediate quiz and the quiz at one week. Survey data were analyzed using the chi-square or paired t test when appropriate. Cohen’s d was used to determine effect sizes, with cutoffs of <.3 considered as small, .3 to .7 as medium, and >.7 as large.¹⁸ Significance was set at p<.05. All analysis was conducted with SPSS (IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp). This study was exempted from review by the University of North Carolina’s Institutional Review Board.

RESULTS

One hundred two students consented to participate in and completed the study, with 51 in the immediate test condition and 51 in the delayed test condition. As for the primary outcome, quiz performance at one week, there was no significant difference between that of students in the restudy group and those in the test group (56% vs 53%, respectively, d=.12, p=.42) (Table 1). Over 96% of students reported not studying the lecture material before this quiz, supporting that student performance reflected long-term retention of material. Conversely, the re-study group performed better on the quiz administered immediately after the review period (71% vs 51%, d=.94, p<.001) (Table 1).

We examined changes over time for students who completed both the immediate quiz and the quiz at one week. This comparison allowed us to investigate the combination of restudy followed by testing or testing followed by additional testing. Compared to the test-test group, the restudy-test group performed better on the quiz at one week, with a small to moderate effect (63% vs 56%, d=.38, p<.010). However, the extent of forgetting between the two conditions differed. The students who restudied and completed the immediate test had a significant decrease in performance over time (71% vs 63%, d=.43, p=.003). In comparison, the students who used retrieval practice followed by the immediate quiz had increased performance over time (51% vs 56%, d=-.34, p=.020) (Table 1).

We also compared quiz performance at one week between those who completed the immediate quiz and those who did not, ie, a between subjects comparison, to determine the effect of the immediate quiz on retention of learned material. For students who restudied, taking the quiz after restudying led to higher quiz performance than those students who did not take the immediate quiz (63% vs 56%, d=.42, p=.034, unpaired t test). However, there was no difference in performance between students who retrieved information and completed an additional quiz and those who just retrieved information once (56% vs 53%, d=.16, p=.42, unpaired t test). Therefore, an additional test did not improve performance.

We examined performance on the individual quiz within the readiness assurance process to identify any effects of additional studying on performance and any potential benefit from prior study or testing. There was no difference between any of the conditions for the in-class, individual readiness test (IRAT) (Table 1).

In addition to performance, we were interested in students’ perception of the two learning strategies immediately after completing each learning strategy. When comparing perceived effectiveness, students perceived the overall effectiveness differently (p<.001), with 82% stating restudy was at least moderately effective compared to 42% perceiving testing to be at least moderately effective (Table 2). Students also felt that rewatching lectures was more boring (d=.36, p<.001) and tiring (d=.48, p<.001) than testing but testing was more difficult (d=-1.2, p<.001) (Figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Perceptions of Learning Strategy Effort. Data presented as mean and standard deviation of a visual analog scale on which 10=extremely (eg, extremely tiring, extremely difficult) and 1=not at all (eg, not at all tiring, not at all difficult). N=102

While one aspect of learning strategies is effectiveness, the other is cost. Students spent significantly more time on the restudy task than on the test task (35 minutes vs 12 minutes, d=2.3, p<.001). We conducted a cost-effectiveness analysis with time on task as the independent variable and performance on the quiz at one week as the dependent variable (Figure 3). The ratio of effectiveness to cost was greatest for testing, with both the test-only and test-test groups having 95% confidence intervals not crossing 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Cost-effectiveness Analysis. (A) Graph of cost in minutes of time against performance at one-week summarized as mean (SD). (B) Ratio of effect to cost (E/C) and its 95% confidence interval

We surveyed students’ self-reported behaviors before and after the semester and, after the study was completed. Prior to the course, a small majority of students reported almost always rereading their notes or textbook, or rewatching recorded classes. This percentage decreased by the end of the semester (Table 3). Over the course of the semester, students self-reported using more retrieval strategies, including self-testing and completing practice problems. When students were asked to rank the effectiveness of different study strategies, there was no change in the percent of students ranking retrieval strategies as effective or rereading/rewatching as a high-impact strategy. Prior to the semester, most students reported attending class and rewatching the class recording. During the semester, however, there was a decline in the percent of students who reported reviewing course recordings. Prior to the semester, students who reported using the recorded lectures did so closer to examinations, and this held true at the end of the semester as well. Finally, student motivation in the areas of mastery approach increased over the semester; the other three constructs did not.

When asked if they would change study habits based on the results of the current study, 37% of students stated they would still use recorded lectures but would self-test afterward. Another 45% stated they would not change their habits. When presented with a theoretical situation comparing three learning strategies, 45% of students would use the strategy that increased long-term retention and took half the time to complete, but resulted in lower short-term performance. Another 4% would use the strategy that took more time and yielded higher short-term results, and 51% would use both strategies. In addition, when asked to define an effective study strategy, 51% of students selected the one that led to better long-term retention, 21% based an effective strategy because it made them feel like they were learning, 19% selected a strategy that yielded higher examination scores, 6% selected a strategy based on ensuring they would not fail, and the final 4% based strategy selection on their comfort with the technique.

DISCUSSION

This is the first study to examine the effectiveness of restudying from recorded lectures as a learning strategy in a quasi-authentic higher education setting. We compared the effects of watching a recording of a previously attended lecture to that of using an evidence-based strategy for retrieval practice (testing). The results indicated that watching the video-recorded lecture increased students’ immediate performance but failed to promote more long-term retention of information than retrieval did. However, rewatching video took students three times longer to complete as compared to retrieval practice. Time on task information plus retrieval yielded better results, and suggested equivalent long-term learning with one-third less time commitment. Although the effectiveness was equal, the time efficiency of retrieval may make it a more valuable study strategy for students. A counter opinion may be that retrieval was not perceived as effective and was more difficult and therefore less appealing for students to use. This is likely because students were unfamiliar with this approach to learning or with the lack of feeling that they were learning.

There were several strengths in this study. The study used subject matter that was relevant to the student population, unlike prior research that used topics unrelated to pharmacy.⁷ This study was completed within the pharmacokinetics class during the first year of pharmacy school. Pharmacokinetics is a unique subject in that most first-year pharmacy students have never encountered this type of content before. This helped prevent the study results from being influenced by students’ prior exposure to or knowledge of the subject matter. Hypothetically, because students were aware that they would later be tested on the material they were learning, they were more likely to pay attention during the lecture and attempt to learn the material. This increased the external validity of the study, bringing the study closer to a real-life scenario for testing purposes. Another way in which we increased external validity was using learning objectives and open-ended questions to add practicality for the students. Not all classes have practice tests or homework for students to use, and, as such, students may feel like they cannot test themselves. By using the learning objectives, students who participated in the study learned to design their own retrieval activities by answering the learning objectives established for the lecture. Another strength of the study was use of a within-subjects design that allowed each student to complete each learning strategy. In addition, the study was limited by the students only attended a single class session before comparing study strategies thus students had limited exposure to the content. This is important because, as mentioned, retrieval works to strengthen memory. Therefore, retrieval would only help reinforce learning that occurred during that single class session. Despite this, restudying did not lead to any improvement in long-term learning; however, restudying followed by a quiz did.

A secondary objective of the study was to examine whether student perceptions of rewatching a recorded class lecture and practicing recall differed. This difference in perception may be an important feature with regards to why students select one learning strategy over another. McCabe demonstrated that undergraduate students were unaware of specific study strategies that would benefit them most according to evidence-based research on study strategies.¹⁹ We wanted to see if this were also true of professional students, who supposedly had been successful in their undergraduate studies in order to have been able to continue their educational pursuits. In post-survey data, we looked at how students rated study strategies as being either low-, medium-, or high-impact. Students recognized rereading or rewatching as a low-impact strategy and retrieval as a high-impact strategy. This may be a result of experience or part of prior courses that discuss learning strategies. Despite that most students believed that rewatching videos was a low-impact strategy, many students still reported rewatching lectures. Also, despite students stating retrieval was a high-impact strategy, their use of self-testing was still relatively low. This inconsistency may reflect students’ desire to balance short-term performance with longer-term retention, or students’ perception that rewatching lectures was less difficult but boring. Although students appeared to be able to distinguish between high- and low-impact learning strategies, whether they thought about the “cost-effectiveness” of the study strategies is unclear. Pharmacy students have increasingly busy schedules as they progress through a curriculum and must weigh their lack of time against opportunities to improve their knowledge and performance. Furthermore, almost half of the students in this study reported using multiple study strategies when studying. More research that compares the use of combinations of strategies for reviewing lecture material and the cost-effectiveness of these strategies to help students best direct how to use their time. This can include the best ways to increase their knowledge base, eg, rewatch lecture or review their own notes, and the best way to reinforce that learning through testing.

There are a few potential implications for this research. First, watching a recording of a class that was attended did not promote greater long-term retention of the lecture content than testing, and watching the recording required more time. For restudying recorded lectures to be an effective study strategy, it needs to be followed by testing. In the classroom setting, instructors could demonstrate effective methods for students to test themselves before assessments to help students retain important information and become more efficient. For example, instructors could provide comprehensive learning objectives to students. This could help students develop their own practice questions to prepare for assessments rather than focusing solely on the lecture content. However, this approach would require more preparation by instructors, and students would have to learn how to monitor their own learning more effectively. This could then enable students to adapt more efficient learning strategies overall, which would be highly beneficial as professional school often requires students to balance multiple commitments with limited time. By using this information, student performance and well-being (eg, time stress) could be improved.

A second potential study implication was that testing appeared only to reinforce the material students left class understanding. Therefore, after a class session, additional study time of the material that students still did not understand may be needed to increase the strength of the student’s memory of that material. Once this is achieved, testing could be used to reinforce those memories.

Third, despite access to evidence that testing is more efficient for learning, most students may not change their study strategies. Theoretically, students want both long-term retention and high acute performance. However, to meet this goal, students may have to add study time and combine study strategies. Often students sacrifice long-term retention in order to increase acute performance through “cramming.” For example, our study found that students were more likely to watch a recording of a class lecture near the time of assessment rather than immediately after lecture. Future research is needed to determine how students can best accomplish their learning goals.

CONCLUSION

This study was the first in a higher education setting to directly compare rewatching a video versus retrieval practice. For short-term retention, students showed higher performance on a quiz administered after the restudy condition from the recorded lecture compared to the retrieval condition. However, long-term retention did not differ between the two groups. Because long-term retention did not differ between the two groups, time spent on each task was an important consideration for selecting the optimal learning strategy. In this study, students spent three times longer rewatching class video than they did on retrieval practice. Therefore, retrieval practice is a more efficient use of students’ time and produces similar long-term results.