Student Performance on Graded Versus Ungraded Readiness Assurance Tests in a Team-Based Learning Elective

DISCUSSION

Team-based learning incorporates multiple levels of student accountability. First, students are individually accountable to perform well on the iRAT. Second, students are accountable to their team members to perform well on the tRAT and team application exercises. Our investigation of adjusting the incentive structure in a TBL classroom using graded or ungraded RATs demonstrated no significant difference in performance on individual overall formative and summative assessments. This important finding is contrary to the widely held belief that a graded incentive structure is needed to extrinsically motivate students to complete preclass preparation. These results confirm previously published data by Koh and colleagues demonstrating that ungraded formative assessments led to a slight decline in formative assessment performance but a sustained performance on summative assessments.²² Our results, along with previously published data, challenge one of the core beliefs in TBL, that graded iRATs incentivize and extrinsically motivate students’ individual class preparation.¹¹

Foundational knowledge established through preclass preparation is crucial within TBL for ensuring proper peer-to-peer learning, supporting team answers, and achieving consensus during team application exercises. While some degree of extrinsic motivation still exists in this structure (eg, accountability to the team and graded exams), this process does not appear to require a graded formative assessment to remain effective. Participating in an iRAT (graded or ungraded) engages retrieval practice that solidifies knowledge and brings awareness to knowledge gaps. Perhaps removing the graded formative assessments allowed students to shift learning motivation from the performance-approach and/or performance-avoidance types to the learning approach, allowing students to focus on mastery of material rather than performance on assessments. Mastery of material may have been indirectly attributed to performance on other graded assessments within the course. Accountability to a team of peers may also motivate students to prepare for class, as the TBL structure emphasizes peer-to-peer learning during the tRAT and team application exercises.

Results from the end-of-course survey of the ungraded RAT cohort confirmed that students felt a responsibility to team members (Table 3). Most reported feeling a need to contribute to the team, and they perceived their contributions as important, indicating a perceived benefit from team interactions that enhanced learning. Regarding class preparation, most students agreed or strongly agreed that class preparation was necessary to do well in the course (Table 3). Most students completed preparation assignments, though this varied by topic and reasons for not preparing. Notably, none of the students reported that they failed to engage in preclass preparation because the RATs did not contribute to their course grade. This may demonstrate that although ungraded RATs did not serve as an incentive for class preparation, students viewed team accountability as an extrinsic motivator, enticing them to maintain engagement with course material despite the lack of a peer evaluation process. Within this incentive structure, individual examination performance may have also been perceived as an extrinsic motivator.

Although there was no difference in overall scores for formative and summative assessments, it is important to note the significant difference in scores between cohorts for the first iRAT (iRAT examination 1) and the first individual examination (individual examination 1), with the ungraded RAT cohort scoring 5% and 7% lower, respectively (Table 2). Although approximately 70% of students from both cohorts reported prior participation in TBL, delivery can vary significantly between courses. Further, the ungraded RAT cohort was adjusting to TBL delivery using live synchronous videoconferencing rather than in-person TBL delivered synchronously across three campuses. This adjustment may have contributed to differences in scores on the iRAT examination 1 and the individual examination 1. For examination 2, perhaps students in the ungraded RAT cohort adjusted to this environment, leading to better assessment performance. This adjustment may not have been observed for the graded RAT cohort because the iRATs were graded and IF-AT cards were used for tRATs, facilitating more initial engagement with material earlier on during the course.

Some limitations of this study must be considered. This was an elective course, so students may have been more engaged in the subject matter and, therefore, intrinsically motivated to prepare for and engage in class compared to a required course. Interest in the course content may have increased the likelihood that students would engage in the mastery approach. Extrinsic motivators (eg, formative graded assessments, peer evaluations) may play a more crucial role in encouraging engagement with course content for a required course. While we did not collect download rates for preclass materials as in previously published data, we attempted to characterize class preparation and perceptions through student survey responses for the ungraded RAT cohort, but we did not collect the same data for the graded cohort. Additionally, some content experts/facilitators differed between cohorts, although new facilitators received the same training for both cohorts. Individual facilitators identified preclass preparation materials, wrote preclass preparation objectives, created RAT questions, developed team-based application exercises, facilitated class discussion, and created examination questions.