Team-based Learning to Promote the Development of Metacognitive Awareness and Monitoring in Pharmacy Students

INTRODUCTION

Pharmacy students are expected to master a plethora of information and develop into lifelong learners. The importance of student learning awareness is supported by the Accreditation Council for Pharmacy Education’s (ACPE’s) Standards for the Doctor of Pharmacy Degree, which emphasizes the importance of actively engaging learners and promoting self-directed learning.¹ Although the ACPE standards do not specifically address metacognition, the guidance acknowledges that life-long learning requires individuals to self-assess their learning needs. Metacognition broadly refers to how individuals think about thinking. It can be described more specifically as “the ability to monitor thinking to use skills and strategies appropriately to achieve a desirable outcome,” which is crucial in controlling and guiding thinking.^2,3 Metacognition consists of three components, including knowledge, monitoring, and control.⁴ It accounts for the students’ awareness of how they learn (metacognitive knowledge and beliefs), how they assess their learning and performance (metacognitive monitoring), and their ability to regulate their understanding (metacognitive control). Students at the lowest performance levels tend to overestimate their abilities compared to high- and mid-level performers (Dunning-Kruger Effect).⁵ Metacognitively aware learners are more strategic, predict their performance more accurately, and excel further than unaware learners.^4,6,7 These students use their study time most efficiently, narrowing their focus to specific areas and maximizing their preparedness.⁸ In meeting the goals of a pharmacy program, students benefit from the use of metacognitive skills to solve new problems and appropriately evaluate their understanding of additional information.⁹ A key instructional strategy for metacognitive skills is “learning through guided experience.” This is achieved when teaching activities focus on both cognitive and metacognitive processes while performing a task.^8,10 Team-based learning (TBL) is a pedagogy that provides a scaffolding, which may enhance metacognition. In TBL, students have the potential to improve their awareness of how they learn, assess their learning by predicting their performance, and build more context-independent knowledge and skills through cooperative learning. During TBL, students complete an individual readiness assessment test (IRAT) followed by a team readiness assessment test (TRAT) using the Immediate Feedback Assessment Technique (IF-AT). When employing the IF-AT, students use a scratch-off card, which makes the assessment process part of the learning cycle and allows them to finish the assessment knowing the correct answers.¹¹ While discussing the TRAT and then completing in-class activities, teams reinforce their knowledge and problem-solving skills through deliberate cooperative processes. As this process is repeated multiple times in a semester, the students gain practice in preparing and then monitoring their level of preparation (ie, metacognitive knowledge, beliefs, and monitoring). Stewart and colleagues reported that students who were given the ability to self-assess using practice quizzes, which provided immediate feedback, saw an improvement in their examination scores.¹² Feedback was given in a way that encouraged students to provide the rationale for each correct and incorrect answer. We hypothesize that the same benefit may be gleaned from the TBL process because the TRAT involves discussion within the group and rationale is frequently provided when a question is answered incorrectly. We are not aware of any reports of the impact of the TBL process on metacognition or on the ability of students to assess their understanding when they are given repeated opportunities to see if their expected performance matches their actual performance.

The purpose of this study was to evaluate the metacognitive skills (including metacognitive knowledge, beliefs, and monitoring) of pharmacy students within a Pharmacotherapeutics of Cardiovascular Disease course, and to determine if introducing metacognition and performing a metacognition inventory along with team-based learning can enhance metacognitive knowledge/beliefs and monitoring. These findings will enhance our ability to promote life-long learning skills in pharmacy students who will be entering a profession that requires these skills for continuing professional competence.

METHODS

RESULTS

One hundred students were enrolled in the course. More than half the class (66%) was female, and the mean grade point average at the beginning of the course was 3.2 (±0.4). Results were organized according to the following three research questions: What is the metacognition of pharmacy students? What is the impact of TBL on metacognition in pharmacy students? What factors impact accuracy of predictions (performance group, hours spent studying, etc)?

The pre-MAI composite score was 77.3% and improved significantly by the end of the course to 84.6% (p<.001). The class significantly improved in all four categories for the knowledge of cognition by the end of the course; however, planning was the only category in which there was a significant improvement for the regulation of cognition (Table 1). There were significant differences in both declarative knowledge and conditional knowledge when evaluating performance groups. For declarative knowledge, there was a difference of 1.3±0.4 between top and bottom performers and 1.1±0.4 between top and middle performers (both p values less than .01). Conditional knowledge had a difference of 0.5±0.2 between top and bottom performers and 0.4±0.2 between top and middle performers, with the p values of .03 and .04, respectively.

The mean pretest global bias for the class was -12±20 (indicating under confidence), and the mean posttest global bias was -11±12 (also indicating under confidence). Absolute global bias was 20±12 for the pretest and 12±11 for the posttest. Concept level bias was -9±24 (underconfident) on the pretest and -7±19 on the posttest (underconfident). Absolute concept level bias was 22±13 for the pretest and 13±15 for the posttest. Relative accuracy for each of the assessments, calculated using the coefficient of determination (R²), are shown in Table 2 (increasing R² values indicated that the accuracy of predicted performance increased with each TBL quiz). The R² also increased from the pre-examination to the final examination, though the R² value was not significant for the pre-examination. Self-assessment of knowledge only correlated to actual performance on the fourth and fifth TBL sessions (R² 0.1 and 0.04, respectively; p<.05).

Students in the middle performance group demonstrated the greatest ability to predict their performance on the final examination (R² 0.1, p<.02). Though not statistically significant, students in the low performance group overestimated their performance, while students in the high-performance group underestimated their performance (Figure 1). There was no correlation between students’ reported hours of preparation and actual performance, but it did correlate with students’ predicted performances on all TBL assessments. Pearson correlations were 0.04, 0.08, 0.16, 0.025, and 0.13, respectively (p<.05). The MAI was not predictive of the final examination score or the final course grade. Baseline GPA was the only factor predictive of the final examination score and the final course grade (R² 0.5 and 0.7, respectively; p<.001).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Mean Performance and Student Predicted Performance on the Final Examination by Performance Groups

DISCUSSION

The major components of metacognition include knowledge and beliefs about cognition (awareness of one’s strengths and weaknesses), monitoring of cognition (ability to monitor performance), and regulation of cognition (ability to implement strategies to increase understanding and performance).⁷ Our study demonstrates that students can improve their ability to predict performance (confidence) with practice. However, in this study, on average, the students’ regulation of cognition did not improve significantly with practice. This may be improved with intentional guidance through an instructor-provided plan of study, but further studies are needed to better understand its effectiveness on metacognitive regulation. We noted improvement in the pharmacy students’ overall composite metacognition and knowledge of cognition over a semester. This improvement was not seen in regulation of cognition, which seems to be consistent with most studies evaluating the major components of metacognition.

Karpicke and colleagues evaluated study strategies in undergraduate students to determine their metacognition. The students were proficient in their knowledge about cognition but were unable to regulate and monitor their own learning.¹⁶ In another study, researchers evaluated pharmacy students’ ability to predict examination performance and identify individual incorrect items. The students accurately predicted their performance but had limited success in identifying which items they missed.¹⁷ Persky and Dinsmore demonstrated that student’s metacognitive judgements do not correlate with improved regulation of cognition, reinforcing the limitation in students’ abilities to regulate their learning.¹⁸ These findings suggest that developing methods to enhance learning (regulation) is not an innate skill and requires intentional interventions within the curriculum.

Team-based learning has been proposed as a successful method to improve regulation of cognition. Richmond suggested that the immediate feedback students receive during TBL with a TRAT may increase metacognitive awareness, allow students to calibrate their cognition in real time, and increase regulation of cognition.^19,20 Several studies have also emphasized the value of immediate feedback for monitoring of learning.^21,22 This study demonstrated that TBL had an impact on prediction performance over the semester, which could suggest that students were evaluating their learning. Our findings support that students predict their performance better over time with TBL. In contrast, regulation of cognition did not improve in our study, with the exception of planning. Planning is the initial step in regulation of cognition and is heavily supported by the TBL structure of completing pre-work to prepare for assessments. Managing information, comprehension, and debugging strategies are higher-level skills and require a greater level of self-regulation and may require more guidance.²³

Although student predictions improved with practice, not all students were equally proficient at predicting their performance. In fact, the students with the lowest performance expressed overconfidence, while those with the highest performance expressed under confidence. Though this may sound like a paradoxical finding, this concept is well elucidated in the literature. Kruger and Dunning found that participants with performance in the bottom quartile had overestimated their performance because of a lack of metacognitive skills. When participants improved their metacognitive skills, they were no longer overconfident and were able to recognize previous errors. Similar to our study, these investigators also found an underestimation of performance by high performers.⁵ Previous research also demonstrates that performance is closely related to high-quality, deliberate practice.²⁴ Such practice requires regulation of cognitive tasks, which could explain why the amount of time that students reported they spent studying did not correlate with improved performance. While students who spent more time studying predicted they would do better on assessments, the lack of higher-level metacognitive skills found in this study further supports the need for deliberate instructional methods to further develop regulation of cognition.

There were several factors that could be confounders in our study, suggesting that differences were not just a result of the TBL and metacognitive activities. First, the volume of pre-work and difficulty of content differed from one TBL session to the next. Upon further analysis, a pyramid learning model was applied for the TBL sessions because some of the previous material served as a foundation for newer topics. Pyramidal sequential processes can either enhance or break down performances depending on students’ foundational knowledge. Next, when students were asked to report their preparation, they were not provided with a clear definition of what constituted “time spent studying.” As such, some students reported studying over 20 hours per day, which is unlikely. Finally, there was missing data for one of the quizzes and the pre-MAI did not provide student-specific data. Although the missing quiz data does not deter from the overall improvement in predictions, the inability to compare pre- and post-MAI data did prevent further analysis of factors that may improve MAI scores over the curriculum.

Despite these limitations, valuable information was gained from this study. Metacognitive skills are emphasized as a critical part of pharmacy curriculum. Accreditation standards promote self-awareness, self-directed lifelong learning, and continuous professional development (standard 4 and 9 of ACPE’s Standards 2016).¹ These are key factors that are essential to the metacognitive process and help students regulate their activities over time to reach desired goals. As such, faculty need to be diligent in enhancing learners’ metacognition in the classroom by aligning metacognitive processes with cognitive processes. Previous researchers have recommended activities that can improve regulation of cognition, including self-questions to promote monitoring and evaluation of cognition, thinking out loud, examination assessments, and written feedback.^2,25 Regulation may also be improved within TBL sessions, first by asking students to predict their performance with immediate feedback. This allows students to more accurately assess their preparation and knowledge of cognition. Additionally, by creating intentional opportunities to improve cooperative learning during TRAT and in-class activities, students can observe the judgements of team members or faculty experts and revise their own thinking. These strategies may be particularly helpful for low-performing students.

CONCLUSION

Developing metacognition skills in student pharmacists promotes their life-long learning skills in a profession that requires these skills for continuing professional competence. In this study, student pharmacists demonstrated improvement in knowledge of cognition but lacked changes in regulation of cognition. Pedagogies such as TBL can support development of these skills. However, deliberate and intentional guidance is required to improve regulation of cognition skills.