Team-Based Learning to Improve Learning Outcomes in a Therapeutics Course Sequence

INTRODUCTION

Beginning in fall 2010, the University of Michigan College of Pharmacy implemented an extensive revision of the doctor of pharmacy (PharmD) curriculum. Curricular transformation was motivated by the desire to enhance students’ critical thinking and problem-solving skills, and to improve long-term learning outcomes by employing more active learning within the classroom setting. Congruent with these overarching goals, faculty members in the pharmacotherapeutic courses chose to use a single active-learning pedagogy across the entire 5-semester course sequence. A faculty subgroup was charged with evaluating different strategies to achieve the college’s overall goal.

This subgroup chose team-based learning (TBL) as a unifying pedagogy because of published reports of improved short-term outcomes in health professions curricula at other institutions, emphasis on self-directed learning, and application of materials instead of rote memorization. Additionally, TBL is a practical pedagogy that can be applied to either a small or large classroom setting (ie, class sizes of 80-100+ students could be taught in a single section).^1,2 TBL “flips the classroom” so that students are responsible for initial exposure to content through guided self-study.

Previous studies have shown that TBL is at least as effective as a lecture-based approach. In a study performed at the Medical University of Vienna, 220 medical students completed first-year course work following a TBL format.³ TBL-taught students had a 25.3% higher score on 230 multiple-choice test questions than students taught with traditional lectures. In another study comparing TBL to lecture-based learning, medical students had similar pre- and post-test scores on 10 questions regarding family planning.⁴ Higher scores were reported in the TBL group with respect to improved problem-solving skills. Another study evaluated the effectiveness of TBL in teaching pharmacology and demonstrated that third-year medical students performed as well as fourth-year students according to an evaluation of 10 multiple-choice questions.⁵ In a study comparing TBL to mixed active learning in a pharmacy ambulatory care elective course, the mean grades of students were similar for the 2 methods.⁶ Overall, the literature in this area is limited but tends to document that TBL methods are at least as effective as traditional lecture-based teaching in assessments of short-term outcomes.⁷

Another major change to the pharmacotherapeutic course sequence was the intentional resequencing of topics taught, such that instruction on the treatment of common chronic conditions changed from the third year (P3) of pharmacy school to the second (P2) year of pharmacy school. Thus, restructuring of the curriculum was completed at the same time as the changes in teaching pedagogy. This provided us with a unique opportunity to evaluate learning outcomes based on 2 teaching pedagogies (traditional lecture and TBL) because, during the transition year, identical content was taught to P3 and P2 students but different teaching methods were used. We hypothesized that the P2 students (taught through TBL) would do as well as the P3 students (taught by lecture) on examination questions that assessed application of knowledge despite having completed 1 year less of experiential and pharmacy course work.

DESIGN

In fall 2010, the college introduced a new PharmD curriculum for students enrolled in their first year of pharmacy school. In the new curriculum, content in the pharmacotherapeutics courses was rearranged so that chronic disease topics were taught earlier in the sequence. This reorganization allowed better alignment with other coursework (eg, medicinal chemistry, pharmacology) as well as with activities in introductory pharmacy practice experiences (IPPEs). The following topics were taught to P2 students in the revised curriculum using TBL pedagogy in fall 2011 (the P2’s first experience with TBL occurred in the winter term of the P1 year) and to P3 students within the former curriculum using traditional lecture or punctuated lecture pedagogy in winter 2012: hyperlipidemia (LPD), hypertension (HTN), heart failure (HF), and coronary artery disease (CAD). Management of substance abuse (SA) was taught to both groups in the winter term of the P2 year.

In the former curriculum, pharmacotherapeutic material was delivered in a traditional lecture or punctuated lecture format. Content was presented by course faculty members to the full class of 80 to 100 students in a lecture hall, often using slides or other visual aids. The 5 faculty members had used a predominantly lecture-based pedagogy for an average of 17 years (range, 3 - 25 years). Some active-learning strategies were incorporated, most frequently through large-class discussion of patient cases. Active-learning techniques employed less often included methods such as “brain dumping” and “think-pair-share.” However, in general, delivery of content was essentially passive in terms of student engagement.

In the revised curriculum, all pharmacotherapeutic faculty members used TBL. As part of this process, students were required to complete a substantial amount of guided self-study before class. In class, a readiness assurance process conducted at the beginning of the class period ensured all students had acquired the requisite knowledge. This process involved use of an individual readiness assessment consisting of 5-10 multiple-choice questions and then a team-based readiness assessment using the same questions. The use of a scratch-off card provided immediate feedback to students regarding right or wrong answers. After these assessments were completed, a short question-and-answer period was conducted for any questions students were unclear about regarding the readiness assessment or self-guided readings. The remainder of class time was dedicated to students working in small groups on activities that required synthesis and application of new knowledge based on the pre-work they completed for that day’s instruction. Faculty members used the basic “4S” approach to TBL during the recitation period: significant problem; same problem; specific choice; simultaneous report. In most instances a “significant problem” involved a complex clinical case. Course faculty members were content experts who designed team-based activities, provided corrective instruction, and facilitated inter-group discussion of application exercises.

In addition to TBL sessions, this new therapeutics course was structured to include a laboratory session 1 day a week. The goal of these sessions was not to introduce any new content but to allow the students to once again apply the knowledge they gained and reinforce key concepts learned in the TBL sessions. These laboratory sessions also allowed for the integration of key curricular threads (ie, cultural competency, communication, evidence-based medicine, etc) with the current content being taught. Laboratory activities included standardized patient instructor interactions, case presentations, and skill assessments (eg, blood pressure monitoring).

In both the new and former curriculum, examinations were administered at different times throughout the semester. All examinations were collected at the end of the examination period and graded examinations were not returned to students. The faculty member who taught the P2 and P3 curriculum for the course also wrote the examination questions used in both offerings of the course.

EVALUATION AND ASSESSMENT

The investigators conducted a retrospective evaluation of examination outcomes for the 2 years. Specifically, those examination questions administered to both P2 and P3 students were identified. These examination questions were administered to P2 students who completed the revised curriculum in semester 3 (except for substance abuse which was taught in semester 4) and to the P3 students who completed the former curriculum in semester 6 (except for substance abuse which was taught in semester 4). Three faculty members from the college’s Department of Clinical, Social and Administrative Sciences who were not content experts evaluated each multiple-choice test question to determine if it assessed recall or application of knowledge. These faculty members were not blinded to the purpose of the study. Recall of knowledge was defined as simple recollection of factual, objective information, while application of knowledge was defined as requiring synthesis of factual data. Most commonly, application questions incorporated patient cases or brief scenarios to evaluate pharmacotherapeutic decision-making skills. For a question to be categorized, at least 2 of the 3 faculty members had to agree on the question subtype. In addition, a 3-part free-form or essay-type question that tested students’ knowledge of heart failure was also evaluated.

Rates of correct responses for each multiple-choice question were compared between P2 and P3 classes using chi-square analysis. Overall mean percentage of correct responses to questions was compared by unpaired Student t test. For essay questions, individual scores were compared between the 2 groups using an unpaired t test. For all tests, P<0.05 was considered significant and data were reported as means and standard deviations.

All 97 P2 students enrolled in the revised curriculum and 85 P3 students in the former curriculum completed the examinations that were compared. Upon admission to the college, the average grade point average (GPA) and Pharmacy College Admission Test (PCAT) score for the P2 class was 3.4 and 79, respectively; and for the P3 class, 3.3 and 82, respectively.

One hundred thirty-one multiple-choice questions were evaluated. Of these, 79 were judged to have tested recall of knowledge in the areas of substance abuse (n=32), coronary artery disease (n=17), hypertension (n=14), heart failure (n=9), and hyperlipidemia (n=7). Application of knowledge was assessed by 52 test questions as follows: hyperlipidemia (n=20), coronary artery disease (n=11), heart failure (n=8), substance abuse (n=7), and hypertension (n=6). Class time allocated to cover these topics was higher for the P3 students. Total contact time was 21 hours for P2 students in the revised curriculum compared with 27 hours for P3 students in the former curriculum.

When evaluating overall examination scores by question subtype, P3 students had significantly higher mean scores on recall questions (Table 1). There was no significant difference in the mean scores of P3 and P2 students in application questions. Further, P3 students’ mean score on the essay question was 5% higher than that of P2 students; however, a higher number of P2 students scored a 100% on all 3 parts of this question.

In addition to evaluating relative performance of the student groups on overall mean scores, an in-depth analysis of individual examination questions was undertaken in order to account for variability in execution of teaching pedagogy. In terms of individual examination questions, P3 students scored higher overall on 29 of 52 application questions, although this difference did not achieve significance (p=0.33) when compared to scores of P2 students. However, P3 students scored higher overall on 48 of the 79 questions that evaluated recall of information (p=0.01). When the data were grouped by therapeutic topic, P2 students performed as well as their P3 counterparts with regard to the number of questions in which the P2 class had a higher mean score for application questions (Table 2). P2 students outperformed P3 students in the heart failure and substance abuse sections and performed equally well in the hypertension section. With respect to questions assessing recall of knowledge, P3 students outperformed P2 students in 3 of the 5 therapeutic areas (Table 3).

DISCUSSION

Revision of the college’s curriculum resulted in several pharmacotherapeutic topics being taught in the same academic year using 2 different teaching pedagogies: TBL for P2 students in the revised curriculum and lecture-based teaching to P3 students in the former curriculum. These unique circumstances allowed direct comparison of the short-term learning outcomes of these 2 pedagogies, as assessed by multiple-choice and essay questions. This study, to our knowledge, is one of the most direct comparisons of learning outcomes between TBL and lecture-based pedagogies. Although P3 students performed better than P2 students overall and significantly better on questions measuring recall, this difference was not apparent when questions assessing higher level and application of knowledge were compared. These results support that active learning, specifically TBL, is an effective teaching strategy, yielding similar results on evaluations of short-term learning of application skills compared to more traditional, lecture-based teaching. Although we are encouraged by the results of our TBL curricular transformation, there are several curriculum-based factors that could have biased our results. We acknowledge that the P3 students in the former curriculum had more pharmacy education than the students in the revised curriculum (5 vs 2 semesters completed, respectively). Notably, students in the former curriculum had completed the entire medicinal chemistry and pharmacology course sequence, and had 3 prior pharmacotherapeutic courses (for 4/5 therapeutic areas); students in the revised curriculum had completed only 1 semester of medicinal chemistry/pharmacology and 1 semester of self-care therapeutics. Students in the former curriculum also had more IPPEs (300 vs 75 IPPE contact hours, respectively), and these experiences included more advanced clinical activities such as patient counseling and medication reconciliation. In contrast, early IPPE activities for students in the revised curriculum involved less clinical activities such as shadowing, interviewing, and service-learning activities. All of these factors, primarily greater repetition of medication knowledge (ie, having completed the entire medicinal chemistry and pharmacology sequence and having had greater exposure to medication prescribing because of the completing their IPPEs), may have allowed the P3 students to perform better than the P2 students on the recall-type questions.

Another major factor that could have biased results was teaching proficiency. The faculty members teaching in the former curriculum using lecture-based pedagogy had an average of over 17 years of teaching experience. These faculty members had honed their units of instruction over several years and were able to write examination questions based on their experiences with previous classes. In contrast, these same faculty members were novices at using TBL. Even though multiple faculty development sessions were conducted over a 2-year period, the execution of TBL in the first year after implementation of the revised curriculum was not optimal. For example, based on formal student feedback performed by the University of Michigan Center for Research on Learning and Teaching (which is outside of the college), the students felt that the recommended guided self-study was too time consuming and that key points and concepts were not always clearly defined by the faculty member teaching the content. In addition, students reported that the assigned self-study readings were often too advanced for their stage in training and that the level of these readings was more appropriate for a clinician than a student. Based on feedback from the course evaluation, there was variability in how well faculty members adapted to teaching in a TBL format. The variability may in part be related to these student concerns and to outcomes (Tables 2 and 3).

Our results are consistent with previous studies, demonstrating that on written examinations testing application of knowledge, students taught using TBL can perform similarly to those taught using traditional lecture-based methods.^3,7 Our study goes further by discriminating between test items that assess simple recall and those that assess higher-level application of knowledge. While incorporation of active, TBL in the pharmacotherapeutics course sequence within the revised curriculum did not produce overall superior results in assessments of short-term learning compared to those achieved with the former curriculum, we plan to continue to refine our use of TBL pedagogy and to reassess learning outcomes. Learning outcomes may improve over time as faculty members become more adept at teaching TBL and continue to hone their educational units. In addition, TBL may produce higher orders of learning, improved critical thinking and problem-solving skills, or more persistent learning not measured by this study. In order to determine if this is the case, additional investigation is needed. For instance, improvements in higher orders and persistence of learning may be noted in advanced pharmacy practice experiences in the fourth year of the curriculum or following graduation. In addition, persistence of learning might be evident on summative assessments, such as capstone or clinical examinations administered later in the pharmacy curriculum.

There are a number of limitations to our study. Having a control group of students enrolled in the same academic year would have allowed more direct comparisons and controlled for the effects of differences in other aspects of the former and revised curricula, both classroom based and experiential. However, such a study design was not practical. In addition, curricular revision pertaining to other non-pharmacotherapeutic coursework could have confounded results and was not controlled. However, at the time the examinations used in this study were administered, the P2 students were enrolled in the fall semester (except for the substance abuse section, which was taught in the winter semester) and the P3 students were for the most part enrolled in the winter semester, somewhat minimizing any influence from the new curriculum. Another consideration is the number of test questions evaluated. Although, to date, this is one of the largest comparisons evaluating learning outcomes between TBL and lecture-based pedagogies, there is a concern for type 2 error. With additional test questions, a significant difference favoring the performance of P3 students might have been detected in regard to application questions. We also did not perform in-depth item analysis for each test question, which may have provided additional insight into our findings. Finally, comparing curricula that had been delivered for a similar amount of time, rather than comparing highly honed lecture-based courses to newly developed TBL courses might have yielded different findings.

SUMMARY

This study demonstrated that P2 students who were taught using TBL methods performed at a similar level than more advanced P3 students taught by traditional lectures in terms of application of knowledge, whereas the P3 student group performed better on questions measuring recall of knowledge. In addition, the P3 students performed better overall when both recall and application of knowledge questions were combined. Overall, these data suggest that TBL allows students to apply therapeutic principles at an advanced level early in their degree program. Additional investigation into longer-term learning outcomes is needed to assess persistence of learning.