A Multiyear Analysis of Team-Based Learning in a Pharmacotherapeutics Course

INTRODUCTION

Transformation of health care delivery models so that they improve quality and reduce the cost of patient-centered care will require that pharmacists take new approaches to care coordination, team-based care, and chronic disease management, and that they assume new health delivery roles.^1-4 These new collaborative models require colleges and schools of pharmacy to ensure that students are prepared to problem-solve effectively as valued members of a patient-centered care team. The Accreditation Council for Pharmaceutical Education (ACPE) Standards 2.0 emphasize that pharmacy programs graduate student pharmacists who “…can contribute to the care of patients and to the profession by practicing with competence and confidence in collaboration with other health care providers.”⁵ Team-based learning in the pharmacy curriculum has been described as a pedagogy that can address this type of professional preparation.^6-16

Team-based learning is a specific instructional strategy and framework that uses intentionally formed teams of learners to deepen student learning and develop high-performing teams. The core principles of TBL are: (1) learning groups must be formed and managed; (2) students must be held accountable for the quality of both their individual work and the group’s work; (3) students must receive frequent and timely feedback on their learning; and (4) team assignments must promote both learning of the content and development of the team.¹⁷

Team-based learning comprises 3 major steps: (1) individual study and preparation; (2) readiness assurance, completion of an individual readiness assessment test (iRAT) in class, followed by the same test as a group, and completion of the group readiness assessment test (gRAT); and (3) application exercises in class where teams work together to solve problems using information gained from the previous 2 steps. Each student evaluates the other members of the group through graded peer evaluations during the semester.

The TBL model has been used extensively in business in the 1970s and, more recently, in health sciences education, with improved learning outcomes in colleges and schools of pharmacy and other health professions.^18-28 Reports of TBL in pharmacy education often use student and faculty perception or performance to examine the initial implementation of TBL. However, no reports have used nationally standardized faculty evaluations of teaching using TBL, examined experiences over a longitudinal implementation of TBL, or reported course outcomes in a pharmacotherapeutics course series that used TBL. More data are needed for faculty members to understand the longitudinal impact of implementing TBL in a required team-taught course.

Medical educators have increasingly advocated for active-learning strategies that allow students to apply problem-solving skills and learn collaboratively. However, student performance and attitudes towards these methods are often mixed, and comparisons are not possible because of the wide variation in teaching models used.^{20,21,24,26,29}

The Drake University College of Pharmacy and Health Sciences (DUCPHS) practice faculty members adopted TBL in 2009 during a college-wide curricular revision in 2008 and in tandem with the release of ACPE Standards 2007. This study describes a multi-year experience with TBL in a required pharmacotherapeutics course series at DUCPHS. The primary objective of this study was to evaluate the impact of implementing TBL on faculty and course evaluations, and the secondary objective was to measure student performance before and after curriculum revision.

We hypothesized that student evaluations of faculty members and the pharmacotherapeutics course series would remain relatively stable and measures of student engagement with learning would increase with TBL. The authors also hypothesized that students’ course performance would not be negatively impacted and that team grades (gRATs) would outperform individual grades (iRATs).

DESIGN

This project was approved as exempt by the Drake University Institutional Review Board. At DUCPHS, the previous model for classroom instruction of pharmacotherapeutics was the live lecture, which focused on delivering content to the entire class, followed in the same week by smaller case-based discussion group sessions. However, faculty members were not satisfied with the students’ engagement with the material, preclass preparation, or postlecture retention of concepts. These assessments were made informally and consistently between faculty members over the years through direct observation of student participation in class, in discussions during annual therapeutics retreats, and during direct interaction with students during their practice experiences. Workload for faculty members teaching in the course was demanding, requiring both a content expert for the lecture component and 3 additional faculty facilitators to conduct recitation groups each week. A comprehensive revision of the professional curriculum at DUCPHS to meet ACPE Standards 2007 offered an opportunity to explore a new pedagogy that would enhance student learning and strengthen development of high-performing teams. Concurrently, delivery of the course was changed from a 2-semester fall-spring sequence for third-year (P3) students to a 3-semester sequence beginning in the spring for second-year (P2) students and fall and spring for P3 students; this sequence more evenly distributed course credit hours (Figure 1). Course faculty members explored new teaching methods and developed a set of criteria that any new method used would have to meet: increase student accountability for learning; increase active learning; develop life-long learning skills; increase retention of knowledge; increase contact with individual faculty members; and decrease the number of faculty members involved in each semester by creating 1-credit hour blocks (modules) for which each faculty member was responsible.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Pharmacotherapeutics course structure before and during team-based learning

The Michaelsen model for TBL was selected as an active learning strategy because it best fit the needs of the course in preparing students for professional roles, emphasized engaged learning and accountability for learning, and reinforced the value of strong team performance on learning and problem solving.¹⁷ Another active learning strategy considered was problem-based learning (PBL), which was first used in medical education in the 1960s. This instructional strategy introduces a problem to the students at the beginning of an instructional module and then requires students to self-discover answers progressively through repeated interaction with a faculty member. The DUCPHS practice faculty members decided that PBL would be time-intensive and require more time teaching outside of class, adding to the challenges of work-life balance and faculty member success in scholarship and service. The effects of PBL on learning outcomes were equivocal.²⁹ Previous experience with a lecture plus case-based discussion did not always meet faculty expectations for learning outcomes and was demanding of faculty teaching time. Based on the extensive literature on TBL, efficiency of course delivery, and potential for improved outcomes of both student learning and student attitudes towards learning, the Michaelsen model for TBL was selected. At the time, neither the students nor the DUCPHS faculty had any previous experience with TBL in the curriculum, though various active learning strategies such as recitation groups and case-based discussion had been used. The change in course pedagogy was approved by DUCPHS faculty members in 2008 and preparation began for delivery of the revised course in spring 2009.

All faculty members at DUCPHS attended a workshop 1 year prior to implementation; however, only the pharmacy practice faculty members decided to fully adopt this model in its entirety for the course. Pharmacotherapeutics faculty members held additional self-directed, hands-on practice sessions prior to implementing this pedagogy. A number of practice faculty members joined the TBL Collaborative to access additional resources and for ongoing support from TBL experts around the country.

The TBL course was taught live as a 3-semester sequence to the P2 and P3 classes of doctor of pharmacy (PharmD) degree students on the university campus. The number of hours spent in class reflected the course credit hours: multiple modules equal to 1-credit hour were taught each semester, and only one faculty expert taught a module in one semester of the course (Figure 1). The first TBL offering was held in a traditional-style classroom located within the college that was not conducive to group work, so this room was subsequently updated to better support engaged active learning. Twelve pharmacy practice faculty members taught in the course series. All of the faculty members maintained a clinical practice in an area of expertise, had completed postgraduate residency training, and had at least 1 advanced credential (eg, Board Certified Pharmacotherapy Specialist).

EVALUATION AND ASSESSMENT

Student ratings of the course and faculty members were used to support the main hypothesis for this study. A Student Ratings of Instruction system, created by the nonprofit IDEA Center, which focused on student learning of 12 different objectives and factored out extraneous circumstances, was used to assess the course. The system also enabled students to rate faculty instruction on 20 different items.³⁰ Students completed these evaluations at the end of each faculty member’s section during both course iterations.

The IDEA Center system has been used by DUCPHS since 2004 for course and faculty evaluations. These evaluations are completed either in class on paper or outside of class online. Because of this history, both faculty members and students were comfortable with the system during the time TBL was introduced. Overall, the faculty evaluation process remained the same both before and after TBL implementation. Using the IDEA Center system, course coordinators selected the objectives believed integral to the course. Prior to TBL implementation, faculty teaching in pharmacotherapeutics selected “gaining factual knowledge,” “learning fundamental principles,” “learning to apply course material,” and “learning to analyze and critically evaluate ideas, arguments, and points of view” as key objectives for the course. For the TBL sequence, key objectives were “gaining factual knowledge,” “learning to apply course material,” and “acquiring skills in working with others as a member of a team.” Progress on these objectives was rated by the students on a 5-point Likert scale (1=no apparent progress, 2= slight progress, ie, “I made small gains on this objective,” 3=moderate progress, ie, “I made some gains on this objective,” 4=substantial progress, ie, “I made large gains on this objective,” 5=exceptional progress, ie, “I made outstanding gains on this objective.”

The IDEA course evaluation system was used for measures of the primary study outcomes. This system has been shown to be both valid and reliable for course-related outcomes and student progress. For evaluation, each cycle of the course sequence was treated as a discrete unit for comparison. This produced 6 cohorts of data (2 years of the course sequence prior to TBL and 4 years using TBL for comparison.

Student ratings for 5 specific approaches to instruction were compared to support the assertion that fundamental teaching changes took place in the course sequence (Table 1). Pre-TBL and TBL courses were compared using Student t tests for these 5 independent activities. In addition, course assignments (reading and nonreading assignments) and student ratings for the difficulty of subject matter were compared across the pre-TBL and TBL courses using student t tests in an attempt to further support changes in format rather than subject matter.

Student progress was analyzed using the 5 IDEA course objectives identified by the faculty as being the objectives they would focus on during the course sequence. Because these course objectives were highly correlated and expected to be simultaneously affected by TBL, multivariate analysis of variance (MANOVA) was used to evaluate differences across the 6 study years. Although these ratings consisted of ordinal data, graphical representation showed adequate dispersion across the scale to warrant this parametric approach. When analyzing independent constructs across the 6 years (such as excellent course ratings and excellent teacher ratings), and when comparing a total score across the 5 objectives, analysis of variance (ANOVA) was used with conservative Scheffe multiple comparison testing. SPSS (IBM Corp. Released 2012. IBM SPSS Statistics for Macintosh, Version 21.0. Armonk, NY: IBM Corp.) was used for all analyses with an a priori alpha of 0.05.

Two student focus groups were conducted at the end of the spring semester in 2009 and 2010. The focus groups were conducted by an individual trained in qualitative focus group analysis who met with 6 P2 students enrolled in each spring semester. In response to themes extracted from these focus groups, purposeful action was taken to improve clarity of expectations, to improve connections between objectives and study questions, and to enhance details on module objectives. Student organizations subsequently provided leadership by presenting mock TBL sessions to inform students on the process and benefits. Minimal changes were made in the amount of reading and preparation required for the course.

Peer evaluations were conducted each semester, but reduced from 3 to 2, ensuring that students continued to receive valuable feedback while minimizing time burden. Manual tabulation was used initially. iPeer, version 3.08 (open-source software, University of British Columbia), was implemented in 2012 to streamline the process.³¹

Student performance during TBL was assessed by comparing gRAT with iRAT performance. On average, 3 iRATs or gRATs were given in each module. Eighty percent of the final course grades in TBL reflected individual performance, 15% reflected team performance, and 5% reflected peer evaluations. In the course pre-TBL, 80% of the grade was based on individual performance on examinations, and 20% on recitation and peer evaluations. Final course grades in the 2 years pre-TBL were compared to those in the 4 years TBL was used.

Across the 6 years the course sequence was evaluated, the mean (SD) class size was 119±11.8 (range 102-135) students. The IDEA Center student evaluation response rates were reliable and averaged 55.6% (range 18-99%) with higher in-class paper response rates. Students’ perceptions of course format suggested that the approaches to teaching the courses did, in fact, change after TBL was implemented. Students in the TBL course noted significantly greater emphasis on 5 preselected teaching approaches that were consistent with TBL (Table 1). Students also noted that the amount of reading increased in the TBL years while the amount of other nonreading assignments decreased. Overall, the students did not see a difference in the difficulty of the subject matter from pre-TBL to TBL implementation in the course (Table 2).

When viewing the cohort of instructors across the 6 years, there was no significant difference in aggregate scores for the item, “Overall, I rate this instructor an excellent teacher” (F=1.988, p=0.09). Scores for the 6 cohorts ranged from 3.6 to 4.3 on a 5-point scale (1=definitely false, 2=more false than true, 3=in between, 4=more true than false, 5=definitely true). When viewing the entire course, 1 year did show a significant decrease in student ratings for, “Overall, I rate this course as excellent” (F=8.93, p=<0.001). A Scheffe multiple comparison test for the statement, “Overall, I rate this course as excellent” found a mean of 4.1 in both pre-TBL years 1 and 2, and means of 3.5, 4.0, 4.4, and 4.3 in years 3 to 6, respectively, of TBL implementation. While the third year was significantly different from all other years, the fifth and sixth years demonstrated progressive improvement in responses compared to pre-TBL or the first 2 years of TBL implementation.

The MANOVA did reveal significant differences between various years and student progress on IDEA course objectives (Table 3). For gaining factual knowledge and learning to apply course material, year 3 had the lowest progress ratings and was significantly lower than years 5 and 6. Both of these objectives were identified as key objectives in both pre-TBL and TBL implementation years. Similarly, the learning fundamental principles objective varied among class years. Again, year 3 (the first year of TBL) received the lowest progress ratings and was statistically lower than years 5 and 6. The first pre-TBL year also received significantly lower progress ratings than year 6.

The IDEA objective of acquiring skills in working with others as a member of a team was used as a course focus for only the TBL years. As expected, this objective showed different progress scores from pre-TBL to TBL years. The lowest score on this objective was in year 1, followed by year 2. The pre-TBL years showed significantly lower progress than all 4 TBL years. In addition, the first year of TBL implementation (year 3) also showed lower progress as compared to the last year of TBL (year 6). Finally, the pre-TBL courses focused on the IDEA objective of learning to analyze and critically evaluate ideas, arguments, and points of view. In a comparison of the 6 years, year 6 had the highest progress rating. This was significantly higher than all but year 4. When analyzing the overall measure of progress on the 5 objectives (total score on the 5), ANOVA showed significant progress in the last 3 years using TBL. Study years 4, 5, and 6 showed higher total scores than year 1, while year 6 was also significantly higher than years 2 and 3 (Table 3).

As postulated in the TBL model, and as the authors hypothesized, students scored consistently better on their gRATs in comparison to their iRATs each year the course was offered (Table 4). Overall, final course grades pre-TBL and during TBL were similar, though there was a higher proportion (5.3%) of D grades in the first course in TBL year 1 (Table 5).

DISCUSSION

The study findings supported our main hypothesis that student evaluations would remain relatively stable and measures of student engagement with learning would increase with TBL. Subjective IDEA Center student evaluations of faculty members and the course did not change drastically and improved in some areas when comparing pre-TBL to TBL evaluations over multiple years. An overall rating of the 5 course objectives showed improved student progress and progressive improvements over time. Ratings for faculty members and the course as “excellent teacher” and “excellent course” were similar, with the exception of a decrease in the first year of TBL implementation. Because students had no previous exposure to TBL before this course, unfamiliarity with this pedagogy likely contributed to lower ratings the first year. Additionally, all faculty members were new to TBL, which also may have contributed to the first TBL year ratings. Over time, increased practice in the pedagogy and constructive responses to student feedback resulted in an improved course.

These findings were consistent with studies that showed similar or higher student ratings over time when comparing TBL to other learning methods.^8,11,14,24 In support of our secondary hypothesis, this study showed improvement in team performance over individual performance and comparable course performance, which was also consistent with previous studies.^7,10,13,14 Lower-performing students benefited more from TBL, or had observed no differences in performance between case-based learning and TBL.^22,25 The proportion of D and F grades was low in both iterations of this course, though a higher proportion of D grades occurred in the first year of TBL. Perhaps these students struggled more in adapting to TBL. In subsequent TBL years, the proportion of low grades was comparable to pre-TBL; however, the small numbers of students in this category preclude definite conclusions.

Although other studies have examined multi-year experiences, to our knowledge this study was the first to focus on IDEA Center teaching evaluations over time. In addition, we used a “pure” TBL model instead of using select components of TBL as many published hybrid models have done.

Most of the pharmacotherapeutics course faculty members were “seasoned veterans,” who had taught the course for more than 5 years and had previously used the more traditional model of lecture and case discussion. These factors likely contributed to faculty members’ ability to successfully adapt to the TBL model. Involvement of faculty champions played a key role in faculty and administrative buy-in, course development, and implementation. Adequate faculty preparation and practice sessions supported successful course delivery.

Teaching circles of all faculty teaching in the course per semester were formed and met regularly to discuss ideas, share success stories, and brainstorm on strategies. An annual faculty retreat allowed course faculty members to review course statistics, ensure coordination of topics, and establish common policies and procedures between semesters. Summary themes from 2 formal focus group sessions were shared and discussed at the retreats during the initial years of TBL. College administration supported the course innovation and provided reassurance that course evaluation scores in the initial years would be evaluated within the context of change. Academic pharmacy support for TBL implementation can also be found in the most recent Center for the Advancement of Pharmaceutical Education outcomes, with specific emphasis on problem solving and communication.³²

This change in educational methodology was initially time intensive, requiring content revision, double teaching for 1 year, and conversion from a 2- to 3-semester model. Students may have felt they were in a constant state of “firsts” and may have had difficulty keeping their minds open to the educational value of change. The advance preparation was a shift in student thinking. Students complained that they did not have as much time to study for other courses. Faculty members from other courses remarked that they felt TBL courses pushed students to study less for their courses and this concern is one of the reasons DUCPHS faculty members have not adopted TBL across the curriculum. Student emotions ran high the first year in particular, and written student comments on course evaluations were extensive and mainly negative. These negative perceptions motivated course faculty members to spend more time orienting students to the rationale and mechanics of TBL during the course and to provide more extensive guidance in the course syllabus, but also in other areas, such as by working with student organizations in the semester prior to TBL.

The first time the TBL class was offered, an existing traditional-style classroom did not easily support group work and could have contributed to some of the negative feedback on TBL initially. Although major physical improvements were subsequently made to the classroom to accommodate TBL, further improvements are needed to optimize team interaction. Past use of detailed PowerPoint slides had to be avoided with the TBL format. For faculty experts with previous experience using extensive slide sets in lecture, reduction to 1 to 3 slides per application exercise question was challenging. Course faculty members have agreed to follow mutually established guidelines going forward, with the understanding that flexibility will ensure students receive the necessary training for success upon graduation.

CONCLUSION

Faculty members of a team-taught pharmacotherapeutics course successfully transitioned to a TBL pedagogy that increased student accountability, reinforced the value of teamwork, enhanced professional development, and optimized faculty workload. Students’ evaluations of the course using the IDEA Center’s rating system showed improved student progress with TBL, while aggregate teacher ratings remained stable. Team performance on gRATs exceeded individual performance on iRATs, and overall course grades remained stable with TBL. Changes in faculty member approaches to instruction with TBL were viewed positively by students and enhanced learning in the course.