Qualitative Analysis of Student Perceptions Comparing Team-based Learning and Traditional Lecture in a Pharmacotherapeutics Course

INTRODUCTION

Health educators recognize that the traditional lecture-based teaching may not optimally achieve high order learning outcomes, such as applying scientific content to clinical scenarios.¹ In addition, the Accreditation Council for Pharmacy Education (ACPE) strongly endorses developing critical thinking and problem-solving skills through active learning that includes self-directed and collaborative learning.² Active learning pedagogies have been advocated and adopted in pharmacy education because they are more compatible with theories of adult learning and emphasize higher orders of learning that could translate into better performance at clinical environments.^3-5

In particular, team-based learning (TBL) is a beneficial active learning strategy because it is scalable and widely adopted in health professions education.^6-8 Studies examining use of TBL in pharmacotherapeutics courses (often compared to traditional lecture) typically assess short-term learning outcomes, and reveal TBL and other similar pedagogies to be similar or somewhat better than lecture.^9-15 In addition, qualitative analyses of “flipped classrooms” have revealed challenges and benefits that occur from the students’ perspective.^16,17 However, these studies employ methodologies that are not as robust as prospective head-to-head comparisons of TBL compared to traditional lecture.

In fall 2010, we implemented a new PharmD curriculum that included a 5-semester therapeutics problem-solving (TPS) sequence. We chose TBL as a unifying pedagogy to be used by all faculty in the TPS sequence. Several investigations were undertaken to measure our students’ outcomes to ensure our new courses were at least as effective as previous courses, which primarily used traditional lecture formats. The purpose of this study is to understand what students value about TBL (compared to traditional lecture) and how students’ experiences with TBL pedagogy changes how they approach and think about learning. These observations could also yield insights into the impact of TBL on other learning outcomes, such as professionalism or lifelong learning.

DESIGN

As part of a greater plan to determine if learning outcomes were impacted by our adoption of TBL, we designed an elective therapeutics problem-solving course to directly compare TBL to traditional lecture in a randomized, crossover design. Learning outcomes from course tests are reported elsewhere, and revealed somewhat better performance among students who learned course topics via TBL compared to lecture (mean test scores 89.2% v. 85.0%, p=0.03).¹⁸ Here, we report qualitative data relating to students’ perceptions about the impact of TBL (compared to lecture) on their learning.

The elective course was offered during the winter semester of 2015. Thirty students took the course (13 and 17 students in each section) and met weekly for two hours. The six faculty members teaching the course were experienced instructors in both lecture and TBL in our TPS sequence, and were each responsible for one topic. Students enrolled were second (P2) and third (P3) year Doctor of Pharmacy students who had two or four semesters of previous TBL experience in TPS. Each topic was delivered twice, one as a traditional lecture and one in TBL format by the same faculty member (Table 1). Assignment of pedagogy to each section was done in a crossover fashion so all students were exposed to lecture and TBL during the course. For topics taught with TBL, students were randomly assigned to teams by the course coordinator to ensure each group was diverse and equal to the other groups. For each TBL session, there were three teams of four to six members each; teams were not altered during the course.

Learning objectives for the course were similar to those in our TPS courses, and included acquiring knowledge related to pathophysiology and pharmacotherapy of disease; gathering information through medical record review or patient interview; interpreting clinical data; applying knowledge of pharmacotherapy to clinical scenarios to create a patient-specific care plan; communicating effectively with other health care professionals and patients; and participating effectively on teams, including the ability to act as a team leader.

The approach to TBL instruction was the same across all faculty. Students completed self-study materials provided by the instructor prior to class. These were designed to be completed in approximately 90 minutes, and could include readings with study guides, recorded lectures, or other materials. At the beginning of class, a readiness assurance test (RAT) was administered to individual students (iRAT), and then was taken by teams (tRAT). After the instructions were given, teams engaged in application problems (tAPP) by discussing cases within their teams and selecting the best solution from a list of viable options, simultaneously reporting their team answers, and participating in inter-team discussions. The readiness assurance process consumed 20-30 minutes of class time, leaving 80-90 minutes for tAPP.

The approach to lecture delivery included providing required or optional readings to be completed before class, and delivering a lecture summarizing pathophysiology and pharmacotherapy related to the topic. Inclusion of audio-visual aids, case examples, and “clinical pearls” was encouraged to be done by faculty.

EVALUATION AND ASSESSMENT

To understand the impact of TBL on student achievement of the course objectives, two exams were given during the semester. In addition, an anonymous post-course survey (using the Likert scale) was conducted to assess student attitudes regarding TBL and lecture and their confidence in mastering the materials. The findings from these assessments are reported elsewhere.¹⁸

To gain a fuller understanding of students’ experience with TBL and assess learning outcomes beyond those measured on the exams, we asked students to provide open-ended responses to questions on the post-course survey (Table 2). We also asked them to complete a written reflection on their experience with TBL by responding to the following prompt: “Describe the biggest challenge you have to adapting to TBL pedagogy and how this has affected the way you approach and think about learning.” The written responses were de-identified and sent to the researchers. Participation in the survey and in the written reflection exercise was voluntary. The study was approved by the institution’s Health Sciences and Behavioral Sciences review board.

All of the 30 students (100% response rate) enrolled in the course completed the post-course survey. Responses to the survey’s first six open-ended questions ranged in length from a word, phrase, or sentence to paragraphs consisting of two to seven sentences. Responses to the seventh open-ended question ranged from one to five sentences in length. Twenty-eight students (93% response rate) completed the written reflection and their reflections ranged from one to six paragraphs in length.

Responses to the open-ended questions and written reflection prompt were analyzed qualitatively using an inductive, grounded theory-based approach in which codes and categories were derived from the data.¹⁹ The investigators decided on inductive category development instead of using preconceived categories in order to allow insights to emerge from the data.²⁰ One investigator conducted a primary analysis of all the responses. All the responses were read several times. An initial coding phase in which each segment of data was examined for analytic ideas to pursue was followed by a focused coding phase, in which the most salient categories were pinpointed. Significant phrases and sections were re-stated and grouped into categories, and the categories were grouped again into overall themes. Several steps were taken to enhance the quality of the research and improve trustworthiness.^21-25 An iterative coding process was used at each stage of the analysis to ensure that the codes, categories, and themes accurately and fully represented the range of responses. The process and findings were discussed with two other investigators to avoid investigator bias. Themes are supported by illustrative quotes from a large sample of students.

Responses to what students liked and disliked about TBL and lecture formats (questions 2, 4, 5, and 6 from the survey) are summarized in Table 3. For TBL sessions, students liked the application exercises and team discussions. For lecture, students liked that there was no preparatory work and that instructors told them what they needed to know. Students’ dislikes tended to mirror the findings from what they like; specifically, they did not like the preparatory work for TBL and did not like that there were no application exercises for lecture sessions. Similarly, when asked about the most helpful aspects of both pedagogies (questions 1 and 3 from the survey), students noted the application problems in the TBL sessions and the emphasis on key points and important information in lecture (Table 4).

The following main themes emerged from analyzing student responses to the survey: student perceptions about the impact of TBL on learning, student perceptions about the value of TBL learning methods, student perceptions about the impact of TBL on their approach to learning, student perceptions about the impact of TBL on the development of professional skills, and student perceptions about enduring challenges with TBL pedagogy.

Students frequently commented on the impact of TBL (compared to lecture) on their achievement of the course objectives, specifically on their understanding and retention of the course material and on their development of clinical reasoning skills. They believed that TBL led to deeper learning, stating, for example, that TBL “allows for a more complete understanding of the material” and that without team discussion of certain topics, “my understanding would have been lacking.” One student wrote that TBL gives “more time to not only learn the material but also to understand and apply it more readily.”

Students also mentioned how TBL is superior to lecture in helping them better retain what they learn. One student wrote, “Through application of new knowledge with critical thinking, the knowledge seemed to stick in my head in a much more permanent way.” Another student wrote, “giving students the resources they need to build a treatment plan … and then guiding and adjusting the application of that plan has personally resulted in much greater retention of required material.” A third student said that working in teams on patient cases “helps to reinforce the information I have learned.” Students remarked that their improved knowledge retention became apparent when preparing for exams, with one noting, “I am able to spend less time studying overall for exams as I retain information better as lessons progress.”

Development of higher order, clinical reasoning skills during TBL sessions (compared to lecture) was another common theme among the responses. Students wrote that the focus on developing clinical decision making skills was a helpful aspect of TBL. One student wrote that “writing case defenses allows for more critical thinking within the team because we actually have to provide legitimate, supportive reasons for our choices using what we know about the case.” According to another student, TBL enhanced critical thinking by “making [students] consider a multitude of factors which affect [a] decision” and “showing … multiple ways to go about treating a patient in an effective manner.”

Many students noted their roles were different in TBL compared to lecture, especially when it came to being responsible for guided self-study, active involvement in case-based learning, and teamwork. Throughout the reflections, students reported that adjusting to these new learning methods posed a challenge initially, but most students grew to enjoy and value these different aspects of TBL.

Students most frequently cited engagement in active learning methods in TBL, such as the team application problems (tAPP), the Individual and Team Readiness Assurance Tests (iRAT and tRAT), and intra- and inter-team discussions, as being beneficial to their learning. Regarding the tAPP, students found that working on cases “provides real application of the material we learn” and reinforced concepts. They found the discussions of RAT and tAPP solutions useful. The following comment is representative: “I have found that I remember things best when I discuss them among a group of people and hear the thought process of others.” Students also saw value in having to explain and defend their opinions. For example, this helped one respondent “gain confidence in … recommending a therapy using evidence-based medicine.” One student said, “I really learned that if you cannot explain your answer or a concept to a group of people then you have not yet mastered the concept and need to study more.”

Another common theme was the value students perceived from learning from their peers and from the diverse perspectives their peers brought to the discussions during TBL sessions, an opportunity that was largely absent during lecture. By working with their peers, students reported gaining insight into the material and expanding their thinking.” Also, working with their peers helped them see that “there is always more than one way to approach these problems.” Another student wrote “I learn a lot from my classmates when we discuss our tRATs or case answers because we all have different perspectives and each of us has something different to contribute, opening my mind to various aspects of the situation I haven’t previously considered and allowing me to see things in a new light.”

Students also reflected on the supportive classroom environment created by TBL. Students valued the open discussion and formative feedback in a low-stakes environment as the instructor interacted with the teams. Several students also commented on how much they enjoy the TBL method overall. One student wrote “Since my first TBL lecture in therapeutics P1 year I have thoroughly enjoyed the interactive and engaging environment Team Based Learning creates when evaluating clinical information and cases.”

Regarding the guided self-study preparatory work, many students noted that this was a new and unfamiliar method for them to learn new material. They wrote about the challenges of making time in their schedules to prepare for class and of being able to pick out the salient points from the self-study materials. However, several students commented on the effectiveness of the preparatory work in supporting their learning. For example, preparing for class ahead of time allowed for “a more complete understanding of the material” and supported “learn[ing] material at a deeper level during class.” Students grew to value the benefit of doing preparatory work because it helped them retain information better and reduced study time prior to exams.

Students also remarked that the requirement to prepare for every class helps students overcome their tendency to procrastinate. One student wrote “With iRAT and tRAT quizzes, people are much more motivated to do their best to learn the material beforehand. For someone who occasionally procrastinates studying for exams (myself included), this forces an earlier understanding of the material.”

Another student appreciated the ability to “learn the pre-class material at my own pace” and “spend as much time on each section of the material that I choose.”

The reflections provided evidence of changes in the way some students think about and approach learning after experiencing TBL pedagogy. For example, several students discussed how TBL led to a realization that learning goes beyond simple memorization of content and that it is important to be able to apply content as well. Further, several students described how they have adapted the TBL format to other classes. One student wrote “I will continue to prepare for courses by understanding fundamental facts and ideas, engage in active discussion with my professors and peers, and use the knowledge I gain before and after class to formulate a more analytical approach to learning the material in order to apply it to a variety of situations.”

In addition, a few students referred to the shift in responsibility for student learning that occurs with TBL moves from the instructor to the students. Although one student perceived this responsibility as stressful, another welcomed the feeling of being “more actively responsible for my education and success.”

Beyond gaining competence in applying therapeutic knowledge, students commented frequently on other skills they had acquired as a result of TBL pedagogy and how these skills would be useful for their career. The most common theme was the preparation students were receiving to work in teams. For example, students noted that the course “mimics the complexity of health care teams we will likely encounter as pharmacy professionals.” One student wrote “Very rarely are decisions made on an individual basis in our modern health care models, and I feel that TBL format exemplifies this well.” Specific teamwork skills that students mentioned were listening to and understanding how other colleagues think, taking others’ opinions into consideration, and coming to meetings prepared and ready to make helpful contributions. A few students wrote about developing cooperative leadership skills, stating for example, “I have adapted to not having to take the lead on everything to get things done, and that as a group, we are jointly making a clinical decision we all agreed upon.” For one student, working on a team had a profound impact, helping the student to become more actively engaged with teammates which led to gains in confidence.

Also, students commented on professional communication skills they had developed as a result of opportunities created through TBL, such as how to advocate for their opinion on a therapeutic issue, or discuss information with a patient. One student noted that “as a health care provider, relaying clinical information to others is our job” and that TBL gives students “great practice for future careers where they will be interacting with colleagues and patients on a daily basis.”

For many students, TBL provided a useful model for approaching patient cases to follow in the future. Two elaborated on the approaches they had developed:“I learned to question more in a clinical perspective. For instance, how will the dosing change if another drug was added, what side effects should we counsel patients on, what studies are clinically relevant or applicable for this patient case? All of these are questions I am starting to build in my mind as I grow as a student.”“Having a TBL pedagogy has given me the opportunity to learn, refine and perfect the way I approach a case study. Questions that I now always ask include: ‘What pertinent information do I look for in the case (age, past medical history, allergies, etc.)? How does this impact the patient’s treatment regimen?’ Thus, TBL learning has allowed me to develop a more step-wise approach to tackling a patient case.”

Finally, students reflected on how TBL presented an effective model for their continuing education and professional development. Learning how to extract relevant information from readings and developing clinical correlations while reading were skills that students felt would serve them throughout their career. One student summarized the connection between TBL and future practice as follows:“As a future health professional, I know that I will be primarily responsible for my own continuing education, and I will have to use my judgment to determine important takeaways from data. Unlike in a traditional classroom, I may not always have access to experts who can package clinical findings in an accessible way, and I will likely need to take on that expert role to convey information to others. TBL forces me to practice the information extraction-process before each class session and examine the learning strategies that work best for me.”

Despite the changes that students reported in their approach to learning as a result of TBL, challenges persist for a few students. Two students reported feeling that certain topics are not covered adequately in TBL, resulting in apprehension about missing critical information and a lack of confidence in making therapeutic recommendations. One student expressed concern that disease information (pathophysiology, epidemiology, etc) was not covered well enough in TBL. Another reported difficulty in applying material without examples provided by faculty during lectures. A fifth student expressed an overall frustration with TBL, stating, “Ultimately, I would like to feel that I am being taught the material, not that I am teaching myself.”

Finally, two students commented on the difficulty of keeping up in TBL because they consider themselves to be slow readers. For one student, this posed a problem primarily in completing the preparatory work, but the student reported accommodating this challenge by developing a method for organizing the information and rewriting notes. For the other student, the challenge involved inability to participate fully in tAPP because by the time the student had finished reading the question, the other group members had already started discussing the answer or had even moved onto the next question.

DISCUSSION

Our results shed light on students’ experiences with TBL (compared to traditional lecture) that can assist faculty using or planning to use this pedagogy. Overall, students report TBL enhances their learning of course content (knowledge and application), teamwork skills, and lifelong learning skills. Our study design allowed head-to-head comparison of TBL and lecture. This design is unique compared to other published reports, and adds to the strength of evidence supporting TBL in pharmacy curricula. Methodologies that employ retrospective or concurrent unmatched controls can be confounded by multiple types of bias and errors that can be controlled for or minimized using a prospective, randomized, crossover design.

In addition to learning outcomes like test performance, students’ perceptions about teaching pedagogy can inform iterative improvements in teaching. Perceptions and attitudes can be particularly important to assess in pedagogies that “flip” the classroom because students’ roles are radically different compared to when traditional lecture is employed. Similar to others, we found students reported differences in roles that affected their behaviors related to the course, as well as insights relating to implementation of active learning pedagogy.¹⁷ However, informal feedback or optional course evaluations are unreliable and can lead to incorrect conclusions about students’ experiences based on bias in sampling or informal analysis. Our crossover methodology with high survey response rate provides a more reliable validation of these findings.

Reports of higher orders of learning, such as application of content to clinical cases, were prevalent among our students. This is supported by improved test performance for students participating in TBL compared to lecture and higher levels of confidence about their ability to make and defend drug therapy recommendations.¹⁸ These qualitative data provide additional support for the benefits of TBL, and help validate that the somewhat small numeric gains made are not simply statistically significant, but are also clinically important. Further, it illustrates the disconnect that can occur between what students “like” and what facilitates their learning. For instance, students like lecture because information is presented by faculty, but students also acknowledge that this format does not allow for application to case scenarios or challenge them to compare and contrast multiple therapeutic options. As faculty evaluate their own teaching and consider pedagogical shifts, they may wish to consider what is popular and what is effective for student learning.

Our students also reported favorable impact on lifelong learning skills and teamwork. Undoubtedly, they drew on their previous experiences with TBL in our curriculum when providing their reflections and evaluations, since these gains were unlikely to be achieved in just three weeks of TBL. Nevertheless, these observations confirm that learning outcomes in these domains can be achieved in parallel with outcomes relating to knowledge and application of course content. Our results are compatible with others who found active learning in teams (not TBL specifically) improved self-directed and self-regulated learning, which are important components of lifelong learning.²⁶ Similarly, TBL favorably affected students’ perception of teams and teamwork compared to students enrolled in similar courses taught using traditional lecture.²⁷

Our data reveal that students perceive some benefit to learning via lecture compared to TBL. Our students raised concerns about the adequacy of guided self-study as initial exposure to course content, in part because it may not convey emphasis and perspective as well as faculty during a lecture. Similar to others, we feel this problem may be related to students’ difficulty adapting to their new role in TBL, problems with guided self-study materials, or both.¹⁵ For faculty teaching via TBL, this disadvantage might be mitigated in a number of ways: curating better self-study materials, providing “clinical pearls” to accompany self-study materials, designing tAPP to illustrate “clinical pearls,” or providing expert perspective during corrective instruction or inter-team tAPP discussions. Another factor that may contribute to this observation is some learning styles may not be as adaptable to TBL as others. While not all students raised these issues, it is still important for faculty to consider and try to accommodate these minority views to create an inclusive learning environment that optimizes outcomes for all students.

There were limitations to our study. While our design was prospective and randomized, the single site, learners experienced with TBL, and a relatively small sample size may limit application to all pharmacy learners. In addition, there may have been bias in our sample because the course was elective, and students with aversion to active learning in general or TBL in particular might have been under-represented.

Overall, these results are congruent with other assessments by us and others, and support the use of TBL as a viable pedagogy to use in pharmacotherapeutics courses.^12-15,18 Nonetheless, additional research in several areas could inform better use of this pedagogy. For instance, evaluating different modalities of self-study materials may help identify methods that optimize efficiency and effectiveness of initial exposure to content, perhaps providing students what they liked about lecture (perspective and commentary provided by a content expert). Similarly, evaluating different types of tAPP could help identify approaches that best help learners bridge the gap between basic knowledge and higher orders of learning (thereby optimizing what students value most in TBL). And, lastly, research on the impact of TBL on long-term learning outcomes, including performance on advanced practical experiences, would be useful.

SUMMARY

By using a prospective, randomized, crossover design, we demonstrated pharmacy students’ attitudes and perceptions about their learning support team-based learning (TBL) as a viable pedagogy compared to lecture. Overall, TBL has positive impact on students’ quality of learning, their approach to learning, and importantly, their ability to build and enhance critical skills that they can take into professional practice. These critical skills include working in a team environment and developing lifelong learning skills.