Implementation and Evaluation of Near-Peer Facilitated Journal Club Activities in a Required MLE Course Series

INTRODUCTION

As medication experts, pharmacists provide drug information (DI) as a foundational aspect of their role on the health care team and within the community.¹ It is critical for all Doctor of Pharmacy graduates to have DI and literature evaluation skills so they can build clinical knowledge and practice evidence-based medicine.² In addition to the desire to develop practitioners to serve patients and the health care team, the Accreditation Council for Pharmacy Education (ACPE) has standards to meet these requirements, including standards related to literature evaluation.³ These standards reaffirm the importance and relevance of these skills for practicing pharmacists.

One way to prepare students for literature evaluation in practice is through journal clubs.^2,4-9 In a 2016 study that examined how medical literature evaluation (MLE) is taught, journal club was the most common active learning method at 68%, which is more than double the rate from a 2006 study.^5,7 Common, however, does not translate into frequent, as the most common frequency of journal club activities reported was only one or two per course.^5,8,9 While the literature provides no specific recommendation on a number of additional journal club activities, it does make clear that this low frequency is inconsistent with experts’ recommendations to increase literature evaluation practice opportunities.⁶

To increase journal club activities in a large, required MLE course series, logistical barriers, including personnel needs, must be minimized. One way to do this is to use the Layered Learning Practice Model and Near-Peer Teachers.^10,11 The Layered Learning Practice Model, sometimes referred to as a hierarchical teaching model, has been widely used in medicine experiential education.^11-16 This model involves an “attending” clinician, residents, and multiple levels of students, with each teaching their respective junior learners. Near-peer teaching fits into this model by incorporating learners as teachers or facilitators when they are at least one year senior to their near-peer learner.^11,13 Near-peer teaching has been reported in the medical literature since the 1980s, and anecdotally, in pharmacy practice and other professions’ practice for some time. But the specific terminology has only recently been incorporated in pharmacy education literature.^10-14,17 The objective of this brief is to describe the implementation and evaluation of journal club activities using near-peer facilitators in a required two-semester MLE course series. The evaluation includes evidence of students’ meeting their educational objectives and perception data from both MLE students and near-peers. This scholarship was designated as exempt through the university’s Institutional Review Board.

METHODS

Concordia University Wisconsin School of Pharmacy (CUWSOP) offers a four-year PharmD program. MLE is a required two-semester course series that includes two 2-credit courses during the second (P2) year. Prior to fall 2012, Medical Literature Evaluation I and Medical Literature Evaluation II (MLE I and II) were offered as traditional lecture-based courses with in-class active learning strategies. However, journal club or similar activities working with peers to verbally discuss, critique, and apply literature findings were absent. A change in personnel led to new course faculty members for the series and a subsequent review of the content, alignment, and application opportunities. Journal club experiences were implemented in this course series to meet student learning objectives shown in Table 1.

Each class session was two hours long and met once per week during the traditional 14-week semester. Ten journal club activities were incorporated into MLE I and II for the first time in the 2012-2013 academic year (Table 2). Journal club and related activities made up 25% of the course grade in MLE I and 30% in MLE II. MLE course faculty members selected all journal club articles to coincide with and build upon previous course content (Table 2). When possible, the therapeutic focus of each article was selected to coincide with concurrent or previous content in the pharmacotherapy curriculum. To appropriately scale this activity, course faculty members targeted group sizes of 8-12 students based on previous practice and education experiences. With 70-100 students in MLE, the activity needed to accommodate about 8-12 simultaneous journal club sessions at once. Like most pharmacy schools, asking 8-12 faculty members to be available every 2-3 weeks was not sustainable given other conflicting responsibilities. The solution to this challenge was a combination of two MLE course faculty and 6-10 near-peer, upper-level P3 students each semester. In the fall semester, these near-peer P3 students selected the near-peer role as one of several required project options in their respective Applied Patient Care Fall P3 course. In the spring semester, near-peer P3 facilitators were those enrolled in the Pharmacy 563 Journal Club elective course. This P3 elective course involved weekly journal club and related advanced literature evaluation skills; the near-peer role in the six MLE II journal club sessions was only a small part of the elective course expectations. All P3 students had previously completed the MLE series. All journal club groups were randomly generated for each session, with different students and different P3 or faculty facilitators each time. This approach allowed attendees to have a variety of journal club experiences.

Each of the 10 journal club activities were divided into four components: preparation, quiz, journal club, and debriefing. Preparation included reading the article thoroughly. P2 students were coached on the appropriate approach to prepare for journal club. P3 student mentors met with faculty members before each journal club to ensure they were adequately prepared and understood key concepts from the article. This included all the near-peer P3 students participating in a P3 student-led, faculty-facilitated full journal club session several days before the MLE P2 journal club activity. This session included reflections on previous journal club sessions, a full journal club session on the article, and time for questions about the article or logistics. In total, each P3 session lasted about 60 minutes.

At the start of MLE on a journal club day, a 10-minute, closed-note quiz was given to the P2 students. The quizzes assessed students’ knowledge of the article and preparation for journal club. Quizzes included questions about the article’s study objectives, design, subject inclusion and exclusion criteria, sample size, other statistical methods, and results related to the primary and secondary outcomes. While students were encouraged to review didactic course material that was relevant to application of the article, the quizzes did not ask about didactic content alone but required didactic knowledge when answering questions about the article’s content. After the quiz, students took 2-3 minutes to transition to their assigned rooms for journal club. CUWSOP has five to six 14-seat breakout rooms surrounding each of its three 100-seat lecture halls, providing adequate space. The MLE I course faculty and near-peer P3 students facilitated the MLE I journal clubs and provided well-modeled facilitation examples, while the P2 students in MLE II served as participants and facilitators. At the beginning of the spring semester, students were randomly assigned to facilitate one journal club session during MLE II. Because of the number of students relative to facilitation opportunities, students were randomly assigned to facilitate either individually or in pairs. For example, in a MLE II offering with 80 students and 60 total journal club sessions (six weeks of 10 individual journal clubs per week), there were 40 individual P2-facilitated and 20 pair P2-facilitated sessions. Facilitators received a guide and were directed to encourage everyone’s participation. Journal club sessions were designed to last about 45 minutes. Up to five participation points were awarded to P2 students by the near-peer P3 or course faculty. After journal club sessions and a transition back to the main lecture hall, the P2 class regrouped for a 10-minute debrief on the article. This faculty-led debriefing ensured that key concepts related to course content were reinforced, take-home points from the article were clear, and any questions from students were addressed. The remainder of class time (usually about 40 minutes) was spent on other teaching activities.

The alignment of evaluative data sources with student learning objectives is shown in Table 1. Data evaluating the success of this educational initiative’s student learning objectives came from quantitative sources (quiz scores and near-peer/faculty rubric data). P3 reflections and P2 course surveys provided indirect evidence of student learning outcomes and perception data.

There were 10 quizzes per year, one for each journal club, ranging from five to eight questions (10-15 points). Quiz scores were converted into percentages to normalize for reporting.

Peer feedback for P2 facilitation in MLE II was formative and in rubric form. The journal club rubric had seven components (introduction, methods, results and author’s conclusions, critique, implications, preparedness, presentation and facilitation) and three anchors (acceptable, needs improvement, and needs significant improvement). All seven components’ anchors included descriptors.

P3 near-peers were responsible for providing a brief reflection after each journal club. They were asked to respond to the following questions: who was present/absent; what participation points should be assigned; what went well; and what was challenging or could be improved. Themes were extracted from these brief reflections by the course faculty members and provided further evidence (albeit indirect) of the P2 students’ meeting student learning outcome expectations as described in the results section. These reflection themes and comments also provided near-peer perception data.

At the end of the MLE sequence, P2 students’ perceptions of the experience were gathered through a 12-question survey about journal club. Initially, survey participation was optional. However, in later years, the course faculty required survey participation to earn points toward the course’s last assignment. In both instances, responses remained anonymous, and a student could get full credit even if he or she elected to electronically submit a blank survey.

The majority of data is summarized using descriptive statistics. Exploratory comparisons of faculty and near-peer ratings were analyzed with the Chi-squared statistical test.

RESULTS

Over four academic years, there were 346 students who completed the MLE course series. Sixty-two P3 students served as near-peers over these four years. This included 25 near-peers in a fall semester as a required project in their P3 Applied Patient Care course, 24 in a spring semester as a required component of the Pharmacy 563 Journal Club elective, and 13 in both a fall and the consecutive spring semester. Along with the two course faculty members, these 62 near-peers covered 419 journal club groups, 180 of which were in MLE I and 239 of which were in MLE II. Course faculty covered 80 of these 419 (19.1%), while near-peers covered 339 (80.9%).

Pre-journal club quiz score means ranged from 78.1% to 92.0%, and the overall average of all 10 quizzes for all students was 87.5%, demonstrating that overall students were reasonably well prepared in their familiarity and understanding of the key features of each article.

There were 239 student rubric-driven formative feedback submissions provided, of which 142 were from solo P2 student facilitators, and 97 were from two P2 students working as partners in facilitation. Results from these rubrics illustrating P2 students’ performance as facilitators in MLE II journal clubs are shown in Table 3. Assignment of a “needs significant improvement” rating was rare, with fewer than 2% of students receiving this rating for any component (range 0.4% to 1.7%). Needs improvement ratings were most common in the journal clubs’ discussion of introduction (24.3%) and methods (18.8%). Acceptable ratings were most common for the two components of implications and preparedness, with both being acceptable at above 90%. One in 50 journal club sessions resulted in a facilitation or presentation component being rated as markedly below expectations. Even in these few instances of less than acceptable journal club sessions, it is not likely that the session was an educational loss, as the P3 facilitators or faculty members were able to assist in cases where the P2 facilitators were struggling.

P2 survey participation ranged from 81.7% to 100% annually. Consistent with performance data, P2 students perceived the journal club activities as educationally beneficial. Ninety-five percent of P2 students agreed or strongly agreed that the journal club activities helped them learn course material, while only 1% disagreed. About 74% of respondents indicated the educational design encouraged them to always or frequently discuss the journal article with their peers before journal club. P2 responses indicated that participation and discussion of peers’ opinions during journal club was the best part of journal club in 131 (43.8%) of 299 free-text comments, about twice as often as any other comment. In addition, 75 (25.1%) indicated that the application of knowledge and skills of the course was the best part of journal club.

Near-peer P3 reflections provide further albeit indirect evidence of P2 student performance. Across all four years, 196 reflections were completed by the 37 P3 near-peer students in the spring journal club elective. Similar to P2 feedback, the P3 reflections described that one of the best parts of journal club was the participation and discussion of the P2 students, noted in 119 of 196 (60.7%) of near-peer reflections. This was the most common theme by almost threefold and consistent with the fact that more than 99% of P2 students received full participation points. Other “best parts” themes included the peer P2 facilitators’ encouragement of their colleagues’ participation (21.4%), the P2 student facilitators’ preparation (18.9%), organization and flow (13.8%), and autonomy in facilitating (as opposed to requiring P3-required intervention) (10.2%). These combined data provide evidence that students were able to and saw value in articulating a critique and identifying practice implications of the literature discussed in the journal club sessions.

However, not all of the P3 reflections suggested optimal P2 facilitation, as was the case with the formative feedback rubric’s data noted above. In addition, while P3 students’ reflections specifically noted that “nothing should be changed” 50 times (25.5%), they did note that P2 facilitators needed to improve their tendency for suboptimal flow/organization (17.3%) and for missed or skimmed important point(s) (13.3%).

Preparation efforts and performance of the near-peer P3 facilitators rated by the P2s were overwhelmingly successful, with 98% of P2s rating their P3 near-peers as adequately prepared. The P2 students also reported that in about 9 of every 10 journal clubs, the P3 students were always or frequently able to teach them something they did not previously know, and all respondents indicated they were able to learn from a P3 facilitator in at least some sessions. Most P2 students (75%) agreed or strongly agreed that the rating and comments provided by their near-peer facilitators via the rubric were useful, with an additional 23% being neutral, and 2% disagreeing. Finally, when asked how the P3 facilitators compared to the course faculty, about half of the students rated them as similarly effective, and 4% rated them as more effective than the faculty. While 38% rated near-peer facilitators as less effective than faculty, none of these respondents rated the P3 near-peers as ineffective.

Course faculty submitted 43 formative feedback rubrics (18% of journal clubs), while P3 students submitted 196 (82% of journal clubs). In all cases, the most frequent rating was acceptable regardless of facilitator type, and when differences in the percent of ratings given did occur, they were between needs improvement and acceptable. Differences in ratings were most notable (about 15% difference or more) in introduction (faculty 58.1% acceptable vs near-peer 78.1% acceptable, p=.01), methods (67.4% vs 82.3%, p=.02), and critique (69.8% vs 86.7%, p=.01); were moderate (about 10% difference) in results (79.1% vs 90.3%, p=.04) and preparedness (81.4% vs 92.9%, p=.02); and were negligible (about 5% or less) in facilitation (79.1% vs 84.2%, p=.42) and implications (90.7% vs 92.3%, p=.72). This data demonstrates that on average near-peer ratings tended to be higher than faculty ratings.

A preliminary exploratory analysis of APPE journal club performance as evaluated by preceptors of students who participated as P2 students in the MLE journal club experiences was performed. Although the components of the APPE journal club rubric matched MLE’s, the APPE journal club rubric components’ anchors mirror other APPE rubric tools instead of the MLE rubric, so a direct comparison to MLE facilitation performance requires caution. However, the components being evaluated had enough similarity to make observations possible. Throughout their APPE rotations, former P2 MLE series students facilitated 717 journal club experiences as fourth-year (P4) students. The percentage of P4 students’ ratings that met or exceeded preceptors’ expectations ranged from 96.9% to 99.3% through all components. In all components, the percent rated as meeting or exceeding expectations by preceptors exceeded the percent rated as acceptable by both faculty and P3 near-peer students. Preceptors did not assign any significantly below expectation ratings. There are many factors related to these ratings, and it would be inappropriate to conclude that MLE journal club is the sole cause of students’ success, especially in the absence of a control group. However, it can be said with reasonable external validation that near-peer facilitated journal club met external partners’ expectations related to reviewing, critiquing, and applying literature to practice more than 95% of the time.

DISCUSSION

Developing and implementing near-peer-facilitated journal club activities arose from the desire to increase the literature evaluation practice opportunities for all students with the limited number of available faculty members. These journal club activities successfully met the learning objectives, providing the MLE P2 students with opportunities to apply course content, critique and discuss article content and patient-care implications, and facilitate journal club sessions. The physical space at the school was a strength, providing the learning environment for 8-12 concurrent journal club sessions in each of the 10 journal club weeks. Providing opportunities for students to develop and hone their literature evaluation skills was rewarding for students and faculty members. The formative, open discussion for student peers to learn from each other encouraged students to engage with the literature. According to student surveys, the open discussion was one of their favorite aspects of the journal club. P2 accountability was an important part of this implementation, and the quiz and facilitation ratings along with P2 and near-peer feedback demonstrated that the P2 students were accountable for their preparation and engagement.

There is little information in the literature about using anyone other than faculty or residents as facilitators for journal club activities in didactic education.⁶ Different from teaching assistants, these near-peers are not participating as employees but as fellow students as part of their training. Although there is interest and evidence in a hierarchical (“near-peer”) approach to teaching in experiential education, to the authors’ knowledge, this study is the first description of a near-peer model in this didactic application.^14,17 This study demonstrates the value of using upper-level students to be near-peer faculty extenders in didactic settings similar to experiential settings. It is possible that near-peer use in other didactic settings would be successful.

There were some limitations to this study, such as the challenges related to logistics of scale, occasional space issues and P3 near-peer scheduling conflicts. Lack of available space may limit other schools’ ability to implement a similar journal club experience in their course. If the number of breakout rooms was limited, the journal club group sizes could be modestly increased. If the proximity of the breakout rooms to the lecture hall was a limitation, quizzes could be administered in smaller groups, and debrief points could be posted online to avoid the need of a main hall. To overcome P3 scheduling challenges, faculty may need to be creative and work with other faculty members to ensure P2 and P3 students are available at the same time for journal club activities. Another limitation of this study was that some groups that had P3 students as facilitators completed discussions earlier, leading to varied lengths of journal club and different opportunities for P2 students to fully articulate their critique and implications of the literature. While the journal club discussions were dynamic, prescribing a precise 45-minute session was not the objective. The in-class debriefs after journal club likely further minimized any impact of variation in journal club length.

Another limitation is the rubric’s implementation. Because of the rubric, the training processes, or near-peer tendencies, several components received higher ratings from near-peers than faculty members. While this is an important area for future focus and improvement, it is not an indication that the use of near-peer facilitators should be avoided. To the contrary, the opportunities afforded to P2 students through these 10 journal club applications was only possible because of the innovative use of near-peers in the MLE series. Near-peers facilitated more than 80% of the 419 journal club sessions. The activities were designed to be formative, and while this does not mean that the feedback would be acceptable if it were poor, it is recognized that the feedback is going to vary similarly to how near-peer vs attending provider feedback might vary in experiential settings. If this innovation had been a high-stakes or summative assessment, this rating discrepancy would have been problematic. Certainly every educational initiative should desire to give optimal feedback, and these areas will be refined in future offerings.

Future opportunities for scholarship also exist in both the P2 students’ and the P3 near-peer facilitators’ abilities to excel in journal clubs and other medical literature evaluation activities in APPEs and in residency. Preliminary data show that students are performing well in APPEs as evaluated by preceptors. However, this data or assessment structure may lack some sensitivity to discern differences within students’ acceptable skills in APPEs. Future studies should look at the potential learning of the near-peer participants, as it may be hypothesized that this should be of benefit to the P3 near-peers. While reflections and anecdotal comments are favorable that this is the case, a more systematic approach to explore near-peer benefits would be appropriate in the future.

CONCLUSION

As pharmacists, the ability to analyze, interpret, and apply drug and medical information to patients is what pharmacists must excel at every day. Pharmacists’ expertise as drug information specialists goes far beyond merely accessing information. The future of pharmacy has been and continues to be the ability to synthesize, evaluate, and incorporate drug and medical information into patient care decision processes. Because pharmacists do not work in a silo as members of the health care team, this includes the ability to articulate their synthesis, evaluation, and interpretation to others. These skills, like any skill, take practice. Ten journal club sessions do not elevate a novice to an expert. This skill is cultivated through years of practice and hundreds of articles. But 10 journal club sessions, purposely tied to course content and facilitated by near-peers, has been shown in this work to be an innovative and successful start.