The Effect of Banning Computers on Examination Performance in a First-Year Pathophysiology Class

INTRODUCTION

Over the past decade, research into the effect of student computer use within the lecture setting has identified potential negative and unintended consequences of technology use on students’ academic performance.¹ Fried demonstrated that students who used laptops in class spent considerable time multitasking, and that laptops posed a significant distraction to both users and fellow students.² Further studies demonstrated that when students in a large biology class setting self-selected whether they wanted to sit in a laptop-permitted or laptop-free zone, the students in the laptop-free zone scored significantly better when normalized for pre-class academic indicators.³ The extent to which students use laptops has been negatively related to several measures of student learning, including self-reported understanding of course material and overall course performance.² Multiple studies have demonstrated that computer notetakers had lower examination grades than hand notetakers,^4,5 which has led to the suggestion that students process and reframe information better when they handwrite notes instead of typing them on a computer.⁵ Yet, despite the potential for distraction to the student (and those seated around them), and the data demonstrating improved student performance when notes are handwritten, student use of computers in the classroom is commonplace, estimated by one study to include almost four out of every five college students in 2016.⁶

The impact of student laptop use in pharmacy education is less clear. Although there has been significant discussion and debate about the use of laptop computers,^7-9 only one study collected data on performance.¹⁰ Prescott and colleagues asked students to self-report their computer usage, which was then compared to their academic performance.¹⁰ Computer usage had no significant effect on second- and third-year students’ performance in a therapeutics course, but when data on the students in the second-year were separated out, those who had refrained from using devices had earned a higher course grade.¹⁰

In this study, we explored the effects of a laptop-free classroom on first-year Doctor of Pharmacy (PharmD) students in a pathophysiology class. Although the “banning” of computers in the classroom is quite controversial, the previously cited studies demonstrate that the distraction of an off-task computer user effects not only the student doing it (a personal choice) but other students near them (less of a personal choice). The performance of the students on four in class examinations was calculated and compared to that of students in a previous year who had access to computers. Student evaluations of the course, including their written comments of the computer “ban” were also examined. The primary objective of the manuscript was to evaluate student performance in the course when they did not have access to their computers compared to performance during a semester in which they did.

METHODS

Within the PharmD program at Western New England University, pathophysiology is a foundational course designed for first professional year pharmacy students. The purpose of the course is to introduce basic knowledge of disease states and disease progression to students before they matriculate into integrated therapeutic module courses where they learn additional pathophysiologic principles as a basis for therapeutics. The course enrolls approximately 65 students each year. Based on previously published literature, it was hypothesized that an electronics-free classroom would lead to improved student examination performance. Thus in 2018, the course syllabus was changed to disallow the use of computers during class lectures.

To compare the effect of the electronics-free classroom on student performance with student performance in a previous iteration of the class, the 2016 course offering was selected. Other modifications to the course had been made in 2017 that were subsequently dropped in 2018, making 2016 the cleanest comparator for this study. In both 2016 and 2018, the author delivered 39 out of the 45 total lectures and was the sole instructor to create the four in-class examinations. As a different instructor had delivered the majority of the lectures covered by a fifth examination in both years and had allowed students to use electronics in 2018, student performance on that examination was also used as a comparison. Beyond the use of computers, the class and assessment structures were quite similar. Students had access to electronic slides to download or print before each class period and the examination questions were of similar difficulty and consisted of a similar percentage of multiple-choice and short-answer questions. The computer-free environment was maintained with gentle reminders given to any student who forgot it, but this was a rare occurrence.

Graphpad software (San Diego, CA) was used to perform statistical analysis. First, the D’Agostine & Pearson omnibus normality test was performed to analyze the normality of the data. Because the data were normal, the two-tailed unpaired t test was used to determine significance. This research was submitted to and approved by the Institutional Review Board of Western New England University.

RESULTS

The average grade on the four examinations in 2016 (when students could use electronics in class) was 77%, compared to an average grade of 80% in 2018 (when electronics were no longer permitted); this increase of 3% was significant (p=.03), with a Cohen d effect size of 0.4, which is considered to be a moderate effect. Of the four examinations, three demonstrated an improvement of at least 3% (Table 1). Final course grades are displayed in Table 2. The percentage of students who earned a final course grade of A or B remained consistent. However, there was a significant change in student performance at the lower end of the scale, as only one student failed the course in 2018 (after computers were not permitted) in comparison to five students in 2018 (when students could use electronics in class). As a curricular baseline to allow for comparison between the groups, we also examined student demographics between the 2016 and 2018 classes, students’ mean pre-pharmacy grade point averages (GPAs), and the cumulative GPA of the class after the first professional year. The two cohorts had a similar ratio of male to female students (63% vs 62%), internal pre-pharmacy students or students who entered the pharmacy program from the university’s pre-pharmacy program (61% vs 60%), and pre-pharmacy GPA (3.4 vs 3.4 on a 4.0 scale). One difference noted was that the 2016 cohort had an overall first year GPA 0.12 points (on a four-point scale) higher than the 2018 cohort, suggesting that the first-year GPA bias would have been lower rather than increased for the computer-free cohort. This is supported by scores on the fifth in-class examination, which primarily covered content taught by a different instructor who allowed the students access to electronic devices. Scores on that examination were 8% lower in 2018 (p<.001) than in 2016.

The instructor evaluations for the course were slightly but not significantly improved in 2018 (4. 6/5 vs 4.7/5 (p=.13) calculated from an average score of 18 different questions. The computer ban was mentioned 10 (out of 60) times in the instructor evaluation free-response fields. Six (60%) of these responses were from students who felt having a computer-free classroom was beneficial to their learning, including two responses were from students who were really against the idea in the beginning of the course but found it beneficial by the end of the course. The four remaining comments were from students who wished they had had access to their laptops for notetaking because of organizational or related preferences.

DISCUSSION

This work explored the possibility that pharmacy student performance is enhanced in a computer-free environment. Consistent with previous studies, the data revealed an improvement in examination performance for the students in an electronic-free environment (3%).⁵ Prescott’s previous study examining pharmacy students differed from this study in multiple ways as Prescott studied two cohorts of students (second and third year) and allowed them to self-identify whether or not they used computers appropriately in the classroom to correlate to grades.¹⁰ Both the current study and the Prescott study found that novice pharmacy students (first year students in this study and second year students in their study) were significantly adversely affected by in-classroom computer use. Prescott postulated that students at an earlier stage in the pharmacy curriculum may still be gaining familiarity with the course sequence, may not have developed sound study habits, and/or may lack the ability to maintain focus on class activities/lecture content while using an e-device,¹⁰ as the benefit they identified in second-year students was not present in third-year students. Considering the present study examined first-year students, our findings would support this hypothesis.

One of the most interesting findings in this study was the difference that removing technology had on the performance of the lowest tier students. While this study found the percentage of A’s and B’s between the 2016 and 2018 course offerings was similar, a notable increase in the percentage of C’s and corresponding decrease in F’s was found. This would support Prescott’s postulation that the students most prone to distraction or those who are the weakest academically may benefit the most from eliminating the distraction that computers can cause.¹⁰ Patterson and Patterson also found that the negative effects of computer use in the classroom were concentrated among the lowest performing students.⁶ Thus, the effect of going computer free may have the strongest benefits on the weakest students, which would be very impactful on student progression in the PharmD program.

One difficulty in banning computers from the classroom is student pushback. Prescott found that almost four out of five students somewhat disagreed or strongly disagreed that use of electronic devices during class should be banned.¹⁰ With our experimental design, students did not have the opportunity to decide whether they would use a computer, but they did have the opportunity to reflect on it. Interestingly, six out of the 10 students who commented about the lack of computers in the classroom stated that they found removing them to be helpful, including two students who stated they never would have done it by choice, but found great value in it. This may help explain the discrepancy between the student perception in the previous study and this one, and supports the idea that students may only see the benefit of the method after they have to do it.

As laptops and tablets become more commonplace within classrooms, instructors need to consider the costs and benefits of allowing students to use them, especially within pharmacy schools, where some programs have embraced laptops to the point that students are required to purchase a laptop for use in class and to take examinations.⁷ One argument for student access to computers is that, within the practice of pharmacy, technology is used every day, and therefore students should have a classroom setting that simulates these conditions.⁸ Studies have shown students perceive educational benefits with computer use ranging from note-taking activities, in-class laptop-based academic tasks, collaboration, improved organization and efficiency, and addressing special needs.¹¹ However, in the same study, students also reported challenges associated with classroom computer use, including distracting laptop behaviors, instant messaging, surfing the web, playing games, watching movies, and decreased focus.¹¹ Kay and Lauricella postulated that faculty members have three choices in regards to student use of laptops: ignore, reject, or actively incorporate them.¹¹ The ignore option was that faculty members would just continue to lecture, ignoring the fact that students were using laptops to take notes instead of writing in a notebook. The problem with the ignore choice was that notetaking on a laptop has been found to lead to extensive off-task behavior by students.^2,11 A separate study found students who were allowed to use computers unregulated in a classroom were off-task up to two-thirds of the time.¹² Anecdotally, the author noted a striking improvement in student engagement in the class that did not use computers, both in terms of engaging in discussions and asking questions about the lecture material. While these differences may have been a reflection of the personalities of students in the class, the other instructors did not notice any discernable difference in class participation or attention during their lectures.

Some educators have advocated for using technology to enhance the learning process, performing active-learning exercises, using novel course-related software, and using case studies.¹¹ While pharmacy education has made great progress and uses many innovative teaching methods and experiential learning,^13-15 traditional lecture is still the primary method of content delivery, especially in pharmaceutical sciences courses such as pathophysiology.^16,17 It is difficult to ignore the potential for distraction that electronic devices may cause in these lecture-based courses, and this may be yet another reason for instructors to consider transitioning their content delivery away from lecture-based methods.

It is unclear whether the selection of the pathophysiology course for this study contributed to the enhanced student performance in the class. Considering the course is taught in the first professional year and is primarily delivered via lecture, the potential exists that the benefit in student performance observed in this study would not transfer to a non-lecture-based course or a course delivered later in the PharmD curriculum. The study by Prescott and colleagues used two similar, primarily lecture-based therapeutics courses.¹⁰ Considering they found a difference in performance between students earlier in the pharmacy curriculum that did not continue one year later, their study suggests that the course choice and content may be less important than the year in the curriculum the course is being offered. However, there is likely advantages to using computers in courses that are not lecture based. For example, in the author’s other courses,^16,18 students often require computers to access information to help them complete the case or problem they are exploring, and that teaching method leads to increased student performance.¹⁶

There are limitations to this study. First, there may still be inherent differences between the student cohorts regarding the amount of time spent studying for examinations and class attendance or innate ability. To examine this possibility, overall demographics between the cohorts as well as first professional year GPA was compared and, from this institutional data, the 2016 cohort would have been predicted to score higher than the electronics-free 2018 cohort by 0.12 points, if all else was equal. Previous studies have demonstrated an increase of 0.17 points (on a 4.0 scale) in student grades if they cannot use laptops in class,⁴ which supports the findings of the current study. Also, improvements in the lecture delivery by the instructor may have affected student performance. In this instance, it was the sixth and eighth offering of the course by the instructor. While the instructor’s teaching aptitude has likely increased over time, it is unlikely that any improvement between years six and eight would be as significant as improvements between years one and three. Finally, although the lecture and examination content were basically the same, the order of the course topics was varied from year to year in an attempt to optimize content delivery.

CONCLUSION

There has been spirited discussion within pharmacy education and beyond regarding the impact of computers in the classroom, but little data exist to support or refute these positions. This study provides evidence that student use of computers within the classroom may have a negative effect on performance. It also demonstrated that the weakest performing students may experience the greatest benefit. Of interest to professors considering a computer-free classroom, there were no overall negative impacts on student evaluations of the instructor. The data presented in this study provides important insights to faculty and administrators as they weigh whether or not to create a computer-free learning environment.