Student Engagement in Pharmacology Courses Using Online Learning Tools

INTRODUCTION

Student engagement is defined as the time and energy students invest in educationally purposeful activities and the effort institutions devote to using effective educational practices.^1,2 It is also the quality of effort and involvement in productive learning activities. The engagement premise has been evolving since the 1930s. One of the earliest reports was a published work in 1969, which showed that time spent on a learning task has a positive impact on improving student understanding.¹ Engagement is emerging as an organizing construct for institutional improvement efforts, assessment, and accountability. The concept of engagement suggests that the more students study a subject, the more they know about it, and the more students practice their learning tasks, the deeper they understand what they are learning.¹ Therefore, the more students engage in learning tasks, the more they benefit from these activities and eventually learn.³

Student engagement involves 2-way communication wherein students and institutions play central roles in creating the environment for engagement and taking advantage of engagement opportunities.^1,4 This broadened perspective highlights the notion that students should be at the heart of the learning process and that institutions aiming to increase student engagement should focus squarely on enhancing individual learning and development.⁴ Student engagement is not a “one-size-fits-all” way of thinking. Nonetheless, student engagement as a concept “provides a practical lens for assessing and responding to the significant dynamics, constraints and opportunities facing higher education institutions. It provides key insights into what students are actually doing and a stimulus for guiding new thinking about good practice.”⁵

A growing body of research into student engagement over the last few years argues that student engagement with traditional classroom lectures and participation in traditional learning activities has significantly declined.⁶ Students are now entering higher education with a diverse range of backgrounds and skill sets, that are different from traditional university entrance criteria.⁷ Moreover, increased student enrolment and the financial costs of higher education have raised concerns about the quality of student learning and experience.⁸ At the same time, education has been undergoing a paradigm shift, moving away from teaching as instruction toward student-centered learning approaches.⁹ Consequently, curricula have been redesigned around learning outcomes rather than content.¹⁰ The rapid expansion and innovation in technology and students’ expectation that technology will be integrated into their learning experiences^11,12 have encouraged higher education institutions to incorporate technology-based teaching into their curriculum to increase student engagement.¹³

Students who have grown up with advances in new technologies have resulted in the emergence of a new concept known as the digital native. Students who are digital natives have novel learning styles that are nonlinear and personalized to individual needs, and they are fluent in “simulation-based virtual settings.”¹⁴ Those students need more than the traditional teaching approaches to engage in the learning process. Technology is thought to provide many advantages to digital-native student learning. Supporters believe that it allows them to direct their own learning by providing flexible learning opportunities.^11,15 The implementation of interactive online teaching materials known as e-tools is increasingly advocated because of their capacity to allow students to learn when, where, what, and how (collaborative or independent learning style) they want.^16,17 E-tools are also anticipated to motivate, engage, and stimulate higher-order thinking for students.¹⁸ However, the impact of implementing e-tools on increasing student engagement has not been extensively investigated. One study in 2011 concluded that commercial e-tools failed to increase the motivation of students when included as part of their teaching and learning. However, to realize a student learning benefit, the content of the e-tools should be aligned with the educational objective of the course.¹⁸ Therefore, a suite of customized animation tools (e-tools) encompassing the basic mechanisms of action for different drug classes were designed for third-year pharmacy students at Griffith University, Gold Coast, Australia. Those e-tools were used to supplement traditional face-to-face lectures in the Human Pharmacology I and II courses.

The e-tools were designed to align with the objectives of the courses to form a system that is beneficial to students.¹⁹ The design process for the in-house e-tools within the framework of a defined pedagogy and relevant teaching theories has been published.²⁰ The aim of this project was to assess student engagement with these e-tools to determine if e-tools increase student engagement. Many measures of student engagement have been used over time, including level of academic challenge, active and collaborative learning, student-faculty interactions,²¹ and student perception. Given that student perception is the most commonly used measure of engagement and has been used by The National Survey of Student Engagement, the present study used this approach by evaluating student comments and feedback.¹ The authors also explored student interaction with the e-tools by analyzing when and how often they accessed the material on course Web site using Blackboard (Blackboard Inc., Washington, DC).²²

DESIGN

This study was conducted at the School of Pharmacy, Griffith University, Gold Coast campus, Australia. A suite of 83 e-tools was designed for the Human Pharmacology I course in semester 1 and 65 e-tools for the Human Pharmacology II course in semester 2 of 2012. These are both 13-week courses normally delivered by means of 3 lectures per week, supported by weekly tutorials and laboratories (2 to 4 hours per week). The e-tools covered the mechanisms of action for the majority of drug classes and were used as a supplement to the standard curriculum. Ethical approval was granted by the Griffith University Human Ethics Committee. Custom animations were sequenced in Microsoft PowerPoint 2010 and narration was added using iSpring Pro 6.1.0 (iSpring Solutions, Alexandria VA) to produce the embedded animation and then convert the animations into an Adobe Flash (Adobe Systems Inc, San Jose, CA) format for ease of delivery and access through Blackboard.²⁰ Participants could easily control the speed of the final e-tools, skip content, and move forward and backward to revisit specific concepts as needed. Each e-tool was accompanied by multiple-choice questions, which were developed to assess stated learning objectives, generated by Question Writer 3 (Professional) (Question Writer Corporation, Torrance, CA) and accessed through Blackboard. Question Writer 3 sent the results anonymously to the researcher’s designated e-mail for evaluation.

The first set of e-tools used during semester 1 were made available to students through the course Web site in Griffith University’s Blackboard interface. Students were informed about the e-tools during the course introductory lecture. Semester 1 assessment items included a midsemester examination; and 4 online quizzes on: (1) genitourinary drugs; (2) the cardiovascular system; (3) drugs affecting blood; and (4) the central nervous system. Students were given 13 weeks to complete the online quizzes. The final examination was administered 2 weeks after the deadline for completion of the online quizzes.

EVALUATION AND ASSESSMENT

Eighty pharmacy students enrolled in the course Human Pharmacology I in semester 1. One student did not pass this course and so was not able to enroll in Human Pharmacology II the following semester.

To evaluate student baseline attributes in semester 1, a paper-based survey instrument was designed to obtain student demographic data and preference for e-tools. One month after finishing the first course, students were approached in person during a Human Pharmacology II workshop and asked to participate in the survey. The timing of the survey gave students a chance to access the e-tools during the first semester and ensured that all students would have the opportunity to participate in the survey. Student participation in the survey was voluntary and anonymous. The survey instrument was designed according to previous studies that examined student preference regarding technology ^21,23-25 and obtained demographic data including gender, grade-point average (GPA), frequency of attending lectures, and difficulty of following topics that cover drug mechanisms of action. It also explored student engagement and perceptions of the e-tools used during semester 1. Students were asked whether they accessed the e-tools, their reasons if they did not, and their behavior and attitude regarding the e-tools if they did. Students also specified whether they accessed the complete set of e-tools or were more interested in accessing only those for certain drug classes, as well as how frequently they accessed the e-tools (eg, daily, weekly, and/or before assessments). Additionally, students were given an opportunity to provide their perceptions, feedback, and additional comments in their responses to an open-ended question. Student preference for technology, in general, was examined by means of a 5-point Likert scale (strongly agree, agree, no comment, disagree, and strongly disagree). Student learning styles were assessed by asking students whether they remembered words and/or pictures in responding to questions related to drug mechanisms of action.

Feedback from semester 1 indicated that students who did not access the e-tools were either unaware of their existence or had forgotten about them. Therefore, in semester 2, a different strategy was followed to motivate students to engage with the e-tools. An initial announcement was made when the e-tools were uploaded into the course Web site, and 7 follow-up reminders/announcements were made through Blackboard and e-mailed to students during the semester, usually prior to assessment deadlines. Semester 2 assessment items included a midsemester examination; 4 online quizzes that were available on Blackboard for a limited period of time, as in semester 1. The 4 quizzes were on: (1) inflammation; (2) antibiotics; (3) the endocrine system; and (4) chemotherapy. The time students had to complete each quiz ranged from 12-21 days. A final examination was administered 6 days after the last quiz deadline.

To evaluate student engagement for semester 1 and semester 2, data from the online course Web site on Blackboard were obtained, including the number of uses for each e-tool and the times and dates of access. The data were de-identified by the course coordinators before analysis. For the survey results, several quantitative analyses were undertaken. Demographic data including gender, English as first language, and student preference for technology and learning style were compared between the students who accessed the e-tools and those who did not. To determine whether the groups significantly differed in these baseline variables, t tests and chi-square tests were used. Student performance in the long-term retention questions across the 2 groups was evaluated using t tests, and the method used to recall information when answering these questions was analyzed using the chi-square test. The survey instrument evaluated participant attitudes toward the technology using a 5-point Likert scale (strongly agree, agree, no comment, disagree, and strongly disagree). To improve sample size per group, these categories were collapsed into 3 types of responses: positive, negative, and neutral. For the course Web site data, t tests were used to compare total e-tool usage in terms of number of hits between the 2 semesters. Analysis of variances (ANOVA) was undertaken to compare the differences in e-tool usage, measured as mean hits per day, in each month during the 2 semesters. The data were analysed using SPSS, version 20 (IBM Corp, Armonk NY). Bar graphs showing usage trends were created in Microsoft Excel. Significance was set at p<0.05.

Forty-three students voluntarily participated in the survey, representing 54% of the total cohort. Of those, 23 students accessed the e-tools (group 1) while 20 did not use the e-tools (group 2) during semester 1. No significant differences were found between the 2 groups in any of the demographic comparisons (Table 1). GPA was also collected from participants through the survey. All recorded values were valid and within the normal GPA range (1.0-7.0).

There was no significant difference between the 2 groups in GPA (p>0.05) or the number of students whose first language was English (p>0.05). Students were also asked to indicate whether they read through the lecture notes before attending lectures (prior lecture study), and no significant difference was seen in this variable between the groups (p>0.05). Additionally, participants were requested to rate the level of difficulty they had in understanding course content that involved drug mechanisms of action. Student responses were split between easy, neutral, and difficult, with no significant differences noted (p>0.05). Student attitude toward online learning tools was also analyzed and compared between the groups. There was a positive preference for the online learning tools regardless of whether students accessed the e-tools. However, the majority of students from both groups were either negative or neutral regarding the substitution of traditional classroom lectures with online learning tools. Student learning style was also compared between the groups. More students who used the e-tools (group 1) preferred animations to reinforce their learning, suggesting a preference for visual learning. The difference between the 2 groups was highly significant (p<0.05) (Table 1).

Feedback was also obtained from students who did not access the e-tools (group 2). The majority of students (n=15) either forgot about or were unaware of the e-tools. Other students indicated lack of time as their reason for not using additional learning materials. Finally, some students preferred to study textbooks, which they cited as more of a match for their learning style.

Analyzing student behaviors toward the e-tools showed that 70% of students who accessed the e-tools were inclined to view the complete suite. Those who viewed certain drug classes indicated that they accessed the e-tools for cardiac drugs, diuretics, antiarrhythmics, and anticoagulants. Students were more likely to access the e-tools before quizzes and examinations (Table 2).

Students included comments and feedback regarding the benefit of e-tools. Two students did not find the e-tools useful because they believed that the information was either too basic or required too much time to view. The remaining comments were positive and were classified into 3 major themes. In the first theme, students appreciated that the e-tools were visual explanation of drug mechanisms of action (7 comments relating to this aspect). In the second theme, students perceived the e-tools as helpful in furthering their understanding of the drug mechanisms of action (4 comments). Finally, students perceived that the e-tools provided additional reinforcement of the lecture materials and that reviewing them was more interesting than repeatedly reading lecture notes (9 comments).

Data from Blackboard provided more in-depth analyses for student engagement with the e-tools. Figures 1 and 2 demonstrate student access to the e-tools during semesters 1 and 2. Students accessed the e-tools in the first semester over 3 months (April, May and June). Time of access for the e-tools time was spread across the day. A similar trend was found during semester 2.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Usage of E-Tools during Semester 1

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Usage of E-Tools during Semester 2

In semester 1, most hits occurred halfway through the semester, particularly on the day before the midsemester examination. There was a steep drop in use during the following month. Use of the e-tools increased again in the study break just prior to the final examination. Analyzing data from semester 2 shows that students started to use and view the e-tools 3 days before the midsemester examination. Usage dropped for the remainder of the term and peaked 3 days before the final examination.

The timing of student access to the e-tools in semester 1 ranged from early morning until almost midnight. Spikes in the number of hits were recorded at 10:00 am and 7:00 pm. Similarly, during the second course, students viewed the e-tools mainly between 7:00 am and 11:00 pm with a small number of hits recorded at 2:00 am and 4:00 am The most popular times to access the e-tools during the second course were at 1:00 pm. and 10:00 pm.

The results showed significantly greater e-tool use during the second course compared to the first course (p<0.05). This difference in use was most apparent in the final month of the semester, with much greater usage in the second course than the first (p<0.05) (Table 3).

Table 4 shows the total usage of e-tools for each drug class covered during the first course. The highest usage (99) was recorded for the diuretic drugs group, which contained 9 e-tools. Only 2 hits were recorded for hypnotic drugs, which contained 5 e-tools (Table 4). During the second semester, the antibiotic drug class consisted of 18 e-tools and received 302 hits. The last drug class that was covered during the semester (antiemetics) recorded the lowest number of hits at 21(Table 4).

DISCUSSION

The aim of the study was to assess student engagement with the set of 148 e-tools across a period of 2 semesters in the School of Pharmacy. Participants in the e-tool user and nonuser groups had similar demographics.

Students’ attitudes toward the application of technology into their learning and teaching were also similar across the groups. In general, students from both groups (72%) were positive toward the technology. This is an expected outcome from digital-native students who were known to have positive preference toward implementing technology into their learning experiences.¹⁴ It also confirms the claim that students expect technology to be integrated into their learning experiences.^11,12 However, 45% (n=14) of students who had positive attitudes toward the application of online tools to learning and teaching did not use the e-tools. Suprisingly, 64% (n=9) of those students did not use the e-tools because they either forgot or did not know about them. Students’ various learning styles affect how they engage with traditional and new teaching methods.²⁶ Therefore, student preference toward studying using text, animation, or both was assessed in the survey. The results showed that the preponderance of students (77%) who used the e-tools preferred to study either animation or a combination of text and animation. This finding can be linked to the qualitative comments from students who stated that they found the e-tools to be a valuable visual explanation for drug mechanisms of action. Students with a visual learning style felt that they learned more easily with diagrams and pictures than with written text.²⁶ This correlates with findings that generation Y students prefer to study and learn using audiovisual materials over textual information.²⁷ This is a good illustration of the benefit of using multiple teaching methods to satisfy various student learning styles, which is in line with previous research showing that one of the advantages of e-tools is to present information to students in different ways, thereby catering to individual learning styles.²⁸ Student feedback indicated that supplementing lectures with e-tools in the Human Pharmacology courses gave them the flexibility to choose the learning method that best suited their needs. Approximately 80% of students had either negative or neutral responses regarding the replacement of traditional lectures with online-learning tools. In a 2011 study that corroborates this finding, 70% of the 359 digital native students surveyed favored attending traditional face-to-face lectures.²⁹ Students still consider face-to-face discussions with lecturers and peer interaction in the classroom to be critical to their learning success.^30,31

While the majority of participants had positive attitudes toward technology and preferences for studying both animations and text, they had either a negative or neutral attitude regarding replacing traditional lectures with those tools. Therefore, we recommend that technology-based teaching methods be used as supplements to traditional lectures.

Despite our intention to increase student engagement, we followed a teacher-focused approach in implementing the e-tools in Human Pharmacology I. With this approach, the teacher focuses on the design of the teaching materials rather than on what the students do.³² We informed the students when the e-tools were ready and uploaded online. No further action was taken to encourage students to use them. Analyzing student reasons for not accessing the e-tools during Human Pharmacology I showed that the majority either forgot or did not know about them, with only a few students indicating that they were not interested in accessing additional learning materials. This minority indicated that the lecture notes and textbook were enough support for their learning. Another study reported similar reasons for student non-engagement with technology, including lack of interest in or desire to use technology. For the minority who did not use the e-tools, as in our study, the main reasons were lack of perceived need or relevance, lack of awareness of the existence of the technology, and lack of knowledge/understanding in relation to how to use the technologies.³³

Apparently, students did not anticipate the expected benefit of using the e-tools during the first semester. Students are known to participate and engage more when they understand the importance and relevance of the learning items to the assessment tasks. Additionally, according to the phenomenography theory, the learner perspective determines what is learned and when is the suitable time to learn.¹⁹ Other researchers have suggested that lecturers need to explain the relevance of the learning tasks to the course to encourage the students to engage and follow a deep-learning approach; ie, to follow a student-focused approach to increase student engagement with the e-tools.¹⁹ The focus of this approach is on student learning and the teacher’s role is to encourage students to lead their self-directed learning and to construct their own knowledge and ideas.³⁴ This phenomenon has been explained by the expectancy-value theory of motivation, which assumes that for students to engage in any learning activity, they need to see the value of the activity toward their ultimate goal: passing the course.¹⁰ Moreover, encouraging students to be independent learners involves them taking responsibility for their learning, monitoring their progress, and seeking help appropriately.³⁵

Students failed to properly engage with the e-tools during semester 1. The assumption that digital-native students will purposely engage with technology is still under question.³⁶ Consequently, we decided to use frequent e-mails and announcements to remind students about the importance of using the e-tools during semester 2 and encourage them to benefit from this teaching approach, as a previous study demonstrated that students appreciate receiving announcements and e-mails about information related to their courses.³⁷ In the announcements, we explained the expected benefit of using the e-tools and encouraged the students to use them. This significantly increased use of the e-tools in semester 2 compared with that during semester 1, suggesting more student engagement with the e-tools.

Student behavior in using the e-tools across both semesters indicates maximum use just before assessment tasks, which raises a question regarding whether accessing the tools immediately prior to examinations is a useful strategy in terms of student learning outcomes. Arguably, students seem to be either taking a surface approach to their own learning by using the e-tools as a short-term memory aid for the examination, or they are studying hard all semester and then using the e-tools as a refresher at the end. As we were not able to measure the length of time students spent watching any given e-tool, it is not possible to draw absolute conclusions regarding the depth of student engagement with the tools.

Another important discussion point is the decreased use of e-tools as the semester progressed (Table 4). This was a clear trend in semester 1 despite that the later e-tools covered the difficult drug classes. Previous research has suggested that academics should not challenge students at the start of their courses but rather focus on introducing them to the environment and challenging them in the final stages of the course.³⁸ Therefore, the Human Pharmacology lecturers structured the courses and started from simple modules to build student knowledge, progressing to more complicated and complex modules. Students may have felt tired as the semester progressed and did not have the energy to engage in the later, more difficult topics. Another possible explanation is that students were following a strategic approach in their study, focusing on the easier topics with the hope of getting easy marks rather than spending more time on hard subjects.³⁹ This is an expected behavior from university students who are purposeful learners.^19,40 Further investigation is needed to better understand this behavior. However, use of e-tools for the latest drug classes in semester 2 was better than that in semester 1, suggesting that frequent reminders to students are important to keep them engaged during the busy time of assessments and examinations.

Students’ qualitative comments indicated their positive preference toward the e-tools, with a few students giving extremely positive feedback. This finding concurs with evidence from other studies that found positive student attitudes toward the implementation of technology into their learning.^21,23-25,41 Students also commented that the e-tools were an interesting additional resource for studying. These comments align with the findings of the EDUCASE Center for Applied Research study, in which 70% of 36,950 students found that technology makes learning the content in their courses more convenient.⁴²

The purpose of this study was to examine student engagement with e-tools over a period of 2 semesters. What we discovered was that the addition of e-tools (or any other resource) alone did not lead to increased student engagement. A student-focused approach, as in the second semester, is needed to improve student acceptance of and engagement with the e-tools. A limitation of this study is that student performance in the final examinations in semesters 1 and 2 was not measured. Other limitations include the small sample size and the potential for nonrespondent and self-reporting bias.

CONCLUSION

Pharmacy students enrolled in a Human Pharmacology course series valued the addition of technology-based teaching strategies as a supplement to classroom teaching methods. However, the development of online teaching and learning resources is ineffective in increasing student engagement unless supported with frequent reminders and encouragement. The provision of online teaching and learning resources were only effective in increasing student engagement after the implementation of a “marketing strategy” comprising e-mail reminders and motivation.