A Reflective Teaching Challenge to Motivate Educational Innovation

INTRODUCTION

Pharmacy faculty members are accountable for developing, delivering, and improving classroom and experiential curricula using a variety of methods, including active learning and assessments that are valid and reliable indicators of student performance. Accreditation Council for Pharmacy Education (ACPE) Standards 9 through 15 ensure that this effort meets the professional requirements for the practice of pharmacy in the 21^st century.¹ Standard 26 emphasizes that faculty members continue to advance their skills and excel in their academic responsibilities, all within the environment of rapidly evolving teaching and healthcare delivery circumstances. Faculty members are also faced with growing expectations to use student-centered instructional methods that improve motivation and retention² as well as adapt to evolving demands of an increasingly digitally oriented student body.³ Such demands may promote innovation by providing new tools for educational experiences. However, they could unintentionally reduce the chances for innovation by leaving little room for experimentation and reflection because faculty members have little time to consider how to use the new tools optimally rather than use them to replicate what is already occurring (eg, record a lecture and make it available online vs revising the material to engage students via short video clips and interactive media tools). Pressure to simultaneously cover required topics, use active-learning techniques, and adopt digital tools can be overwhelming. The willingness to experiment and innovate may be further hindered by the competing teaching, scholarship, and administrative responsibilities many faculty members have.

In 2010, the Department of Pharmacy Practice at Northeastern University School of Pharmacy completed a strategic planning exercise that identified priorities related to ACPE Standards 9 through 15 and 26, including 1 to evaluate and identify preferred teaching methods while working to design courses that encouraged students’ accountability for their own learning. The department curriculum task force assigned to this priority focused on strategies to develop faculty members as educators, realizing that the varied backgrounds, interests, professional trajectories, subject areas taught, and settings represented in the department posed challenges for identifying faculty development topics, especially given the wide range of years faculty members had spent teaching (1 to 40 years). Although the substantial diversity among faculty members posed challenges in terms of what types of faculty development programs were needed, if this diversity were harnessed, it could result in content that would enrich the experiences of all faculty members. We wanted to challenge faculty members to try something new that could improve teaching and learning in order to encourage innovation in a way that was applicable to all faculty members, regardless of their backgrounds and create synergy for faculty development from the individual experiences.

DESIGN

The faculty development activity embodied 3 components: diffusion of innovation theory, decomposed theory of planned behavior, and evidence-based insights for effective student learning. The diffusion of innovation theory inspired our effort. Everett Rogers defined diffusion as the process by which an innovation is communicated through certain channels over time among the members of a social system.⁴ It has been used for decades to study the adoption of a wide range of behaviors and programs. Others have applied it to the adoption of technology in teaching.⁵ The theory describes 5 attributes of an innovation that affect its adoption: relative advantage, compatibility, complexity, trialability, and observability. The theory addresses communication channels for sharing the innovation and that people rely on “near peers” for evaluation of information that we have strong feelings about; however, mass media channels influence responses to innovation that we are neutral about. The “effects over time” and “impacts of the social system” elements of the theory address factors associated with the adoption process (knowledge, persuasion, decision, implementation, and confirmation) and the norms associated with innovation adoption.

We also sought to create a faculty development activity that was consistent with the decomposed theory of planned behavior.⁶ This theory identified the following components leading to behavioral intention: attitudes based on perceived usefulness; perceived ease of use and compatibility; subjective norms based on student, peer, and superiors’ influences; perceived behavioral control based on self-efficacy; and facilitating conditions. These factors affect adoption of Web 2.0 technologies and the biggest obstacle to the application of technology in teaching has been the faculties’ reluctance to use it. These concepts are also consistent with numerous theories regarding adults’ (and children’s) preferences for self-directed learning. Others have applied these concepts to predicting faculty members’ attitudes towards, interest in, and use of Web 2.0 capabilities.⁷

The remaining component that we wanted to address was how to help faculty members connect with student experiences that were potentially less comfortable and familiar to them. We used the learning principles outlined in “How Learning Works: Seven Research Based Principles for Smart Teaching” as a foundation for our work.⁸ It offered advancements in applying the science of learning to education to help faculty members see how evidence-based findings could be applied to improve teaching effectiveness. The 7 principles are: (1) students’ prior knowledge can help or hinder learning; (2) how students organize knowledge influences how they learn and apply what they know; (3) students’ motivation determines, directs, and sustains what they do to learn; (4) to develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned; (5) goal-directed practice coupled with targeted feedback enhances the quality of students’ learning; (6) students’ current level of development interacts with the social, emotional, and intellectual climate of the course to impact learning; and (7) to become self-directed learners, students must learn to monitor and adjust their approaches to learning. Dr. Ambrose, vice provost for teaching and learning at our institution, has conducted workshops on campus and faculty members have numerous materials available to them on these principles.

The 2 theories and 7 principles reinforced each other and guided us in the creation of a faculty development activity with the following characteristics: (1) personalized to account for the different starting places of any 1 person to improve its compatibility with faculty members’ comfort level and the extent to which it can be tried/tested relative to their prior experience; (2) use of “near peers” to share experiences in a safe environment with voluntary presentations at department meetings and encouragement for the sharing of failures as well as successes; (3) trialability with the requirement that something must be tried while anything could be tried, with allowance for use of small-scale, low-risk activities; trying something was critical to gaining a better understanding of the process and reflecting about its value or lack thereof; (4) principles derived from evidence-based teaching. By coupling the challenge with evidence-based principles that were not tied to a specific strategy, a faculty member could see how innovations in teaching and use of technologies could be aligned with relevant, proven concepts independent of the innovative strategies themselves; (5) a systematic reflection on the experience to help bring forward faculty members; underlying attitudes, norms, and perceived behavioral control. The purpose of this study was to describe the New-for-You Reflective Teaching Challenge and evaluate results from the first year of its implementation in which we conducted curriculum improvement and faculty development under the circumstances routinely faced by faculty in colleges and schools of pharmacy.

EVALUATION AND ASSESMENT

Two Web-based survey instruments were used to evaluate this initiative. The first survey instrument was designed and deployed at the start of the New-for-You Reflective Teaching Challenge in January 2013 to capture a baseline profile of the teaching methods faculty members were using. The survey instrument contained examples of teaching techniques obtained from the literature and assessed the type and frequency of use during the 2012 calendar year. These data were used to establish the breadth of teaching strategies used by the department prior to the challenge, as well as to give faculty members an opportunity to reflect on the strategies that they had used. The survey instrument also asked respondents to identify several strategies they were considering trialing in the 2013 challenge.

The second online survey instrument was used to gather data regarding challenge participation, specific new-to-you strategy or strategies used during 2013, and information about the setting in which the trials were conducted. Faculty demographic data, including generational data, were collected to ascertain any potential descriptive subgroup differences in the use of different types of strategies, including educational technology-based experiences. This post-challenge survey instrument also included questions to assess perceived effectiveness of the newly tried educational strategy and motivating factors behind participation in the challenge and technique selection, and provided a forum for each faculty participant to reflect on lessons learned.

The reflective component was a critical part of the challenge, as we sought to facilitate a careful review of the experimentation and potentially achieve a metacognitive educational experience by asking faculty members to share and discuss their experiences with their peers at faculty meetings.⁹ To lead each participant through the reflective process, the online post-challenge survey instrument asked faculty members to describe what activity was new for them and its role in teaching, which educational principles were linked to their activities, what happened overall, what were the implications of incorporating the new technique in the learning arena, and whether they would continue to use the activity as-is or with modifications in the future.

Additional questions solicited faculty member perceptions of the value of the brief reflection sessions at department meetings. No cost information was collected as we did not anticipate additional costs to be incurred beyond time spent. Survey logic redirected faculty members who did not participate in the challenge to questions asking for reasons for nonparticipation, and intent and motivation to participate in the future. This post-challenge survey instrument was administered in November-December 2013. Study investigators with educational and assessment expertise developed and pilot-tested both survey instruments.

Descriptive statistics were used to summarize the results of both surveys. In the second survey, 1 of the study investigators qualitatively reviewed all open-ended questions in the reflective component of the survey and summarized major themes identified. The Fisher exact test was used to specifically compare flipped classroom use. The analysis describes how faculty members met the challenge and identifies patterns associated with implementation. Sharing of the data was approved by the Northeastern University Institutional Review Board.

Thirty-one members of the department were employed by the university for the entire 2013 calendar year and had assigned teaching responsibilities. Five were in co-funded positions and 11 were tenured or tenure-track. Five specialized in the area of social and administrative sciences and the remaining department faculty members specialized in pharmacy practice. The mean number of years in academia was 13.7 (median 12, range 1 to 40). Sixteen participants (52%) completed the initial baseline assessment survey instrument, which captured the frequency of use of a variety of teaching techniques during the 2012 calendar year (Figure 1). The most frequently used classroom techniques were traditional lecture, the Blackboard Learning Management System (Blackboard Inc, Washington, DC) discussion board feature, and audience response software (clickers). In the experiential setting, the most frequently used techniques were peer teaching, case discussion, and team-based learning.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Frequency of various teaching techniques used by faculty members in classroom and experiential settings in 2012 baseline survey instrument.

Twenty-eight (90%) faculty members completed the post-challenge survey instrument, including the reflection. Of the 28 who completed the survey instrument, 25 completed the challenge (81%), while 3 (19%) did not. Forty percent of respondents who completed the challenge identified themselves as members of the baby boomer generation (1946 to 1964), 40% were members of Generation X ( 1965 to 1979), and 20% were from Generation Y (1980 to 1999). When responses were separated into groups based on self-identified generations, we found no significant differences in strategies selected based on the descriptive data comparison, but more Generation X participants (n=4) tried a flipped classroom strategy (p>0.5 for comparison with Generation Y (n=0) and baby boomer (n=1) generation groups using the Fisher Exact Test).

The majority (67%) of participants reported trying 1 new strategy, while 26% tried 2, and 6% tried 3 new strategies during 2013. When asked to rank the factors that motivated them, the top 3 ranked factors were “to improve student engagement” (17 times [68%]), “to improve student learning” (15 times [60%]), and “to improve teaching” (15 times [60%]). The motivating factor that was least frequently cited on the survey instrument was related to assessment of student achievement relative to learning outcomes (4 times [16%]). When selected, it was the third most relevant choice for implementation of an innovative teaching method. Only 4 respondents ranked “to meet 2013 challenge” in their top 3 motivating factors. The final choice, “to improve patient care,” was only selected 5 times (20%).

Nineteen faculty members (56%) tried their 34 new strategies most commonly in face-to-face classes, 10 (29%) tried them in experiential settings (APPEs and service learning), and 3 (9%) tried them in laboratories/seminars. The face-to-face class size was >50 in 83% of reports and <19 in 18%, and the class was required in 72% of reports. Table 1 reports the types of new-to-you strategies reported in the post-challenge survey instrument. Themes discovered from qualitative review of the survey responses included frequent use of flipped classroom strategies (or related activities), novel presentation software, addition of video media, and use of social media. As a group, faculty reported applying each of the 7 principles of research-based teaching. The most frequently reported principle was “how students organize knowledge influences how they learn and apply what they know” (14 [56%]). Other principles cited were “to develop mastery, students must acquire component skills, practice integrating them and know when to apply what they learned” (13 [52%]) and “students’ motivation determines, directs and sustains what they do to learn” (12 [48%]).

The 25 respondents who completed the challenge also answered the reflective questions. When asked how the success of the new technique was evaluated, 20% reviewed student-learning outcomes, 24% obtained peer feedback, and the remainder reviewed student evaluations or university-administered course survey instruments. When asked about the implications of the challenge on their faculty development (respondents could check all that apply), 60% reported that they explored the pedagogy of the new method, 68% learned the operation of the new method, and 84% reported learning the advantages and disadvantages of the new technique. Ninety-two percent of faculty members reported that the technique met their intended goals. Ninety-six percent stated that participation in the challenge was worth the effort and that they planned to participate in 2014. Forty-four percent of respondents planned to continue to use their new technique without modifications and 56% with some change. For a majority of respondents (83%), the average time spent on the challenge, including background research and implementation, was 9 hours. The remaining faculty members indicated that their respective novel strategies were developed and implemented over the course of a year, so average time spent could not be included as no specific estimate was provided.

The 3 faculty members who responded to the post-survey instrument but did not participate in the challenge cited that they were too busy with other responsibilities, and 1 cited lack of appropriate faculty development. All were motivated by other faculty members’ presentations in department meetings and indicated they would participate in the next year’s challenge.

Nineteen (76%) respondents attended at least 1 of the brief presentations by a colleague describing a new-to-you strategy in departmental meetings. Eighteen (95%) respondents strongly agreed or agreed that the presentations were interesting and 12 (63%) strongly agreed or agreed that they found a strategy that they planned to try in the future. Of those who attended their colleagues’ presentations, 6 (32%) reported developing a new idea for a subsequent new-to-you challenge based on these presentations.

DISCUSSION

The New-for-You Reflective Teaching Challenge accomplished its goal of motivating most of the participants to explore methods that were new to them in a variety of settings. However, faculty members in our department had trialed educational technologies even before the challenge (Figure 1). Participants were at different starting places in terms of adopting new techniques as evidenced by the diverse activities in Table 1, which represented a cross-section of faculty teaching in the department.

Faculty willingness to require nominal documentation of participation in the challenge as part of an annual performance evaluation, combined with the high survey response rate on the post-challenge survey, suggests a culture of commitment to improving teaching in the department. This culture has evolved over the past decade, with adoption of mandatory, formative, peer-faculty teaching assessments,¹⁰ and faculty opinion leaders adopting various teaching strategies and demonstrating what is possible while acknowledging the shortfalls.^11-14 The finding that 26% of respondents tried more than 1 strategy also provided evidence of this continuous teaching quality improvement culture and the overall perceived value of the challenge. Interestingly, 100% of respondents voluntarily disclosed their names on the post-challenge survey instrument, indicating a culture of openness to continuously improve that was consistent with total quality management principles.¹⁵ Lastly, the culture of assessment was also evident in that all faculty members assessed the technique in some form upon completion of the learning session. The responses to the baseline survey instrument also suggested a culture of quality improvement as faculty had already been adopting a range of techniques before we initiated the challenge.

When reviewing the faculty responses, it was noted that 75% of all teaching innovations required the use of technology in one form or another. However, the specific innovations used varied in nature, with 13 separate categories represented. The most frequently cited categories of innovative techniques were the use of flipped classrooms, the use of some method of social media, and the use of a new assignment or project. Additionally, approximately half of all respondents indicated that their teaching strategies were successfully implemented and did not require any future modifications. Faculty members were comfortable exploring new teaching methodologies and were generally successful in their approaches during the challenge.

The challenge enabled most faculty members to stretch themselves as what would be expected by the theories and evidence that guided the design and implementation of the challenge. The importance of faculty members’ feeling comfortable, regardless of the starting place, was an essential component to having their teaching methods evolve. That aspect also contributed to the value of sharing by “near peers” that occurred during faculty meetings. Faculty members were forthcoming about what did not work as planned and these outcomes were received in the mindset of continuous quality improvement, which contributed to the success of the program. The fact that 63% heard about a technique that they would use in the future during a brief presentation supported the importance of the sharing component in the implementation of the challenge. The sharing as well as mandatory reporting of participation on merit documentation also created a sense of accountability among department faculty members.

Limitations of this study included the self-reported nature of all data with no independent validation of what occurred. Additionally, it was unclear whether these results would be generalizable as elements of the social system context enabled the challenge to occur. While some faculty development activities about various topics, including course assessment and transitioning courses to the online setting occurred in 2013, they were not explicitly linked to the challenge. The department did not provide proactive faculty development related to the 7 principles. We provided no definitions or classifications for the techniques included in baseline or post-challenge survey instruments and faculty members could have interpreted names of activities and strategies differently. Lastly, the online survey instrument was tested by the coauthors but not by others, and no reliability-related or validity-related studies were performed.

While there may not have been direct outcomes data to support the impact of the new-to-you approaches identified through the challenge, the positive anecdotal feedback indicated that they benefited both the faculty members and their students. The department agreed to continue the new-to-you challenge for the next academic year with the goal that faculty members will continue to seek out innovative approaches and integrate them into their teaching activities. While faculty members may run into limitations with identifying new approaches to teaching, the possibilities for continued quality improvement are endless. Department task forces will use our study findings to help guide future faculty development offerings to ensure that faculty members feel confident and competent to implement any of the described techniques.

SUMMARY

The New-for-You Reflective Teaching Challenge provided a method for faculty members in our department to learn, implement, integrate, and evaluate new and different teaching strategies into their classroom and experiential offerings. These various teaching approaches in differing settings had a high level of practical application and strengthened links between evidence-based educational principles and day-to-day classroom activities in a practical context. Not only did a number of faculty members move outside of their previous comfort zones in their educational approaches, students experienced new learning methods that they may not have otherwise encountered. The new-to-you design placed accomplishing the challenge within reach of each faculty member, regardless of background, subject, or experience in academia. It also allowed the department to meet the goals of curricular improvement, faculty development, and student-centered instruction concurrently.