Qualitative Analysis of Written Reflections during a Teaching Certificate Program

INTRODUCTION

Reflection is an intentional, dynamic process that allows improvement in one’s actions, abilities, and knowledge by learning from past experiences.^1-4 While this process can be useful in almost all aspects of life, reflection in the workplace can be particularly beneficial. Successful professionals must be able to reflect on their experiences in order to find solutions to complex problems encountered on a daily basis.^1-2 Such reflection is not only necessary for pharmacists and other health care professionals to improve their practice, but also to further hone their expertise as educators.⁵ Reflection should be used by pharmacists committed to professional growth as lifelong learners.⁶

The practice of reflection during residency programs offers a valuable opportunity to observe and guide residents in this process at the beginning of their careers. Teaching certificate programs within pharmacy residency programs were founded on the idea that having specialized pharmacy knowledge does not necessarily equate with being an excellent teacher.⁷ Teaching certificate programs give participants general pedagogical knowledge to combine with their existing content knowledge. Participants complete didactic and experiential teaching activities to develop such teaching skills. A critical component of the teaching certificate program at the University of Arkansas for Medical Sciences (UAMS) College of Pharmacy is the extensive reflection required throughout the year-long program. Because reflection is such an integral part of professional development, evaluation of the content and extent of participants’ reflections is imperative.

Research on the topic of reflection is extensive, but investigation of reflection on teaching by pharmacy faculty members or faculty members in training is not described in the literature. Additionally, methods to assess teaching certificate programs focus on surveys but lack the details offered by more in-depth analysis. Qualitative analysis of reflective essays could provide better understanding of program benefits and participant growth. Our mixed-methods evaluation is the first to examine thematic composition of reflections as well as the extent of reflection evidenced in the written essays of potential future pharmacy faculty members as they participate in teaching certificate programs.

DISCUSSION

Reflection is vital in the life of an educator.¹¹ Even more, reflection is the key to learning, which occurs when we create meaning from a past event and use this to shape future experiences.^4,12 While other disciplines such as education have been using reflective practices for some time, the health sciences professions have more recently adopted this concept in the training of future health care professionals. Medicine, nursing, and pharmacy are among the disciplines that are adapting these types of reflective processes in curricula to aid in learning as well as improve patient care.^13-21

The extent of medical residents' reflection has been explored ^13-15 and showed that physicians' decision-making skills were improved in complex clinical cases if they were able to critically reflect on those experiences.¹⁵ An evaluation of nursing students concluded that interview sessions (individual, paired, and group) on reflective practice were viewed as beneficial for the participating students and encouraged them to practice reflective thinking on their own.¹⁷ Like many other professions over the past few decades, accreditation standards for pharmacy education require student reflection and subsequent assessment of these skills.^18-20

Additionally, the American Society of Health-System Pharmacists Required Competency Areas, Goals, and Objectives, for Postgraduate Year One Pharmacy Residencies address the need for residents to reflect on their personal performance and professional development.²¹ Published works on reflection in pharmacy center on pharmacy students, but the literature suggests reflection is necessary in the day-to-day practice of a pharmacist as well.^22,23 However, little is known about the reflective abilities of pharmacy educators and how these skills affect teaching-related self-development.

Assignments in this teaching certificate program served as tools to facilitate and encourage reflective thinking in order to promote growth in teaching abilities, and participants achieved high levels of reflection when completing these assignments. Almost 50% of essays evidenced critical reflection by describing how change occurred in their practice of teaching (behavior change). This high level is rarely achieved by so many.¹⁰ This could be because participants in this program were highly achievement-oriented as supported by their admission to and completion of pharmacy school as well as pursuit of residency training.

Additionally, preceptors in the program were seeking teaching development and voluntarily participated. The teaching certificate program also emphasized reflection throughout the program. In addition to the reflections reviewed in this evaluation, participants were asked to reflect immediately after each teaching activity; this practice of reflective thinking and writing may have facilitated deeper reflection in the more summative midpoint and final essays. Another factor that could have contributed to the high levels of reflection was the mentorship of program faculty members, who reviewed reflection drafts and further prompted participants to consider their experiences, personal characteristics, and teaching development goals.

When analyzing themes discussed by participants, it was evident that they considered the reflection prompts when writing their essays because strengths, weaknesses, assessment, and feedback all appeared often. A closer look at the references to these categories revealed that participants wrote most frequently about the areas of assessment and feedback. These areas stood out because they were the areas in which participants lacked confidence early in the teaching certificate program, but they were also the areas in which participants recognized the most growth at the conclusion of the program. Collectively, the extent of reflection significantly increased from the midpoint essays to the final essays in the feedback category.

There was no intervention on reflective writing between the midpoint and final essays, so the change evidenced by our evaluation seems to have occurred naturally. Participants gained more practice with these skills over the course of the program year, and they subsequently reflected more and explained their growth in this area. This finding of deeper reflection in the area of feedback validated the design of the UAMS teaching certificate program and its focus on self-reflection as a method of teaching skills development.

The area discussed by all participants in the final essays was their confidence and comfort in teaching. This linked nicely with the significant increase in the extent of reflection seen in the strengths category. Participants seemed to be more deeply aware of their strengths as the program progressed, and they discussed areas of perceived confidence in these essays. This is consistent with our previous findings that demonstrated increased self-perceived teaching abilities during the program.⁹

Because the teaching certificate program provides the opportunity for gaining practical experience in teaching, confidence in these abilities is expected to grow. Gaining confidence can allow a deeper awareness of personal strengths. Growth was evidenced as experiences led to increased confidence, and prompted reflection on these experiences led to gains in self-perceived teaching abilities and strengths. Increased confidence through experiences was the aim of the teaching certificate program and explains why such programs are beneficial to participants, future employers, and academic institutions.

Participants most often solicited feedback from students and faculty members. Feedback from peers and patients were reported much less frequently. Because it is a teaching certificate program requirement for participants to receive feedback from faculty members, peers, and students, it is not surprising that these perspectives were discussed in the reflections. However, patient teaching also is encouraged by the program, and the perspective and feedback of patients should be valued. Although the number of participants soliciting feedback from patients was not collected in our evaluation, few discussed these perspectives in their reflections.

Twenty participants (30%) listed patient counseling as a teaching activity, but only three participants discussed receiving teaching evaluations from patients. This could be because this information was not solicited from participants and, therefore, they do not reflect on it specifically or because the participants perceived patient evaluations to be lower quality or less importance than those from faculty members or peers. Additionally, participants might have viewed the patient as a different type of learner than students and not seen the need for reflection on their evaluations. Curricular emphasis of patients as learners increased in later program years to encourage variation of feedback from learners. To foster the view of patients as learners, programs should consider requiring patient teaching activities along with assessment of patient learning and completion of teaching evaluations by patients.

It is also noteworthy that participants wrote more frequently about receiving feedback from students and faculty members and less frequently about feedback received from peers. This corresponds to the small number of participants who reported giving feedback to faculty members (n=1) and peers (n=4). Giving and receiving feedback from peers may be an area of discomfort for participants despite the teaching certificate program requirement to give and receive this type of feedback. Additionally, only one participant reflected on the program requirement to provide feedback to at least two faculty members after observing their teaching. Overall, reflection on giving feedback to peers and faculty members was lacking, and participants did not reflect on receiving feedback from all sources.

The extent to which feedback was sought is not known, so additional research is needed to clarify this finding. Many factors could contribute to this pattern, including the perceived need to obtain feedback from certain groups, the feasibility and convenience of sampling, the perceived relative importance of feedback from more experienced or more educated groups, or the comfort level of the participants to give feedback to and receive it from certain groups. Teaching certificate programs should consider increased emphasis on 360-degree evaluations of teaching and both giving and receiving feedback to help participants understand the necessity and value of feedback from others. If participants gain comfort in this activity, they could provide higher quality feedback to others and potentially could benefit more from the feedback they receive.

Participants were not prompted to write about opportunities they did not get to experience; therefore, it can be assumed that participants discussing this theme must have been expecting to participate in such activities. By identifying these patterns, program directors can get an idea of skills that some participants might want to gain from the program. As expected, more than 70% of the themes coded for no opportunity were expressed in the midpoint essays. Of the remaining nine items discussed in the final essays, grading students and writing test questions accounted for the most themes discussed. Our program has taken these comments into consideration, incorporating additional opportunities for examination item review sessions to give participants insight on how to develop and evaluate examination items.

Qualitative evaluation of participant reflections can provide feedback on teaching certificate program effectiveness. These reflections provide a detailed view of how participants develop teaching skills throughout the program and impart realizations of the effectiveness of the program. By using reflective essays as quality indicators, program directors can shape program content to better develop teachers. At the same time, the reflection itself can aid participants in the development of a well-informed, highly individualized written statement of teaching philosophy, another potential quality measure of the teaching certificate program. This research did not evaluate participants’ teaching philosophy statements. However, qualitative evaluations of these documents also could provide a proxy of the teaching certificate program’s effectiveness in increasing participants’ awareness of their own teaching style and ideals.

Although our qualitative evaluation was rigorous and followed the guidelines presented by Anderson,²⁴ care must be taken when generalizing the results to other programs. Our data represent the products of a single teaching certificate program and are specific to the participants and experiences of this program. The characteristics of the residents and preceptors completing the program are varied, but they may reflect these groups at other institutions.

The use of quantitative and qualitative measures to evaluate this teaching certificate program is a strength of the study, and additional analyses are needed to determine if participants’ reflective abilities remain constant over time and predict future teaching performance. Finally, because of the extensive nature of our evaluation and the volume of the qualitative data, the potential exists for coding errors and inconsistencies between coders. Several mechanisms were in place to limit these inaccuracies and preserve the integrity of the data.