A Model for Assessing Reflective Practices in Pharmacy Education

INTRODUCTION

Learning never ceases in a professional career, and the pharmacy profession is no exception. The introduction of new medications and recognition of new and complex medication interactions along with diagnostic and technical developments in pharmacy is ongoing. Clinical decision-making is integral to pharmacy education, and reflective thinking may lead to better informed clinical decisions.¹ Critical thinking,^2-5 problem solving,^6-8 self-directed learning,¹ and skills to overcome communication barriers⁹ are essential skills for continuous evaluation of professional practice.¹⁰ Furthermore, increased public concerns over medication errors highlight the importance of critical thinking and decision making.¹¹ It is with reflective thinking that these skills can be enhanced.

Reflective-thinking skills need to be a focus in pharmacy education so graduates have the ability to view clinical situations from different angles to gain new insights and perspectives. It is with reflection on practice that we challenge our own firmly held beliefs and assumptions.¹ This process is thought to underpin informed judgments and clinical decision-making. Thus, integrating these processes in pharmacy education can better prepare the pharmacy student for the intricacies of future clinical practice. In addition to integrating reflective thinking into pharmacy curricula, using sound methods of assessing students’ reflective-thinking skills is also important. Yet, there is a significant gap in the literature regarding the use of reflective rubrics to assess levels of reflecting thinking in pharmacy education. Furthermore, there have been limited provisions for the development of and research on the reliability of such rubrics in this field. This paper reviews the literature to date surrounding assessment strategies employed in health professions education, with the goal of identifying a model to assess reflective practices in pharmacy education.

Reflective practices involve multifaceted processes and form the foundation for deep learning from previous experiences.^1,12 Reflective and critical reflective thinking can be used as tools to promote the retention of knowledge one is exposed to in a traditional didactic classroom setting. For example, students do not simply acquire new information and add it to their memory rather, they integrate and deduce meaning by relating their new found knowledge with prior experiences.¹³ Previously, learning was focused narrowly on purely knowledge gained¹³ However, learning involves a number of complex processes, which may include reflection and “draws on a reservoir of accumulated experience as a rich resource of learning.”¹³ Therefore, if reflective practices assist with deeper learning, better judgments, and clinical decision-making, integration of reflective-practice activities into curricula may benefit future pharmacists. For reflective practices to be incorporated into curricula, appropriate assessment strategies must also be addressed.

METHODS

Four researchers conducted a simple systematic review using a 5-step approach¹⁴ (Figure 1). Relevant articles for this review were obtained through searches of: Scopus; ERIC (Educational Resources Information Center); PubMed; Medline via OvidSP; and PsychINFO via OvidSP. Keywords included reflective practice, assessing reflective writing, reflective rubric(s), and reflective learning. The inclusion criteria were publications relating to health disciplines (pharmacy, medicine, nursing, dentistry, and allied health sciences) in the higher education field. English, peer-reviewed journal articles published in the last 20 years were considered. Journal articles that specifically identified strategies for assessing reflective activities were included. Articles meeting inclusion criteria were further screened by reading the abstract.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Simple Systematic Review Using a 5-step Approach¹⁴

RESULTS

The extensive literature review revealed a number of outcomes. First, assessing reflective practices is complex. Second, despite the complexity of this assessment, there are benefits to the process. Third, assessment strategies such as rubrics (to measure levels of reflection) have been derived from theories of seminal educators of reflective practices and can be tailored for use in pharmacy education. Finally, there are considerations required by pharmacy educators when introducing assessment strategies to assess levels of reflection.

Fifty-three published papers reported the use of specific reflective tools designed to enhance the reflective process (Table 1). Twenty-six papers reported either proposing or developing a reflective rubric for specific health disciplines (Table 2). Although the literature supports the use of reflective tools in pharmacy education,^1,10,15-24 only 2 studies provided a reflective rubric. One of these studies involved pharmacy students, the other involved pharmacy interns. The study of pharmacy interns reported on the use of a model by Kember et al, which evaluated the level of reflection evident in reflective essays.¹⁰ The study of pharmacy students used videos and portfolios as the platform for assessment with 4 levels of categorization for reflection.²⁴

THE COMPLEX PROCESS OF ASSESSING REFLECTIVE PRACTICES

Assessment of reflective practice skills is complex.^16,25,26 This complexity is a result of researchers’ multiple interpretations regarding the conceptual meanings of “reflection.”^1,27,28 Despite this obstacle, assessment of reflective practices is an important consideration for pharmacy educators. The ability to reflect on experiences can advance practice by developing critical-thinking and problem-solving skills to deal with convolutions of clinical practice.^1,5 As reflective practice often draws from previous experience, the act of reflecting on a critical incident may induce personal and emotional responses. For example, reflecting on an incident can bring to the forefront personal feelings, thoughts, and responses as such reflection may trigger emotions from that event or from previous experiences, thus the assessment manner may be considered a subjective process. Although reflecting on an incident may trigger subjective elements such as emotions, feelings, and deep thoughts, the process of assessing reflection is objective, the primary reason being that reflection on a critical incident is a learning process concerned not only with content of an experience but also with how one processes and learns from that experience.²⁵ Therefore, the process rather than the content of the experience can be assessed objectively.

There are many benefits of assessing reflective practices. Research shows that assessment tends to drive student learning as students view assessment marks (grades) as a critical outcome to achieve a level of attainment in a course.^29-31 Thus, students will often devote a greater amount of effort to assessed tasks compared to those that are not.²⁵ In addition, evidence shows students will adopt a certain approach to learning depending on the type of assessment tasks set by educators.²⁹ Furthermore, assessment in higher education has a direct effect on student learning.³² Therefore, assessment of reflective practice requires careful thought and consideration in pharmacy education. Carefully constructed assessment strategies of reflective practices may enhance student engagement in a task and, in turn, student learning—the ultimate goal for educators.^29,33,34

HOW REFLECTION LEVELS HAVE BEEN ASSESSED

Several models for assessing reflective practices were identified as suitable for use in pharmacy education. Assessment strategies designed to measure reflective-thinking ability have been derived from seminal educators in the field. Conceptual frameworks that underpin reflective practice have been derived from Dewey,^35,36 Schon,³⁷ Boud et al,³⁸ and Mezirow.^39,40 Each played a significant role in the evolution of the theory of reflection.¹ All educators agree on the complexity of assessing reflection process. For example, Dewey argues that “reflective thought’ is an important part of the learning process, yet implies that it is difficult to measure because reflection implies something is believed.³⁵

The models we examined help to evaluate reflection levels in health professions students. Their strengths and weaknesses depend on the type of reflective tool used and what stage of the student’s education is being assessed. For example, Schon’s model emphasized the importance of measuring reflection in and after practice,³⁷ and his construct is used as an instrument for measuring reflective components in journals.²⁶ However, this model primarily assesses reflection levels during and after clinical practice. Therefore, it may not be as useful a model for the novice pharmacy student to assess levels of reflection, particularly during the earlier pharmacy education years. Conversely, this model would be beneficial for assessing levels of reflection during clinical placements and in early or established clinical careers.

Boud et al used 7 elements of the reflective process noted in journals and blogs.^1,5,41,42 Their model has been at the forefront of educational literature for the last 30 years. This model’s strength is its depth of reflection, shown as 7 stages. The stages may not proceed in a linear progression, nor are they independent of each other. Rather, the process of reflection involves a continual movement back and forth between the stages and, as a result, sometimes, stages are omitted.³⁸

Mezirow’s model describes assessing levels of reflection in 3 distinct categories: nonreflector, reflector, and critical reflector.³⁹ In this model, reflective action relates to reflection and critical reflection and nonreflective action relates to habitual action, thoughtful action, or understanding and introspection. His model has been used to measure reflection primarily with journals and blogs.^4,26 If paired with Boud’s model, a deeper, conceptual framework can be used.^4,5,41,42

Two studies examined registered nurses’ level of reflection using reflective journals.^41,42 Both studies used Boud et al’s conceptual framework and Mezirow’s models, identifying a 2-stage process of reflection. The first process utilizes the stages of reflection,³⁸ followed by a second process which identifies the categories of reflection.³⁹ Furthermore, the stages of reflection identifies reflective and critically reflective processes: Stages 2 through to 4 are considered reflective and Stages 5 through to 7 as critical reflective.^41,42 Reflective thinking involves looking from different perspectives and arriving at new insights and understandings. Critical reflective thinking goes beyond reflective thinking and occurs when a person considers one’s firmly held beliefs, attitudes, approaches, and biases and discovers faults they previously believed to be correct.⁴³ Thus, behavior is changed to improve future outcomes.¹ Another study in dental hygiene students also used the combination of Boud et al’s and Mezirow’s framework to form a rubric. However, these researchers used a 6x4 matrix, rather than a 2-stage process of reflection.⁵

While these studies combined the 2 frameworks, we propose a different model. In our version, Stage 1 (Returning to experience) is an important part of the reflective process. We also take into account Boud et al’s explanation that the process of reflection may move between stages and some stages may be omitted. Considering that idea, we propose a matrix rubric, mirroring the matrix rubric used in dental education,⁵ so that it is possible for students to be evaluated as either nonreflector, reflector, or critical reflector at different stages. One variation in our matrix from that used in dental education is a 7x3 rubric that identifies the 7 stages of the reflective process and the 3 categories of reflection.

The strength of our reflective rubric, as opposed to the 2-stage process models, is that ours includes both depth and quality of the type of reflection. For example, a student can be assessed on this rubric at different stages of reflection. For example, a student may be assessed as a critical reflector at Stage 3 but may also be considered a nonreflector at Stage 4. This conceptual framework provides a more sophisticated level of measurement accounting for not only a level of reflection, but also depth and strength of critical reflection. It also accounts for different stages of reflection and how well a student adapts to these stages. A more precise measurement can be derived from this distinction. Furthermore, pairing of these 2 models yields more precise assessment of levels of reflection reached at different stages in a reflective writing task. The rubric can, therefore, be an effective tool during feedback sessions between an educator and student, as it allows the educator to explain distinctively at what stages the student requires extra assistance to improve their reflective thinking.

Kember’s model,⁴⁴ derived from Mezirow’s model of transformative learning in adults, formed the basis of Kember et al’s Reflective Thinking Questionnaire (RTQ).⁴⁵ Prior to the development of the RTQ,⁴⁵ the model identified levels of reflection involving 6 coded stages (habitual action, thoughtful action, introspection, content reflection, process reflection, and premise reflection). This previous model evaluated reflective ability of allied health students from reflective journals. However, the Cronbach alpha score, which measures internal consistency of the raters, for the reflective coding system in this study was less than desirable (0.65).⁴⁴

Kember et al developed a more refined questionnaire (with a 4-stage categorization: habitual action, understanding, reflection, and critical reflection) to assess and measure the reflective ability of a student.⁴⁵ Questions in the RTQ related to nonreflective processes as well as reflective processes. Questions related to habitual action and understanding are considered nonreflective, while questions related to reflection and critical reflection are considered reflective. The thought behind this distinction is that habitual action is done without thinking and, therefore, nonreflective, and understanding of theory content is not necessarily a reflective process. Understanding is only related to reflection when a personal experience is involved; however, if it is only understanding of content and classroom material, then reflection is not necessarily involved. Thus, when assessing reflective thinking ability of students with this tool, emphasis on the questions related to reflection and critical reflection (the reflective process questions) should only be considered. As this tool has good psychometric properties⁴⁶ and has been used in the health professions education context,⁴⁵ it may be a beneficial tool to assess the reflective thinking ability of pharmacy students in a pretest/posttest context.

Although the refined Kember et al RTQ can be considered an effective tool to measure reflective thinking ability, it may not be useful for measuring the level of reflection on a writing task. The tool asks students to indicate a level of agreement to specific questions on a Likert-type scale regarding reflection and critical reflection (as well as questions related to nonreflective activities) and does not measure specific examples of reflection or critical reflection directly from writing tasks.

The extensive use of Boud et al’s and Mezirow’s models for assessing levels of reflection in writing tasks demonstrates their success and, if paired to make a matrix rubric (Table 4), could be preferred better model over Kember’s refined model for assessing reflective writing tasks. Moreover, the strength of using a matrix rubric combining the stages of reflection with categories of reflection allows for cross-tabulation and not purely linear results, thereby contributing to the robustness of the educational tool.

ASSESSMENT STRATEGIES TO PROMOTE THE REFLECTIVE PROCESS

Examination of the published literature yielded a number of assessment strategies to promote reflection. Of these, we propose 2 that could be used in pharmacy education: a questionnaire to assess reflective thinking ability of pharmacy students and a reflective rubric that draws on the earlier work of Boud et al Mezirow, with elements drawn from the Wetmore et al⁵ rubric.

Kember’s RTQ (Table 3) reports good psychometric properties⁴⁶ has been tested in health professional student cohorts,⁴⁵ and provides an indication of reflective and critical reflective thinking ability. The RTQ in Table 3 can be adapted for pharmacy students. By evaluating the questions related to reflection and critical reflection on a Likert-type scale, this tool can be used to analyze the impact of reflective and critical reflective thinking ability. Furthermore, if it is provided as a pretest (prior to reflective practice activities) and posttest (after reflective practice activities), reflective thinking ability of students can be effectively measured.

Reflective rubrics can assess levels of reflection. They are designed based on theories of seminal educators of reflective practices and for specific discipline use to assess the levels of reflective thinking (Table 2). Reflective rubrics provide a means to assess reflection levels and can guide students self- and peer reflection.⁴⁷ When made available to students prior to reflective writing activities, reflective rubrics may further enhance reflection. For example, students who have access to their assessment rubrics prior to an assessment task have guidelines of what is expected from them. Once the task has been completed and assessed, students can return to the rubrics and reflect on how they can improve practice. Based on the theories of Boud et al’s stages of reflection,³⁸ Mezirow’s categories of reflection,³⁹ and elements drawn from Wetmore et al’s rubric used in dental education,⁵ we propose a reflective rubric found in Table 4. Alternative reflective rubrics can be accessed from journal articles found in Table 2.

ASSESSMENT OF REFLECTIVE PRACTICES IN PHARMACY EDUCATION AND THEIR IMPLICATIONS

If reflective practices are to be incorporated into pharmacy curricula, constructive alignment of these practices to assessments is required.^33,48 Effective constructive alignment involves a learning environment that encourages students to perform learning activities and appropriate assessment strategies that enable performance to meet intended learning outcomes.³³ Reliable and appropriate assessment strategies provide means for improving pharmacy education. This is emphasized in Standards 24 and 25 in the 2016 Standards of the Accreditation Council for Pharmacy Education (ACPE).⁴⁹Assessment of pharmacy students’ reflective ability involves a number of considerations: educators must be clear about the curricula learning outcomes; concepts of reflection and what constitutes reflective practice and reflective practitioner (see Schon’s work³⁷) should be defined; clear, structured (but not “recipe-following”) reflective activities⁵⁰ used as practice examples prior to assessment can encourage honest reflection;⁵¹ often reflection discloses personal matters and may be challenging for both the learner and assessor, so sensitivity toward these matters is required;⁵⁰ time must be allocated for assessment and practice reflection activities;^32,51 what will be assessed should be concisely identified; and reflective assessment activities must be constructively aligned³³ so that assessment also aligns to long-term and future learning.⁵²

In addition to these considerations, for effective assessment of reflective practices in pharmacy education, 2 other issues must be addressed. First, pharmacy educators must have adequate training in assessing this complex and fundamental skill and second, limitations in terms of time allocated in the curriculum need to be considered when designing reflective practice assessment tasks. Regarding assessor training, they need to be familiar with terms, concepts, principles, and tools of reflective practices. Furthermore, assessors should have knowledge of assessment criteria for the required outcomes of reflection. Understanding the rubrics and outcomes involved in this complex process are essential for assessment consistency. To further improve the process, assessors may want to reflect on their own practices prior to assessing others.³²

Allocating sufficient time for effective student feedback and grading is one of the limiting factors in introducing reflective practice assessments into pharmacy curricula. As reflective writing requires a deeper thought process through an essay, blog, journal, statement, or portfolio, assessing each student on levels of reflection requires a number of hours. One way around this is to use a peer-review process, which enables students to peer assess using a reflective rubric and provide formative feedback. This process can also enhance both self- and peer reflection without compromising grading time. If these activities take place in a laboratory or tutorial setting, they could stimulate a deeper class discussion on the reflective process and potentially further enhance peer reflection. Furthermore, facilitating tutorial discussions using a reflective dialogue could enhance deeper, reflective learning.³²

CONCLUSION

Assessment can drive student learning. If reflective practices are not assessed, students may not see them as significant compared to other assessed tasks.²⁵ Assessing reflective practice allows instructors to review strategies of reflective thinking. This provides a foundation for students to make better informed decisions.¹ Furthermore, assessment of reflective skills enables educators to gain a better understanding of their students’ thought processes as they problem-solve to make clinical decisions. Using a validated questionnaire or proposed reflective rubric is a useful model for such assessment in pharmacy education as either measures reflective thinking ability as well as reflective writing skills.