Current State of and Opportunities for Enhancing Peer Evaluation Practices Across the Pharmacy Academy

DISCUSSION

To prepare faculty to meet the dynamic needs of the student population and to provide a premier student experience, schools and colleges of pharmacy must consistently evaluate their teaching effectiveness and quality. Over the decades, PET has become a prevalent way for faculty to measure and improve their teaching quality to better meet the needs of students, especially in countries with the most widespread use of PET, namely the United States, Australia, and the United Kingdom.²⁰ Because PET has become an additional component of the comprehensive assessment of teaching (in addition to other measures of teaching effectiveness),²¹ teaching has moved from a singular to a community enterprise for improvement.²² While ACPE Standard 10 requires that the curriculum be delivered via teaching methods that incorporate strategies to address the learning needs of students and facilitate the achievement of course expectations and outcomes, it does not mandate a quality assurance process for faculty to provide feedback on the teaching and learning process.³

Our study found that most institutions have a PET/IC program. However, we found that the variability in the PET/IC programs across the Academy is broad. As this may be appropriate due to different schools having different pedagogical models and incentive structures for faculty, the lack of a universal standard for how to analyze and foster peer review is also potentially problematic, as this may result in a lack of awareness about whether instructors are using evidence-based teaching practices. Thus, opportunities exist for the Academy to improve PET.

Historically, PET has been applied using a formative or summative approach in higher education. In our study, most institutions with PET/IC programs reported using a combination of formative and summative evaluations. Both assessment approaches to PET have been shown to help faculty reflect on their teaching, increase their confidence, feel less isolated, enhance student learning experiences, improve their teaching, create community and collegiality, and critically reflect upon the social constructs that play a role in teaching.^23-30 Thus, PET programs should be viewed as a holistic process where pedagogy is able to be discussed, critiqued, transformed, and preserved.

In pharmacy education, limited literature has aimed to establish best practices for PET, but other disciplines have published guidelines. Successful PET programs need an explicit framework to ensure success. Evaluations should incorporate appropriate evidence and processes to ensure fairness, consistency, and the reliability of data.³¹ Evaluation methods should be reflective of the college’s mission, values, and practices. Rubrics and other assessment methods are recommended to be identified and validated so that the measure can accurately assess teaching impact and allow for faculty to track their progress for continual improvement of instruction. This is a particular area in which the pharmacy Academy can improve, as only 16.7% of institutions surveyed reported using a validated instrument in their PET/IC process. Reynolds and colleagues suggested that rubrics should be developed to assess four distinct criteria: disciplinary expertise, design and development skills, instructional practices and performance, and teaching environment.³² To further explain each one in detail, each criterion is broken down into subcategories with descriptive statements.

Many higher education institutions provide a central teaching and learning center to support faculty instruction regardless of discipline. However, colleges of pharmacy are encouraged to develop their own center that reconnects teaching to the discipline. By doing this, efforts can be made to value and reconnect teaching as a vital part of the student experience by engaging faculty in the scholarship of their profession.^23,33

After reviewing the literature, we have developed some suggested best practices for creating and maintaining a PET program, in conjunction with the steps recommended by Trujillo and colleagues, outlined in Table 1.³⁴ Reflection and feedback should be incorporated and seen as a valuable part of the review process, preferably by including a meeting to foster discussion rather than by only providing written feedback. Feedback should appear as nonthreatening, and there should be sufficient time to discuss findings.³⁵ Creating this safe space will encourage faculty to experiment with new and innovative ideas without feeling threatened, anxious, or embarrassed and without causing negative internal dialogue.³⁵ Some examples of providing feedback as nonthreatening and creating a safe space include programs trying to make PET as transparent as possible by using video to record the reviewee and the pre- and post-discussions with the evaluators and faculty being reviewed.³⁶ Other programs allow for the faculty being evaluated to observe and assess their own teaching prior to the post-review meeting to allow them the opportunity to identify challenges and solutions, which creates the conditions for a two-way dialogue and for the feedback session to be more conversational.²⁹ In other programs, feedback is provided only to the faculty member being evaluated, and then they have the choice to share the feedback with others (eg, their department chair).¹⁹ In our study, it was apparent that in many institutions, opportunities exist to improve PET by enhancing the feedback provided. This can be done by incorporating additional elements as part of the process, such as a pre-observation meeting and consistent timelines and expectations for post-observation meetings and feedback. Additionally, programs should consider incorporating the review of assessments as part of the process to ensure the quality of both teaching and learning.

Few formal training programs exist that prepare faculty to conduct peer reviews. Most faculty receive informal mentoring from committee members or staff.³⁶ In some instances, more experienced faculty are selected as observers.^37,38 Regardless of previous teaching experience or type of faculty appointment, peer reviewers should receive guidance and training to better understand pedagogical approaches. As each institution is unique, we recommend that each college of pharmacy and department encourage a formal training process based on best practices and establish a culture where performance measures are created through faculty consensus, ongoing mentorship, and where evaluations are linked to meaningful outcomes.³⁶ In our study only 31.9% of the institutions reported formal training for peer observers. Peer reviewers should receive appropriate training and orientation on how to rate peers against a professional standard, collect evidence, and provide feedback. Training should be developed by senior faculty in conjunction with educational experts. All of these are ways to increase trust, enhance mutual respect, consider sociocultural differences, and minimize bias.

The lack of a thoughtful and comprehensive approach toward evaluation standards can often result biases from evaluators. This can present disadvantages to historically underrepresented groups where structural and institutional barriers exist. To combat this, the reliability of PET can be improved by using standardized evaluation tools, training evaluators, engaging in reflections and discussions, and intentionally recognizing and celebrating sociocultural norms, values, and conventions. Some literature recognizes the impact of sociocultural contexts in the peer review process. Sociocultural perspectives should be recognized and discussed during the process, as they contribute to the department and college’s norms, rules, and attitudes as well as shape the ways that faculty communicate, support, and engage with each other and students. A climate built on trust, support, and common goals facilitates open communication to allow for peer review to be successful.³⁷ As institutions have increasingly been encouraged to acknowledge and separate themselves from parts of their discriminatory history, the review process should articulate, embrace, value, and respect faculty who have different cultural intersections within the department or institution. Incorporating peer reviewers who belong to historically underrepresented groups are one way to mitigate patterns of discrimination that may appear in the review process.

Our study revealed that very few institutions have evaluated the effectiveness and impact of their PET/IC programs, which remains one of the largest opportunities to identify how these programs can have a meaningful impact on teaching and learning across the Academy. For one institution that did evaluate their PET/IC program, this evaluation first involved administering a pre-implementation survey assessing faculty needs and attitudes related to peer evaluation. Then, two years after implementation, the survey was repeated and additional questions were asked regarding adherence to peer observation policies and procedures, feedback received, and impact on teaching.¹³ Overall, little remains known on the direct connection between PET/IC programs and student attainment of knowledge and skills. Along with evaluating PET/IC programs, we also suggest that PET be used in combination with and to supplement other teaching effectiveness methods, such as student evaluations, self-assessments, and teaching artifacts, as teaching is multidimensional and cannot be captured by one or two measures.

Several important limitations to our study exist. First, out of 141 institutions contacted, 91 completed the survey for a response rate of 64.5%. Despite having a response rate of less than 80%, our respondents were proportionally representative of the institutional control (p=0.8) and age of pharmacy schools (p=0.58). Second, the study was limited to the responses obtained from the respondent for each institution and may not reflect all elements of that institution’s PET/IC program accurately, particularly those aspects that the respondent was not aware of. To mitigate this potential limitation, when the survey was sent to respondents, they were asked to consult others as necessary and/or forward the survey to another individual more knowledgeable of the PET/IC program at their institution to complete, as appropriate. Third, the survey did not meaningfully distinguish between PET and IC. Future research should explore the differentiation between summative and formative approaches to improving teaching effectiveness in pharmacy education.