Letters

Peer Evaluations Can Be Problematic

To the Editor. We read with interest the article, “A Peer Review Process for Classroom Teaching.”¹ We commend the authors for setting up and reporting the results of such a process. Despite the paucity of publications on the matter, there is likely much interest in the academic pharmacy community about peer evaluations of classroom teaching, and many schools, including our own, have attempted to establish similar programs.

We agree with the authors that student evaluations can be subject to evaluation bias, specifically, the “mirror effect” (ie, when student evaluations tend to mimic their grades). However, the peer evaluation process is fraught with many philosophical and practical concerns of its own. In addition to the authors' point that the process is time and labor intensive, we have encountered the following types of questions and concerns:

• What is the purpose of peer reviews? Are they to be used in a formative or summative manner, or both? Faculty members should be required or forbidden to submit them as part of annual reviews or promotion and tenure applications, as allowing them to choose whether or not to do so would bias the process. Also, what weight should be given to peer reviews versus student evaluations?

• How objective is the process? Neither the process that the authors describe nor our own is conducted anonymously. Often the evaluator and the one being evaluated have worked with one another for years, or at the very least, have the prospect of working with one another for many years to come. Not surprisingly, it has been our experience that faculty members are generally given relatively high marks in the peer evaluation process. Because of the resultant ceiling effect, it is difficult to use peer evaluations to differentiate teaching abilities of faculty members.

• Who is qualified to conduct the reviews? Having junior faculty members review other junior faculty members is arguably the proverbial “blind leading the blind.” Yet if only senior faculty members conduct the reviews, this only accentuates the problem of time/labor intensity. Furthermore, many newer schools may not even have sufficient numbers of senior faculty members to conduct these reviews at all.

• What teaching domains are best to be evaluated by peers (rather than students) and in what manner? Should evaluation criteria be the same for both student and peer evaluations? Should criteria be norm- or referenced-based? From a philosophical standpoint, classroom teaching is meant for students. As such, a faculty member cannot possibly perceive a lecture like the typical student because of differences in knowledge and experience. While we agree with the authors' teaching domains, we think that the chief concern is whether or not those items translate into student learning, which of course, is best judged by students.

• How often and when should peer reviews be done? Should they be conducted only for new faculty members or as part of a new course, or should they be part of a continuous process of improvement? As opposed to student evaluations, which are typically meant to measure the faculty member's performance over a lecture series or course, peer evaluations tend to be one-time measurements, and thus they may not be truly representative of the faculty member's usual performance.

Peer review of teaching plays an important role in the improvement and progress of the academy. Hopefully, this will evolve as a mechanism for faculty members to help one another via formative feedback as opposed to a mechanism for faculty members to offset poor summative student evaluations. However, we feel that the process would benefit from standardized guidelines specific to pharmacy education in order for the peer evaluation process to be most helpful within the overall assessment of a faculty member's teaching. We encourage the American Association of Colleges of Pharmacy to be a major driver in the further discussion and systematic development of such guidelines that identify both shortcomings and advantages of peer evaluation.

The Change of the Guard: Are We Ready?

To the Editor. A good number of articles devoted to academic leadership in pharmacy have stressed the “graying” of the American dean. One of the projects on succession planning by last year's cohort of the Academic Leadership Fellows Program (ALFP) stated that 30% of academic pharmacy deans were approaching retirement. Recent American Association of Colleges of Pharmacy (AACP) meetings have also had forums on the need for leaders in the next 5 years or so.

The question therefore is, are we ready? At the associate deans' meeting at the annual AACP meeting in Boston in July 2009, a senior associate dean from an established state school lamented the fact that “young-uns” without much academic experience were either thrown into or self-selected themselves for administrative positions. Now this was certainly not a good augur. In contrast, this person had spent years climbing the academic ladder to the rank of full professor, had worked in 2 state schools, and then moved into administration. Certainly, there is no substitute for experience. However, with the expansion of pharmacy programs across the country, the need for administrators has certainly led to academia picking its share of “young-uns.”

Is pharmacy a microcosm of the United States or an aberration? Well, we have a fairly-young President, a Southern governor who is in his late 30s, several CEOs of silicon valley come to mind who are yet to touch 40, the youngest Super Bowl winning coach at 36, and most recently a 33-year-old head coach calling the “wild horses” play for the Denver Broncos.

This, however, does not mean that “young-uns” should consider it their right to lead their institutions but rather they are inheriting a privilege that their predecessors have forged. When I look around at some of my assistant/associate deans colleagues, I see a growing number of younger people. I do not know the exact age range that classifies people as young, but yes, most of these “young-uns” are in their late 30s to late 40s. Needless to state, I fall within these ranks, something that was not the norm even 10 years ago.

So what are we “young-uns” doing to claim our place and show ourselves worthy of the responsibilities given to us? Just being young and brave is not the only criteria. Some of my friends across the country have mentioned that we bring a different side to leadership—something that was not considered de rigueur in deans. We display the characteristics that Cohen has reported in his article titled “Today's' Mandate for Pharmacy Deans.” (AJPE 2009; 73:1, Article 19). What are we doing to become the “Clooneys” and “Pitts” of academic leadership? Well, speaking for myself, I have started taking more leadership and management training—not a far cry from my PhD in Pharmacy Administration. I am also a current member of the Academic Leadership Fellows Program (ALFP) 2009–2010 cohort. Not surprisingly, many of the fellows in the ALFP cohort, both past and present, seem to fall within the definition of “young-uns.”

Speaking to friends across the country, most of us “young-uns” are learning on the job as well as taking courses/classes to be more effective leaders. Kudos go to the current deans—including the 3 who hired me—who see our latent potential and then nurture us, guide us, and help us get ready for the No. 1 job. There is a need for people at the helm, and true to the spirit in which this great country was founded, we are prepared to meet the challenge. Passion and hard work is not a substitute for experience, but we are trying and the deans are behind us all the way!

Unstandardized and Defective Evaluation Practices in the Examination System in Pharmacy Institutes of Pakistan

To the Editor. An ideal examination system determines approaches to both teaching (what and how teachers teach) and learning (what students learn).^1,2 Apart from student assessment, examinations help teachers modify their teaching methodologies or adopt new ones to impart the desired academic outcomes. A fair and uniform examination process is important, but many attempts to standardize the process remained fruitless until the late 1990s when the Pakistani government began reforming the examination process. Stringent measures were imposed, and some success was achieved to bring uniformity and eradicate malpractices in the examination system, especially at secondary and higher secondary levels.^3,4 Thus far, there is no report of such attempts at reforming the examination system at the graduate and postgraduate levels. The same problems were encountered while reviewing the pharmacy examination system in Pakistani institutes.

At present, a number of pharmacy institutes in both public and private sectors are practicing 2 types of examination systems – semester and annual. The mode of assessment is different in these 2 systems and is causing uneven academic outcomes. Surprisingly, different institutes practicing the same system are not uniform in terms of grading, which can deprive capable students of obtaining jobs since in government and the private sector, candidates are short-listed based on their grades in pharmacy school. Hence, there is a real need to standardize the examination system in all pharmacy institutes of Pakistan.

In comparing semester and annual grading systems, the former is found to be easier, less time-consuming, and more economical; however, the sole internal assessment is making the fairness of this system hazy due to a higher probability of malpractices by examiners themselves, or because of the influence of students, parents, politicians, and institutional heads. Inconsistency is most problematic in the private sector where teachers often try to remain in good favor with their students by showing leniency in evaluation, and in good stead with the administration of the institutes by showing good results.

In the annual evaluation system, assessment is conducted externally for the theoretical examination and internally for the practical examination. While this system seems fair, in fact it raises serious concerns including the methods used to appoint the external examiners, the non-standardized grading system used, and the cost and inefficiency of the system. Students often are concerned and inquisitive about the grading criterion used (eg, why they received a grade of 50% to 60% on a question they attempted to answer completely) and delays in receiving their grades. These are sensitive areas requiring the focus of academicians and authorities controlling the examination process.

The methods of administering and grading practical examinations are also ambiguous and un-standardized. The process is conducted by 1 external and 4 internal examiners in a strange way because there are no standardized examination equations. This is worth noting in the case of viva, a part of the practical examination, in which more consideration is given to theory and not just practical aspects. This part of the examination is one of the most un-standardized parts in terms of the types of questions given, the number of questions per student, the grading scheme per question, and the number of examiners and their approach to grading. The practical examination is also given at different times, which adds another variable. Unlike theoretical examinations, the record of viva assessment is not maintained professionally so students are not able to check their paper if they are not satisfied with the evaluation. If students are not satisfied with the evaluation they may check their paper. Students often fear viva assessments, complaining of less time to review the syllabus and the evaluation process. After the examination, they remain uncertain until grades are posted. In one of the universities where only students with the highest grades are admitted, the failure rate is more than 60%, and those who pass have marks in the range of only 50%-60%. The newspaper The Daily Dawn has reported that students fear examinations, and upon failing, some have committed suicide.⁵ It is a serious concern because students are blaming the higher failure rate on 2 things: the monetary benefits that the examiners receive for conducting supplementary examinations, and the university's desire to keep students submissive. There is no official/unofficial body to analyze and correct these problems with examinations, which is an important matter to academia not only for improving teaching and learning but also to produce professionally and psychologically sound pharmacists to serve humanity.

Wikis and Open-Access Education?

To the Editor. We read with great interest the review of Web 2.0 applications by Cain and Fox.¹ The tone of the discussion on wikis seemed skeptical and enshrined with “fear-mongering” as we have experienced with some contemporaries. As reported by Clauson and colleagues, Wikipedia is not a panacea of drug information.² However, authors appeared optimistic in the accuracy of information despite its incomplete scope for drug information. Wikipedia and “Darwikism” (socially writing and editing common knowledge among users) appears to be a good source to begin a wider search for a medical condition or education-related terminology such as “pedagogy” or “constructivism.” Though omitted in the investigation by Giles,³ content within Encyclopedia Britannica is written by 1 author, including only that author's expert viewpoint on a given entry topic. We in academia are aware of numerous ongoing debates and recognize 1 viewpoint does not reflect the “correct” or “right” answer to all problems. Through social collaboration of entries, Wikipedia appears to enunciate multiple perspectives if some only superficially (and can prompt further exploration by readers). Might Wikipedia be a post-modernist evolution of the modernist Encyclopedia Britannica (ie, including diverse viewpoints as opposed to a single viewpoint describing all truth in the world according to their authority)? In comparing Wikipedia with Encyclopedia Britannica, it should also be pointed out that errors in Wikipedia are edited by other readers relatively quickly, while errors in Encyclopedia Britannica do not change until/if a new edition is obtained.

In recent years, a related movement has been towards open-access forums for educational materials. While this movement was initiated within K-12 schools and has been recognized in undergraduate studies applications, little has been discussed or disseminated in the pharmacy literature. We commend this Journal for its continued efforts in making content freely available over the Internet for all pharmacy educators. The community property aspect of Web 2.0 applications seems a laudable impetus towards open-access in pharmacy education.