Skip to main content

Main menu

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers: Moving from Injustice to Equity
  • Reviewers
    • Reviewer Instructions
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Other Publications

User menu

Search

  • Advanced search
American Journal of Pharmaceutical Education
  • Other Publications
American Journal of Pharmaceutical Education

Advanced Search

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers: Moving from Injustice to Equity
  • Reviewers
    • Reviewer Instructions
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Follow AJPE on Twitter
  • LinkedIn
Research ArticleINSTRUCTIONAL DESIGN AND ASSESSMENT

Assessment of a Revised Method for Evaluating Peer-graded Assignments in a Skills-based Course Sequence

Tara Storjohann, Erin Raney and Kelsey Buckley
American Journal of Pharmaceutical Education October 2015, 79 (8) 123; DOI: https://doi.org/10.5688/ajpe798123
Tara Storjohann
Midwestern University College of Pharmacy, Glendale, Arizona
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Erin Raney
Midwestern University College of Pharmacy, Glendale, Arizona
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kelsey Buckley
Midwestern University College of Pharmacy, Glendale, Arizona
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Objective. To evaluate the modified peer-grading process incorporated into the SOAP (subjective, objective, assessment, plan) note sessions in a skills-based pharmacy course sequence.

Design. Students assessed a de-identified peer’s SOAP note in a faculty-led peer-grading session followed by an optional grade challenge opportunity. Using paired t tests, final session grades (peer-graded with challenge opportunity) were compared with the retrospective faculty-assigned grades. Additionally, students responded to a survey using 4-point Likert scale and open-answer items to assess their perceptions of the process.

Assessment. No significant difference was found between mean scores assigned by faculty members vs those made by student peers after participation in 3 SOAP note sessions, which included a SOAP note-writing workshop, a peer-grading workshop, and a grade challenge opportunity. The survey data indicated that students generally were satisfied with the process.

Conclusion. This study provides insight into the peer-grading process used to evaluate SOAP notes. The findings support the continued use of this assessment format in a skills-based course.

Keywords
  • peer grading
  • peer assessment
  • SOAP notes

INTRODUCTION

The formulation and documentation of patient care plans is a skill central to the provision of patient-centered care, supported by the Center for Advancement of Pharmacy Education (CAPE) 2013 Educational Outcomes.1 To teach and evaluate this skill, Midwestern University College of Pharmacy – Glendale (MWU-CPG) incorporated a series of active-learning experiences relevant to pharmacy practice using the “subjective, objective, assessment, plan” (SOAP) written documentation format. Supplementing each SOAP note-writing workshop, students assessed and scored an anonymous peer’s written SOAP assignment using a faculty-developed rubric in a required faculty-led workshop, followed by a self-reflection opportunity.

This assessment method was designed to meet the Accreditation Council for Pharmacy Education (ACPE) Draft Standards 2016 recommendation that schools and colleges should “utilize teaching/learning methods that facilitate achievement of learning outcomes, actively engage learners, promote self-directed learning, and foster collaborative learning”.2 In addition, this method helps address Domain 4 of the CAPE 2013 Outcomes, which focuses on educating future pharmacists to be self-aware by providing a structured opportunity to develop the skills of peer- and self-assessment.1

Peer assessment not only serves as a grading procedure, but a method to practice self-evaluation skills. Peer assessment is a valuable form of assessment in higher education for a variety of assignments, and students generally perceive it as beneficial.3 Peer assessment is used in pharmacy education for student case presentations, laboratory courses, medication management assignments, patient interviews, and advanced practice experiences.4-9

A proposed benefit of the peer assessment strategy is providing an enhanced learning environment for students in which a high order of thinking is utilized and guided by evaluation. Other benefits include self-reflection, peer interaction, and a decreased faculty workload.4 Finally, peer assessment can help to foster higher levels of responsibility among students by requiring them to be fair and accurate with the feedback and evaluations they make of their peers.3 Limitations identified in previous research include a lack of student understanding of the purpose of the peer-grading session, lack of student experience in grading, and lack of confidentiality. In these studies, peer grades were generally higher than faculty grades for the assignments.4-9

A previous study comparing student and faculty scores for 3 peer-graded assignments at the college resulted in the students assigning significantly lower scores than faculty members to student peers, which was inconsistent with other research.10 One goal of using peer grading was to replace the need for traditional faculty SOAP note grading, so several changes were made to improve the consistency of faculty and peer-assigned scores. A standardized format, titled the Comprehensive Medication Management Plan (CMMP), was adopted for guiding SOAP-note writing, which detailed the expectations for each section and served as the basis for the grading checklists used in the peer-grading sessions. This approach attempted to provide the students clear and consistent expectations among different faculty members when formulating their written SOAP note.

Additionally, a “challenge” opportunity was implemented to allow each student the chance to self-evaluate their peer-graded note prior to the final grade assignment and submit a grade challenge if appropriate.10 Outcomes of a “challenge” strategy, including the accuracy of final grades after this opportunity, has not been formally evaluated in the literature. The intent of these changes was to improve the scoring process so the peer-assigned scores would be similar to those assigned by a faculty grader, thus validating the peer-grading process used for the SOAP notes sessions as a stand-alone assessment.

The purpose of this study was to evaluate the modified peer-grading process incorporated into the SOAP note sessions. The specific aims were to compare the final recorded student grade (given after the SOAP note session and “challenge” opportunity) with a retrospective traditional faculty-assigned grade, and to gather the student’s perceptions about the process.

DESIGN

Professional Skills Development (PSD) is a required 8-quarter, workshop-based course sequence in a 3-year didactic curriculum (PSD 1-8). In the second year of the sequence in 2012-2013, there are 3 SOAP note sessions. Each session consists of 3 parts: a writing workshop, a peer-grading workshop, and a challenge opportunity that build on the foundational knowledge taught in the therapeutic course sequence (PSD 5, 6, and 8). The final score received after the 3 sessions is considered a summative assessment of this knowledge.

This study involved a single cohort of students completing the second year of their PSD sequence. All students enrolled in PSD were required to participate in the 3 SOAP note sessions. The case content for each session corresponded to the material taught by the subject matter expert in the concurrent therapeutics course during each of the 3 quarters (hyperlipidemia, psychiatry, and cardiovascular). The subject matter expert also designed the case and grading checklist, led the peer-grading workshop, was present at the challenge opportunity, and graded the challenge submissions.

Organization of Sessions

For each SOAP note writing workshop, students were given a template prior to writing their SOAP note, which included a list of medical problems students were required to address for the patient case. Given the subjective and objective data, students were asked to individually write the assessment and plan for the patient. The first SOAP note-writing workshop was completed outside of class and submitted via Safe Assign, a program found in the Blackboard learning system (Blackboard Inc., Washington, DC), to check for plagiarism. The other 2 SOAP note-writing workshops were completed within a 2-hour time block in the university’s testing center. The subject matter expert was not present at the writing workshop. To aid in developing their assessment and plan, students were given a CMMP grid (Appendix I). Completion of the CMMP was optional during the SOAP note-writing workshops, although highly encouraged, as the grading checklists for each note were based on the same categories.

Peer-grading workshops occurred during class time within one week of completing the writing workshop. Each of the 3 peer-grading workshops were held in 2 sections (2 hours each) on the same day to accommodate one-half of the class at a time. This ensured that students who wrote the SOAP notes being evaluated were not present in that particular section of the peer-grading workshop. In an effort to hold the students accountable for attending the session, for grading the SOAP note to the best of their ability, and for providing quality feedback to their peers, students wrote their names on the cover sheet of the evaluation form. The course coordinator kept track of who graded each SOAP note, but students and the subject matter expert were blinded throughout the entire process. The grading checklist was distributed first, prior to the cases to be graded. The subject matter expert directed the students to review the grading checklist point by point. This was an opportunity for students to reflect on what they had individually written on their own note and ask questions for clarification, prior to being held responsible for grading a peer’s note. Peer notes then were distributed, and students were given approximately 20 minutes to read through their assigned note and grade based off of the supplied grading checklist. Once students had completed grading, they again reviewed the rubric. At this time, students could request whether an alternative written response not included within the grading checklist be considered for points by the subject matter expert facilitating the session. Once the grading was complete, students were instructed to provide anonymous constructive written feedback to their peers.

A 1-hour challenge opportunity was offered within 24 hours after each peer-grading workshop for students to review their peer-graded note and to challenge their peer-assigned grade. Only students who attended the challenge opportunity had the ability to submit a challenge form. If students wanted to proceed with a challenge, they completed a preprinted form specifying which point(s) they wanted to challenge and providing justification for each. The subject matter expert was present at the challenge opportunity to answer additional student questions regarding the case or their responses. After the challenge opportunity, the subject matter expert reviewed the challenge forms and addressed only the inaccuracies noted by the student. However, the subject matter expert reserved the right to regrade the assignment if necessary. Grade challenges were not accepted after this session was completed.

Evaluation of Sessions

To evaluate the scores assigned during each of the 3 SOAP note sessions, the subject matter expert retrospectively graded each assignment using the standard grading checklist used in the peer-grading workshop which modeled the traditional grading method used at the college. This traditional method did not include a peer-grading workshop or challenge opportunity and resulted only in a scored rubric with written comments. The subject matter expert was blinded to the student names and all results of the peer-grading workshop and/or challenge opportunity.

The final session grades (peer-graded with challenge opportunity) were compared to the retrospective faculty-assigned grades for each session. Evaluations were made using paired t tests with a p value of ≤0.05 being considered significant. An anonymous survey was also administered to all students enrolled in the final quarter of the PSD course sequence. The survey was voluntarily completed during a course period after the last of the three SOAP note sessions. The survey assessed the students’ opinions of the SOAP note sessions utilizing 4-point Likert scale statements, multiple-choice questions, and open-ended questions. Descriptive statistics were utilized to analyze the findings of the survey. Approval for the study was obtained from the Institutional Review Board (IRB) at Midwestern University.

EVALUATION AND ASSESSMENT

Scores for 453 peer-graded assignments (151 for session 1, 152 for session 2, and 150 for session 3) were analyzed. No difference was found between the mean scores for final recorded grades and retrospective faculty assigned grades (Table 1). Differences between the mean scores for each of the 3 individual sessions were 0.8, 0.9, and 1.3, respectively. A subgroup analysis was performed to compare the final recorded grades and the retrospective faculty-assigned grades based on participation in the challenge opportunity (Table 2).

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1.

Final Recorded Grades Using Peer-Grading Method vs Retrospective, Traditional Faculty Grades for 3 Sessions

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 2.

Final Recorded Grades Using Peer-Grading Method vs Retrospective, Traditional Faculty Grades for 3 Sessions Based on Participation in the Challenge Opportunity

We found that for all 3 sessions combined, 158 students (35%) challenged their peer-graded SOAP note for faculty review (93% of which resulted in a score change), 243 (54%) reviewed their peer-graded note but did not challenge for faculty review, and 52 (11%) did not review their assignment. No difference was found if the student reviewed their submission and either opted to challenge or not to challenge. However, there was a significantly lower score assigned for students who did not review their assignments in the challenge opportunity.

One hundred twenty students (80%) participated in the survey assessing their perspectives on the SOAP note sessions. (Table 3) Responses from 2 surveys were not included because the respondents did not meet the criteria of having attended at least two challenge opportunities. Regarding the SOAP note-writing workshop and subsequent peer-grading workshop, the majority of students either “agreed” or “strongly agreed” that the CMMP helped them structure their SOAP notes (89%), the instructions for participating in the sessions were clear (96%), the grading checklists were easy to follow (95%), the peer grading sessions enhanced learning (70%), and faculty guidance during the peer-grading workshops allowed them to effectively grade a peer’s note (90%). Regarding the challenge opportunity, the majority of students either “agreed” or “strongly agreed” that the challenge opportunity allowed them to self-reflect on their work (73%), that the challenge sessions are necessary to receive a fair score on peer-graded assignments (89%), and that faculty members awarded points fairly after the grade challenge opportunity (77%). The lowest score was in response to whether peer-provided comments were constructive and useful in improving SOAP note skills. However, 54% of the students“agreed” or “strongly” agreed with this statement as well.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 3.

Students’ Opinions Regarding SOAP Note Sessions in the Professional Skills Development Course Sequence

The open-ended questions asked what the students liked most and least about the SOAP note sessions and suggestions for improvement. (Table 4) Students most commonly stated that their learning was enhanced by exposure to the various ways their peers approached the case. Other commonly reported responses included enhancement of disease state knowledge through immediate faculty feedback, fairness in grading, and improvement of general SOAP note-writing skills.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 4.

Student Comments Regarding Peer Grading Process in the Professional Skills Development Course Sequence

The majority of the responses regarding what students liked least included the process taking too long or classmates asking too many questions. Other common responses included inconsistency in points awarded by peer graders, rubrics being too strict, and reviewing the rubric prior to starting taking too much time or being too repetitive. The most frequent responses for how to improve the process included structuring the sessions to take less time and/or limiting the number of questions asked by other students. Other suggestions included revising the rubrics to be less stringent, holding the graders accountable for inaccuracy in grading, and allowing the notes to be typed instead of handwritten.

DISCUSSION

The revised peer-grading process for SOAP note sessions in the PSD course sequence resulted in the same overall score when compared to traditional faculty grading. This contrasts to a previous study, which showed higher traditional faculty-assigned scores than student peer-assigned scores.8 This result strengthens our confidence in the use of the process in place of traditional faculty grading. The most likely contributor to this result was the institution of the challenge opportunity; allowing students to self-assess their work, review peer comments, and submit a challenge form if inaccuracies were noted. In the subgroup analysis, we noted a difference in the final recorded grade vs the traditional faculty-assigned grades for those students who did not participate in the challenge session.

The use of the CMMP document was another possible contributor to the equalized scores, since it served as a structure by which all grading checklists were formatted. This may have streamlined the grading process by improving the clarity of the grading checklists used in the peer grading sessions.

Advantages and disadvantages of the peer-grading process from student and faculty perspectives were noted. Student noted that learning from their peers’ approach to the case, obtaining immediate faculty feedback that enhanced disease state knowledge, and fair grading were advantages. From the faculty perspective, an advantage to this process was the reduced faculty grading workload. Scoring student-submitted challenge forms (n=158) took a total of approximately 3 hours for the subject matter experts to complete. In contrast, scoring each submission separately (n=453) for the purposes of this research approximated 10 minutes per submission.

This process is not without disadvantages. Some students said the peer-grading workshop took too long because other students asked too many questions even though the workshop was limited to 2 hours and was conducted during normal course time. Some also indicated an inconsistency among peer graders and may have viewed the challenge as an opportunity to double-check their peers rather than an opportunity for self-reflection. Finally, some students indicated a lack of constructive feedback from peers despite receiving examples from instructors of appropriate vs inappropriate feedback. One disadvantage from a faculty perspective was the negative connotation implied by the terms “peer-grading” and “challenge opportunity.” Despite an formal orientation on the benefits of peer and self-assessment, some students perceived this process as a way for faculty members to decrease their workload and transfer the burden to students.

There are several limitations to this research. The 3 SOAP note sessions were each designed, conducted, and graded by 3 different subject matter experts and focused on a wide array of therapeutic topics. Given this, the students’ perceptions about the process could be influenced by the therapeutic topic or faculty member. Additionally, the SOAP note sessions were conducted in the fall, winter, and summer quarters, while the survey was administered at the end of the final session, possibly reflecting the students’ opinions of the most recent session. Finally, this study is reflective of a specific cohort of pharmacy students enrolled in a workshop-based course sequence that may not be applicable to other schools or other disciplines.

The SOAP note sessions in PSD have continued with subsequent student cohorts, without the use of traditional faculty grading. Several adjustments have been instituted based on the findings of this study. For example, in addition to the formal orientation, a more formal method for providing peer feedback has been instituted. After the peer-grading workshop, students are asked to state 3 things the author did well and 3 areas for improvement. Additionally, the procedure during the peer-grading workshop has been streamlined to allow students to grade peers’ submissions at the beginning of each session in pencil, eliminating the initial faculty review of the rubric with the class. Once all individual grading is complete, the subject matter expert reviews the rubric and entertains student questions and feedback. This change was based on feedback that students found it redundant to review the rubric twice and that it lengthened the overall session time. Finally, the description and name of this process has changed from “peer grading” to “peer assessment” to further convey the positive academic intent of the process. In addition, the term “challenge” opportunity was changed to “self-reflection and review workshop” to promote the individual benefits of participating in the process.

CONCLUSION

This study provides insight into the peer-grading process used in the PSD course sequence at the college. Our findings indicated no difference between grades assigned to students using the peer-grading process and traditional faculty grades for all 3 sessions. Additionally, students perceived that the process was beneficial to their learning, supporting the continued use of this assessment format in our curriculum. We feel the process has the potential to be successfully used in other professional programs as well.

Appendix

View this table:
  • View inline
  • View popup
  • Download powerpoint
Appendix 1.

Comprehensive Medication Management Plan (CMMP)

  • Received September 11, 2014.
  • Accepted March 18, 2015.
  • © 2015 American Association of Colleges of Pharmacy

REFERENCES

  1. 1.↵
    1. Medina MS,
    2. Plaza CM,
    3. Stowe CD,
    4. et al
    . Center for the Advancement of Pharmacy Education (CAPE) educational outcomes. Am J Pharm Educ.2013; 77(8):Article 162.
    OpenUrl
  2. 2.↵
    Accreditation Council for Pharmacy Education. Guidance for the accreditation standards and key elements for the professional program in pharmacy leading to the doctor of pharmacy degree. Guidance for standards 2016. https://www.acpe-accredit.org/pdf/ GuidanceforStandards2016FINAL.pdf Accessed November, 13 2015.
  3. 3.↵
    1. Dochy F,
    2. Sergers M,
    3. Sluijsmans D
    . The use of self-, peer and co-assessment in higher education: A review. Study Higher Educ. 1999;24(3):331-350.
    OpenUrl
  4. 4.↵
    Blooms BS. Taxonomy of Educational Objectives. 1st Ed. The cognitive domain. NewYork: David McKay, Inc.; 1956.
  5. 5.↵
    1. Basheti IA,
    2. Ryan G,
    3. Woulfe J,
    4. et al
    . Anonymous peer assessment of medication management reviews. Am J Pharm Educ. 2010;75(5):Article 77.
    OpenUrl
  6. 6.↵
    1. Krause JE,
    2. Popovich NG
    . A group interaction peer/self-assessment process in pharmacy practice course. Am J Pharm Educ. 1996;60:Article 136-145.
    OpenUrl
  7. 7.↵
    1. O’Brein CE,
    2. Franks AM,
    3. Stowe CD
    . Multiple rubric-based assessment of student case presentations. Am J Pharm Educ. 2008;72(3):Article 58.
    OpenUrl
  8. 8.↵
    1. Planas LG,
    2. Er NL
    . A systems approach to scaffold communication skills development. Am J Pharm Educ. 2008;72(2):Article 35.
    OpenUrl
  9. 9.↵
    1. Wagner ML,
    2. Suh DC,
    3. Cruz S
    . Peer- and self-grading compared to faculty grading. Am J Pharm Educ. 2011;75(7):Article 130.
    OpenUrl
  10. 10.↵
    1. Storjohann T,
    2. Raney E,
    3. Pogge E,
    4. et al
    . Evaluation of peer-graded assignments in a skills-based course sequence. Curr Pharm Teach Learn. 2013;5(4):283-287
    OpenUrl
PreviousNext
Back to top

In this issue

American Journal of Pharmaceutical Education
Vol. 79, Issue 8
25 Oct 2015
  • Table of Contents
  • Index by author
Print
Download PDF
Email Article

Thank you for your interest in spreading the word on American Journal of Pharmaceutical Education.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Assessment of a Revised Method for Evaluating Peer-graded Assignments in a Skills-based Course Sequence
(Your Name) has sent you a message from American Journal of Pharmaceutical Education
(Your Name) thought you would like to see the American Journal of Pharmaceutical Education web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
9 + 0 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Assessment of a Revised Method for Evaluating Peer-graded Assignments in a Skills-based Course Sequence
Tara Storjohann, Erin Raney, Kelsey Buckley
American Journal of Pharmaceutical Education Oct 2015, 79 (8) 123; DOI: 10.5688/ajpe798123

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Assessment of a Revised Method for Evaluating Peer-graded Assignments in a Skills-based Course Sequence
Tara Storjohann, Erin Raney, Kelsey Buckley
American Journal of Pharmaceutical Education Oct 2015, 79 (8) 123; DOI: 10.5688/ajpe798123
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • DESIGN
    • EVALUATION AND ASSESSMENT
    • DISCUSSION
    • CONCLUSION
    • Appendix
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

Similar AJPE Articles

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Transformation of an Online Multidisciplinary Course into a Live Interprofessional Experience
  • Enhancing Student Communication Skills Through Arabic Language Competency and Simulated Patient Assessments
  • Student Self-Assessment and Faculty Assessment of Performance in an Interprofessional Error Disclosure Simulation Training Program
Show more Instructional Design and Assessment

Related Articles

  • No related articles found.
  • Google Scholar

Keywords

  • peer grading
  • peer assessment
  • SOAP notes

Home

  • AACP
  • AJPE

Articles

  • Current Issue
  • Early Release
  • Archive

Instructions

  • Author Instructions
  • Submission Process
  • Submit a Manuscript
  • Reviewer Instructions

About

  • AJPE
  • Editorial Team
  • Editorial Board
  • History
  • Contact

© 2021 American Journal of Pharmaceutical Education

Powered by HighWire