Skip to main content

Main menu

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Other Publications

User menu

Search

  • Advanced search
American Journal of Pharmaceutical Education
  • Other Publications
American Journal of Pharmaceutical Education

Advanced Search

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Follow AJPE on Twitter
  • LinkedIn
Research ArticleRESEARCH

Comparison of Answer-Until-Correct and Full-Credit Assessments in a Team-based Learning Course

Michelle Z. Farland, Patrick B. Barlow, T. Levi Lancaster and Andrea S. Franks
American Journal of Pharmaceutical Education March 2015, 79 (2) 21; DOI: https://doi.org/10.5688/ajpe79221
Michelle Z. Farland
aUniversity of Florida College of Pharmacy, Gainesville, Florida
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Patrick B. Barlow
bThe University of Iowa Carver College of Medicine, Iowa City, Iowa
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
T. Levi Lancaster
cUniversity of Tennessee Health Science Center College of Pharmacy, Knoxville, Tennessee
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrea S. Franks
cUniversity of Tennessee Health Science Center College of Pharmacy, Knoxville, Tennessee
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Objective. To assess the impact of awarding partial credit to team assessments on team performance and on quality of team interactions using an answer-until-correct method compared to traditional methods of grading (multiple-choice, full-credit).

Methods. Subjects were students from 3 different offerings of an ambulatory care elective course, taught using team-based learning. The control group (full-credit) consisted of those enrolled in the course when traditional methods of assessment were used (2 course offerings). The intervention group consisted of those enrolled in the course when answer-until-correct method was used for team assessments (1 course offering). Study outcomes included student performance on individual and team readiness assurance tests (iRATs and tRATs), individual and team final examinations, and student assessment of quality of team interactions using the Team Performance Scale.

Results. Eighty-four students enrolled in the courses were included in the analysis (full-credit, n=54; answer-until-correct, n=30). Students who used traditional methods of assessment performed better on iRATs (full-credit mean 88.7 (5.9), answer-until-correct mean 82.8 (10.7), p<0.001). Students who used answer-until-correct method of assessment performed better on the team final examination (full-credit mean 45.8 (1.5), answer-until-correct 47.8 (1.4), p<0.001). There was no significant difference in performance on tRATs and the individual final examination. Students who used the answer-until-correct method had higher quality of team interaction ratings (full-credit 97.1 (9.1), answer-until-correct 103.0 (7.8), p=0.004).

Conclusion. Answer-until-correct assessment method compared to traditional, full-credit methods resulted in significantly lower scores for iRATs, similar scores on tRATs and individual final examinations, improved scores on team final examinations, and improved perceptions of the quality of team interactions.

Keywords
  • team-based learning
  • active learning
  • pharmacy education
  • answer-until-correct
  • partial credit

INTRODUCTION

A variety of methods are used to grade assessments in courses that use team-based learning (TBL). Pharmacy educators use of the Immediate Feedback Assessment Technique, or theIF-AT (Epstein Educational Enterprises, Cincinnati, OH), on tRATs and on team examinations.1-7 In Allen and colleagues survey of faculty members from colleges’ and schools of pharmacy in the United States, 74% of respondents who had implemented TBL were using the IF-AT forms for assessment.8 The IF-AT card is a response form with a preset answer key designed to be used with multiple-choice questions.9-11 Students work in teams to determine the best answer to an assessment item and scratch the area under the answer they think is correct. A star is revealed when students answer correctly, and a blank box is revealed when students answer incorrectly. Following an incorrect answer, students have the opportunity to scratch the area under another answer choice until the correct answer is revealed. This method of answer-until-correct allows for partial credit to be awarded for each item of the assessment. It also provides immediate feedback for students and faculty members.

Of those who report using IF-AT forms in TBL sessions, only 2 specify using a partial grading method.3-4 However, the grading method is not compared to other methods in these reports. Persky and colleagues assessed the impact of IF-AT forms compared to traditional assessment methods (ie, multiple-choice, true/false, short-answer) for individual course examinations in a pharmacokinetics course not using TBL.12 They found no differences in student performance but did find that students preferred IF-AT forms over traditional assessment methods. Published comparisons of student performance in pharmacy education using IF-AT forms for team assessments in TBL courses do not describe the impact on the quality of team interactions.

The purpose of this study was to assess the impact of awarding partial credit to team assessments using an answer-until-correct method compared to traditional methods of grading on both student performance and quality of team interactions.

METHODS

Team-based learning was implemented in 2008 in an elective course titled Ambulatory Care.13 Since this time, the course has been offered annually during the fall semester to third-year student pharmacists and has used TBL as the sole instructional method. The course directors consistently incorporate the core elements of TBL in the course structure.14 During the 2013 fall semester, IF-AT forms were introduced to the course to be used as the assessment tool for tRATs.

We conducted a retrospective longitudinal study that included students enrolled in the ambulatory care elective in the 2010, 2011, and 2013 fall semesters. The control group (full-credit) consisted of students enrolled in the course in the 2010 and 2011fall semesters. The course used a traditional, full-credit assessment method for tRATs at those times. Students from those course offerings were identified as the control group because they had completed the Team Performance Scale as a component of the course to assess quality of team interactions. The intervention group (answer-until-correct) consisted of students enrolled in the course in the 2013 fall semester when IF-AT forms were used to grade tRATs. Topics included in each course offering varied slightly from year to year based on instructor availability. Table 1 displays the topics included in each course offering. Topics were consistent in the course except for anticoagulation part 2, women’s health, and chronic noncancer pain. Students were divided into 5-6 member teams. At the start of each class session, students completed an iRAT, and immediately after, the tRAT. The topic instructor then led a discussion of misconceptions identified by the tRAT. Students completed the team application activity, which was followed by interteam discussions led by the topic instructor. Students completed one course examination at the conclusion of the course, first as individuals, then as teams. All assessments comparing individual and team performance used the same items.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1.

Topics Addressed in the Course by Semester

All iRATs and tRATs consisted of 10 multiple-choice questions with 4 answer choices. The total possible points earned on tRATs changed when the answer-until-correct method was implemented. The full-credit group could earn up to 10 possible points on the tRAT, 1 point per question. The answer-until-correct group could earn up to 40 possible points on the tRAT. Partial credit was awarded based on the number of answer choices revealed on the IF-AT form. Full credit (4 points) was awarded when students revealed 1 answer choice. Half credit (2 points) was awarded when students revealed 2 answer choices. Students who revealed 3 and 4 answer choices were awarded 1 and 0 points, respectively. Grades from the answer-until-correct group’s tRATs were scaled to a 10-point scale for comparison.

Study outcomes included student performance on iRATs, tRATs, individual final examination, team final examination, and student-rated quality of team interactions using the Team Performance Scale, an 18-item survey that has evidence for reliability (Cronbach α=0.97) to assess the quality of team interactions.15 Each item was assessed on a 0-6 Likert scale with 0 representing none of the time and 6 representing all of the time (highest possible composite score was 108). Higher composite scores indicated higher performing teams.

Descriptive statistics were used to summarize student performance on the iRAT and tRAT by topic and overall and on the Team Performance Scale. For statistical comparison, number of correct items (raw score) was used as opposed to percent correct. Total possible points for the iRAT and tRAT were 110 points each (1 point per item, 10-item RATs, 11 RATs in each course). Total possible points earned for the individual and team final examinations were 50 points each (1 point per item, 50-item examination). Preliminary Pearson correlations were conducted to assess whether cumulative grade point average (GPA) was significantly associated to any of the outcome variables, and would therefore need to be controlled as a possible confounder. When such a confounding relationship was found, analysis of covariance (ANCOVA) was performed to compare the 2 study groups’ performance on each outcome while controlling for cumulative GPA. Independent t tests were used to compare the study groups when no confounding effect was discovered. All tests were 2-sided, and an alpha value below 0.05 was considered significant. Prior work established reliability of the Team Performance Scale in a sample of medical students.15 Cronbach alpha was calculated as a measure of reliability for the Team Performance Scale in a sample of student pharmacists to confirm it could be used in this population. Analyses were conducted using IBM SPSS v.21 (IBM Inc., Chicago, IL). The study was deemed exempt by the Institutional Review Board.

RESULTS

Eighty-four students enrolled in the courses were included in the analysis (full credit, n=54; answer-until-correct, n=30). Cumulative GPA prior to enrollment in the course was similar between groups (full-credit mean 3.1 (0.3), answer-until-correct mean 3.1 (0.4), p=0.693). However, GPA was significantly related to both iRAT and tRAT scores, so it was controlled for in these analyses. The average iRAT scores are listed in Table 2. Students enrolled in the full-credit course offering scored significantly higher on iRATs overall (full credit mean 88.7 (5.9), answer-until-correct mean 82.8 (10.7), p<0.001) with differences found on the following topics: heart failure, hypertension, anticoagulation part 2, women’s health, smoking cessation, and dyslipidemia. The average tRAT scores are listed in Table 3. There was no difference between study groups in overall performance on the tRAT (full-credit mean 105.0 (2.9), answer-until-correct mean 104.7 (3.9), p=0.635). However, there was a significant difference found in student performance on tRATs for heart failure, hypertension, anticoagulation part 2, asthma, women’s health, and dyslipidemia.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 2.

Individual Readiness Assessment Test Scores by Topic and Study Group

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 3.

Team Readiness Assessment Test Scores by Topic and Study Group

There was no statistical difference found in student performance on the individual final examination (full-credit mean 38.0 (3.7), answer-until-correct mean 39.4 (4.4), p=0.147). Students in the answer-until-correct group scored significantly higher on the team final examination (full-credit mean 45.8 (1.5), answer-until-correct 47.8 (1.4), p<0.001).

The reliability testing using Cronbach alpha indicated the team performance scale was a sufficiently reliable instrument (Cronbach α=0.94). Quality of team interactions differed significantly between study groups (full-credit 97.1 (9.1), answer-until-correct 103.0 (7.8), p=0.004). Table 4 summarizes the results for each item of the Team Performance Scale. Statistical comparison of each item was not conducted as the tool was not intended to be used in this manner. However, the mean score reported for each item was higher in the answer-until-correct group.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 4.

Team Performance Scale Item Summary

DISCUSSION

Student performance was mixed depending on type of assessment and method of grading. Students performed significantly better on iRATs when using traditional methods of grading for tRATs. This contradicted the hypothesis that different methods of grading team assessments would not impact performance on individual assessments. One theory why this may have occurred is that student preparation prior to class sessions may have declined when partial credit was available for team assessments. Part of the rationale behind using both an individual and team assessment prior to the application phase of TBL is that students are held accountable to themselves in the individual assessment and to the team in the team assessment. This process may entice students to prepare more thoroughly before class to not let themselves or the team down. Thus, when partial credit on team assessments is introduced, some pressure for team accountability is potentially alleviated, resulting in less optimal preclass preparation; however, this theory needs to be scientifically tested.

Overall student performance on tRATs was similar between grading methods used. However, students in the full-credit group performed significantly better on 4 of the 11 course topics, whereas the partial-credit group performed better on 2 of the 11 course topics. This outcome was unexpected, as one would assume the opportunity for partial credit would have potential to increase the grade for the assessment, and indicates that use of partial credit using an answer-until-correct method did not inappropriately inflate grades for the tRAT. There were, however, anecdotal observations made by course faculty members that students who used the answer-until-correct method spent more time in team discussion while completing the tRAT compared to students in prior course offerings. Persky and colleagues also observed that students required additional time to complete assessments when using the answer-until-correct method for individual course examinations.12 Prior to the implementation of IF-AT forms in the pharmacokinetics course, examinations consisted of open-ended problems, multiple-choice, true/false, and short-answer questions and were scheduled for 2 hours. Following implementation of the IF-AT forms, the entire examination was made up of multiple-choice questions, and its duration increased to 3 hours to allow sufficient time for completion.

Student performance on individual course examinations did not differ between groups as was expected. Performance on the team course examination was significantly higher in the answer-until-correct group. One possible reason for the observed but small difference in team final examination performance was the emphasis on team development throughout the course that led to improved team performance. The Team Performance Scale scores were higher for teams that used the answer-until-correct assessment method compared to the full-credit group. However, overall tRAT scores did not differ significantly between methods. Future research should be conducted to determine if either assessment method used in this study could help teams work through Tuckman’s stages of team development (forming, storming, norming, performing).16 It can be hypothesized that teams who reach the performing stage would perform better on team activities such as a tRAT or team final examination.

A potential limitation of the study was that topics and faculty facilitators changed slightly in the course each year, although 80% of the topics and faculty facilitators remained the same. Also, differences in RAT items among courses might have contributed to the differences observed in iRAT and final team examination scores. While the content was similar in each course offering, the RAT questions did not remain exactly the same, based on feedback from students during the appeal process that faculty members used to improve questions for future offerings. This may have limited the ability to compare findings from different offerings of the same course. However, constant revisions and updates to course content reflect the reality of teaching practice. Another limitation was that the sample size was relatively small, but the analysis was conducted in an elective course with a smaller enrollment than required courses in the curriculum. A similar comparison in a course with higher student enrollment should be conducted to confirm the results of this study.

CONCLUSION

The answer-until-correct assessment method compared to traditional methods using full-credit resulted in lower iRAT scores, similar tRAT and individual final examination scores, and improved team final examination scores and perceptions on team interaction quality. Additional research is needed to identify reasons for decreased performance on iRATs and to assess the impact answer-until-correct assessment methods have on time to complete the assessment. We also recommend that additional research focus on assessing progression through stages of team development to help identify activities that assist teams in reaching the performing stage.

  • Received May 6, 2014.
  • Accepted August 3, 2014.
  • © 2015 American Association of Colleges of Pharmacy

REFERENCES

  1. 1.↵
    1. Persky AM
    . The impact of team-based learning on a foundational pharmacokinetics course. Am J Pharm Educ. 2012;72:Article 31.
  2. 2.↵
    1. Persky AM,
    2. Pollack GM
    . A modified team-based learning physiology course. Am J Pharm Educ. 2011;75:Article 204.
  3. 3.↵
    1. Kolluru S,
    2. Roesch DM,
    3. de la Fuenta AA
    . A multi-instructor, team-based, active-learning exercise to integrate basic and clinical sciences content. Am J Pharm Educ. 2012;76:Article 33.
  4. 4.↵
    1. Redwanski J
    . Incorporating team-based learning in a drug information course covering tertiary literature. Curr Pharm Teach Learn. 2012;4(3):202-206.
    OpenUrl
  5. 5.↵
    1. Pogge E. A
    team-based learning course on nutrition and life-style modification. Am J Pharm Educ. 2013;77(5):Article 103.
  6. 6.↵
    1. Grady SE
    . Team-based learning in pharmacotherapeutics. Am J Pharm Educ. 2011;75(7):Article 136.
  7. 7.↵
    1. Bleske BE,
    2. Remington T,
    3. Wells TD,
    4. et al
    . Team-based learning to improve learning outcomes in a therapeutics course sequence. Am J Pharm Educ. 2014;78(1):Article 13.
  8. 8.↵
    1. Allen RE,
    2. Copeland J,
    3. Franks AS,
    4. et al
    . Team-based learning in US colleges and schools of pharmacy. Am J Pharm Educ. 2013;77(6):Article 115.
  9. 9.↵
    1. Epstein ML,
    2. Epstein BB,
    3. Brosvic GM
    . Immediate feedback during academic testing. Psychol Rep. 2001;88(3 Pt1):889-894.
    OpenUrlCrossRefPubMed
  10. 10.↵
    1. Epstein ML,
    2. Brosvic GM
    . Students prefer the immediate feedback assessment technique. Psychol Rep. 2002;90(3 Pt2):1136-1138.
    OpenUrlPubMed
  11. 11.↵
    1. Parmelee D,
    2. Michaelsen LK,
    3. Cook S,
    4. Hudes P
    . Team-based learning: a practical guide: AMEE Guide No. 65. Med Teach. 2012;34:e275-e287.
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Persky AM,
    2. Pollack GM
    . Using answer-until-correct examinations to provide immediate feedback to students in a pharmacokinetics course. Am J Pharm Educ. 2008;72(4):Article 83.
  13. 13.↵
    1. Zingone MM,
    2. Franks AS,
    3. Guirguis AB,
    4. George CM,
    5. Howard-Thompson A,
    6. Heidel RE
    . Comparing team-based and mixed active-learning methods in an ambulatory care elective course. Am J Pharm Educ. 2010;74(9):Article 160.
  14. 14.↵
    1. Farland MZ,
    2. Sicat BL,
    3. Franks AS,
    4. Pater KS,
    5. Medina MS,
    6. Perksy AM
    . Best practices for implementing team-based learning in pharmacy education. Am J Pharm Educ. 2013;77(8):Article 177.
  15. 15.↵
    1. Thompson BM,
    2. Levine RE,
    3. Kennedy F,
    4. et al
    . Evaluating the quality of learning-team processes in medical education: development and validation of a new measure. Acad Med. 2009;84(10 Suppl):S124-S127.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Tuckman BW
    . Developmental sequence in small groups. Psychol Bull. 1965;63(6):384-399.
    OpenUrlCrossRefPubMed
PreviousNext
Back to top

In this issue

American Journal of Pharmaceutical Education
Vol. 79, Issue 2
25 Mar 2015
  • Table of Contents
  • Index by author
Print
Download PDF
Email Article

Thank you for your interest in spreading the word on American Journal of Pharmaceutical Education.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Comparison of Answer-Until-Correct and Full-Credit Assessments in a Team-based Learning Course
(Your Name) has sent you a message from American Journal of Pharmaceutical Education
(Your Name) thought you would like to see the American Journal of Pharmaceutical Education web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
1 + 2 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Comparison of Answer-Until-Correct and Full-Credit Assessments in a Team-based Learning Course
Michelle Z. Farland, Patrick B. Barlow, T. Levi Lancaster, Andrea S. Franks
American Journal of Pharmaceutical Education Mar 2015, 79 (2) 21; DOI: 10.5688/ajpe79221

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Comparison of Answer-Until-Correct and Full-Credit Assessments in a Team-based Learning Course
Michelle Z. Farland, Patrick B. Barlow, T. Levi Lancaster, Andrea S. Franks
American Journal of Pharmaceutical Education Mar 2015, 79 (2) 21; DOI: 10.5688/ajpe79221
del.icio.us logo Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

Similar AJPE Articles

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Multi-institutional Analysis of Student and Program Variables as Predictors of Performance on the NAPLEX
  • Exploring Factors That Influence Student Engagement in Community-Engaged Learning Activities Within a Pharmacy Context
  • Self-Assessment of Therapeutic Decision-Making Skills in Pharmacy Students
Show more RESEARCH

Related Articles

  • No related articles found.
  • Google Scholar

Keywords

  • team-based learning
  • active learning
  • pharmacy education
  • answer-until-correct
  • partial credit

Home

  • AACP
  • AJPE

Articles

  • Current Issue
  • Early Release
  • Archive

Instructions

  • Author Instructions
  • Submission Process
  • Submit a Manuscript
  • Reviewer Instructions

About

  • AJPE
  • Editorial Team
  • Editorial Board
  • History
  • Contact

© 2022 American Journal of Pharmaceutical Education

Powered by HighWire