Skip to main content

Main menu

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Other Publications

User menu

Search

  • Advanced search
American Journal of Pharmaceutical Education
  • Other Publications
American Journal of Pharmaceutical Education

Advanced Search

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Follow AJPE on Twitter
  • LinkedIn
LetterLETTERS

A Plea for Psychometric Rigor

Michael J. Peeters and Lisa M. Hayes
American Journal of Pharmaceutical Education May 2017, 81 (4) 79; DOI: https://doi.org/10.5688/ajpe81479
Michael J. Peeters
University of Toledo College of Pharmacy and Pharmaceutical Sciences, Toledo, Ohio
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lisa M. Hayes
University of Toledo College of Pharmacy and Pharmaceutical Sciences, Toledo, Ohio
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • PDF
Loading

To the Editor: We wish to commend Dr. Cowart and colleagues for their report of a multiple mini-interview (MMI) and its correlation with PharmD students’ grade-point averages.1 Activities described in this investigation were considerable work to accomplish. Necessarily so, non-traditional (also known as non-cognitive2) factors are becoming increasingly relevant and being considered within pharmacy school admissions.3 An important facet overlooked in this MMI report is educational psychometrics, including reliability. Foundational to understanding rigor in this report’s numbers, authors must quantify measurement error (ie, reliability), so that readers can appreciate whether score variations are true differences or simply noise from variations secondary to measurement error. Measurement error can occur with variability among interviewers, inconsistency in rubric scoring and context specificity by using too few stations.4 Unfortunately, without reporting measurement error, readers cannot differentiate between true signal variation and noise. That is, inferential statistics in this report do not seem actionable for us.

Regarding reports of pharmacy education research, Peeters and colleagues have recommended that authors using educational measurement report their reliability.5 Subsequently, Peeters and colleagues6 as well as Cor and Peeters4 have shown how Generalizability (G) theory can be an excellent model to report entire process reliability with interviews (including the multiple mini-interview, and other objective structured clinical examinations). Notably, G theory is not new and was reviewed over a decade ago in medical education.7,8 A recent step-by-step primer also is available.9

Knowing this, we urge pharmacy education stakeholders – authors, peer reviewers, and editors – to seek and insist on reporting this rigor in future study descriptions that used educational measurement. Furthermore, we caution generalizations based on any single performance-based assessment of a student. Psychometrics is a key science for educational assessments that educators should all understand and use adeptly.10

  • © 2017 American Association of Colleges of Pharmacy

REFERENCES

  1. 1.↵
    1. Cowart K,
    2. Dell K,
    3. Rodriguez-Snapp N,
    4. Petrelli HMW
    . An examination of correlations between MMI scores and pharmacy school GPA. Am J Pharm Educ. 2016;80(6):Article 98.
    OpenUrl
  2. 2.↵
    1. Norman G
    . Non-cognitive factors in health sciences education: from the clinic floor to the cutting room floor. Adv Health Sci Educ Theory Pract. 2010;15(1):1-8.
    OpenUrlPubMed
  3. 3.↵
    1. Latif DA
    . Including the assessment of nontraditional factors in pharmacy school admissions. Ann Pharmacother. 2005;39(4):721-726.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Cor MK,
    2. Peeters MJ
    . Using generalizability theory for reliable learning assessments in pharmacy education. Curr Pharm Teach Learn. 2015;7(3):332-341.
    OpenUrl
  5. 5.↵
    1. Peeters MJ,
    2. Beltyukova SA,
    3. Martin BA
    . Educational testing and validity of conclusions in scholarship of teaching and learning. Am J Pharm Educ. 2013;77(9):Article 186.
    OpenUrl
  6. 6.↵
    1. Peeters MJ,
    2. Serres ML,
    3. Gundrum TE
    . Improving reliability of a residency interview process. Am J Pharm Educ. 2013;77(8):Article 168.
    OpenUrl
  7. 7.↵
    1. Crossley J,
    2. Davies H,
    3. Humphris G,
    4. Jolly B
    . Generalisability: a key to unlock professional assessment. Med Educ. 2002;36(10):972-978.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Crossley J,
    2. Russell J,
    3. Jolly B,
    4. et al
    . I’m pickin’ up good regressions: the governance of generalisability analyses. Med Educ. 2007;41(10):926-934.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Bloch R,
    2. Norman G
    . Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. Med Teach. 2012;34(11):960-992.
    OpenUrlCrossRefPubMed
  10. 10.↵
    1. Norman G
    . Is psychometrics science? Adv Health Sci Educ Theory Pract. 2016;21(4):731-734.
    OpenUrl
PreviousNext
Back to top

In this issue

American Journal of Pharmaceutical Education
Vol. 81, Issue 4
1 May 2017
  • Table of Contents
  • Index by author
Print
Download PDF
Email Article

Thank you for your interest in spreading the word on American Journal of Pharmaceutical Education.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
A Plea for Psychometric Rigor
(Your Name) has sent you a message from American Journal of Pharmaceutical Education
(Your Name) thought you would like to see the American Journal of Pharmaceutical Education web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
16 + 0 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
A Plea for Psychometric Rigor
Michael J. Peeters, Lisa M. Hayes
American Journal of Pharmaceutical Education May 2017, 81 (4) 79; DOI: 10.5688/ajpe81479

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
A Plea for Psychometric Rigor
Michael J. Peeters, Lisa M. Hayes
American Journal of Pharmaceutical Education May 2017, 81 (4) 79; DOI: 10.5688/ajpe81479
del.icio.us logo Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • REFERENCES
  • Info & Metrics
  • PDF

Similar AJPE Articles

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Response to a Plea for Psychometric Rigor
  • Expanding Dress Code Requirements in the Doctor of Pharmacy Program
Show more LETTERS

Related Articles

  • No related articles found.
  • Google Scholar

Home

  • AACP
  • AJPE

Articles

  • Current Issue
  • Early Release
  • Archive

Instructions

  • Author Instructions
  • Submission Process
  • Submit a Manuscript
  • Reviewer Instructions

About

  • AJPE
  • Editorial Team
  • Editorial Board
  • History
  • Contact

© 2023 American Journal of Pharmaceutical Education

Powered by HighWire