Understanding and Interpreting Pharmacy College Admission Test Scores

INTRODUCTION

The PCAT is a norm-referenced standardized test developed by NCS Pearson, Inc, to measure content knowledge and cognitive abilities that pharmacy schools consider to be essential for success in their programs. In an effort to ensure the continuing relevance and usefulness of the test for assessing candidates’ prerequisite knowledge and skills, the American Association of Colleges of Pharmacy PCAT Advisory Committee works closely with Pearson to review annual score data and to suggest periodic revisions to the contents of the test. These efforts are intended to ensure that the PCAT accurately reflects current pharmacy school prerequisite requirements in the sciences, math, and language arts, and that candidates’ scores accurately suggest their abilities in these areas.

Immediately upon completing the PCAT at a Pearson Vue test center, candidates are given a preliminary score report. After Pearson verifies that no irregularities occurred that could have affected candidates’ performances, and after candidates’ essays have been scored, each candidate receives an official score report, and each school designated by the candidate receives the candidate’s scores either on a printed official transcript directly from Pearson or electronically from the Pharmacy College Application Service (PharmCAS).

Each official score report and official transcript lists the following information for a candidate’s most recent PCAT test results: scaled scores and percentile ranks for the current four multiple-choice subtests: biological processes, chemical processes, critical reading, and quantitative reasoning; a composite scaled score, indicating an average of the four multiple-choice subtest scaled scores, and a composite percentile rank; a score for the writing subtest and a mean writing score representing the average score earned by all candidates taking the test during the 12 months prior to the candidate’s test date. The official transcript also lists the candidates’ four most recent previous PCAT scores earned during the past 5 years.

The PCAT scaled scores are standardized scores that represent equal units on a continuous scale, ranging from 200 to 600, with a designated mean and standard deviation (originally established in 2004 as 400 and 25, respectively). These scaled scores are derived from candidates’ raw scores, which are the number of items answered correctly for a given subtest. The use of psychometric procedures to equate raw scores on a common scale also facilitates the determination of percentile ranks ranging from 1-99, which represent the percent of candidates who received a scaled score lower than a given score in the current norm group, which is the 64,652 candidates taking the PCAT for the first time from July 2011 through January 2015.

Because they represent equated scores, scaled scores earned for a given subtest during one PCAT test administration are comparable to scaled scores earned for the same subtest during other test administrations, even though different forms of the test are administered. Unlike scaled scores, the percentile ranks are comparable across subtests because they are based on performances relative to the current norm group. For this reason, the percentile ranks are more useful than the scaled scores for comparing individual candidates to one another and for determining a candidate’s strengths and weaknesses relative to other candidates. The subtest percentile ranks are most useful for comparing abilities in the specific subject areas assessed by the PCAT, and the composite percentile rank is most useful for general comparisons.

All multiple-choice scores are reported on official transcripts as scaled scores and 2015 percentile ranks, with all percentile ranks obtained before July 2016 converted to 2015 equivalents. For all PCAT scores earned prior to July 2016, composite scaled scores have been recalculated based on the four subtests included on the PCAT, rather than the five in use when candidates originally took the examination. Additionally, composite percentile ranks for scores earned prior to July 2016 are based on these recalculated scaled scores. The PCAT percentile ranks listed on transcripts issued prior to the July 2016 PCAT administration may be compared to the current percentile ranks by using the compendium tables included in the 2016 edition of the PCAT Technical Manual.¹

Each raw-score point on a test form subtest corresponds to a unique scaled score which, in turn, corresponds to a percentile rank. Because each PCAT multiple-choice subtest includes 40 operational items (items on a test form that are used to determine candidates’ scores), and there are 401 possible scaled-score points (200-600), a difference of one raw-score point necessarily results in a difference of more than one scaled-score point. This does not mean, however, that an increase of one raw-score point (one more item answered correctly) necessarily results in an equal unit increase in scaled-score points (eg, 10 scaled-score points for each raw-score point).

Each test form subtest is composed of a unique set of operational items that vary in difficulty, with no two items requiring the exact same estimate of ability for candidates to answer them correctly. For this reason, a raw-score point total (1-40) earned on one test form subtest reflects a slightly different estimate of the ability required for a candidate to answer the same number of operational items correctly on another test form. It follows, then, that the scaled-score increase associated with each increase in raw-score point will also vary.

A scaled score is assumed to indicate the ability needed to answer a given number of operational items correctly for a given subtest, and the corresponding percentile rank indicates how a given subtest scaled score relates to all candidates from the current normative sample. Table 1 shows average raw-score to scaled-score to percentile-rank correspondences for all test form subtests administered during the 2011-2015 PCAT normative sample period (except for verbal ability, which is no longer part of the PCAT). This table illustrates how 1-point increases in raw score have resulted in corresponding increases in scaled-score points and percentile ranks. Each row in Table 1 shows how a 1-point raw-score increase corresponds to a given average increase in scaled score and percentile rank during the normative sample period. For example, for biological processes, an increase from a raw score of 23 to 24 (shown as “23-24” in the table) corresponds to an average 3-point scaled-score increase and a 5-point increase in percentile rank. Note that the correspondences shown in Table 1 are averages that illustrate patterns observed across multiple test forms administered over several years rather than exact score correspondences for a specific test administration.

Table 1 illustrates what is typically seen in a subtest raw-score to scaled-score relationship: a 1-point difference in the raw score near the lower and upper ends (ie, 1-5 and 35-40) results in greater scaled-score differences than a 1-point raw-score difference near the median where corresponding differences in scaled scores are less. This occurs because there are more candidates earning scores close to the median, which results in greater scaled-score precision (with smaller standard error of measurement), and fewer candidates earning scores at the low and high ends where there is less scaled-score precision.

Table 1 also shows that the pattern in the relationship between scaled-score and percentile-rank correspondences is the opposite of the pattern seen between raw scores and scaled scores. The scaled-score to percentile-rank relationships illustrated in Table 1 reveal that around the subtest means, relatively slight differences in scaled scores result in greater differences in percentile rank. The reason for this is that scaled scores are derived from an assumption that the candidates taking the PCAT constitute an approximately normal distribution of abilities. The large size of the normative sample used to determine the percentile ranks has indeed resulted in the data forming a classic bell curve characteristic of a normal distribution, with the majority of scores clustered around the mean. This cluster of scaled scores around the mean is illustrated in Figure 1 by showing the bell curve formed when these scores are plotted for the current normative sample.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Distribution of Candidates Earning Composite Scaled Score Points for the 2011–2015 Normative Sample (With the Solid Line Indicating the Normal Distribution)

As shown in Figure 1, the percent of candidates earning a given scaled score (indicated on the y-axis) is greatest near the middle of the scaled-score distribution (ie, around 400 on the x-axis). This illustrates why relatively small differences in scaled scores near the mean correspond to greater differences in percentile rank than do scaled scores near the extremities, where there are fewer candidates and where each difference in scaled score results in a smaller difference in percentile rank. For example, as illustrated in Figure 1, a composite scaled score of 405 includes approximately 14% more candidates scoring below that point than does a score of 399 (a 6-point increase), whereas a scaled score of 447 only includes less than 1% more candidates scoring below that point than does a score of 441 (also a 6-point increase).