Abstract
Objective. Studies have examined possible predictors of success on the North American Pharmacist Licensure Examination (NAPLEX). This systematic review investigated the literature on potential predictors of success on the NAPLEX.
Findings. Articles were included in the review if they studied student characteristics and academic performance as independent variables and NAPLEX scores and/or pass rates as an outcome. Data were extracted from each article for students’ demographics or variables, sample size, methods of statistical analyses, and results reporting correlation or predictability. From 40 articles retrieved from the initial search and sorting, 20 studies were included in the final review per inclusion criteria. Three studies included all the pharmacy programs, 15 were single-institution studies, two were multi-institution studies, and four had been published as posters. Among 30 different variables identified as potential predictors of success on the NAPLEX, the most examined variables were student age at matriculation, having a prior degree, Pharmacy College Admission Test (PCAT) scores, cumulative pharmacy school grade point average (GPA), overall Pharmacy Curriculum Outcomes Assessment (PCOA) scores, and PCOA content areas scores. Positively correlated factors included PCAT scores, not having a prior degree, prepharmacy and pharmacy school GPA, institutional characteristics, and PCOA scores. Negatively correlated factors included older age at admission.
Summary. Cumulative pharmacy school GPA and PCOA scores were predictors of NAPLEX success consistently in the studies. The effects of preadmission student characteristics on NAPLEX success varied and were not consistently correlated or predictive.
INTRODUCTION
Passing the North American Pharmacist Licensure Examination (NAPLEX) is required for pharmacists to obtain licensure to practice in the United States. The NAPLEX measures a doctor of pharmacy (PharmD) candidate's entry-level knowledge and competence to practice as a pharmacist, and serves as one of several programmatic outcomes that the Accreditation Council for Pharmacy Education (ACPE) evaluates for accreditation criteria of a pharmacy program. However, concerns regarding graduates’ suboptimal performance on the NAPLEX began to surface when a decrease in the overall pass rate for the NAPLEX from 92.6% in 2015 to 85.9% in 2016 was reported.1 Since then, the pass rate has remained relatively stable with 88.03% in 2018 (first-time pass rate, 89.46%), 86.74% in 2019 (first-time pass rate, 88.34%), and 87.03% (first-time pass rate, 88.43%) in 2020.2 The proliferation of new pharmacy programs,3,4 the changing applicants’ demographics,4 and changes to the NAPLEX format in 2016,5 including changes in passing standards, extension of the length of the examination, and more application-based questions, may have contributed to the lower NAPLEX pass rates in recent years. Additionally, the National Association of Boards of Pharmacy (NABP) published its revised NAPLEX competency statements in January 2021, which have expanded from two main areas to six, each including a range of 4-12 subareas and added expanded contents such as pregnancy or lactation. This change may likely affect the NAPLEX performance of the first cohort of pharmacy graduates in 2021.6
While pharmacy programs make efforts to increase the pass rate of their graduates, there are no clear predictors of success for NAPLEX identified to date. Compiled and described in this review, a score of studies has examined possible predictors of NAPLEX first-time pass rates, such as student demographic, preadmission criteria, Pharmacy College Admission Test (PCAT) scores, pre-NAPLEX scores, the pre-professional curriculum, Pharmacy Curriculum Outcomes Assessment (PCOA) scores, and remediation status. These studies have attempted to provide data-driven direction for admissions-related and programmatic decisions, and collectively have shown the challenges of predicting students’ success on NAPLEX outcomes, partly due to the variability in student sample size and characteristics, study design, outcome variables, and period of evaluation. Only one study to date has comprehensively reviewed previous studies for these variables7; however, it included only third-year pharmacy grade point average (GPA) as a predictor of PCOA scores and a subsequent predictability of PCOA scores on NAPLEX scores. The use of multiple indicators or predictors to identify students at risk of not passing NAPLEX would provide an opportunity for programs to strategize student support prior to taking the NAPLEX. Therefore, the purpose of this systematic review is to comprehensively review previously published data regarding predictors of NAPLEX success.
METHODS
Based on the principles from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA),8 a literature search was conducted to retrieve articles regarding student characteristics at admission and NAPLEX pass rate using PubMed, Embase, American Journal of Pharmaceutical Education, and Currents in Pharmacy Teaching and Learning. Keywords used for the search included one or a combination of the following terms: NAPLEX (both spelled out and the abbreviation), licens* exam, pharmacy, board exam, graduation year, factor, variable, predict*, correlat*, personal characteristic, student demographic, and student profile. An additional search was conducted via the internet search engine Google (Google Scholar) to capture and retrieve studies that were not yet published in a journal or indexed by databases not found in the search. Citations of each retrieved article was reviewed to find any potential articles missed from the database search.
Articles were included in the review if they used NAPLEX passing at the first attempt as an outcome, reported results from pharmacy programs in the United States and/or a US territory, included at least one student characteristic as an independent variable potential for correlating with or predicting the outcome, was published in a peer-reviewed journal or as a poster at a professional conference, and investigated either a correlation or predictability of the student characteristic for NAPLEX passing. Articles were excluded if they were thesis/dissertations, review articles, commentaries, editorial, or non-peer reviewed publications, or published only as an abstract with minimal data; did not study NAPLEX passing as an outcome; or reported NAPLEX passing rates after an instructional intervention or a curriculum revision (eg, incorporation of skills laboratory, modification of an instructional design) was made; or did not report any correlation or predictability of NAPLEX passing.
Three investigators independently searched for relevant articles and discussed their merit for inclusion via consensus-building. Articles were included in the review if all three investigators agreed on their merit for inclusion based on the inclusion criteria. The investigators comprehensively reviewed the retrieved literature and extracted the following data: study year, study institution, student sample size, preadmission variables, first-attempt NAPLEX pass rate, any other assessments reported as independent variables (eg, PCOA scores), and any statistical analyses and results reporting correlation or predictability. Preadmission variables were defined as any independent variable that students possessed without any intervention before they began their pharmacy program such as prepharmacy GPA and PCAT scores. Prior degrees were used for any undergraduate or graduate degrees attained before entering the pharmacy program. If the degree was specifically noted as an undergraduate degree, it was reported as such. The NAPLEX success or NAPLEX passing was defined as passing of NAPELX at the first attempt unless otherwise noted. If the authors reported NAPLEX scores as outcomes instead of NAPLEX success/passing, they were reported here as such. Statistical significance was confirmed by reported p values or effect estimates with confidence intervals. If any statistical significance was claimed but no statistical data were provided to verify the claim, it was noted as “not reported.”
RESULTS
The initial search yielded 40 articles. From these, 20 articles were included based on the inclusion and exclusion criteria. The details of the included studies and their findings are summarized in Appendix 1.9-28 Of the excluded articles, three contained duplicate information (ie, a poster which was later published as full-text); eight articles were commentaries, editorials, or non-peer reviewed publications; five articles did not have passing the NAPLEX as an outcome, three articles correlated NAPLEX passing after a change in the curriculum, and one article reported correlation between the student characteristics with pre-NAPLEX scores. The included studies were published over a period of 15 years from 2005 to 2020, investigated NAPLEX passing from pharmacy programs ranging from a single institution to 137 institutions, and collectively covered over 13,000 individual students’ NAPLEX results. Four of the 20 studies were published as posters with extractable data.15-17,19 Two publications by Shaya and colleagues15,16 were of the same student cohort from one institution but had results for different predictor variables. Common predictors investigated in the studies were age, gender, having a prior degree, prepharmacy GPA, PCAT scores, pharmacy GPA, PCOA scores, pre-NAPLEX scores, and on-time graduation. Each predictor is discussed in more detail below. Other variables included attending different campuses of the same program, type of program or institution, critical-thinking test scores, personality type, and remediation.
Six studies10,15,16,19,25,28 evaluated age as a predictor of performance on NAPLEX. Of these, four found age to be an independent predictor for NAPLEX success.15,16,19,25 Two of these studies were from one institution but they reported using two separate analyses of the same cohort of students, and found that student age ≥25 years was associated with lower scores on both the NAPLEX and the Multistate Pharmacy Jurisprudence Examination (MPJE).15,16 A three-institution study19 found that age ≥25 years was a negative predictor of NAPLEX scores in all three areas as well as the total score (p < .01). This result was also confirmed in another single-institution study in which age at graduation >28 years was the only significant factor that had a negative correlation to NAPLEX score (p < .014) and was predictive of poorer outcomes on the NAPLEX (p < .002).25 McCall and colleagues10 found that, when controlled for composite PCAT and prepharmacy GPA, age at admission was negatively correlated with NAPLEX (r = −.19, p < .001). While age was not significantly associated with NAPLEX success, Spivey and colleagues28 found that increased age was predictive of decreased likelihood of on-time graduation (p < .001).
Six studies assessed the correlation between gender and NAPLEX performance.15,16,19,20,22,28 Only one three-institution study found that male students had higher mean NAPLEX scores than female students, but only by 1.65 points (p = .017).19 No other studies reported positive correlation between male gender and NAPLEX scores.
Eight studies investigated the correlation between students having a prior or other degree at admission on NAPLEX performance.10,12,15,16,19,20,25,28 Five of these studies found that having an undergraduate degree(s) had no association with or predictability of NAPLEX performance.10,12,19,20,25 However, having a prior degree was negatively correlated with NAPLEX performance in a study with one cohort of students from one institution.15,16 Shaya and colleagues16 found that not having a prior degree was positively predictive of the score on the pharmacotherapy section of NAPLEX (p < .05). Relatedly, Spivey and colleagues28 found that having an undergraduate degree was a negative predictor of on-time graduation.
Eight studies evaluated PCAT performance as a predictor of NAPLEX performance and attempted to investigate both the PCAT composite score as well as PCAT section scores.10,15,16,19,20,24,25,28 All of the studies except one found a positive correlation with NAPLEX scores; however, in the study by Spivey and colleagues,28 PCAT composite percentile was not associated with NAPLEX passing. McCall and colleagues10 found that PCAT composite score had the highest correlation value with NAPLEX score (r = .40, p < .001) and each of the five PCAT subscores showed a positive correlation with NAPLEX score (all p < .001). Shaya and colleagues15 found that PCAT chemistry, reading comprehension, and verbal ability correlated with NAPLEX scores. In a sub-analysis that examined the relationship between PCAT scores and specific sections of NAPLEX, Shaya and colleagues16 found that scores in area 1 were explained by PCAT chemistry and verbal ability, scores in area 2 were explained by quantitative ability and reading comprehension, and scores in area 3 were explained by biology, quantitative ability, and verbal ability (Appendix 1). These results were confirmed by another study using a three-institution cohort19; both NAPLEX and MPJE pass rates were associated with higher chemistry, reading comprehension, and verbal scores on the PCAT. Chisholm-Burns and colleagues20 also found a positive correlation with PCAT section scores and composite scores with NAPLEX total scaled scores (p < .05 for both). Shah and colleagues25 found that low PCAT composite scaled score (<74) was a predictor of poor NAPLEX performance (p < .05).
Seven studies investigated the relationship between prepharmacy GPA and NAPLEX success.10,12,15,16,19,24,28 Of these, two studies found a positive correlation.10,12 McCall and colleagues10 found that prepharmacy PGA was positively correlated with NAPLEX score (p < .001); however, neither the type of institution where organic chemistry was completed (2-year vs 4-year) nor taking advanced chemistry, biology, and math courses was significantly correlated with NAPLEX score. Additionally, the best predictor of NAPLEX score was a combination of composite PCAT score, prepharmacy GPA, and age.10 Allen and Diaz12 found that prepharmacy cumulative GPA, math-science GPA, transfer admission, and having no unsatisfactory grades at admission positively correlated with NAPLEX success (p < .001). No other studies corroborated this correlation or predictability. Two other studies found no significant relationship between NAPLEX scores and prepharmacy major or NAPLEX scores and prior institution’s ranking, size, or location.15,16
Among the nine studies that investigated the relationship between pharmacy GPA and NAPLEX success,12,15,16,19-21,24,25,28 seven of them found a positive correlation between the two.12,15,16,19-21,25 Allen and Diaz12 found that cumulative pharmacy GPA had a positive correlation with NAPLEX success (p < .001). They also found that a total number of unsatisfactory grades, having no unsatisfactory grades, and on-time graduation were positively correlated with NAPLEX success (p < .001). Cumulative pharmacy GPA was positively correlated with NAPLEX15 and MPJE success,19 and with all three areas of NAPLEX scores.16 Hein and colleagues24 found a positive association between P3 pre-APPE GPA with NAPLEX success (p < .05), while Shah and colleagues25 found no correlation with P3 cumulative GPA. Nevertheless, having a final course percentage less than 74% in three or more courses was positively correlated with poor NAPLEX scores (p < .05).25 Chisholm-Burns and colleagues20 suggested that cumulative pharmacy GPA was one of the significant predictors of NAPLEX success in their predictability equation where NAPLEX total scaled score was 3.514 + 25.479 (pharmacy GPA)+0.205 (pre-NAPLEX score). Garavalia and colleagues21 found that GPA and PCOA together accounted for 39% of the variation in the NAPLEX scores (p < .001), with GPA (14%) explaining more than PCOA scores (8%). While not shown to have a positive correlation with NAPLEX success, Spivey and colleagues28 found that P1 GPA was a positive predictor of on-time graduation and a negative predictor of dismissal from the program.
From five studies investigating academic progression as a predictor of NAPLEX success,11,12,20,25,28 Madden and colleagues11 found that the NAPLEX passing rate significantly differed between students who required remediation and those who did not (70% vs 97%, respectively; p < .001). Two studies found that on-time graduation was positively correlated with NAPLEX success.11,20 However, Shah and colleagues25 found that graduation delay was not one of the predictors of poor NAPLEX performance. Spivey and colleagues28 found that appearing before an academic progression committee was a negative predictor of NAPLEX passing at first attempt (p < .02), and that age at graduation and having an undergraduate degree were negative predictors of on-time graduation (p < .001 and p < .01, respectively).
All six studies14,17,21,23-25 found a positive correlation between NAPLEX and PCOA scores. Naughton and Friesner14 found a significantly positive correlation between PCOA scaled score and four subtopic domain scores on the NAPLEX total score and scores in areas 1, 2, and 3 (range: r = .167 to .59; p < .05). However, the PCOA Social, Behavioral, and Administrative Science subtopic score did not significantly correlate with NAPLEX competency area 2 scores consistently, with mixed results between Pearson (p = .084) and Spearman (p = .033) analyses. Hutchinson and colleagues17 found that combined scores of the Pharmaceutical Sciences and Clinical Sciences content areas explained 32.8% of the variation in the NAPLEX scores (r = .572; p < .0005) and those areas were predictive of NAPLEX scores by 51.4 (Pharmaceutical Sciences) and 0.70 (Clinical Science). This association was also corroborated in two other studies in which PCOA score explained 26% of the variance in NAPLEX scores,21 and PCOA total and contents scores explained 30%-33% of variance in total NAPLEX scores.23 Basic Biological Sciences, and Social, Behavioral, and Administrative subtopic score were not predictors of NAPLEX scores23 and did not correlate with NAPLEX Competency area 2 scores.14 Additionally, Hein and colleagues24 found that PCOA scores were positively associated with P3 GPA prior to starting advanced pharmacy practice experiences (APPEs).
The results were split between the two studies that investigated the association between pre-NAPLEX test scores and NAPLEX scores.20,28 Chisholm-Burns and colleagues20 found that pre-NAPLEX was a significant predictor of NAPLEX score and explained 44% of the variance in NAPLEX total scaled scores (p < .05). Spivey and colleagues28 found no correlation or predictability with pre-NAPLEX score.
McCall and colleagues10 found a positive correlation between NAPLEX scores and the California Critical Thinking Skills Test, which was taken during the admissions interview (p < .001). Interestingly, Ware22 found that Myers-Briggs Type Indicators were predictive of first-attempt NAPLEX scores with 17% of the variation explained by the types (adjusted r2 = 12.4%); Introversion personality types scored 9.5 points higher than Extroversion types (p < .01), and Feeling types scored 6.0 points higher than Thinking types (p = .03).
Based on one study by Prabhu and colleagues,18 NAPLEX pass rates did not differ between graduating students from a traditional PharmD program versus those who were internationally trained pharmacists graduating from an international postbaccalaureate PharmD program. These international graduates performed better compared with graduates from the traditional group in average total scaled score (115.1 vs 107.6; p < .001), in area 1 scores (13.6 vs 13.1; p = .013), and in area 2 scores (13.5 vs 12.5; p < .001), respectively; however, no significant difference was found in area 3 scores.
Zaremski and colleagues9 compared NAPLEX first-time pass rate between 74 programs accredited before 1992 and nine programs accredited in or after 1992, and found small but significant difference between the two groups: 19,124 (96.4%) passed from the pre-1992 while 1,070 (94.9%) passed from the post-1992 (p < .05). While no statistical difference was found on the first-time pass rate in 2000, 2001, or 2002, a difference was found for 2003 (p value not reported). Additionally, a significant difference was found between NAPLEX scaled scores and pre-1992 vs post-1992 groups (p < .001) and testing year (p < .015). Approximately 14 years later, Jimenez and colleagues26 conducted a comprehensive review of 128 accredited programs with PharmD graduates as of May 2018 and compared their NAPLEX pass rates over a three-year period with the programs’ nonmodifiable factors (eg, graduating class size <100, public vs private, four-year vs three-year curriculum, GPA vs pass/fail grading system). Positive correlations were found between public institution vs private and graduating class size ≥100 in the years 2015 and 2017. No correlation was found for four-year vs three-year curriculum, graduating class size ≥100 in 2016, or GPA vs pass/fail grading system. However, Williams and colleagues27 evaluated 137 programs accredited as of July 2017 and found a positive correlation between NAPLEX total scaled scores and public institution, location of the program in an academic health center, establishment before year 2000, and traditional four-year curriculum. For institutions having more than one campus, the location of the campus (eg, main vs satellite) had no correlation with NAPLEX scores or pass rates in one study,13 whereas another study found a 5.7 point higher score in one campus vs the other.22
Williams and colleagues27 also found a positive correlation between NAPLEX total scaled score and the percentage of P4 students who matched for a first PGY-1 residency. However, the NAPLEX scores were negatively correlated with the percentage of out-of-state students. Shaya and colleagues found a similar result where the students from the western region of the country was negatively predictive of NAPLEX scores (p < .05)15 and that the students were negatively predictive of the NAPLEX area 1 result (p < .05).16
DISCUSSION
This is the first comprehensive review of potential student-related factors correlated with or predictive of NAPLEX success. Given the number of publications on this topic and the breadth of data collection and evaluation, NAPLEX passing rate is a critically important measure of programs’ success and an area of ongoing interest.29,30 This review elucidates the efforts and curiosity of pharmacy programs to define potential variables of NAPLEX success amid the vague and subjective nature of those variables in answering the question. McCall and colleagues10 demonstrated that while composite PCAT, prepharmacy GPA, and age had the strongest correlation with NAPLEX success, they explained only 21% of the variance in NAPLEX performance and the other 79% of the variance was explained by other variables. While the results of this review may not provide a precise answer to the question of student-related variables that would increase their NAPLEX success, they provide an overview of the factors and congruency among different studies investigating similar variables. For example, cumulative pharmacy GPA and PCOA scores were one of the most frequently studied variables and were also consistently correlated or predictive of NAPLEX scores. Interestingly, these results corroborated a recent report by NABP, which reported a comprehensive analysis on the relationship between PCOA scores of 49,510 students and their NAPLEX outcomes over a three-year data period.32 While the analysis was considered preliminary, the report concluded that PCOA was a strong predictor of NAPLEX performance.
The results of this review should be carefully and critically interpreted given their limitations. The most significant limitation is that the data do not represent all pharmacy programs or their entire student populations; thus, some of the positive or negative correlation data may not be generalizable to all programs. The three nationwide, multi-institution studies evaluated only the characteristics of the programs and not student variables.9,26,27 To comprehensively determine the predictability of student-dependent characteristics for NAPLEX success, a large-scale study of all pharmacy programs would be warranted.
Moreover, several changes have occurred in pharmacy academia recently that may affect the generalizability of this result. First, many programs have decided to eliminate the PCAT requirements for pharmacy school admission due to competition among programs to recruit students and ease of admission application process for the candidates. Thus, the correlation of PCAT data with NAPLEX success may not be relevant to some programs. Given that several studies did demonstrate that PCAT scores were correlated with NAPLEX scores, the PCAT requirement may be reconsidered as part of the admission criteria. Second, NABP revised its NAPLEX competency statements in January 2021.6 This change evokes curiosity and a concern for NAPLEX pass rates for the class of 2021. Using the previous NAPLEX change in 2016 as an example, four studies from our review included NAPLEX performance data from both pre- and post-2016 NAPLEX blueprint.25-28 All four studies reported a decrease in the NAPLEX pass rate in the class of 2016 compared with other years: the graduating class of 2016 from 128 programs performed significantly worse compared with the graduating classes of 2015 and 2017 (p < .05).26 Additionally, the NAPLEX pass rate of graduates in 2016 was significantly lower than that of graduates in 2014 and 2015 (p < .001). Furthermore, 29% of schools experienced more than a 10% drop in NAPLEX pass rates between 2015 and 2016.27 Given these results, we can presume that any changes to the NAPLEX blueprint may negatively affect pharmacy students’ NAPLEX performance, especially during the first year after the change. Moreover, students’ preparedness for and familiarity with the test or lack thereof may potentially affect their program’s first-time performance. Given that NAPLEX score reporting is also changing from scaled scores to a pass/fail system in 2021,33 the correlation or predictability of the variables with NAPLEX scores may not be relevant or helpful moving forward.
Nevertheless, there is a need for a more consistent and systematic approach to studying the effectiveness of student demographic and performance variables to inform academic support decisions, especially for NAPLEX preparation. Given the uncertainty of some of these variables in predicting NAPLEX outcome, more judicious and cautious use of these results is recommended, ie, overtly accepting or dismissing these variables as NAPLEX predicators for all students would be precarious and unreliable. While there are inherent challenges and limitations to considering a single predictor for this purpose, the use of multiple variables and predictors may assist in identifying students at risk for not passing NAPLEX. The results of this review may serve as a tool for pharmacy programs to supplement or support their own assessment data in an effort to prepare for early intervention programs to change the trajectory of “at-risk” students in the program.
CONCLUSION
Cumulative pharmacy school GPA and PCOA scores were consistent predictors of NAPLEX success in the studies included in this review. Being older, having a prior degree, having lower PCAT scores, and experiencing a delay in graduation negatively affected pharmacy students’ NAPLEX scores. The findings on the effects of other preadmission characteristics, pre-NAPLEX scores, and institutional characteristics on NAPLEX success varied because of differences in student population and sample sizes used in the studies. The predictive variables highlighted in this review, supplemented with each institution’s own data, may help schools to identify at-risk students for early intervention and prepare for the changing NAPLEX landscape.
Summary of Literature Investigating Student-related Factors Correlated with or Predictive of NAPLEX Success9-28, a
- Received February 13, 2021.
- Accepted May 4, 2021.
- © 2021 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.