Impact of Pharmacy School Characteristics on NAPLEX First-time Pass Rates

INTRODUCTION

The North American Pharmacist Licensure Examination (NAPLEX) is the standard for which state boards of pharmacy assess the entry-level competence of candidates for pharmacy licensure.^1-2 Several studies have examined possible associations between the pass rates of students taking the NAPLEX for the first time and various factors such as preadmission criteria, pre-NAPLEX scores, the pre-professional curriculum, Pharmacy Curriculum Outcomes Assessment (PCOA) scores, remediation status, and licensure preparation tools.^3-9

McCall and colleagues noted that a higher NAPLEX score was associated with higher pre-pharmacy grade point average (GPA), younger age, and higher composite Pharmacy College Admissions Test (PCAT) scores.³ Chisholm-Burns and colleagues reported that both pharmacy GPA and Pre-NAPLEX score were significant predictors of NAPLEX score. Other variables such as race and ethnicity, PCAT composite and section scores, on-time graduation, and undergraduate GPA also correlated with higher NAPLEX scores.⁴ The PCOA total and subtopic domain scaled scores have been correlated with higher NAPLEX scores, as demonstrated by Naughton and colleages.⁶ Madden and colleagues determined a significant difference in first-time pass rate between students who required remediation in pharmacy coursework and those who did not.⁷ Notably, most published studies on variables affecting NAPLEX first-time pass rates included data from only one school. Recently, Whittaker and colleagues examined modifiable predictors of institution-wide first-year postgraduate (PGY1) residency match rates among 121 colleges and schools of pharmacy and noted that average NAPLEX pass rate, academic health center affiliation, admit-to-applicant ratio, and U.S. News & World Report rankings positively predicted match rates, while larger class size and higher minority enrollment were significant negative predictors.¹⁰

Beginning in 2015, the NAPLEX has undergone several changes, including a revision to the passing standard in 2015, and starting in 2016, extending the length of the examination from 4.25 hours to 6 hours, increasing the number of questions from 185 to 250, and moving from a computer adaptive format to a preassembled model with certain percentages of questions varying in difficulty. Most of the questions on the NAPLEX are scenario-based, using patient profiles or medical records that require analysis in order to answer related questions.¹¹

Overall first-time mean passing rates for NAPLEX have seen a downward trend since 2014, with the most significant drop occurring between 2015 and 2016 (92.64% vs 85.86%).¹² There was wide variability within the 2016 institution scores. While approximately 36% of schools had over 90% passing scores, first-time pass rates dropped as low as 59% for two programs.

In 2005, Zarembski et al compared first-time NAPLEX pass rates between programs ACPE accredited prior to 1992 (n=74) and those accredited after 1992 (n=8), and found there were small differences between the two groups, favoring established programs.¹³ Since then, the number of accredited pharmacy programs has continued to increase. As of July 2017, 138 US programs had full or candidate status and four had pre-candidate status.¹⁴ Many of the more recently accredited programs are located at private institutions, and several have an accelerated three-year curriculum. The continued expansion of pharmacy programs, changes to the NAPLEX, and recent trend of lower overall first-time NAPLEX passing rates raise the question of whether relationships exsit among these variables.

The objectives of this study were twofold: to determine the relationship between structural characteristics of individual colleges of pharmacy (ie, location in an academic health center, established before or after 2000, traditional or accelerated curriculum, public or private) and NAPLEX first-time pass rates for 2014, 2015, and 2016; and to determine factors that are predictive of the 2015 and 2016 NAPLEX first-time pass rates.

METHODS

Online public sources were used to collect data concerning all US colleges of pharmacy accredited as of July 2017 for the academic years 2013-2014, 2014-2015, and 2015-2016. Information on pharmacy programs including established year and campus locations were obtained from the Accreditation Council for Pharmacy Education (ACPE). The NAPLEX pass rates were obtained from the National Boards of Pharmacy (NABP) website.¹² The American Society of Health-System Pharmacists (ASHP) supplied National Match Service reports on PGY1 residency match statistics. All other data was retrieved using the reporting feature found on the American Association of Colleges of Pharmacy (AACP) Trends Interactive Dashboard.¹⁵ The study was approved by the University of Tennessee Health Science Center Institutional Review Board.

During the summer of 2017, the above sources were searched and the following data were collected for each US school or college of pharmacy: part of an academic health center; one or multiple campuses; established (defined as receiving precandidate status) before or after 2000; program structure (traditional four-year curriculum vs accelerated three-year curriculum); program type (public vs private); student-to-faculty ratio; percentage of out-of-state students enrolled in the PharmD program; percentage of the fourth-year (P4) class that matched for a PGY1 residency; and NAPLEX first-time pass rates for 2014, 2015, and 2016. These variables were selected as they are common data points across pharmacy schools and are publicly available. The year 2000 was selected because of the rapid increase in the number of new pharmacy schools since that date. An additional variable was created based on whether colleges of pharmacy experienced a 10% or greater decrease in NAPLEX first-time pass rate between 2015 and 2016.

Statistical analyses were performed using IBM SPSS Statistics 24.0 (Armonk, NY). Categorical variables were summarized using frequencies, and continuous variables were summarized using means and standards deviations. Independent samples t tests were conducted to examine the relationships between categorical independent variables (academic health center, multiple campuses, established before or after 2000, program structure, and program type) and NAPLEX first-time pass rates. Paired-samples t tests were conducted to compare NAPLEX first-time pass rates for 2014, 2015, and 2016. Independent samples t tests were also conducted to examine the relationships between categorical independent variables and the variable for the 10% or greater decrease in first-time NAPLEX pass rate (coded as 0=no decrease of greater than or equal to 10% between 2015 and 2016, or 1=decrease of greater than or equal to 10% between 2015 and 2016). Pearson r and point biserial correlations were calculated to determine the associations between academic health center, multiple campuses, established before or after 2000, program structure, program type, student-to-faculty ratios, percentage of out-of-state PharmD students, percent of total graduating class who matched for a PGY1 residency, and NAPLEX first-time pass rates. Separate correlational analysis was conducted for each study year.

Multiple linear regression analysis was conducted using 2015 NAPLEX first-time pass rate as the dependent variable and the following independent variables: whether schools were located at an academic health center (coded as 0=no, 1=yes); whether schools had multiple campuses (coded as 0=no, 1=yes); whether schools were established before or after 2000 (coded as 0=before, 1=after); program structure (coded as 0=traditional four year, 1= accelerated three year); program type (coded as 0=public, 1=private); student-to-faculty ratio for the 2014-2015 academic year; percent out-of-state student enrollment for the 2014-2015 academic year; percentage of the P4 class that matched for a PGY1 residency in 2015; and NAPLEX first-time pass rate for the prior year (2014). Multiple linear regression analysis was also conducted using 2016 NAPLEX first-time pass rate as the dependent variable and the following independent variables: academic health center; multiple campuses; whether schools were established before or after 2000; program structure; program type; student-to-faculty ratio for the 2015-2016 academic year; percent out-of-state student enrollment for the 2015-2016 academic year; percentage of the P4 class that matched for a PGY1 residency 2014 NAPLEX first-time pass rate; and NAPLEX first-time pass rate for the prior year (2015). To conduct each regression analysis, variables were entered in one block. The a priori significance level was .05. Since the focus was to determine factors associated with NAPLEX first-time pass rates following the 2015 revisions, multiple linear regression analysis was not conducted for the 2014 data.

RESULTS

Characteristics of US colleges of pharmacy (N=137) are summarized in Table 1. The majority of schools were not located at an academic health center (59.9%), did not have multiple campuses (77.4%), were established before 2000 (60.6%), had a traditional 4-year curriculum (91.2%), and were private (51.8%). Of the 137 existing pharmacy schools, 120 participated in the NAPLEX in 2014, 124 in 2015, and 128 in 2016; nonparticipating schools in each year were new and did not yet have a graduating class. Between 2015 and 2016, 29% of schools experienced a more than 10% drop in NAPLEX first-time pass rates.

Compared to 2014 (M=94.8, SD=4.6), NAPLEX first-time pass rates in 2015 (M=92.6, SD=5.4; p<.001) and 2016 (M=86.1, SD=8.2; p<.001) were significantly lower. Likewise, NAPLEX first-time pass rate in 2016 was also significantly lower than 2015 (M=86.4, SD=8.1 vs M=92.3, SD=5.7, respectively; p<.001).

For each year included in the study, significantly higher NAPLEX first-time pass rates were found for schools located within an academic health center compared to schools that were not located within an academic health center; schools established before 2000 compared to after 2000; and public schools compared to private schools (p<.05; see Table 2). Significantly higher 2016 NAPLEX first-time pass rates were also found for schools with a traditional program structure vs an accelerated structure (p=.01). No differences were found in NAPLEX first-time pass rate based on whether a school had multiple campuses (p>.05).

A lower percentage of schools with a traditional four-year structure vs an accelerated three-year structure experienced a decrease in first-time pass rates of greater than or equal to 10% between 2015 and 2016 (c²=5.54, p=.02). No other significant differences were found for this variable.

As displayed in Table 3, the variable NAPLEX first-time pass rate was consistently (ie, across all study years), positively, and significantly correlated (p<.05) with percentage of P4 students matched for a PGY1 residency and being located at an academic health center. A school’s NAPLEX first-time pass rate was consistently, inversely, and significantly correlated (p<.05) with percentage of out-of-state students enrolled in the PharmD program (in other words, as the percent of out-of-state students increased, the NAPLEX first-time pass rate decreased), whether schools were established before or after 2000 (eg, schools established before 2000 had higher NAPLEX first-time pass rates), and whether schools were public or private (eg, public schools had higher NAPLEX first-time pass rates). The 2016 NAPLEX first-time pass rates were positively correlated to 2014 and 2015 NAPLEX first-time pass rates (p<.01). The 2016 NAPLEX first-time pass rates were also inversely, significantly correlated (p=.02) with whether schools had a traditional program structure or an accelerated structure (ie, schools with a traditional four-year structure had higher first-time pass rates on the NAPLEX).

In the multiple linear regression analysis for 2015, the following independent variables were significant (p<.05) predictors of 2015 NAPLEX first-time pass rate: 2014 NAPLEX first-time pass rate and 2015 percent of total graduating class who matched for a PGY1 residency. Nonsignificant variables were removed from the analysis, and a second multiple linear regression was conducted. In this model, the significant predictors in the final model remained 2014 NAPLEX first-time pass rate and 2015 percent of total graduating class who matched for a PGY1 residency (p<.01). This model explained 56.8% of the variance (adjusted R²=.568; p<.001) in 2015 NAPLEX first-time pass rates, with 2014 NAPLEX first-time pass rate alone explaining 54.4% of the variance.

In the multiple linear regression analysis for 2016, the following independent variables were significant (p<.05) predictors of 2016 NAPLEX first-time pass rate: 2014 NAPLEX first-time pass rate, 2015 NAPLEX first-time pass rate, and 2016 percent of total graduating class who matched for a PGY1 residency. Nonsignificant variables were removed from the analysis, and a second multiple linear regression was conducted. In this model, 2014 NAPLEX first-time pass rate was not significant; the remaining significant predictors in the final model were 2015 NAPLEX first-time pass rate and 2016 percent of total graduating class who matched for a PGY1 residency (p<.01). This model explained 51.1% of the variance (adjusted R²=.511; p<.001) in 2016 NAPLEX first-time pass rates, with the 2015 NAPLEX first-time pass rate alone explaining 42.4% of the variance.

As tolerance was greater than 0.1 and the variance inflation factor (VIF) was less than 10, multicollinearity was not an issue in either of the final models.¹⁶ Other assumptions of multiple linear regression were also met.

DISCUSSION

As changes in the NAPLEX were implemented in 2015 and 2016, average first-time pass rates dropped in each of the corresponding years. The 2.5% drop from 2014 to 2015 coincided with revision of the NAPLEX standards in 2015. The next 6.3% drop from 2015 to 2016 coincided with the increase in the number of questions and length of the examination, changes which were implemented in 2016. Indeed, almost 30% of colleges experienced a 10% or greater decrease in NAPLEX first-time pass rate between 2015 and 2016. While other factors may have played a role in the decrease of the NAPLEX first-time pass rates nationwide, the changes made to the NAPLEX format may account for a good portion of the decrease seen.

In the current study, NAPLEX first-time pass rates for each of the three years studied were significantly higher for schools located at academic health centers, in schools established before 2000, and in schools that were part of public institutions. Additionally, the 2016 NAPLEX first-time pass rate was also significantly higher in schools with a traditional four-year program structure. Considering these results collectively, many schools likely have all four of these characteristics. Of the schools established before 2000, 54 of 83 (65% of this subsample, or 39% of the overall sample of 137 schools), are four-year programs at public institutions; 38 of this group of 54 (70% of this subsample, or 28% of the overall sample) are located at academic health centers.

Differences between first-time pass rates between schools may exist for different reasons. As data on average admission GPAs and PCAT scores were not publically available for all schools, it was impossible to determine whether certain types of schools are starting with an academically stronger cohort of students than other types of schools. Additionally, comparing the academic rigor of one program with that of another to determine if certain types of schools are better preparing their students for licensure and practice is difficult. Although some debate the usefulness of the NAPLEX for any assessment other than licensure determinations, the pharmacy literature routinely uses NAPLEX scores as a measure of a program’s success.^4,7,13,17-19 Likewise, NAPLEX scores are traditionally used as an assessment measure for ACPE accreditation, and programs are required to publish their NAPLEX pass rates.²⁰

In this study, first-time pass rates were found to be inversely, significantly correlated with percentage of out-of-state students in the program. Without more information on the academic strength of out-of-state cohorts, this finding is difficult to explain. Students may attend an out-of-state program for a variety of reasons.

Notably, no differences were found in first-time NAPLEX pass rates between schools that had multiple campuses versus those that did not. The number of distance education programs in pharmacy education has increased significantly in the last 20 years, so it was reassuring to see this phenomenon has not affected the NAPLEX first-time pass rates for those programs

The first-time pass rates on the NAPLEX in 2014 and the percent of total 2015 graduating class that matched for a PGY1 residency remained significant predictors in the final model when multiple linear regression analysis was performed for 2015 NAPEX first-time pass rates. Similar findings were seen in the multiple linear regression model analysis for 2016 NAPLEX first-time pass rates, with 2015 NAPLEX first-time pass rates and percent of total 2016 graduating class who matched for a PGY1 residency remaining significant in the final model. Together these two factors were responsible for more than 50% of the variance each year, indicating that how well a class does in the PGY1 residency match has been an excellent predictor of NAPLEX first-time pass rates for that class. Finding a significant correlation between first-time pass rates and percent of total graduating class who matched for a PGY1 residency was not surprising as we would expect graduates from programs with higher NAPLEX pass rates to be highly sought after candidates for PGY1 positions. Additionally, a program’s NAPLEX first-time pass rates performance proved to be an excellent predictor of performance on the NAPLEX in the following year. These correlations between NAPLEX pass rates over the three years could be an indicator of the consistency of pharmacy programs in producing students who pass the NAPLEX at similar rates across years.

The findings of this study are important for pharmacy programs that have seen significant drops in their first-time NAPLEX pass rates between 2014 and 2016, as well as for programs that are concerned about possible drops in the future. Identifying programs with like factors could be beneficial as programs may choose to investigate what methods other programs that have not seen significant drops in first-time pass rates are using for NAPLEX preparation.

This study has limitations. Although the multiple linear regression models that we conducted explained more than 50% of the variance seen in the 2015 and 2016 data, we were unable to pinpoint other factors responsible for the remaining variance. If additional student or program variables were available for analysis, such as measurements of academic strength of students, use of board review courses, average PCAT scores, percent of students with undergraduate degrees, average age of students, then future studies may be able to pinpoint additional factors that contribute to NAPLEX outcomes.

CONCLUSION

Understanding predictors of NAPLEX first-time pass rates is important as programs try to identify why their pass rates have dropped in the last two years. While it is easy to point to changes in the NAPLEX as the reason for decreases, certain programs are experiencing significant drops while other programs are not. Careful analysis of the decreases in pass rates seen in the last two years show differences between schools in certain program characteristics. In our study, programs not located within an academic health center, those established after 2000, accelerated programs, and programs that are part of private institutions experienced significant decreases in NAPLEX first-time pass rates after changes in the examination were implemented. While many of these factors are not modifiable, programs can proactively and critically evaluate their educational programs and the readiness of their students to sit for the NAPLEX.