Abstract
Objective. To determine yearly (phase 1) and cumulative (phase 2) publication records of pharmaceutical science faculty members at research-intensive colleges and schools of pharmacy.
Methods. The publication records of pharmaceutical science faculty members at research-intensive colleges and schools of pharmacy were searched on Web of Science. Fifty colleges and schools of pharmacy were randomly chosen for a search of 1,042 individual faculty members’ publications per year from 2005 to 2009. A stratified random sample of 120 faculty members also was chosen, and cumulative publication counts were recorded and bibliometric indices calculated.
Results. The median number of publications per year was 2 (range, 0-34). Overall, 22% of faculty members had no publications in any given year, but the number was highly variable depending on the faculty members’ colleges or schools of pharmacy. Bibliometric indices were higher for medicinal chemistry and pharmaceutics, with pharmacology ranking third and social and administrative sciences fourth. Higher bibliometric indices were also observed for institution status (ie, public vs private) and academic rank (discipline chairperson vs non-chairperson and professor vs junior faculty member) (p<0.01 for each). The median number of cumulative publications per faculty member was 34 (range, 0-370).
Conclusion. Significant differences exist in yearly and cumulative publication rates for faculty members and bibliometric indices among pharmaceutical science disciplines and academic ranks within research-intensive colleges and schools of pharmacy. These data may be important for benchmarking purposes.
INTRODUCTION
The 1980 Argus commission report identified the disparity of research productivity among the 72 accredited colleges and schools of pharmacy in the United States. Using the level of funding by the National Institutes of Health (NIH) and publications as a measure of research activity, the study suggested that as many as 75% of existing US colleges and schools of pharmacy at the time were not engaged in sufficient research activity to be considered successful. This report provides not only an impetus for the importance of scholarship in pharmaceutical education but also the first significant benchmark for scholarship in academic pharmacy.1
Benchmarking is becoming an important tool for colleges and schools of pharmacy as they collect and evaluate assessment data.2 In 2009, Thompson and colleagues3 summarized the available benchmarking data on publication rates from pharmacy and medicine faculty members, along with new information on pharmacy practice chairpersons. Data interpretation and comparison challenges were also outlined in this paper. While the findings of this study provide a useful start to developing normalized scholarship benchmarks, additional research is needed.
New scholarship indices have been introduced in the past few years. A review of these indices provides their definitions and descriptions as well as potential advantages and disadvantages.4 The most interesting of these is the h-index. A faculty member with an h-index of 10 has published 10 papers, each of which has ≥10 citations. The h-index incorporates both quantitative (number of papers) and potentially qualitative (number of citations) aspects into a single whole number. Of course, sufficient normative data in pharmaceutical education must be generated before these indices can be useful as routine benchmarking indices. Another index is the m-quotient, which is the h-index normalized by the number of years in a researcher’s journal article publishing career. A researcher’s publishing career is generally determined by the date of the first publication in a particular database to the time of the current literature search. Other indices include the creativity index and productivity index, first introduced by Soler in 2009.5 Each of these 2 indices requires the generation of additional data to establish normalized values for the pharmacy education academy; however, they are unique in providing normative, quantitative methods for measuring creativity and productivity with respect to journal publications.
The purpose of this research was to explore descriptive scholarship indices for pharmaceutical science faculty members in research-intensive colleges and schools of pharmacy by collecting publication data and calculating bibliometric indices from the Web of Science database. Discipline, academic title, and other demographic differences were also compared. Unique aspects of this study included yearly and cumulative publication data, calculation of several new scholarship indices (h-index, creativity index, productivity index), and comparative data that can establish benchmarking tools for pharmaceutical science faculty. These aspects of the study were included to provide descriptive data for deans, discipline chairpersons (usually referred to in US schools as a department chairperson), and faculty members to use as benchmarks of successful performance in research-intensive colleges and schools of pharmacy.
METHODS
This study involved pharmaceutical science faculty members at research-intensive colleges and schools of pharmacy in the United States. Figure 1 outlines the basic methodology used in the study protocol. Colleges and schools of pharmacy were defined as research intensive if they met at least 2 of the following 3 criteria: classification as a Carnegie I research university6; ≥ $500,000 in total research funding over the period of 2005-2009, as listed on the American Association of Colleges of Pharmacy (AACP) Web site7; and a PhD program, as verified by the 2009 Survey of Professional Graduate Degree Programs conducted by the AACP.8 The Roster of Faculty published by the AACP was the primary instrument used to identify faculty members.9 Fifty-three research-intensive colleges and schools of pharmacy met the study criteria. From this pool, 50 colleges and schools of pharmacy were randomly chosen for further analysis. These 50 institutions were selected using a nonreplacement randomization scheme in which 10 different colleges and schools of pharmacy were selected for evaluation each year from 2005-2009 (Table 1). Only 3 eligible research-intensive colleges and schools of pharmacy were not represented in the study. Eligible faculty members from the study institutions included the discipline categories of medicinal chemistry, pharmaceutics, pharmacology, and social and administrative sciences. Additional exclusion and inclusion criteria are described in Figure 1. After initial filtering, the study was divided into phase 1 and phase 2.
Standardized University Names Used in Web of Science (WoS) Searches
Study Algorithm
The database used for the study was the Institute for Scientific Information Web of Science, which covers over 10,000 journals since 1900. The Web of Science was chosen because of its comprehensive coverage of the scientific and technical sciences and inclusion of the Science Citation Index (7,100 journals), Social Science Citation Index (2,100 journals), and the Arts and Humanities Index (1,200 journals). It provides an excellent database to cover the pharmaceutical and administrative sciences for pharmacy.10 It is also superior to both Scopus and Google Scholar as a source of bibliometric and citation data.11-13 Web of Science provides a comprehensive indexing system for each author’s address of papers included in the database. While Web of Science does not use a controlled vocabulary, it does provide a standardized, consistent method of indexing university names. A listing of the Web of Science standardized search names used in this study is provided in Table 1.
The bibliometric FILTER and MERIT programs used in this study were created and developed by Soler.14 The FILTER program uses a mathematical algorithm to break down a listing of publications with authors having identical names into clusters based on similar topics. The researcher using the program is presented with each cluster and a representative sample publication and is asked to include or reject the cluster. Once a cluster of publications is accepted, the program calculates the distance from the previously selected cluster. The number of iterations necessary for a particular group of papers can vary. The MERIT program categorizes publication and bibliometric data by total number of papers, type of paper (article, review, proceedings, abstract, letter, editorial, note, or other category), author productivity (sum of citations/number of authors), author creativity (calculation involves sum of citations, sum of references, and number of authors), h-index, average number of authors per paper, average number of citations per paper, average author creativity per paper, average number of citations per year, average author productivity and creativity per year, and m-quotient. A more thorough review of these indices and their advantages and disadvantages is available.3
Since the roster is published as an academic-year publication and the collection of phase 1 data was on a calendar-year basis, consecutive roster lists were compared and only those faculty members who were listed in both the preceding academic year and the following academic year were used in the calendar-year search. For example, to be eligible for a 2007 calendar year search, a faculty member’s name must have appeared in both the 2006/2007 and the 2007/2008 rosters. Faculty and college information captured from the roster and the AACP Web site included primary degree, department, academic title, and college classification (ie, public or private). Data were also categorized and analyzed by academic rank, discipline, chair or non-chair status, public or private university status, and the level of both the department and the college or school.
The advanced search option was used to search the Web of Science. For example, a search would include all articles published by a given author at a given institution in a given year. All publications by the author produced by the search were counted regardless of author order on the paper (eg, first author, second author) and collaborators’ location (ie, same or different colleges or schools of pharmacy). Using this search strategy involves floating subheadings. The author and address fields did not necessarily have to correspond. Therefore, records were inspected for similar research content and connection of author and address fields. If no publications were returned for a search, the time restriction was removed and the search was run again. If verification of at least 1 publication was not confirmed, the author’s name was checked for spelling errors on the college or school of pharmacy Web site. This step allowed for identifying possible author misspellings in the roster. Initial searches yielding more than 12 publications or any search in which a homologue (authors with the same name in different disciplines) was discovered were processed through the Soler FILTER program to remove homologues. Once the initial searches were evaluated and/or filtered, the files were processed through the MERIT program for generation of bibliometric indices. These indices were transferred to an Excel (Microsoft, Redmond, WA) spreadsheet for analysis. A flow chart describing this process is provided in Figure 2. The estimated accuracy and precision of the phase 1 methodology were both 98%.15
Phase 1 Methodology Algorithm
Phase 2 of the analysis involved evaluation of cumulative publication records and metrics from a sample of faculty members representing the 50 research-intensive colleges and schools of pharmacy included in phase 1 of the study. Cumulative publications were defined as the first publication by an author on the Web of Science search to the present time. A stratified, random sample of 10 faculty members was chosen from each faculty rank (assistant professor, associate professor, and professor) for a total of 30 faculty members in each academic discipline (pharmaceutics, pharmacology, medicinal chemistry, and social and administrative sciences). This phase provided a sample of 120 faculty members for a comparison of cumulative publication and bibliometric indices. The search strategy for this phase began with a general search of the author and the US state where the college or school of pharmacy was located (eg, AU= Smith J* and AD=OH). This approach allowed the searcher to identify how the faculty member generally authored publications (eg, Smith J or Smith JP). After identifying how the faculty member’s name was typically listed on publications, a specific broad search without geographic restriction was conducted. These publication files were then captured and screened through the FILTER program to remove homologues. After filtering, the files were compiled and analyzed using the MERIT program. These data were then entered into an Excel spreadsheet for further analysis. The estimated accuracy and precision of the phase 2 methodology were 95% and 98%, respectively.15
Average iterations for phase 1 data (N=1,042) were 4.2 ± 2.6 (range, 0-14), with 37% of initial papers removed by filtration. Average iterations for phase 2 (N= 120) were 4.2 ± 6.7 (range, 0-37), with 20% of initial papers removed by filtering. One hundred percent of phase 2 data and 8% of the phase 1 data were filtered through the Soler program. Two faculty names were identified as being misspelled in the roster, and 1 faculty member published under a name that was different from what was listed in the roster (identified through a Web site search).
Data were analyzed using the statistical software package Analyse-It (Analyse-It Software, Ltd. Leeds, UK) from the Excel (Microsoft, Redmond, WA) spreadsheets. Initial statistical analysis involved descriptive and inferential statistics. The Kruskal-Wallis analysis of variance for nonparametric data was used to assess differences between 2 groups. The Bonferroni post hoc test was used to determine differences among 3 or more groups. The a priori level of significance was set at p< 0.01.
RESULTS
Table 2 describes a subgroup analysis of the phase 1 data. Disciplines and academic titles had broad representation, but public colleges and non-chair status dominated the other subcategories. Figure 3 provides the frequency of publications per year. Overall, 6% of all pharmaceutical sciences faculty members had more than 10 publications and 22% had zero publications in any given year (Figure 3). Discipline-specific percentages for publishing zero publications were medicinal chemistry, 19%; pharmaceutics, 23%; pharmacology, 23%; and social and administrative sciences, 29%. Analogous data by academic title include assistant professor, 22%; associate professor, 26%; and professor, 20%. About two thirds (67%) of total publications were articles, 19% were abstracts, 6% reviews, 4% proceedings, 2% editorials, 1% letters, and 1% other publications. Overall, 15% of all faculty members published 50% of all the publications. Faculty members authoring 50% of all publications in their discipline represent pharmacology (18%), pharmaceutics (17%), medicinal chemistry (16%), and social and administrative sciences (13%).
Faculty Subgroup Analysis by Discipline, Academic Title, Public vs Private, Chair vs Non-Chair Status
Number of faculty vs number of publications per year in phase 1. Percentage figures are the percent of all faculty members publishing various numbers of publications per year.
Average author productivity was highest for pharmaceutics at 10.9 (95% CI, 8.0-13.8), then medicinal chemistry at 9.8 (95% CI, 7.9 -11.7), pharmacology at 6.0 (95% CI, 4.8-7.3), and social and administrative sciences at 2.6 (95% CI, 1.5-3.7). Average author creativity was highest for pharmaceutics at 2.7 (95% CI, 1.5-4.0), then medicinal chemistry at 2.4 (95% CI, 1.7-3.1), pharmacology at 1.2 (95% CI, 0.9-1.5) and social and administrative sciences at 0.76 (95% CI, 0.4-1.1). Average h-index results were highest for medicinal chemistry at 2.4 (95% CI, 2.2-2.7), pharmaceutics at 2.1 (95% CI, 1.9-2.4), pharmacology at 1.6 (95% CI, 1.4-1.7), and social and administrative sciences at 1.0 (95% CI, 0.8-1.2).
Significant differences in all publication metrics are listed in Table 3. For example, the median values for various disciplines ranged from 1 to 3 publications per year, 1.5 to 17 citations per year, 1 to 2 h-indices per year (m-quotient), and 1.1 to 3.3 for the sum of creativity scores on all published papers. The creativity index involves counting the number of citations a paper receives and the number of references the paper cited, normalized by the number of authors of the paper.3 Overall, medicinal chemistry and pharmaceutics were the pharmaceutical science disciplines with the highest publication indices. Pharmacology tended to be third in most indices with social and administrative sciences ranking fourth. Medicinal chemistry was higher in most bibliometric indices than both social and administrative sciences and pharmacology. Pharmaceutics was generally higher than social and administrative sciences. Professors were consistently higher than both associate and assistant professors, with generally no differences between assistant and associate professors.
Pharmaceutical Science Faculty Publication Metrics: Phase 1 Yearly Data (N=1,042)
Because of the skewed nature of publication metrics, means and medians are listed for each index. As the study was conducted over a 5-year period, metrics involving citation counts would not be considered normative. While differences were obtained for the variable of chair status vs non-chair status, this finding is likely an artifact of academic title differences. Fully 86% of faculty members who were discipline chairpersons were full professors, while only 1% were assistant professors. Several indices varied between public and private colleges; however, distribution of private colleges tended to congregate in the latter years (2007, 2008, and 2009). This finding could explain some differences in publication metrics involving citations (eg, productivity, creativity, and h-index) but would have no effect on the descriptive indices of publications per year or articles per year.
Data were also evaluated with relation to colleges and schools of pharmacy. Figure 4 illustrates the significant variability in publications per faculty member per year by individual colleges and schools of pharmacy. While means and medians were useful in describing central tendency, faculty expectations of publications per year were highly dependent on the individual college or school of pharmacy. Twenty-five colleges and schools of pharmacy had fewer than 3 publications per faculty member per year, and 25 had 3 to 6.5 publications per faculty member per year. Similarly, the percentage of faculty members with zero publications per year by colleges and schools of pharmacy was highly variable. With 3 exceptions, most research-intensive colleges had ≤ 50% of faculty members with zero publications. At the other extreme was 1 college of pharmacy with only 3% of its pharmaceutical science faculty members with zero publications per year.
Average publication per faculty per year by individual college of pharmacy.
Table 4 provides a subgroup analysis of the phase 2 data. As a result of the stratified randomization process, disciplines and academic titles were equally represented; public colleges and non-chair status were similar to those in the phase 1 data. For various academic ranks (assistant, associate, and full professor), the median metrics ranged from 14.5 to 79.2 cumulative publications, 182.5 to 1429 cumulative citations, 5.5-20 h-index, 7 to 143.1 cumulative author creativity, and 41.8 to 401.8 cumulative author productivity. The number of highly cited papers (>500 citations) by discipline were medicinal chemistry (3), pharmaceutics (1), pharmacology (0), and social and administrative sciences (0). Highly cited papers (250 to 500 total citations) by discipline included medicinal chemistry (20), pharmaceutics (16), pharmacology (10), and social and administrative sciences (0). Analogous data (>500 citations), categorized by academic title, showed 4 papers by the full professor rank and none by associate professors or assistant professors. Highly cited papers (250 to 500 total citations) were attributed to the academic title of full professor (27 papers), associate professors (14), and assistant professors (5).
Pharmaceutical Science Faculty Publication Metrics: Phase 2 Cumulative Data (N=120)
Significant differences in publication metrics for phase 2 data are listed in Table 4. Overall, as with the phase 1 data, medicinal chemistry and pharmaceutics both vied for the top spot in most bibliometric indices. For most indices, medicinal chemistry and pharmaceutics were higher than pharmacology, and medicinal chemistry, pharmaceutics, and pharmacology were generally higher than social and administrative sciences. Professors had higher indices than assistant professors, and for a few bibliometric indices, associate professors were higher than assistant professors. Again, because of the skewed nature of publication metrics, means, and medians are listed for each index. Cumulative publication records and indices are listed along with per-paper and per-year metrics. While many of these data can be considered normalized by number of faculty members, they are the product of “point in time” analyses and must be interpreted in this context.
DISCUSSION
We observed marked variability in both yearly and cumulative bibliometric indices for faculty members and disciplines in various pharmaceutical sciences at colleges and schools of pharmacy, with the number of publications and citations, h-index, author creativity, and author productivity being substantially different for the various disciplines. Some of this variability may be explained by differences in the mission and goals of the colleges and schools. However, the finding that 22% of research-intensive faculty members had no publications during a 5-year period was surprising. In any given year, some faculty members at each institution may have roles for which there is no expectation of publication. Higher publication indices for full professors may be explained by senior faculty members having well-developed research programs and greater opportunities to collaborate at their institutions. The misappropriation of authorship by senior faculty members has been described in the publication ethics literature in other disciplines.16,17
The generation of publication and scholarly activity data among faculty members in the academy should be useful to begin developing benchmarking norms for faculty members. This will be particularly important for young faculty members who are unfamiliar with how to establish scholarship goals for their careers. National benchmarking data would be beneficial to chairpersons and deans in monitoring and setting goals and standards for improving research and scholarship. A paper on benchmarking2 concluded that “…reliable national benchmarking data would be a powerful tool for academic pharmacy department chairs and deans, and further initiatives to gather, assess, and share such information…,” suggesting that efforts to expand and improve upon benchmarking data will continue in academic pharmacy. In applying these data, important considerations include the limitations of bibliometric data and the significant variation in mission, standards, and foci of individual colleges and schools of pharmacy.
It is important to understand that publication counts and other bibliometric data are not normally distributed. Lotka’s law, 1 of the basic principles of bibliometrics, describes the skewed production of publications by authors often characterized by the “80-20” rule18: approximately 80% of publications will be produced by 20% of authors (or faculty members). In our study, 15% of all faculty members published 50% of all publications. Among the pharmaceutical science disciplines, percentages ranged from 13% for social and administrative sciences to 18% for pharmacology, although the differences between the 2 disciplines are modest at best. Developing broad conclusions becomes more complex when combining the skewed nature of these data with the significant variability among individual colleges and schools of pharmacy. As shown in Figure 4, the number of total publications in our study was highly dependent on the colleges and schools of pharmacy. Thus, analyzing measures of central tendency for benchmarking purposes must be done with care.
Caution must be exercised when comparing bibliometric indices among different disciplines.19 For example, comparing h-indices between biologists and economists may not be generally appropriate. We chose to compare all the pharmaceutical science disciplines (pharmacology, medicinal chemistry, and pharmaceutics) along with the discipline of social and administrative science. The placement of social and administrative sciences in pharmacy colleges and schools is highly variable, as it can be listed under pharmaceutical science or pharmacy practice or as an independent department of social and administrative sciences. We included social and administrative sciences under the general rubric of pharmaceutical sciences, which may not be appropriate. Just as biologists and economists should not be compared using bibliometric indices, perhaps social and administrative faculty members should not be compared with those in medicinal chemistry, pharmaceutics, and pharmacology.
A similar study published in 200520 covered a 5-year period (1999-2003) and involved pharmaceutical science faculty members for only 5 randomly selected research-intensive colleges and schools of pharmacy. While the methodology of this study was not as rigorous as that of the present study, it also found a higher number of publications for both medicinal chemistry faculty members and pharmaceutics faculty members than that for pharmacology faculty members. Social and administrative faculty members were not evaluated in the earlier study. Additionally, full professors were found to have published significantly more papers than had associate and assistant professors, as was found in the current study.
There are potential limitations to the study. First, there are several databases available as potential sources of bibliometric studies. A partial list of these would include MEDLINE, EMBASE, Web of Science, International Pharmaceutical Abstracts, Google Scholar, and Scopus. Web of Science, which was used in this study, however, is the only database that provides comprehensive coverage of current and historical pharmaceutical sciences journals with citation counts and addresses for all journal publication authors. It is important to use the same database when comparing publication counts and bibliometric indices of pharmaceutical scientists, as citation counts from other databases (eg, Google Scholar or Scopus) would likely yield different results.12,13 Second, source documents themselves can have inherent errors. Source journals and Web of Science have publication errors, although they are low considering the number of citations that are indexed yearly.10 Additionally, self-citations, negative citations, and citations from emerging new bibliometric sources (online-only journals) can complicate results.21 Additional source errors could come from the AACP Roster, as it was used to initially identify the pharmaceutical science faculty members, and homologues are always a potential problem in bibliometric research.14,22 Our methodology, using the Soler filter program, reduced the impact of homologues and other sources of error to 98% for both accuracy and precision in phase 1 and 95% accuracy and 98% precision in Phase 2.15 Phase 2 data may be less accurate than phase 1 data because of the longer timeframe involved (cumulative vs yearly). Another source of error in the methodology involved the “floating subhead” inherent in Web of Science searches (Figure 2). In the methodology used, the author (eg, AU=Johnson J*) and author address (eg, AD=Univ Arizona) are not necessarily linked. Therefore, citations were visually inspected for author–institution agreement. Sampling error is also possible because we chose 50 of the 53 colleges and schools of pharmacy to evaluate and also sampled 10 colleges and schools of pharmacy from each year rather than using the entire sample over each time period.
Our data demonstrate marked differences in the yearly and cumulative publication rates and other bibliometric indices among pharmaceutical sciences faculty members and disciplines, as well as academic ranks and type of institution. These observations may be useful for stimulating further research to explain the differences and for benchmarking purposes at various colleges and schools of pharmacy in the United States and abroad. If these data can continue to be updated and refined, practical applications can be made at various levels of the Academy. Our data may be useful to all pharmaceutical sciences faculty members with a desire to target a level of academic research publications, citations, and scholarship indices necessary to achieve academic promotion, recognition, or salary increase. Of course, various colleges and schools of pharmacy can be highly individualized; thus, the variability in data among research-intensive colleges must be considered for proper interpretation and application at a specific institution. A discipline chairperson could use these data to review aggregate scholarship from faculty members and apply benchmarking records from other research-intensive colleges to determine future goals for the discipline. The dean can use these data similarly and perhaps with greater focus on strategic planning, resource allocation, and vision for the entire research enterprise. Finally, bibliometric indices may not consistently measure quality and impact of publication.
CONCLUSION
Scholarship and research are important components of academic pharmacy. Significant differences exist in yearly and cumulative publication rates for faculty members and bibliometric indices among pharmaceutical science disciplines and academic ranks within research-intensive colleges and schools of pharmacy. These data may be important for benchmarking purposes.
ACKNOWLEDGEMENT
The authors thank Dr. Robert (Buzz) Kerr, Dr. Ceclia Plaza, and Mr. Brad Miller for their guidance and helpful discussions and Dr. Sarah J. Ramsey for her review of the paper. The support of the Brodie Scholar-in-Residence program at the American Association of Colleges of Pharmacy is appreciated, as is the valuable assistance of Dr. Radwan Al-AJarrah with mathematical concepts.
- Received November 22, 2011.
- Accepted May 20, 2012.
- © 2012 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.