Skip to main content

Main menu

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Other Publications

User menu

Search

  • Advanced search
American Journal of Pharmaceutical Education
  • Other Publications
American Journal of Pharmaceutical Education

Advanced Search

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers - Intersectionality of Pharmacists’ Professional and Personal Identity
  • Reviewers
    • Reviewer Instructions
    • Call for Mentees
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Follow AJPE on Twitter
  • LinkedIn
Brief ReportBRIEF

Long-term Association of Altmetric Attention Scores With Citations in Selected Major Pharmacy Journals

Dave L. Dixon and William L. Baker
American Journal of Pharmaceutical Education February 2022, 86 (2) ajpe8606; DOI: https://doi.org/10.5688/ajpe8606
Dave L. Dixon
aVirginia Commonwealth University, School of Pharmacy, Richmond, Virginia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
William L. Baker
bUniversity of Connecticut, School of Pharmacy, Storrs, Connecticut
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Objective. To determine the long-term change in the Altmetric Attention Score (AAS) and its components, as well as the impact of higher AAS on citation count for articles published in major pharmacy journals.

Methods. This study evaluated articles from pharmacy journals ranked in the top 10% according to their AAS in the year 2017. Correlation between the AAS and number of citations (through November 2020) was assessed using the Spearman’s correlation test. A Kruskal-Wallis 1-way analysis of variance was used to compare the AAS across journals.

Results. The median three-year AAS and citation count per article was 20 (25th, 75th percentile=15, 28) and 11 (6, 18), respectively. Between November 2018 and November 2020, there was no significant change in the median AAS for the 137 included articles. The only change in the AAS components was an increase in the number of Mendeley readers (22 [13, 34]). The median number of citations per article also increased (8 [4, 14]). We found a significant association between the three-year AAS and the three-year number of citations. The three-year number of Mendeley readers was associated with an increase in the 3-year number of citations. The mean three-year AAS was highest with articles published in the Journal of the American Pharmacists Association, while the mean three-year number of citations was highest for articles published in Pharmacotherapy.

Conclusion. Higher AAS scores appear to be associated with the number of citations for articles published in major pharmacy journals within three years of publication.

Keywords:
  • publications
  • bibliometrics
  • alternative metrics
  • journal impact factor
  • pharmacy

INTRODUCTION

Publishing in peer-reviewed journals is the primary process by which research is evaluated by one’s peers to determine its scientific merit.1 It is also an important consideration when making promotion and tenure decisions for faculty.2 The evaluation of a faculty’s scholarship is generally based on the quantity and quality of the articles published. The quality of peer-reviewed publications is typically dependent on the journal impact factor (JIF); however, the JIF largely reflects the reputation of the journal rather than that of its authors.3 The raw number of citations an article receives may also be considered an important metric, but the value of this metric is limited as there is large variation among disciplines as to what is a significant number of citations for an article. The Hirsch index, or h index, has largely been touted as a metric that evaluates both quantity and quality since it reports the number of articles that have been cited h number of times.4 For example, an author with 15 publications, each cited 15 times, would have an h index of 15. Of course, this favors more senior authors who have had more time to publish articles and accrue citations.

In addition to traditional metrics, there is an interest in recognizing online publications. Alternative metrics was coined as a term in 2010 and refers to many types of attention a published article receives online, including but not limited to mentions on social media platforms (eg, Twitter), blogs, Wikipedia, and news outlets.3 Journals have largely embraced alternative metrics and primarily use either Altmetric or Plum Analytics to source, aggregate, and report alternative metrics. One difference between these two platforms is that Altmetric provides a score, while Plum Analytics does not.5 The Altmetric Attention Score (AAS) serves as a quick and easy indicator of the amount and type of attention an article receives online. The score is based on an automated algorithm representing a weighted count of how much attention the article receives.6 More weight is given to news outlets, blog posts, policy documents, patents, and Wikipedia, while less weight is given to mentions on social media platforms. Further, more prominent news outlets with a national reach (eg, New York Times) are weighted more heavily than local news. A colored donut shape around the score represents the sources of attention. The author of the online post also matters as more credit is given when peers share your work than when the journal automatically shares a link to the article. Although alternative metrics have not replaced traditional measures of scholarship quality, perhaps they can serve as a complement to highlight the attention one’s scholarship receives online through social media and news outlets, which is not measured by traditional metrics.3

An area of uncertainty around alternative metrics is how effective they are at increasing citation counts, which affects both the JIF and the author’s h-index, and are still widely accepted measures of scholarship quality.7 Traditionally, one might only find an article if it was identified through a PubMed search, but more researchers are now sharing their articles online, especially through social media. This broader reach to the public and the scientific community could translate into more citations for articles that are shared and discussed online. There is little evidence regarding whether the AAS contributes to higher citation counts among pharmacy practice journals. Our own analysis of major pharmacy practice journals found no short-term correlation between the AAS and citation count.8 Therefore, the objective of this study was to determine if the AAS is associated with citation counts over an extended period of time in select major pharmacy practice journals.

METHODS

Our previous study8 included articles published in 2017 from pharmacy practice journals indexed in Web of Science or Scopus with a 2017 JIF of at least 1.9 Journals considered to focus primarily on international pharmacy practice or pharmacy education were excluded. We extracted data for all articles that ranked in the top 10% for each journal according to their AAS using Dimensions (as collected in November 2018), an online platform that provides citation counts and altmetrics data in aggregate form.10 Each article was reviewed to categorize its article type, study design, topic, and whether it was an open access article. We updated the AAS and relevant citation data for each included article through November 2020, providing up to three years of information from date of original publication. Sources of attention for each articles AAS were obtained from the Altmetric website.

Both AAS and the number of citations were log-transformed given skewed distributions. We used linear regression to evaluate the relationship between AAS and number of citations. The Kruskal-Wallis test estimated differences in AAS and number of citations across journals. We also explored non-linear associations using polynomial regression with 10-fold repeated cross-validation to identify the optimal model (highest explained variance, lowest mean squared error). We then conducted multiple regression to identify predictors of number of citations. Variables to include in the models were identified using least absolute shrinkage and selection operator (LASSO) regression.11 The optimal value for the tuning parameter (λ) to optimize model performance was selected using 10-fold repeated cross-validation. We performed all analyses using SAS 9.4 (SAS Institute) and R 4.0.3 (The R Foundation, https://www.r-project.org/) using the “caret” and “glmnet” packages, with a p value <.05 considered statistically significant.

RESULTS

Characteristics of the included articles have been published previously.8 As shown in Table 1, the median 3-year AAS and number of citations per article was 20 (25th, 75th percentile=15, 28) and 11 (6, 18), respectively. While many AAS components remained unchanged over time, including the median AAS, the median number of citations per article increased by 8 (4, 14) and the number of Mendeley readers increased by 22 (13, 34).

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1.

Long-term Characteristics of the Top 10% of Pharmacy Articles According to Altmetric Attention Score (N=137)

An increase in the log-transformed three-year AAS was associated with an increase in the log-transformed three-year number of citations (adjR2=0.06, p=.005) (Figure 1). Because the component of the AAS with the largest change from one-year to three-year follow-up was the number of Mendeley readers, we additionally examined its relationship with citation counts. An association between increasing log-transformed three-year number of Mendeley readers and an increase in the log-transformed three-year number of citations (adjR2=0.36, p <.001) is shown in Figure 2. The mean three-year AAS was highest with articles published in the Journal of the American Pharmacists Association (JAPhA) compared with other journals, with Research in Social and Administrative Pharmacy being the lowest (p<.0001). The mean three-year number of citations was highest for articles published in Pharmacotherapy, with American Journal of Health-System Pharmacy (AJHP) being the lowest (p=.013).

Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.

Relationship between log-transformed 3-year Altmetric Attention Score and log-transformed 3-year number of citations.

Figure 2.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 2.

Relationship between log-transformed three-year number of Mendeley readers and log-transformed three-year number of citations.

The LASSO model identified the following predictors, which we then included in the multiple regression: log-transformed number of citations: log-transformed AAS, AJHP journal, Pharmacotherapy journal, opioid epidemic topic, pharmacy education topic, miscellaneous topic, systematic review/meta-analysis design, and case-report design. Predictors (coefficient intercept [95% CI]) associated with a higher log-transformed number of citations included log-transformed AAS (0.178 [0.0007 to 0.356]), Pharmacotherapy journal (0.595 [0.148 to 1.043]), opioid epidemic topic (0.385 [−0.099 to 0.870]), and systematic review/meta-analysis design (0.167 [−0.293 to 0.628). Those (coefficient intercept) associated with a lower log-transformed number of citations included AJHP journal (−0.229 [−0.616 to 0.157]), pharmacy education topic (−0.432 [−1.389 to 0.524]), miscellaneous topic (−0.144 [−0.682 to 0.394]), and case-control design (−0.770 [−1.915 to 0.374]).

DISCUSSION

Contrary to our previous study,8 a longer follow-up period resulted in an association between the AAS and the number of citations for articles published in major pharmacy journals in 2017. This longer follow-up period likely allowed time for more citations to accrue, resulting in a greater chance of identifying an association. We also observed no significant change in the AAS during this longer follow-up period, and Mendeley readers was the only component of the AAS associated with higher citation counts. While studies of journals from other disciplines have observed similar associations,12⇓⇓-15 we believe this is the first time such an association has been identified among pharmacy journals.

The sharing of articles online seems to occur primarily around the time the article is originally published. This is reasonable given that journals and authors are probably more likely to share newly published work rather than older articles. As such, we were not surprised to see the AAS did not change over the course of this longer follow-up period. Likewise, this phenomenon may also explain why we did not observe an association between the AAS and number of citations in our short-term analysis.8 When individuals see an article posted on Twitter or another platform, it is likely saved or tagged for later reading or use in their own work, which may not be published for months (or years) later resulting in a delay in the article shared online receiving a citation.

Given the AAS is based on multiple components, we explored which of these components were associated with higher citation counts. The only AAS component associated with higher citation counts were articles captured by Mendeley readers. Mendeley is a free reference manager with a built-in academic social network where individuals can share articles or entire reference libraries. As such, articles saved or shared by Mendeley readers were more likely to be cited since saving an article in a reference manager may suggest the user intended to cite the same article in a future publication of their own. While other reference managers exist (eg, EndNote), Mendeley is the only one captured by the AAS since it has a built-in social network.

In addition to Mendeley, Twitter has become a frequently used platform to share articles online.16 Although we did not find an association between Twitter promotion and citation counts, two large, randomized studies17,18 of articles published in cardiology and thoracic surgery journals demonstrated that Twitter promotion was associated with a significant increase in citation counts. However, neither study controlled for authors who may have had a significant online presence and self-promoted their published articles. The lack of an association between Twitter promotion and citation counts in pharmacy journals may be due to variability in each pharmacy journals’ presence and reach on Twitter, available support staff to manage the account, and approach to using Twitter to promote articles. Nevertheless, the body of evidence largely favors promoting published articles via Twitter to increase citation counts. Pharmacy journals may need to enhance their presence on Twitter or consider a dedicated Social Media Editor.

Our analysis also has implications for faculty and academic institutions. Faculty should consider sharing their published work online through social media. This may be particularly helpful to junior faculty who are trying to establish themselves and build their professional network. In addition to sharing at the time of publication, faculty may consider re-sharing their relevant articles in response to queries from their online peers or current events. There also appears to be a role for alternative metrics in the promotion and tenure process, and institutions should consider incorporating language around the role of alternative metrics in their promotion and tenure policies. Regardless, faculty who want to include alternative metrics in promotion and tenure dossiers may need to explain what they are and understand how they will be perceived by their department chair and institution.

CONCLUSION

There is increasing evidence that the AAS, which captures the attention an article receives online, is associated with increased citation counts. Until now, there has been no evidence to support such an association among articles published in pharmacy practice journals. Although our previous work showed no association within the short-term, we did find an association between the AAS and citation counts during a three-year follow-up period. Additional study is warranted to determine whether promotion on specific platforms (eg, Twitter) contributes to higher citation counts. There is now some evidence that authors and pharmacy journals should consider promotion of their articles online with the goal of raising the articles AAS score, which may result in future citations.

ACKNOWLEDGMENTS

The authors thank Madeleine Wagner for assisting with data collection.

  • Received February 19, 2021.
  • Accepted May 25, 2021.
  • © 2022 American Association of Colleges of Pharmacy

REFERENCES

  1. 1.↵
    1. Bosso JA,
    2. Chisholm-Burns M,
    3. Nappi J,
    4. Gubbins PO,
    5. Ross LA
    . Benchmarking in academic pharmacy departments. Am J Pharm Educ. 2010;74(8):140. doi: 10.5688/aj7408140
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    1. Schimanski LA,
    2. Alperin JP
    . The evaluation of scholarship in academic promotion and tenure processes: past, present, and future. F1000Res. 2018;7:1605. doi:10.12688/f1000research.16493.1
    OpenUrl
  3. 3.↵
    1. Dixon DL,
    2. Baker WL
    . Measuring research impact in the 21st century: are alternative metrics the answer? Minerva Cardioangiol. 2020;68(4):279-281. doi:10.23736/S0026-4725.20.05241-X
    OpenUrl
  4. 4.↵
    1. Hirsch JE
    . An index to quantify an individual’s scientific research output. Proc Natl Acad Sci. 2005;102(46):16569-16572. doi:10.1073/pnas.0507655102
    OpenUrlAbstract/FREE Full Text
  5. 5.↵
    University of Western Australia Library. Guides. Citations, Altmetrics and Researcher Profiles: Alternative Metrics. https://guides.library.uwa.edu.au/rim/altmetrics. Accessed February 14, 2022.
  6. 6.↵
    The Donut and Altmetric Attention Score. Altmetric. July 9, 2015. Accessed February 14, 2022. https://www.altmetric.com/about-our-data/the-donut-and-score/
  7. 7.↵
    1. Burghardt KJ,
    2. Howlett BH,
    3. Fern SM,
    4. Burghardt PR
    . A bibliometric analysis of the top 50 NIH-Funded colleges of pharmacy using two databases. Res Soc Adm Pharm. 2020;16(7):941-948. doi:10.1016/j.sapharm.2019.10.006
    OpenUrl
  8. 8.↵
    1. Dixon DL,
    2. Baker WL
    . Short-term impact of Altmetric attention scores on citation counts in selected major pharmacy journals. J Am Coll Clin Pharm. May 2019:10-14. doi:10.1002/jac5.1141
  9. 9.↵
    1. Minguet F,
    2. Salgado TM,
    3. Santopadre C,
    4. Fernandez-Llimos F
    . Redefining the pharmacology and pharmacy subject category in the journal citation reports using medical subject headings (MeSH). Int J Clin Pharm. 2017;39(5):989-997. doi:10.1007/s11096-017-0527-2
    OpenUrl
  10. 10.↵
    Dimensions. Accessed February 12, 2021. https://app.dimensions.ai/discover/publication
  11. 11.↵
    1. Freijeiro‐González L,
    2. Febrero‐Bande M,
    3. González‐Manteiga W
    . A critical review of LASSO and its derivatives for variable selection under dependence among covariates. Int Stat Rev. Published online August 30, 2021:insr.12469. doi:10.1111/insr.12469
  12. 12.↵
    1. Huang W,
    2. Wang P,
    3. Wu Q
    . A correlation comparison between Altmetric Attention Scores and citations for six PLOS journals. PLOS ONE. 2018;13(4):e0194962. doi:10.1371/journal.pone.0194962
    OpenUrlPubMed
  13. 13.↵
    1. Mullins CH,
    2. Boyd CJ,
    3. Corey BL
    . Examining the correlation between altmetric score and citations in the general surgery literature. J Surg Res. 2020;248:159-164. doi:10.1016/j.jss.2019.11.008
    OpenUrl
  14. 14.↵
    1. Patel RB,
    2. Vaduganathan M,
    3. Bhatt DL
    . Characterizing high-performing articles by Altmetric score in major cardiovascular journals. JAMA Cardiol. 2018;3(12):1249-1251. doi:10.1126/sciadv.1500183
    OpenUrl
  15. 15.↵
    1. Barakat AF,
    2. Nimri N,
    3. Shokr M,
    4. et al.
    Correlation of Altmetric Attention Score With Article Citations in Cardiovascular Research. J Am Coll Cardiol. 2018;72(8):952-953. doi:10.1016/j.jacc.2018.05.062
    OpenUrlFREE Full Text
  16. 16.↵
    1. Dixon DL,
    2. Reed BN
    . To tweet or not to tweet? A primer on social media for pharmacists. J Am Coll Clin Pharm. 2019;(April):554-562. doi:10.1002/jac5.1120
  17. 17.↵
    1. Ladeiras-Lopes R,
    2. Clarke S,
    3. Vidal-Perez R,
    4. Alexander M,
    5. Lüscher TF
    . Twitter promotion predicts citation rates of cardiovascular articles: a preliminary analysis from the ESC Journals Randomized Study. Eur Heart J. 2020;41(34):3222-3225. doi:10.1093/eurheartj/ehaa211
    OpenUrl
  18. 18.↵
    1. Luc JGY,
    2. Archer MA,
    3. Arora RC,
    4. et al.
    Does tweeting improve citations? One-year results from the TSSMN prospective randomized trial. Ann Thorac Surg. 2021;111(1):296-300. doi:10.1016/j.athoracsur.2020.04.065
    OpenUrl
PreviousNext
Back to top

In this issue

American Journal of Pharmaceutical Education
Vol. 86, Issue 2
1 Feb 2022
  • Table of Contents
  • Index by author
Print
Download PDF
Email Article

Thank you for your interest in spreading the word on American Journal of Pharmaceutical Education.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Long-term Association of Altmetric Attention Scores With Citations in Selected Major Pharmacy Journals
(Your Name) has sent you a message from American Journal of Pharmaceutical Education
(Your Name) thought you would like to see the American Journal of Pharmaceutical Education web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
3 + 3 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Long-term Association of Altmetric Attention Scores With Citations in Selected Major Pharmacy Journals
Dave L. Dixon, William L. Baker
American Journal of Pharmaceutical Education Feb 2022, 86 (2) ajpe8606; DOI: 10.5688/ajpe8606

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Long-term Association of Altmetric Attention Scores With Citations in Selected Major Pharmacy Journals
Dave L. Dixon, William L. Baker
American Journal of Pharmaceutical Education Feb 2022, 86 (2) ajpe8606; DOI: 10.5688/ajpe8606
Reddit logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • ACKNOWLEDGMENTS
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

Similar AJPE Articles

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Teaching and Assessing Pharmacy Students on Sterile Compounding Accuracy Checks
  • Assessment of a Mindfulness Meditation Elective in an Accelerated Doctor of Pharmacy Program
  • Development and Initial Evaluation of an Advanced Pharmacy Practice Experience Readiness Assessment Plan
Show more BRIEF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Keywords

  • publications
  • bibliometrics
  • alternative metrics
  • journal impact factor
  • pharmacy

Home

  • AACP
  • AJPE

Articles

  • Current Issue
  • Early Release
  • Archive

Instructions

  • Author Instructions
  • Submission Process
  • Submit a Manuscript
  • Reviewer Instructions

About

  • AJPE
  • Editorial Team
  • Editorial Board
  • History
  • Contact

© 2023 American Journal of Pharmaceutical Education

Powered by HighWire