Abstract
Objective. To evaluate the progression of pharmacy students’ knowledge of black box warnings across 3 years of didactic training, and to determine how they stay current with new warnings.
Methods. A cross-sectional survey instrument was administered to pharmacy students in their first (P1), second (P2), and third (P3) professional years. The survey assessed student awareness of medications possessing a black box warning and familiarity with the warning content for 20 medications (15 with and 5 without warnings).
Results. Mean number of correct responses identifying the presence or absence of a black box warning among the 20 medications were 5.8 ± 3.3, 9.6 ± 4.0, and 14.8 ± 2.8 for the P1, P2, and P3 students, respectively. Knowledge of black box warning content was variable. Students were least aware of the warning content for stavudine and enoxaparin. Students were most familiar with the warning content for paroxetine and estrogen.
Conclusion. Students' awareness and understanding of black box warnings was proportional to their educational progression, but their knowledge level was inconsistent across drug groups.
INTRODUCTION
Black box warnings are the most serious warnings imposed by the Food and Drug Administration (FDA) for prescription medications and highlight potentially fatal, life-threatening, or disabling adverse effects for prescription drugs.1 In addition, black box warnings may include information regarding restriction of use and/or distribution of medications. The FDA mandates that black box warnings be separated and highlighted from the other text in the package insert and typically are characterized with a black box border.
Most black box warnings are not the result of controlled clinical trials but often the result of postmarketing surveillance.2 Lasser and colleagues found that most warnings were imposed within 7 years of a drug's introduction to the market.3 Their study also found of the 548 new medications introduced from 1975–1999, 10% received a new black box warning or were withdrawn from the market. They projected that the rate of warnings for new medications in the next 25 years would approach 20%.
Wagner et al conducted a retrospective study using claims data from 10 different health plans in the United States to assess the prescribing frequency for 216 medications with black box warnings, and to evaluate adherence with the warning's recommendations.4 Medications with a black box warning had been prescribed to more than 40% of the patients. Additionally, prescribers' adherence rate to the warning criteria ranged from < 1% to 50% depending on the medication.
Currently there are approximately 350 drug entities with a black box warning.5 With this large number of medications, it is difficult for clinicians to stay knowledgeable regarding the recommendations given in warnings. How much pharmacy students learn about black box warnings and how they stay current with new warnings once they are in practice is not known. The goals of this study were to evaluate the progression of pharmacy students' knowledge of black box warnings across 3 years of didactic training, and to inquire how they stay current with new warnings.
METHODS
A cross-sectional survey instrument was administered to pharmacy students in their first (P1), second (P2), and third (P3) professional years at the end of the spring 2007 semester. The survey instrument assessed students' awareness of medications possessing a black box warning and familiarity with the warning content for 20 medications (15 with and 5 without).
The survey instrument was developed as a collaborative effort by faculty members in the schools of pharmacy and medicine. Medications were chosen to represent a wide range of indications and organ systems, low and high frequency of use in generalist practice, brand and generic medications, prescription-only and nonprescription medications, number of years on the market, and medications with and without recent publicity about the addition of black box warnings or serious adverse effects. Eighty percent of the medications on the survey originated from the top 200 branded or generic drug lists.6,7 The primary outcomes of the survey were students' awareness of medications possessing a black box warning (yes or no response) and their familiarity with the warning content (using free-text response). Respondents were asked their current year of training, and to self-report how they stay current with black box information. Students were allowed 10 to 15 minutes to complete the survey instrument at the end of class and were not allowed to use any reference materials or discuss the survey instrument with other students. This study was approved by the University of Kansas Medical Center Human Subjects Committee.
Free-text responses for the reason for the black box warning were judged by 2 independent raters, each of whom classified the responses as correct or incorrect. Kappa scores for interrater reliability were accessed for each medication and ranged from 0.8–1.0, with most drugs demonstrating complete agreement regarding correctness of the reported reason for the black box warning. Afterwards, any remaining areas of discrepancy were adjudicated and consensus was reached. Descriptive statistics were obtained, including percentage of correct responses for each individual drug, the mean percent correct for presence/absence of a black box warning for all 20 drugs, and the mean percent correct for the reasons cited for each of the 15 drugs carrying warnings. Analysis of variance was used to compare total scores across the P1, P2, and P3 students. Statistical significance was defined at a p < 0.05 level. Data was analyzed using Excel and SPSS (SPSS, Inc, Chicago, Illinois) for Windows.
RESULTS
An 82% survey response rate was observed (230 of 278 students; P1 = 50/72, P2 = 77/103, P3 = 103/103). The remainder of the students opted not to complete the survey form or returned it without answers. The ability to identify correctly the presence or absence of a black box warning on the list of 20 medications increased in association with years of education. Mean (± SD) number of correct responses identifying the presence or absence of a black box warning was 5.8 ± 3.3, 9.6 ± 4.0, and 14.8 ± 2.8 for the P1, P2, and P3 students, respectively (p < 0.05). Correct responses by professional year and specific medications are detailed in Figure 1. More than 90% of the P3 students correctly identified amiodarone, estrogen, infliximab, and paroxetine as medications with a black box warning. The only medication with less than a 50% accuracy rate for the presence of a warning in the P3 population was metformin, with a correct response rate of 36%.
Percentage of pharmacy students by curricula year correctly stating whether the labeling of specific medications contains a black box warning.
For the 15 medications carrying black box warnings, students' knowledge of the reason for the warning varied across drugs but correct responses increased with advancing year of education (Figure 2). Students were least aware of the black box warning content for stavudine (P1 = 0%, P2 = 4%, P3 = 5%), enoxaparin (P1 = 0%, P2 = 1%, P3 = 8%), and olanzapine (P1 = 0%, P2 = 0%, P3 = 12%). In contrast, students were most familiar with the warning content for estrogen (P1 = 0%, P2 = 33%, P3 = 71%), paroxetine (P1 = 0%, P2 = 46%, and P3 = 78%), and warfarin (P1 = 6%, P2 = 35%, and P3 = 71%). Additionally, many free-text answers which were scored as incorrect did mention an important drug safety concern, but not necessarily one specifically provided in the black box warning.
Percentage of pharmacy students by curricula year correctly identifying the reason for specific medication's black box warnings.
Self-reports about methods for staying current with black box warning information were diverse. Unfortunately, most students responded that they did not keep up with the literature (P1 = 92%, P2 = 62%, P3 = 77%). Web sites (eg, lexi-comp, FDA), e-mails from organizations, and professional journals or magazines were the most commonly identified responses for staying current. Media (eg, television or news shows), work, and school were also identified as sources of information.
DISCUSSION
Pharmacy students' knowledge of black box warnings improved significantly as they progressed through the curriculum. At our institution, the classroom instruction on black box warnings currently occurs in the P3 year. It consists of one 2-hour lecture in the drug information/biostatistics class, accompanied by 2 examinations in the pharmacy skills laboratory class. The lecture consists of reviewing the history, definition, FDA criteria, and types of information included in black box warning. Additionally, the lecture focuses on limitations of black box warnings and the use of warnings as a communication tool. Last, medications with black box warnings are discussed in the lecture.
As expected, low knowledge levels about warnings were found among P1s. P2 students, however, showed knowledge gains compared to P1s, even though formal classroom introduction on this topic did not take place until the P3 year. This suggests that informal exposure is occurring as students progress toward their P3 year, and that some learning on this topic is already taking place. Earlier formal classroom instruction on this topic might enhance learning outside the classroom. In addition, our students are now being exposed early in the curriculum to pharmacy practice (eg, introductory pharmacy practice experiences), thus some knowledge of black box warning issues prior to initiating pharmacy practice experiences would be beneficial.
Recent revisions or additions of black box warnings significantly impacted students' knowledge of them. Paroxetine, warfarin, and estrogen were recognized most consistently as drugs that possessed black box warnings and were recognized most accurately for warning content. These prescription medications all had black box warnings introduced or modified within 5 years prior to the survey. Recent publicity (eg, news coverage) about warning additions may have been the reason why students were able to identify the warnings. For example, not long before the survey was administered, outcomes of hormone replacement therapy from the Women's Health Initiative had been widely discussed in the media, as had studies demonstrating an association between antidepressants and suicide in young people. Previous studies had shown that highly publicized warnings make an impact on prescriber's behavior as opposed to less publicized material.8
The scoring system required that students give the correct reason for the black box warning, and citation of other important safety concerns were not credited with a correct score even if they were important information for pharmacists. For example, olanzapine's warning about the risk of diabetes, weight gain, and hyperlipidemia had been publicized in the media near the time that this survey was administered, and that media coverage may have influenced learners. Only 12% of the P3 students correctly identified the reason for olanzapine's black box warning (increased mortality in older patients with dementia-related psychosis); many students indicated olanzapine's warning as new-onset diabetes or weight gain. The scoring system underrepresented students' overall knowledge of drug safety issues for various medications because they were not given credit for knowledge of legitimate non-black box warnings.
The study also found that a majority of the pharmacy students reported not staying current with new black box warning releases or additions. To what extent this reflects registered pharmacists' awareness of black box warnings is not known. Mailings to prescribers have little impact on prescribers' behavior. 9–11 Several venues for black box warning information include the FDA Web site and a free Web site dedicated to providing a list of medications with warnings.5 The identification of effective methods to disseminate information regarding black box warnings requires further study.
A major limitation of this study was that it was conducted at only 1 institution. While this institution has a faculty member who was a recognized expert in black box warning-related issues, and who releases hospital-wide notifications on new warnings, additional studies should be done at other institutions to determine whether these results are generalizable. Other schools of pharmacy may introduce black box warnings at different points in the curriculum, potentially producing different results. Another limitation is that this survey instrument was not validated.
Currently the study of black box warnings is not mentioned in the Accreditation Council for Pharmacy Education (ACPE) Accreditation Standards and Procedures; however, this may be assumed under the umbrella topic of medication safety.12 All pharmacy schools should offer formal training to their students in the area of black box warnings. Surveying registered pharmacists would help to determine how practicing pharmacists stay current with new black box warning information.
CONCLUSION
Our cross-sectional survey of 20 medications (15 with and 5 without black box warnings) administered to P1, P2, and P3 students determined that pharmacy students' awareness and understanding of black box warnings was proportional to their educational progression; however, knowledge deficits remain. Early training in black box warnings is warranted in pharmacy schools' curricula. Discovering effective ways to educate pharmacy students and other health care professionals, and helping them stay current with new black box warnings once in practice, is vital to improving patient quality of care and prescription medication safety.
- Received June 5, 2009.
- Accepted August 2, 2009.
- © 2010 American Journal of Pharmaceutical Education
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.