Pharmacy Students' Ability to Identify Fall Risk-Increasing Drugs Using an Innovative Assessment Tool

INTRODUCTION

Focusing on falls risk reduction in the elderly is of great value because falls are not only costly, they can also be deadly.¹ Approximately one in three adults aged 65 years and older will fall each year, with up to 20% of falls resulting in serious injury or death. Hip fracture is a common injury that, 95% of the time, is caused by a fall and can therefore be used as a surrogate marker. These types of injuries require hospitalization and can cost an average of $35,000 per stay.¹ Furthermore, with a one-year mortality rate of 50%, even non-fatal falls imbue a grave prognosis.²

The National Council on Aging (NCOA) released an updated 2015 Falls Free National Action Plan, which strives to implement a nationwide falls prevention program and raise awareness among patients and healthcare professionals on the risks of falls in the elderly.³ This Falls Free National Action Plan includes the following medications management strategies: “incorporate attention to falls prevention in curricula related to medications and pharmacology in medical schools, allied health education, and health care professional continuing education” (goal B, strategy 3), “develop a systematic method for predicting how various combinations of medications interact with patient characteristics to increase risk of falls, and then add to existing pharmacy software...” (goal B, strategy 2), and “maximize the opportunity to address falls issues as part of the Medication Therapy Management (MTM) Services within the Medicare Part D benefit.” (goal B, strategy 4).³

Seeking a way to address the above strategies relating to the NCOA Falls Free National Action Plan, the researchers developed a novel clinical decision support tool, the Medication Falls Risk Assessment Tool (MFRAT), in 2015 (Figure 1).⁴ The MFRAT was developed with a literature supported scoring system which allows medication risks from multiple FRIDs to be combined for a cumulative score. This prototype was designed within a Microsoft-Excel (Microsoft Excel, Redmond, WA) environment to be integrated as part of a MTM encounter to help pharmacists screen for and address the use of FRIDs in older adults and is used as a teaching tool within the doctor of pharmacy curriculum.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Example of a Completed Medication Falls Risk Assessment Tool (MFRAT)

Few studies have been published that describe the use of clinical decision support tools by students within a pharmacy curriculum.^5,6 Cutler and colleagues were able to test students’ abilities to use the Medicare Part D Plan Finder Tool using a pretest/post-test study design before and after health policy education.⁷ A literature search did not identify any use of a clinical decision support tools to prevent falls. Our study evaluates the ability of third-year pharmacy students (P3s) to identify fall risk-increasing drugs (FRIDs) in patient medication regimens when the MFRAT was applied to an actual patient case.

METHODS

In this pre/post study, P3 students were asked to find and collect de-identified data related to an actual patient that met Medicare Part D MTM criteria. The data were used to document an MTM encounter during the required geriatrics section of their pharmacotherapeutics sequence during the spring of 2017. The students were given approximately one month to gather the data in preparation for an application class scheduled within the geriatrics section. These students had completed the American Pharmacists Association (APhA) MTM certificate during the previous semester (fall 2016).

Prior to the application class, the students viewed the Medication Falls Risk Education (MFRE) consisting of an online presentation entitled “Medication Management and Fall Prevention in Older Adults.” Learning objectives for this presentation included identification of fall risk-increasing drugs (FRIDs), identification of health care system barriers related to medication use and falls, and strategies and tools related to deprescribing FRIDs.

Using a flipped-classroom approach, students were given access to the MTM worksheet containing the MFRAT and were guided through the tool’s use during the application class by the instructor, which further reinforced identification of FRIDs. Upon completion of the falls-risk focused MTM assignment using the MFRAT, students submitted the following de-identified documentation: patient history form, pharmacist action plan, personal medication record (PMR), medication-related action plan (MAP), as well as the MFRAT.

An assessment was developed and administered both before use of the MFRAT during the application class (pre-assessment) and after completion of the MTM documentation (post-assessment). The assessment consisted of 10 case vignettes with individual medication regimens. Chronic conditions represented in the assessment included chronic obstructive pulmonary disorder (COPD), dementia, hyperlipidemia, congestive heart failure (CHF), atrial fibrillation, chronic pain, diabetes, and cancer. There were five possible responses for each of the 10 chronic conditions. Thus, the assessment required students to make 50 distinct drug choices. Each question was in a "select all that apply" format, and the number of correct selections for questions ranged from one to five. Thus, of the 50 choices presented in the assessment, 30 were correct responses and 20 were incorrect responses (distractors).

The assessment was administered via Blackboard (Blackboard Inc., Washington, DC). Students received a participation grade for completing the pre-assessment; however, the correct answers to the assessment were not disclosed at that time. The post-assessment was conducted immediately following the application class and counted toward the course grade. Students’ grades on the assessment were calculated based on the number of correct answer choices, but they were not penalized for selecting distractor medications. However, the assessment scores included in the data analysis were: number of correct answers selected (maximum score of 30; higher scores indicated greater FRID identification ability), number of incorrect answers (distractors) selected (maximum score of 20; lower scores indicated greater discrimination of non-FRIDs), and combined total score (maximum score of 50; higher scores indicated greater combined identification and discrimination). This total score was calculated by assigning one point for every correctly chosen FRID and one point for every distractor that was correctly not selected. Thus, each question was potentially worth five points. Data from the pre- and post-assessment were collected, paired, and de-identified. Because, from a clinical standpoint, it is important for a pharmacist to be able to deprescribe true FRIDs and not deprescribe false FRIDs, this scoring system, which accounts for both true positives and true negatives, was devised to investigate students’ ability to accurately identify FRIDs. The study was deemed exempt by the University at Buffalo Institutional Review Board.

Students who had used the MFRAT tool already as part of an experiential education component (eg, APPE, IPPE, wellness clinic) were identified as experienced users while those that had not were considered inexperienced users. Because the pretest scores of the two groups were compared using a t test and determined to be significantly different, assessment scores were analyzed separately. The reliability of the obtained scale scores was estimated using Cronbach alpha. When checking a scale for internal consistency, social science research considers an alpha of 0.70 or higher as acceptable as it represents high internal consistency reliability.⁸ Data were analyzed using paired t tests in Minitab, version 17 (Minitab Statistical Software. Version 17. State College, PA) ⁹ to determine if use of the MFRAT resulted in a significant difference in assessment scores. In addition to examining the significance level, the Cohen's d measure of effect size was reported. In this analysis, a d value less than 0.2 was considered a very small effect; greater than 0.2 and less than 0.5 was considered a small effect; greater than 0.5 and less than 0.8 was considered a medium effect; and greater than 0.8 was considered a large effect.¹⁰ The proportion of students that selected each drug on the pre- and post-assessment were also examined using the McNemar test to determine statistical significance, except when a choice received either a 100% response rate or 0% response rate, in which case a two-proportion test was used. The authors assumed the data were non-normally distributed, so the Fisher exact test p value was used to interpret significance from the two-proportion test. A p< .05 was considered statistically significant for all tests.

RESULTS

Of the 136 enrolled students, 15 did not have both pre- and post-assessment data (13 students did not complete the pre-assessment and two students did not complete the post-assessment) and were subsequently excluded. The student co-investigator was excluded from the analysis, as were six students who selected all 50 answer choices on the post-assessment, apparently in an attempt to obtain the best grade without risking any loss of points from missing correct answers. Thus, the study sample was comprised of 114 students. Correct FRID scores between inexperienced (n=104) and experienced users (n=10) were compared via two-sample t tests. Results demonstrated that experienced users had significantly higher baseline scores (24.9 vs 27.3, p=.03). Therefore, the two groups were analyzed separately on all measures.

The mean and standard deviation for all scores, Cronbach alpha estimates, results for the paired sample t tests, Cohen’s d values, and corresponding effect sizes are presented in Table 1. Both groups demonstrated significant changes on all of the assessment variables (correct, distractor, and total); the effect sizes for the changes ranged from small to large.

The percent selecting each of the 50 medication choices on the pre- and post-assessments is shown in Table 2. On the post-assessment, 15 of the 30 correct FRID choices received 100% response rates. All correct FRID choices showed improvement from the pre- to the post-assessment, whereas some distractor choices had decreased response rates. Distractors that were chosen more on the post-assessment than on the pre-assessment included amiodarone, rosuvastatin, diltiazem, low-dose aspirin, and pantoprazole. Fifteen of the 30 FRID choices were selected by all students on the post-assessment, and vitamin D was not selected by any student on the post-assessment. For these 16 drugs, a two-proportion test was performed to compare the number of students selecting them between pre- and post-assessment rather than the McNemar test.

DISCUSSION

Overall, pharmacy students’ ability to identify drugs that increase fall risks significantly improved after using MFRAT. The improvements seen in all three scores (ie, correct, distractors, and total) among both student groups confirmed that the addition of the MFRAT to the standard MFRE increased the ability of students to correctly identify FRIDs, and reduced students incorrectly identifying distractors as FRIDs. This innovative teaching mechanism addresses many of the Medications Management strategies of the NCOA 2015 Falls Free National Action Plan.

As this study focused on the addition of a clinical decision support tool (ie, the MFRAT) reinforcing FRID identification, instructors can be confident that the use of similar tools will improve students’ retention of material instead of them having to repeatedly refer to peripheral resources to retrieve drug information. Other investigators have reported that student learning is enhanced through the use of computerized tools, and our data support previous findings that students retain the knowledge presented to them by the tool and are able to apply it to an actual patient case.^11-14

Benzodiazepines, zolpidem, and diphenhydramine were identified by >90% of students on the pre-assessment as FRIDs. This is to be expected as these drugs have been identified in other coursework, including the APhA MTM certificate program, as potentially inappropriate medications for elderly patients. However, because many of the students already knew this, it caused a ceiling effect on the assessment. This ceiling effect limited the change from pre- to post-assessment on these drugs and on the total assessment. In addition, the lack of variability on these items caused the Cronbach alpha estimates to be low. Future studies could include more items and items that are more difficult (thus increasing variability), as both of these would improve the assessment reliability.

Classes of FRIDs that showed the greatest improvement from pre- to post-assessment included skeletal muscle relaxants, selective serotonin reuptake inhibitors, serotonin norepinephrine reuptake inhibitors, opioid analgesics, antipsychotics, and antiepileptics. While this implies that the MFRAT aided in identifying these FRIDs, alternatively, it could suggest that students may have watched or re-watched the video presentation between pre- and post-assessments. To eliminate this possibility, the link to the video should have been removed after the MFRAT was released, forcing students to view it during the intended time before exposure to the MFRAT and not have the option to refer to it while taking the assessment. However, this would have prevented students from being able to review the presentation as they prepared for an examination, which was given the day after the post-assessment closed, and therefore was impractical. Because of this, the improvement seen in the selection of these FRIDs is thought to be the net effect of both MFRE and the MFRAT. In contrast, recognition of some lesser known agents (eg, doxylamine, imipramine, zaleplon) showed significant improvement with relative differences of >12% from pre- to post-assessment. Because these agents were not specifically named in the video presentation, their improvement is more likely associated with the use of the MFRAT.

The percent of students who selected 36 of the drugs increased from pre- to post-assessment, while the percent of students who selected the other 14 drugs did not change. Of these 14 drugs, four of them were correct answer choices and 10 were distractors. Student selection of 26 of the 30 correct choices showed significant improvement, while the four that did not had already been identified as FRIDs by 90% or more of the students at baseline. Therefore, students’ ability to identify FRIDs after MFRAT use improved.

However, the changes in selection of only half of the distractor choices were found to be significant from pre- to post-assessment, suggesting there was not an appreciable change in the proportion of students who selected them from pre- to post-assessment. This could be because the proportion of students who identified them as non-FRIDs was low on both pre- and post-assessments which would be considered a good outcome. Conversely, some non-FRIDs selection rate were high on both the pre- and post-assessment and therefore students did not successfully learn that they were not true FRIDs. An example of this can be seen in the results for insulin detemir (Levemir), which was selected by 53% of students on the pre-assessment and 45% of students on the post-assessment (p=.21). Some distractors (eg, low-dose aspirin, diltiazem, pantoprazole, rosuvastatin, and amiodarone) worsened between pre- and post-assessment and may be explained by the conflicting data reported in the literature. Furthermore, incorrect selection of amiodarone as a FRID worsened significantly from pre- to post-assessment (56% to 76%, p=.001). This may be explained by the conflicting data on antiarrhythmics and their potential to cause falls, even though it was not explicitly mentioned in the MFRE video presentation. While not intentional, results such as these help to identify focal points in the curriculum that need to be better addressed during class presentations to teach students why certain medication classes are not considered FRIDs.

A limitation of this study was that students only received credit in the course for completing the post-assessment, while completing the pre-assessment was counted as a participation grade. Thus, the effort students put in to taking the initial assessment is unknown. Additionally, aspirin (81 mg) was included as an answer choice on three separate questions, which complicated the data analysis as the intention was to have distinct, non-repeating distractors. This was an oversight made while creating the assessment questions, so a suggestion for future studies involving pre- and post-assessments is to ensure all answer choices are unique. The study was designed to provide MFRE to all students at the same time to ensure a consistent baseline of knowledge was established before they were assessed on FRIDs. However, data on when the students viewed the MFRE video presentation were not collected. Therefore, students may have deviated from the methodology and watched the presentation after taking the pre-assessment, after being exposed to the MFRAT, or even not at all, making it impossible to know whether all students received the same amount of education on falls risk prior to the assessments.

CONCLUSION

Use of the MFRAT resulted in a significant improvement in students’ ability to identify drugs that can lead to falls after examining an elderly patient’s medication regimen. When data were adjusted for outliers and analyzed for significance, the ability of both inexperienced and experienced users of the tool to select the correct FRIDs and incorrect distractors significantly improved. Therefore, pharmacy students increased their ability to identify FRIDs and discern them from distractors when MFRE was augmented with an actual patient case exercise that used the MFRAT clinical decision support tool.