Abstract
Objective. To design and implement an elective course in information mastery and assess its impact on students' ability to identify information needs and formulate clinically relevant, evidence-based answers.
Design. A semester-long (15-week) elective course was offered to third-year (P3) doctor of pharmacy (PharmD) students that outlined the necessary knowledge and skills for using information mastery in evidence-based practice.
Assessment. Results of a pre- and postcourse survey instrument demonstrated an increase in students' knowledge of information mastery and confidence in and familiarity with the practice of evidence-based medicine in pharmacy. Students who had completed the elective were able to provide higher quality search strategies and evidence-based answers to a clinical question than other P3 students, and P4 students who did not participate in the elective.
Conclusion. An elective course in information mastery improved students' knowledge and understanding of information mastery as it pertains to practicing evidence-based medicine.
INTRODUCTION
Because opportunities for pharmacy students, graduates, and practitioners extend beyond the traditional roles of pharmacists, drug information skills and the practice of evidence-based medicine (EBM) are vital components of pharmacy education. Watanabe's systematic 5-step approach to drug information serves as the foundation for pharmacists to provide drug information to patients and other professionals.1 The goal is to provide patients and health care professionals the most useful information possible, rooted in EBM. Curricular guidelines offered to colleges and schools of pharmacy through the American Association of Colleges of Pharmacy (AACP) Center for the Advancement of Pharmaceutical Education (CAPE) and the Accreditation Council of Pharmacy Education (ACPE) encourage teaching EBM in pharmacy education.2,3 However, with the growing number of resources and quantity of information available, identifying the most useful information becomes increasingly difficult. Instructing students in how to identify useful resources after employing their skills to navigate and find information also can be difficult for educators.
Information mastery is an approach to practicing EBM that focuses on identifying and using the most “useful” information in the clinical decision-making process. Information mastery defines useful information as information that is relevant to the situation, valid in its design and timeliness, and easily attainable through print and electronic databases. Slawson and colleagues identify that as the volume of medical and drug information grows, navigating and choosing the best evidence to apply to patient care becomes more difficult. Information mastery is a tool for all health care practitioners to navigate the “information jungle” of medical evidence/information with various practice tools, and emerge with the most useful information for clinical practice.
In colleges and schools of pharmacy, information mastery may be a valuable tool for students to identify useful information. It is not a replacement for the currently accepted 5-step model for drug information skills in colleges and schools of pharmacy. Rather, the tools used throughout information mastery allow pharmacy students and providers the opportunity to enhance the systematic approach and help users process these steps more quickly, the end result being a useful evidence-based answer. For example, step 2 of the systematic approach requires users to “determine the true information need.”4 However, the process by which a user should attempt to identify the true need has not been defined/explained adequately in the literature. The information mastery approach uses a model that encourages providers to identify a specific population, intervention, comparison, and outcome (PICO) before moving forward with their information search. Thus, PICO helps clinicians determine their true information need. When integrated into the systematic approach, the PICO model gives practitioners a template of the basic information needed before they search for evidence-based drug or medical information or gather more patient-specific information. More examples of how information mastery can support and enhance the systematic approach are shown in Table 1.
Use of Information Mastery to Enhance Pharmacy Students' Systematic Approach to Researching Drug Information
We attempted to identify other colleges and schools of pharmacy that incorporate information mastery into their academic program. While colleges and schools of pharmacy may use information mastery tools as part of their drug information curriculum, we thought it important to identify other institutions that specifically highlighted information mastery in pharmacy education. A search of PubMed and the American Journal of Pharmaceutical Education for information mastery in pharmacy education with regard to course offerings, curricular design, or formal instruction on the subject yielded no results. A general search also was performed using Google and Google Scholar (Google Inc, Mountain View, CA) using the terms information mastery, pharmacy, and school. Although results from this search directed us to information mastery textbooks, Web sites, online resources, and continuing education programs in colleges and schools of pharmacy, we were unable to identify any formal mention of information mastery to instruct students on EBM. Expanding the search to medical and other allied health professional education returned minimal results. Most of the information regarding information mastery was limited to graduate medical education.5-7 The lack of results does not mean that the tools or practice of information mastery previously have not been incorporated into a drug information course or curriculum in a college or school of pharmacy, but assessing its use without a formal description of the process is difficult.
Our objective was to develop an elective course in information mastery that would serve a twofold purpose. The first was to introduce students to the concepts of and tools used in information mastery and assess their performance and comfort with this approach to practicing EBM. The second was to use the elective course curriculum as groundwork for teaching an additional approach to drug information and EBM in a school of pharmacy. We included competency assessments in the elective course to assess whether enrolled students demonstrated higher proficiency in information mastery and EBM compared to their peers. Survey assessments also were included to evaluate student comfort with and practice of EBM.
DESIGN
The information mastery elective was designed using the fundamental approach and tools provided by Rosser and colleagues8 and the University of Virginia School of Medicine Information Mastery Practicum & Course.9 The elective was offered to P3 students. The objective of the course was to help students develop strategies and thought processes necessary to become lifelong learners and prepare for clinical practice in advanced pharmacy practice experiences (APPEs), residency, and/or employment. Learning objectives for the course included: (1) view a clinical situation and develop a relevant clinical question using the PICO model; (2) identify valuable and available resources for gathering information regarding a clinical question; (3) evaluate clinical information and categorize as disease-oriented evidence or patient-oriented evidence that matters (POEM); (4) critically appraise various sources of information for appropriate research technique and outcomes reporting, validity, and applicability to clinical practice; (5) compare and contrast treatment recommendations from various resources (guidelines, consensus statements, and proprietary data) with available POEM data; and (6) investigate and appraise point-of-care resources for usefulness in clinical practice.
All students had been instructed in the systematic approach to drug information and medical literature evaluation in a series of practical laboratory experiences completed prior to taking the elective course. Knowledge and skills gained from this series of laboratories formed the foundation needed for learning the skills and tools of information mastery.
Educational Environment
The elective course used an ability-based outcome philosophy.10 The goal was to identify the knowledge, skills, and attitudinal objectives taught in specific class periods that related to the school and course outcomes. The elective course, which met for 2 hours every week for 15 weeks, was facilitated by 2 pharmacy practice faculty members and the school of pharmacy librarian. Each student was provided with a personal netbook computer with wireless Internet access, an information mastery textbook, and a subscription to a “hunting tool”— either Dynamed, (EBSCO Publishing, Ipswich, MA), or Essential Evidence Plus (John Wiley & Sons, Inc, Hoboken, NJ). A hunting tool is an evidence-based information repository where practitioners are able to search for relevant and valid information. The tool does not provide the clinician with an answer, but rather the necessary information to evaluate and make a clinical decision. These resources were made available to students through the Wolters-Kluwer Health Facts & Comparisons (Philadelphia, PA) Academic Partnership Program Advancement of Clinical Education (ACE) Curriculum Grant. Each class period consisted of 60 to 90 minutes of instruction followed by 30 to 60 minutes of active-learning exercises based on current and previous class material. Active-learning exercises increased in complexity throughout the semester. Some exercises required students to develop questions from clinical case scenarios and then share their questions with other students or the entire class (think-pair-share) to demonstrate the variety of questions that might arise from a clinical encounter. In later classes, students developed questions and then used search engines and hunting tools to identify information that would be useful to answer the question. The goal was to begin a discussion about the benefits, limitations, and application of each of the methods for seeking information.
Course Content
In the initial class periods, students were instructed on how to formulate appropriate clinical questions using the PICO model and were required to demonstrate with in-class examples and worksheets. After formulating clinical questions appropriately, students received a refresher and broad overview on searching electronic resources effectively including PubMed, TRIP Database, Micromedex, Facts & Comparisons, and other traditional drug information references.11-14 In addition, students were introduced to various hunting tools for identifying information at the point of care. They were introduced to the ideas of patient-oriented evidence that matters as research that highlights outcomes that patients may care about most (eg, morbidity, mortality, quality of life), and disease-oriented evidence as research that reports surrogate outcomes and markers of disease.8 After focusing on the students' ability to identify and locate information for clinical questions, the course focused on how to evaluate the data for relevance, validity, and clinical application. Students learned how to identify clinical significance, evaluate company-sponsored data and marketing tactics, and compare therapies using a “STEPS” analysis. STEPS is an acronym for safety, tolerability, efficacy, price (preference), and simplicity, and is used to determine the benefit or risk of a particular intervention in a patient or patient care scenario.15,16 A more comprehensive list of course topics is presented in Table 2.
Course Topics for an Information Mastery Elective
Within each class period, students were presented with 1 or more clinical scenarios that required them to use their learned skills to make an informed and evidence-based clinical decision. For example, when students were scheduled to receive instruction on how to evaluate therapeutic dilemmas or compare therapy options, each student was asked to develop a clinical question based on an introductory pharmacy practice experience (IPPE) or internship experience before coming to class. During the first part of class, students learned the components of a STEPS analysis and how to use previously-acquired knowledge and skills in this process. After a brief description of a STEPS analysis and how to conduct one, students were asked to share their clinical questions. Three questions were identified to research in class. The class then separated into groups based on the question that most interested them and conducted the appropriate research using Internet databases and hunting tools. Because information mastery encourages the use of highly relevant and valid information that requires limited effort to find, students were given a limited amount of time to complete the task. At the end of class, each group reported on its research using STEPS and attempted to make a general clinical conclusion.
EVALUATION AND ASSESSMENT
In-class Assessments
During the semester, students were given 2 cumulative practical examinations (weeks 5 and 9) to assess their knowledge and skills. In addition, each student was paired with another classmate to prepare a capstone presentation for the final weeks of class. The capstone presentation focused on a clinical question developed by the students and approved by course faculty members. The presentation was designed to follow the Family Practice Inquiry Network's template for clinical inquiry.17 Presentations were recorded using Profcast (Humble Daisy Inc, Ann Arbor, MI) so students could include them in their electronic student portfolios if desired. Each presentation was graded by the course faculty members, and the grading rubric was shared with students prior to their presentations. The 4-point grading scale and 10 grading criteria were outlined in a rubric: formulating a clinical question, search strategies, resources used, evaluating the information, providing an evidence-based answer, identifying the next information need, overall presentation skills, organization, ability to address audience questions, and supporting materials.
Eleven students completed the elective course. Capstone presentation grades ranged from 86% to 97% (mean 92%). Students most often lost points for formulating an incomplete or poorly structured clinical question or not identifying relevant follow-up questions based on their initial clinical question. Students regularly excelled in using hunting tools and secondary resources (Pubmed, TRIP Database) to identify relevant information regarding their clinical question.
Preassessment and Postassessment Survey
At the beginning of the course, students were given a survey instrument about their comfort and familiarity with concepts relating to information mastery, application of medical evidence to patient care, and what resources they typically used to answer patient and/or provider questions. The survey instrument was developed by the course instructors for the purpose of this elective and had not been previously validated. The same survey instrument was re-administered at the completion of the elective course. The first portion of the assessment asked students to rate their comfort level with several different information mastery concepts using the following 5-point Likert scale: 1 = strongly agree, 2 = agree, 3 = unsure, 4 = disagree, and 5 = strongly disagree. The median score for each concept was determined to compare pre- and postassessment answers.
The most notable change from precourse to postcourse Likert-scale rating was observed in students' comfort and familiarity with developing a relevant clinical question using the PICO model (improved from a median score of 4 to 1) and ability to retrieve, evaluate, and use medical information (improved from a median score of 4 to 1). When students were asked to assess their comfort with identifying clinical research to further evaluate, and their ability to critically appraise an article independently, a change from a median score of 4 to 2 was reported. All other questions had a change on the Likert scale of less than 2 (Table 3).
Pharmacy Students' Assessment of Information Mastery Knowledge and Skills Before and After Completing an Elective Course
The preassessment and postassessment included a list of references and asked the students to choose all that they used when searching for information (Table 4). One hundred percent of students chose Lexi-Comp and Micromedex in both the pre- and postassessment survey. Fewer used Clinical Pharmacology and Facts and Comparisons; however, the change in percentage using each was less than 10% between preassessment and postassessment data (100% to 91% for Clinical Pharmacology and 67 to 63% for Facts & Comparisons). Use of the Cochrane Database of Systematic Reviews, Dynamed, and Essential Evidence Plus increased from 0 usage to 90.9%, 100%, and 73%, respectively. The use of PIER, The Pharmacists Letter, and National Guidelines Clearinghouse increased by a small percentage on the postassessment.
Pharmacy Students' Assessment of Database Usage Before and After Completing an Elective Course in Information Mastery
After completing the course, the students were asked to provide feedback about how the course prepared them for future practice. Overall, student comments were positive and stated that the course was beneficial in preparing them for APPEs and future pharmacy practice. They stated that they had gained confidence and were more comfortable performing timely and efficient searches, interpreting the evidence for validity, and applying the information gathered to practice. A student in the elective stated, “One of my biggest fears about rotations was being able to answer a physician's question and have the evidence to support it. I now know where to look for information and how to interpret it. I feel more prepared and a little less stressed about rotations because of this class.”
Information Mastery Exercise
After completing the elective course, the information mastery skills of students who had completed the course were compared to 2 other groups of students via an exercise conducted using Survey Monkey (SurveyMonkey, Palo Alto, CA). An e-mail asking the students to participate in an information mastery exercise and describing the purpose, content, and expectations of the exercise was sent to all P3 and P4 students. Students also were informed about how the results would be used and ensured that their participation was optional and anonymous. Participants were given a brief clinical scenario and asked to create a clinical question from the scenario and an answer to the clinical question, and to describe the quality of the resources used to answer the question.
The 3 areas were assessed for appropriateness using a 6-point scale and worth 6 points each for a possible total score of 18 on the exercise. In addition, the amount of time that students spent completing the exercise was evaluated. All submissions were blinded to faculty instructors, and each submission was reviewed independently. After independent reviews, the 2 primary reviewers compared their assigned scores. Scores that differed by 1 point were averaged and recorded (ie, if reviewer A gave the submission a score of 5, but reviewer B scored it as 4, the recorded score was 4.5). If an initial score differed by greater than 1 point between reviewers, the answer was reviewed and a score was agreed upon. After completing the initial faculty review, an additional faculty member (not involved in the course) was asked to evaluate student submissions based on the evaluation criteria. The outside faculty member's scoring was used to validate and confirm the scores of the primary reviewers.
Two students who were enrolled in the elective chose not to participate in the competency assessment. Nine students enrolled in the elective, 10 P3 students not enrolled in the elective, and 5 P4 students participated in the postclass competency assessment. After review and adjudication, the mean score ± SD for P3 students who completed the course was 16.7 ± 1.7 compared to 7.3 ± 3.1 for P3 students who did not complete the elective and 8 ± 5.37 for P4 students. Mean scores for individual areas are shown in Table 5. The external reviewer's scoring agreed with that of the primary reviewers' scoring 86.1% of the time (62/72 scores). Of the 10 scores that differed between the initial reviewer and the external reviewer, 4 were scored higher by the external reviewer (mean 2.4 points) and 6 were scored lower (mean 2 points). In addition to competency scores, students were asked to self-report the time it took to complete the assessment. The mean times reported for each group were 13.4 ± 5.6 minutes for students who completed the elective, 14.4 ± 6.3 minutes for P3 students who did not complete the elective, and 19.8 ±9.7 minutes for P4 students.
Competency Area Scores for Each Group Assessed, Mean (SD)
DISCUSSION
Since its inception in 1996, the school of pharmacy at Wilkes University has used the systematic approach to drug information to instruct students about gathering, analyzing, and disseminating information. As a basic approach to formulating a thought process and attempting to work through questions of varying complexity, the systematic approach has served the profession well. Nevertheless, despite multiple revisions, since it was first published, the systematic approach is open to interpretation with regard to completing each of its steps. In writings published since the last revision, other authors point out that the systematic approach implies a level of deception in all questions and that the question asked is not always the question that the requestor needs answered.4 These uncertainties may cause students to focus on trying to decrypt the question rather than gather information to provide an insightful answer. Unfortunately, while the systematic approach instructs students to find the true question, it offers little assistance to first-time users on how to achieve this step.
In addition to identifying the requestor's needs and the true clinical question, the systematic approach encourages pharmacists to conduct an inclusive and efficient search of the medical literature to find the answer. With the increasing number of references and publications available each year, traditional search strategies require more and more effort to identify valid information relevant to the situation at hand. Conducting an inclusive and detailed search is always important, but some circumstances or situations may not allow time for this. In those situations, a resource that has already conducted the initial search is highly useful.
Introducing and using information mastery in a school of pharmacy poses some barriers and adaptations. One of the first considerations is that information mastery is not meant to replace drug information, but enhance how students gather and interpret information. Students are still expected to conduct appropriate information searches and evaluate the information themselves. Information mastery teaches students additional skills and how to expedite these searches.
One apprehension about using an information mastery approach is the risk of students being less proficient with some of the more universally available drug information resources. Our results did not show a notable decline in use of these initial resources and showed that the students' searches had the ability to be more inclusive with additional database knowledge. Faculty members and other stakeholders in pharmacy education also may be hesitant to adopt an information mastery approach because it could be viewed as a “short cut” with regard to identifying, retrieving, and evaluating medical and drug information. We are aware of this limitation and choose to address it in 2 ways. First, the skills of identifying, retrieving, and evaluating medical and drug information are not eliminated by information mastery as these are essential skills for all practitioners and life-long learners. They are the foundation for being able to practice evidence-based medicine. The perceived “short-cut” provided by information mastery is misinterpreted and should be viewed as a way to ensure EBM is provided at the point of care. Information mastery allows providers to make informed decisions at the point of care rather than withholding or making an uninformed decision. These point-of-care decisions then should be further researched as part of professional development.
This also raises another concern about whether information mastery should be taught to pharmacy students or only to pharmacy residents/postgraduate students. Pharmacy students are encouraged to make well-informed decisions and avoid coming to conclusions based solely on the information in abstracts or summaries. The goal is for students to be able to identify the information source, retrieve the source, evaluate it in its entirety, and come to a conclusion. These skills are a necessity when learning information mastery because, even though hunting tools summarize information for use at the point of care, students, trainees, or practitioners should evaluate the research on their own for future encounters. Also, when using hunting tools as a part of information mastery, the tools should be reputable resources that provide evidence-based answers. As part of an information mastery curriculum, students should be instructed on how to identify high-quality versus low-quality hunting tools. Tools that use a systematic or evidence-based approach to finding the most relevant and valid information should be encouraged, while opinion-based and consumer-based resources should be discouraged.
Teaching the information mastery approach in the pharmacy school curriculum offers great opportunity to assist students in completing tasks and developing skills associated with drug information retrieval. When introducing new hunting tools to the available drug information resources, we were concerned that students would use other vital information resources less. Fortunately, we did not see a decline in the spectrum of resources students considered when answering questions, but an increase in their awareness and potential use of more resources.
There were additional limitations that we identified with the evaluation of the course. Better performance on the competency assessment by students in the elective course may have been biased by their recent completion of the course. We attempted to control for this by comparing their results to their peers and graduating fourth-year students, but ideally should compare the students who completed the elective to their peers, 1 year later, prior to graduation, to evaluate whether their performance remained at a higher level. We did not perform a precourse competency assessment, thus, we cannot determine whether students who enrolled in the course may have been performing at a higher than expected level already. Finally, the low sample size in the elective and comparison groups makes it difficult to extrapolate this information to an entire class, curriculum, or other schools of pharmacy. With further course offerings, we hope to have more results to help in our evaluations.
While expanding database availability and familiarity with information databases is ideal, it also poses a dilemma for institutions. Most of the tools for identifying high-quality medical and drug information require individual and/or site subscriptions which may pose a significant cost burden to students and schools of pharmacy. While some of these databases already may be available through academic medical centers, justifying subscriptions to these databases for educational purposes (as opposed to patient care purposes) may be difficult. However, for the information mastery approach to drug and medical information, these tools are essential and cannot be substituted. The decision of which hunting tool is appropriate should be a decision made by faculty members and students as they will be the primary users of this resource. During informal discussions, students who participated in the elective course commented that an individual subscription to these services would not be unreasonable as the cost of an annual subscription is approximate to the cost of a course textbook.
Faculty members or other practitioners may be familiar with information mastery and its practice, and instruct students on APPEs or IPPEs. However, it may not yet be accepted as part of a drug information curriculum. The first step in facilitating this transition is ensuring that all faculty members are familiar with the practice of information mastery, and preceptors are aware of the change.
The next step in this process is to begin developing an information mastery curriculum to use throughout the school of pharmacy when instructing students about drug and medical information. This change will take significant effort as it requires all members in the school of pharmacy to be familiar and proficient with information mastery when practicing EBM. Also, a new set of goals and outcomes for the School of Pharmacy Educational Outcomes document will be developed and proposed to the school for discussion. Pending approval, it then will require 1 to 2 academic years to completely integrate into the program as students still are being instructed solely using the 5-step systematic approach. The elective course will continue to be offered to P3 students.
CONCLUSION
The information mastery elective course served as an excellent pilot program to demonstrate the effectiveness of the approach to identifying, understanding, and using medical and drug information. The course's structure will serve as a foundation for development of other courses in the school of pharmacy. The practice of information mastery has the potential to enhance the way pharmacy students gather information and use it in clinical decisions and information dissemination.
- Received October 5, 2010.
- Accepted November 22, 2010.
- © 2011 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.