Case-based Assessment in an Online Nursing Pharmacotherapy Course

INTRODUCTION

Pharmacotherapy is a complex topic that requires several years of study in both classroom and experiential contexts for pharmacy students to master. Other health professional training/degree programs often limit pharmacotherapy education to 1 semester or term. Because of their expertise in pharmacotherapy, pharmacy educators often are asked to teach the pharmacotherapy component of the curriculum within other health professional programs. Comprehensively covering the wide range of topics within the field and tailoring the content to a particular health professional audience presents significant challenges for the instructor. Increasingly, pharmacotherapy courses in nonpharmacy health professional programs are offered online, which adds a further challenge to creating a meaningful, yet manageable, educational experience.

Within health professional programs, a variety of related educational strategies are used to provide authentic learning opportunities, such as case-based, anchored, problem-based, project-based, and cognitive apprenticeship instruction.¹ Teaching cases, an active-learning strategy commonly used in health professional education, are used to expose students to important and common patient situations that have no definitive correct solution. The use of teaching cases is based on the premise that learning from a specific case discussed in the classroom can be transferred to an actual patient encounter in practice, and that exposure to teaching cases helps students to develop the complex skills and flexible thinking needed to adequately manage novel patient cases.² But effective use of case-based strategies requires a lot of instructor time and effort and careful development and planning.¹ For this reason, case studies often are not used in high enrollment courses, even though this strategy is recognized as an important tool for building clinical competency.¹

Case-based reasoning agrees with a constructivism position that students must create their own understanding within a situated learning experience.³ Through case-based design, the goal is to create learning opportunities that require active engagement of the learner; the learner is given opportunities to infer and apply judgment, and to detect similarities and differences between situations.³ The ultimate goal is that students develop reasoning skills that can be transferred to patient situations later encountered in practice. The story aspect of case-based learning reinforces the idea of treating the whole patient, and recognizes that the patient's lifestyle as well as social and work history can impact treatment decisions and success.³

Transfer of knowledge, “the ability to extend what has been learned in one context to new contexts,”^2(p51) is a complex area of cognitive science, but in health professional education generally refers to the transfer of knowledge from a course to a real-life situation. Because every possible patient scenario cannot be taught, transfer is a primary skill for health professionals and the ability to transfer determines clinical competency.² Transfer requires that content knowledge not be overly situational and be learned in a meaningful way, allowing students to personally construct their knowledge and self-assess their learning. While disagreements in the literature concerning transfer is outside the scope of this manuscript, the importance for health professional educators to consider the goal of knowledge transfer in the design of educational activities and assessments is noteworthy.

As more health professional programs explore distance education formats, the challenges of providing engaging activities to a student audience separated from instructors and fellow classmates by distance and time is further complicated by high enrollment numbers. While synchronous distance activities are an option, they too, have their challenges, including: finding a specific time when all members of a course or group can participate; making adjustments for students who are not able to attend the synchronous event due to illness, technology issues, etc; dedicating the additional instructor time and effort that synchronous events require; and navigating through/maintaining continuity during the often disjointed discussion that occurs within a synchronous event (eg, different typing speeds, side conversations).⁴

Since 2004, the University of Minnesota College of Pharmacy has offered an online pharmacotherapy course to students in the bachelor of science in nursing degree program; in 2007, a graduate level pharmacotherapy course was offered to students in the master of nursing degree program. Teaching pharmacotherapy online and outside of the context of the pharmacy profession can be challenging. One complaint consistently mentioned in course evaluations was that the overwhelming amount of content covered in the course was too much to learn in 1 semester. While students have traditionally performed well (primarily A's and B's), there was concern among students and instructors that the volume of content was too great and that students were not retaining the information and thus not able to transfer their knowledge to practice after the course concluded.

The objectives of this study were to develop and evaluate a case-based approach to learning materials and assessment for an online pharmacotherapy course for nursing students; a design to foster in-depth understanding of common clinical scenarios and to facilitate transfer of reasoning skills to novel patient situations encountered in practice. The primary goal of this study was to design case-based assessments in the course, structured in a way that did not require synchronous course activities or individualized grader time and effort. The assessments aimed to address potential barriers to maintaining academic integrity, by using multiple parallel assessments that would perform equivalently. The use of case-based assessments and a transfer learning activity also were informally evaluated in this study.

DESIGN

In 2009, the instructional team redesigned the course, significantly decreasing the amount of content within the course and focusing it on the most common pharmacotherapy scenarios encountered by nurses within the ambulatory practice setting. Eleven nurses were interviewed to gain their perspective on the most relevant topics in the ambulatory practice setting. The nurses were given a list of previously covered topics in the course and asked to choose the areas and conditions most commonly encountered in practice. They also were asked which aspects of drug therapy were most critical to know (eg, mechanism of action, adverse effects, administration, monitoring, drug interactions, etc). The nurses’ comments were combined with the firsthand experience of the primary course director, a pharmacist who practices in a family medicine ambulatory care clinic, and the past experiences of instructional team members with teaching pharmacotherapy to nursing students. From this input, a list of pharmacotherapy topics was generated and shared with nursing program faculty members who made suggestions about what should be covered in the course.

Reducing the number of topics covered in the course allowed each to be covered in greater depth, with the intention that solid content knowledge and extensive practice making clinical judgments within those disease/condition areas would foster the transfer of knowledge and clinical skills to future novel patient scenarios. The course learning materials were presented in a case-scenario format – students “followed” 2 patients, a man and a woman, throughout each learning module. The patients in the scenarios in subsequent modules continued to develop conditions as the semester progressed. By the final module of the course, both patients had a complex combination of medication conditions and therapies. Evaluation of the redesign using the new assessment strategy was performed in spring 2010, during the second offering of the course.

EVALUATION AND ASSESSMENT

Evaluation measurements of the assessment strategy were collected, including midterm examination averages, final examination averages, final grade distributions, and student perceptions of the measure of knowledge for the midterm examination and final examination. All examination scores and questions are reviewed after the examinations were delivered to evaluate question performance. No questions were dropped from the examinations. Data collected from the semesters using the new assessment strategy (fall 2009 and spring 2010) were compared to the 2 previous offerings of the courses in which proctored examinations were used. The 4 versions of the midterm examinations and final examinations also were compared for both courses using ANOVA tests, with main effect of test.

DISCUSSION

The use of online, case-based assessments for high-enrollment, online pharmacotherapy courses for graduate and undergraduate nursing students was successful in providing 4 parallel examinations that assessed students’ case-based decision making skills, while maintaining academic integrity and minimizing the grading burden for instructors. Graduate and undergraduate students’ performance on the 4 midterm and final examination was similar. The ease of administering and grading these examinations provided a manageable assessment method that we continue to use in these courses. Minimal to no technical issues were reported during the delivery of the examinations.

While scores on the final examinations for the undergraduate course detected a difference between versions, scores on the identical final examinations for the graduate course did not differ significantly. Averages for the examinations were largely in the A grade range; however, this aligns with the redesign of the courses. One goal of reducing the amount of content and focusing on a smaller number of topics was to allow students to master the material; thus, the scores may be a reflection of this mastery. A larger percentage of students felt the online versions of the midterm and final examinations were better able to measure their knowledge compared to the previous face-to-face proctored examinations. Although there is a body of evidence suggesting that student perceptions of efficacy may not be indicative of increased learning outcomes,⁶ student satisfaction related to the course design and activities is relevant to student learning and has been found to be an important component of motivation.⁷

The act of completing the case-based assessments and learning activity may have furthered student learning and provided them with additional practice with clinical decision making. The idea that students often learn from an examination, especially from mistakes made on an examination, is not new; as Rash states “a primary goal of assessment is to enhance learning.”^8(p90) Further, in their recent study evaluating the role of information retrieval in learning, Karpicke and Blunt argue that recalling information to answer an examination question requires a number of complex steps, including reconstructing content in a way that is relevant to the question at hand, and “thus mechanisms involved in retrieving knowledge play a role in producing learning.”^9(p774) While the act of retrieval was not evaluated in this study, further investigation on the influence of using case-based assessments as an additional learning strategy is warranted.

Using this assessment strategy of parallel case-based examinations delivered online has limitations. The creation of multiple examinations is time consuming for faculty members.^10,11 The time to create the first case-based 20-item multiple-choice examination was approximately 20 hours, with subsequent versions of the examination taking approximately 10 hours or less. The examinations also must be comparable in difficulty and content. This study showed that the examinations were equally difficult, and this was accomplished using a systematic approach to ensure parallel concepts were covered in each version of the examinations. Using a chart or keeping a tabulation of the concepts that have been covered and repurposing examination questions by applying them to new patient cases are suggestions that may aid instructors in creating subsequent versions and revisions of an examination.

The case-based reasoning literature states that failure and reflection are essential components to educational design; however, neither was incorporated in this study, which primarily focused on the ability to create parallel, case-based examinations, administered online in a way which maintained academic integrity. Incorporating specific opportunities for students to fail in a low- to no-stakes learning environment and adding a reflection component will be the next steps in the continuous quality improvement of this course.

While this particular course was delivered to students in professional nursing programs, the concept can be applied to other high-enrollment courses in other professional healthcare programs, included pharmacy. As the number of pharmacy programs grow and enrollment increases, online delivery of courses may play a larger role in the curricula.

SUMMARY

Using an online, case-based assessment strategy proved to be an effective and feasible method for evaluating the clinical application knowledge of undergraduate and graduate nursing students in a high-enrollment online pharmacotherapy course. The teaching strategy satisfied both the case-based educational needs of the health professional students and incurred a manageable workload for the instructional team.