Methods for Optimizing Student Pharmacist Learning of Clinical Note Writing

INTRODUCTION

When students are learning to solve problems, two important instructional strategies are commonly used: studying a worked example and practicing problem-solving. A worked example is a step-by-step explanation or demonstration on how to solve a particular problem or situation.¹ If a student was asked to solve for x given the equation 2x+10=24, then the worked example would take a stepwise approach to show the solution (eg, Step 1: subtract 10 from both sides; Step 2: divide both sides by 2).¹ Problem-solving practice is retrieving information from one’s memory to solve a given problem. For example, after a lesson on basic algebra, a student may be asked to solve the same problem, 2x+10=24, on their own, without any guidance.

Selection of instructional strategies can be difficult and is often contextual. For instance, when a first-year medical student is learning to perform a procedure, problem-solving practice (eg, retrieval practice) may be the most effective method of instruction because retrieval practice strengthens the student’s memory of the fixed sequence of steps required for the procedure. However, for acquiring and developing the initial cognitive framework to organize the key concepts of the procedure, a more effective approach may be a worked example. This instructional strategy has been used for areas of problem-solving that are more algorithmic, such as math, physics, or computer programming, but have not often been applied to less-algorithmic activities such as writing.¹ Because clinical note writing requires a procedure for breaking down a patient case into defined parts, the task of clinical note writing may be very adaptable to these instructional approaches. As such, we examined two approaches to teaching clinical note writing: a worked example paired with problem-solving practice (WE-PS) and a worked example paired with another worked example (WE-WE). The differences between a utilizing a worked example versus problem-solving for writing a clinical note are illustrated in Figure 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Differences Between a Worked Example and Problem-Solving Practice Teaching Modalities to Teach Clinical Note Writing

With regards to problem-solving, an individual must first have a schema. Schemas are generic knowledge frameworks with respect to a specific situation that guide recognition and understanding of a new problem. A schema can help in the problem-solving process by helping learners recognize the problem structure. Acquisition of a schema may be improved by providing learners with worked examples.² Providing novices with more worked examples is often more effective for learning than relying heavily on problem-solving practice^2-5; however, once a schema is acquired, problem-solving practice may be more beneficial.⁶ Studies have found that students provided with worked examples outperformed students who used problem-solving practice.^5,7,8 These findings contrast somewhat with the generation effect, which states that people are better able to recall information that they generate themselves.⁹ In certain instances (especially for novices), the benefits of worked examples may outweigh the benefits of learning by problem-solving. However, it is unclear how easy it is to develop the schema necessary for clinical note writing. If easy, the worked example-problem-solving approach may prevail; if difficult, the worked example-worked example approach may be best. Research suggests that studying additional worked examples may be more effective than solving additional problems in learning a procedure like clinical note writing.

In addition to the problem-solving domain, learners must also develop the ability to transfer their skills to new problems. Few studies have compared the effects of only worked example study to only problem-solving study. Of these, some found no difference and some found a benefit from problem-solving, but most found that worked examples were more effective for transfer.¹⁰ Most studies, however, have alternated worked examples with problem-solving (example-practice approach). Several studies have shown that worked example and problem-solving pairs were more effective for learning and transfer than problem-solving alone.^2,3,11-13 However, when the results of studies examining worked example followed by worked example (WE-WE) were compared to studies where worked example was followed by problem-solving practice (WE-PS), the findings were mixed.^14-20 Problem variability may also help with the ability to transfer skills and knowledge.^21-25 Thus, we introduced problem variability to assess the impact on transferring clinical note writing to different disease states.

Some research studies have investigated different teaching methods on clinical note writing. Within these studies there was an emphasis on assessing the clinical notes written and less on teaching the actual skill of note writing. In one study, Andrus and colleagues investigated the implementation of a standardized rubric into a preexisting practice for clinical note writing during advanced pharmacy practice experiences. Students who received formative feedback and further instructional direction had significant improvements in their scores and in their confidence in clinical note writing, and received more consistent grading from various preceptors.²⁶ Divine and colleagues explored the impact of curricular integration on clinical note writing. Their study incorporated guided instruction, peer grading, and recorded simulations into their didactic courses to address an observed deficiency in clinical note documentation. The researchers found that these changes provided more opportunities to practice documentation skills, while producing significant improvements in the assessment and plan domains of the clinical note. While this research has shown improvements, there is limited data regarding the use of explicit instruction in teaching clinical note writing.²⁷

The purpose of our study was to investigate the role of worked examples and problem-solving practice in student pharmacists learning and reinforcement of the skills for writing a clinical note. Specifically, we tested the effectiveness of the worked example-problem-solving (WE-PS) approach against that of the worked example-worked example (WE-WE) approach to learning to write clinical notes. Furthermore, we examined the effects of variability on the worked examples or problem-solving practice, relative to initial learning and transferability skills.

METHODS

We used a 2x2 factorial study design to examine both instructional activity (worked example or problem-solving practice) and problem variability (low or high). The study design consisted of three phases: learning, intervention, and assessment (Figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Flow Chart of Study Design

An a priori sample size calculation was performed based on a large effect size (d=0.8), alpha of 0.05, and beta of 0.2. The sample size needed was 26 participants per group as determined using G*Power 3.1 software (Universität Kiel, Germany). Participants were recruited from the first professional year (P1) at one school of pharmacy, and the study was conducted during the first month of the PharmD program. First professional year students (P1) were selected because of the lack of any formal instruction on clinical note writing and minimal background in patient care. Participants were compensated with a $15 gift card. Students within the first professional year were similar in age (M=22 years, range 18-31 years) and educational background (91% with a college degree, mean grade point average=3.5, mean Pharmacy College Admission Test score=88%).

The first part of the study was conducted in a classroom where participants were given 90 minutes to complete the learning phase and intervention phase. Students were provided with a link to an online platform (Qualtrics) that contained all the study materials. In the learning phase, students worked individually to study a worked example on constructing a clinical note. For standardization, a traditional SOAP (subjective, objective, assessment, and plan) note structure was selected. The initial worked example started with a brief overview of each section of a clinical note. Following this brief overview, a patient case was presented. The final task was translating the patient case into a worked example. Here the worked example explained and guided the participants on how to breakdown the case, in a step-by-step fashion, into their respective clinical note domains (Figure 1). The first worked example was a patient with stage 2 hypertension and no other comorbidities. To account for the P1 students’ limited clinical knowledge, they were provided with abridged therapeutic guidelines, pertinent to the patient case, to help them understand the relevant drug therapy, dosing, and monitoring parameters. All case vignettes and their solutions and worked examples were developed by the authors and vetted by content experts who taught in the respective therapeutic areas.

The intervention phase was conducted in the same classroom, immediately after the first worked example was completed. Participants were randomized into four conditions by the online platform: worked example with low problem variability; worked example with high problem variability; problem-solving with low variability, and problem-solving with high variability (Figure 2). In the first two conditions, participants were presented with and studied another worked example (WE-WE). For the latter two conditions, participants were asked to generate a clinical note (WE-PS). For the problem-solving conditions, feedback was provided to students upon submission of their practice problem by providing them with the same worked example used in the other conditions.^28-30 Two other conditions were nested within the WE-WE and WE-PS conditions. Half of the students assigned to each condition (WE-WE or WE-PS) received a second case about hypertension that represented low problem variability because of the case’s similarity to the initial case (stage 2 hypertension). The other half of the students assigned to the WE-WE and WE-PS groups received a case about type 2 diabetes that represented high problem variability (Figure 2). For all participants, regardless of the condition to which they were assigned, backward navigation in Qualtrics was disabled so that they could not go back to re-study the original materials provided. Day one of the study concluded with the completion of the intervention (ie, second worked example or the practice problem). As in the learning phase, students were provided with abridged, relevant guidelines during the intervention. Following day one, links were deactivated and students could not review the experimental material in preparation for the next phase.

Three days later, participants returned to the classroom and began the assessment phase of the study. Consistent with day one, participants were given 90 minutes to complete the assessment. The assessment phase consisted of students completing a quiz and generating two clinical notes from vignettes. The quiz assessed the lower levels of Bloom’s taxonomy (knowledge and comprehension) using a mixture of single best answer multiple-choice questions and select-all-that-apply items. For the select-all-that-apply items, participants received one point (+1) for correct responses and lost one-half point (-0.5) for incorrect responses. For each section of the clinical (SOAP) note there was a definition question and select-all-that-apply question regarding what information goes into which sections (eg, family history into the S; laboratory values into O, and so on). Following the quiz, all participants then completed two clinical notes, one for a patient with stage 2 hypertension and another for a patient with a Clostridium difficile infection. These two different disease states were used to examine the ability of participants to transfer their skills near (to another hypertension case) and far (to the C. difficile case). Clinical notes were scored using a modified 25-point checklist based on the American Pharmacists Association guidelines and adopted from others.^27,35 This modified checklist only evaluated the presence or absence of material in each section of the note rather than relevancy of the material as that would have required participants to have more clinical expertise.

The primary analysis was the participants’ average performance on the two clinical notes (eg, hypertension and C. Difficile) under the conditions of either worked example or problem-solving. The secondary analysis was the average performance on the two clinical notes under conditions of low or high problem variability. We also subsequently examined these conditions for the quiz scores and performance on each note to examine transferability of skills. Each note was graded individually by two assessors and any discrepancies were discussed until consensus was achieved.

A p value of 0.05 was used as the threshold for statistical significance. A 2x2 ANOVA (worked examples/problem-solving and low/high variability), with planned comparisons using Bonferroni adjustment for three comparisons when appropriate. Statistical calculations were conducted using SPSS Statistics for Windows, version 22.0 (IBM). We calculated an effect size, when appropriate, using Cohen’s d with d<0.5 as a small effect, 0.5≤d≤0.8 as a medium effect, and d>0.8 as a large effect. This study was approved by the University of North Carolina Institutional Review Board.

RESULTS

Eighty-one P1 students were initially enrolled in the study; however, only 79 remained through completion (n=79, 98%). Two of the participants were excluded because they provided incomplete data or did not finish all components of the study.

The primary study outcome was the application of learning by generating two clinical notes, with the outcome measured being the average of these two notes. Participants who engaged in problem-solving practice (WE-PS) outperformed those who studied a second worked example (WE-WE) (WE-PS, 72%; WE-WE, 64%, difference -8% [95% confidence interval: -12%, -4%], p<.001, d=0.93) (Table 1). Specifically, differences were seen in performance on the subjective (d=0.79, p=.001), objective (d=0.64, p=.005), and plan (d=0.46, p=.04) sections, but not on the assessment section (d=0.28, p=.13; Table 1). When comparing low- to high-variability, we found no significant effects on note writing performance, except in the subjective section (d=0.48, p=.039) (Table 1). There were no interactions between instructional strategies and variability conditions (p=.83). Students randomized to the WE-WE group with low variability practice (65%, n=23) performed no differently in note writing than the WE-WE group with high variability (64%, n=19) (difference 1% [95% CI: -4%, 6%]). Students randomized to the WE-PS group with low variability practice (72%, n=14) performed no differently in note writing than the WE-PS group with high variability (72%, n=23) (difference 0% [95% CI: -7%, 7%).

We assessed knowledge retention and comprehension via a quiz administered three days after the learning phase but no further reinforcement of learning was provided. The mean score on the knowledge quiz was 75%. However, there was no significant difference in quiz scores between participants who received the worked example and those who received the problem-solving practice conditions (Table 1). Additionally, there were no significant differences in quiz scores between the low- and high-variability groups (Table 1). There was no interaction on quiz score performance between learning strategy (worked example or problem-solving practice) and variability conditions (high or low).

We looked further at performance on individual cases to examine differences in the ability to transfer skills. Participants performed better on the hypertension case than on the C. difficile infection case (hypertension, 72%; C. Diff: 65%, difference, -7% [95% CI, -10%, -5%, p<.001, d=0.63), especially in the subjective (hypertension: 84%, C. Diff: 69%, p<.001, d=1.1) and assessment (hypertension: 50%, C. Diff: 39%, p<.001, d=0.5) sections. First, we examined near transfer of learning based on variability in the learning, ie, all students were presented with an example of a hypertension case; thus, we looked at performance on a hypertension case. There was no difference in overall performance between participants who were assigned two hypertension cases (low variability) and those who were assigned two different disease states (high variability) (Table 2). For far transfer, we observed performance on a novel case (C. difficile). No significant differences in transferability were observed between the low- and high-variability conditions (Table 2). However, participants who were given a practice sample performed better on both cases than those who completed two worked examples. In other words, the WE-PS group performed better on the near- and far-transfer tasks compared to students in the WE-WE group.

DISCUSSION

This study examined the use of worked examples and problem-solving to facilitate the learning of clinical note writing. We found that giving students a worked example followed by a problem-solving example (the WE-PS approach) was superior to giving them a worked example followed by a second worked example (the WE-WE approach). The WE-PS model yielded higher student performance on both near- and far-transfer tasks. We did not find any variability in student performance or knowledge transfer when the cases used for each type of example were varied.

Our findings that the WE-PS approach was superior was consistent with the findings of Yeo and Fazio, who suggested that the use of problem-solving and worked examples aligns with the knowledge-learning instruction (KLI) framework.^31-33 The KLI framework consists of both testing to support one’s goals for stabilizing facts (ie, problem-solving practice or retrieval practice) and reviewing worked examples if the goal is to learn flexible procedures (eg, schemas). In our model, we observed that flexibility in procedure was present in the problem-solving group with their ability to apply and transfer the task. Both groups received similar quiz scores, indicating they acquired the desired knowledge, but students’ ability to apply the knowledge learned from the worked example varied depending on the learning approach to which they were assigned.

The use of variable learning problems may support more durable learning of a procedure. Past studies have shown benefits of variable worked examples,^34-39 while some have not.⁴⁰ Researchers have proposed that presenting participants with two or more different worked examples supports transfer of learning by allowing students to link analogous solutions to different problems.^{2, 36} In this study, we found no benefit of providing students with a variety of worked examples or practice problems. We focused on two nonidentical problems for practical reasons (time, realism to classroom practice). Providing students with additional nonidentical problems may impact the results. The use of multiple nonidentical problems may be more consistent with the effects of a curriculum where clinical notes are constructed for a variety of disease states over time.

A primary strength of this study was the use of learners who had not received prior formal instruction in clinical note writing on how to develop a set of essential, complex clinical skills. In addition, it is one of the few studies that has looked at using worked examples vs more formulaic problem-solving to teach clinical note writing. While worked examples and problem-solving seem to have helped students with learning the formulaic part of writing a clinical note (ie, what goes into the four sections of the SOAP note), clinical writing requires a basic level of therapeutic knowledge and exposure. In this study, we did not assess the appropriateness of the information or recommendations included in the examples, but rather whether the student provided the information in the appropriate sections of the SOAP note. The former requires clinical judgement, which is a limitation when using novice learners. For this reason, clinical note writing is usually taught in the context of therapeutics. For the therapeutic recommendation students’ proposed in their clinical note, the appropriateness of the included material was more straightforward since abridged guidelines were provided; that is, they could not recommend an alternative sub-optimal, therapy. The WE-WE approach may have been beneficial if pertinent information had been included in the grading rubric. The WE-WE approach may help with schema generation, instead of emphasizing the exact steps in a process. Additionally, because the sample size was based on large effect sizes, the study was not able to detect some of the small to medium differences that were observed (eg, when examining transfer effects). From a practical classroom intervention standpoint, this intervention allowed students to study once before assessing their abilities. In an authentic classroom, students would study the material repeatedly, which could improve performance.

This study also used a composite score from two different clinical notes crafted using the SOAP note format. Both the number of notes written and the format used may have been a strength of this study (ie, multiple outcome measures and a clinically relevant format) or a potential limitation. Having study participants write only two clinical notes may be viewed by some as low sampling compared to studies in which an examination with multiple questions is administered. Given the large effect size, it is unlikely that requiring more clinical notes would have changed the outcome. As for format, the format used for writing clinical notes varies by institution as well as by practice setting. Some institutions may vary the structure (eg, assessment and plan first), and some practice settings may only write a progress note and not a full note. Regardless of format, the skills required to write a clinical note following any format may be strengthened by studying worked examples and problem-solving.

The literature pertaining to instruction in clinical note writing is limited; thus, this study helps to augment our current understanding in this area. In addition, it also extends the previous findings around worked examples and problem-solving practice, which is typically focused on more algorithmic problems and math-based disciplines. By applying this literature to a less algorithmic task like writing, our findings augment the benefits of both worked examples and problem-solving. Further studies are needed to differentiate under what conditions the use of WE-WE or WE-PS is optimal, especially considering the role of clinical knowledge and the appropriateness of recommendations.

CONCLUSION

When the goal was for novice student pharmacists to generate a clinical note from information provided in a written case, the WE-PS approach was associated with better performance. When student pharmacists were asked to recall foundational knowledge about a clinical note on a quiz, there was no difference in retention of the content between those who had completed the WE-PS approach and those who had completed the WE-WE approach. The WE-PS model also was beneficial in helping students transfer their clinical note writing skills to a novel disease state. Hence, our findings suggest that use of the WE-PS approach provides optimal retention and transfer of knowledge when a procedure needs to be followed. Results from this study can help guide future instruction and provide insight into helping student pharmacists write clinical notes.