Student Pharmacists' Perceptions of Testing and Study Strategies

INTRODUCTION

An important responsibility in educating student pharmacists is instilling in them a commitment to lifelong learning so that as pharmacists they will be able to maintain and expand their knowledge and skills to better serve patients, the profession, and society as a whole. Ironically, it is the responsibility of educators to place more responsibility for learning on student pharmacists as they progress through PharmD curricula. Jungnickel and colleagues¹ state that “self-directed learning will be required for pharmacists to be viewed as pharmacotherapy experts by other health professionals.” ACPE Standard 11 states that teaching and learning methods should “foster the development and maturation of critical thinking and problem-solving skills… and enable students to transition from dependent to active, self-directed, lifelong learners.”² Pharmacy students should be able to determine their individual developmental needs, a domain encompassed within self-regulated learning.¹ Regardless of which term is preferred, be it self-directed learning, professional development, self-regulated learning, or lifelong learning, all point to the need for student pharmacists to take control of their learning, not only in pharmacy school, but also in the years that follow as pharmacy practitioners.

Monitoring cognition or metacognition is essentially knowledge about knowledge and is used to monitor and regulate cognitive processes.³ Pharmacy students should possess the skills necessary to employ metacognitive strategies prior to completing the PharmD degree and, preferably, early in the curriculum. Content knowledge in the pharmacy profession and in health care in general expands at a rate much faster than that at which curricula could be updated to ensure that learners are exposed to and comprehend all knowledge regarding topics important to the profession. Effective professional development relies on students' abilities to monitor learning and make changes as necessary to increase learning. However, college students receive minimal instruction on how to learn or regulate learning.⁴ Research regarding the extent to which self-regulated learning strategies are taught to and employed by pharmacy students as compared with the general college student population is lacking. Student pharmacists, in general, are perceived to be high achievers; however, high achievement scores are not necessarily predictive of learning strategies suitable for a profession that requires lifelong learning. In other words, the type of learning strategies students use to perform well academically and achieve above average grades may not be the type of learning strategies that best serve student pharmacists as they develop into knowledgeable practitioners.

Popovich has suggested that negative learning experiences in pharmacy school could result in students developing a mindset inconducive to lifelong learning.⁵ The amount of and importance associated with examinations in pharmacy curricula could result in student pharmacists who have a short-term (eg, learning for examinations) rather than long-term perspective on learning, eg, learning for a lifetime. Short-term learning is commonly driven by deadlines or an upcoming critical event as opposed to systematic monitoring of learning.⁴ Whereas this method of learning may allow many students to succeed from a grade/achievement perspective, the amount of long-term retention garnered from some strategies used to study for tests is questionable.^6,7

Pharmacy courses often involve few administered tests (eg, administration of only a midterm and final examination or 3 examinations per semester) that comprise a large percentage of students' course grades. Tests often are used for 2 purposes: to determine how much students have learned, and to assign grades. While these are valid reasons for administering tests, tests can serve an additional purpose: increasing learning.^8,9 The increased retention resulting from testing is referred to as the testing effect.⁸

Roediger and Karpicke posit that testing in the classroom and student self-testing can have a greater impact on retention than spending an equivalent amount of time restudying material “even when performance on the test is far from perfect and no feedback is given on missed information.”⁹ This concept is not novel. Literature dating from the 1600s recognizes the role that self-testing can have in retention.¹⁰ However, a test often is perceived only as an assessment of learning and not a learning event,⁴ while strategies such as reading notes, attending lecture, and highlighting important points are perceived as learning events.^11,12 From a cognitive psychology perspective, testing involves retrieving information from memory or retrieval. Testing also can assess higher-order thinking, but retrieval of knowledge is still necessary. It is the act of retrieving information stored in memory (eg, testing/self-testing) rather than the encoding of information in the brain (eg, rereading notes) that is believed to have positive effects on retention.^13,14 Bjork hypothesized that the testing effect could be related to desirable difficulties that enhance learning.^15,16 Techniques that promote rapid initial learning often lead to poor long-term retention, while techniques that involve slower, more effortful learning often enhance long-term retention.

Research by Karpicke and colleagues has examined metacognitive control and learning strategy selection of psychology students with regard to testing.^{8,9,11,13,17-19} In one of their studies, only 11% of psychology students asked to monitor their own learning reported using retrieval techniques when studying.¹⁷ Karpicke and colleagues also found that repeated retrieval (eg, via self-testing) resulted in increased long-term retention as compared with repeatedly studying material.^8,11,13,19 Intermittently placing testing events in learning environments also facilitated learning more that repeated studying.^4,7,12,20

Research regarding student pharmacists' use of learning strategies and the use of testing in pharmacy curricula is scant. Garavalia and colleagues²¹ examined student perceptions of self-regulated learning strategies across first-year (P1) and third-year (P3) students. P1 students relied more on recall ability and external regulation for learning than did P3 students in the study. Decreases in external regulation scores were supportive of development of self-regulation skills. Smith and colleagues^22,23 conducted 2 studies that examined Australian pharmacy students' approaches to learning using a learning styles inventory and found students' preferences for self-regulated learning and knowledge construction did not increase as they progressed through the pharmacy curriculum. The investigators found no previous research that examined student pharmacists' perceptions of self-regulated learning related to testing or the use of retrieval practices in study strategy selection. The objectives of this study were: (1) to solicit student perceptions of testing in the pharmacy curriculum; (2) to investigate pharmacy students' perceived study strategies; (3) to determine student pharmacist perceptions of recall ability; and (4) to assess the extent to which student pharmacists employ retrieval practices in their metacognitive learning strategies.

RESULTS

Out of 435 survey instruments collected, 425 were usable, yielding a response rate of 91.2%. The distribution of students by gender was 35.3% male and 64.6% female. Data regarding highest previous degree earned and professional program grade point average (GPA) also were collected. Eighty-two percent of respondents had no prior degree, while 13.2% had earned a bachelor's degree. Regarding GPA, 56.9% of respondents had a GPA of 3.51 or greater on a 4.0 scale; 33.6% had a GPA of 3.0-3.5; and 8.5% had a GPA of 2.51-3.0.

DISCUSSION

The manner in which tests are perceived has much to do with the purposes to which tests are put. Likewise, the purpose to which tests are put in the pharmacy curriculum has much to do with the attitudes and beliefs of student pharmacists (and pharmacy educators) toward tests. While student pharmacists are traditionally high achievers, pharmacy educators should not assume that student pharmacists employ learning strategies that complement the lifelong learning characteristics the profession desires of its future pharmacists. One of the goals of the study was to determine the manner in which student pharmacists view tests, specifically whether students view tests as learning events. Only 7.3% of students indicated that the primary purpose of tests is to improve their learning. However, in a separate instrument item, 65.3% of students indicated that tests in courses do increase the amount of material learned. A majority of students indicated that tests are used primarily to assess the amount of material they have learned. Indeed, one function of testing is to assess student learning. Roediger and Karpicke⁹ argue that the fact that tests improve knowledge retention is often overlooked by both the test taker and the test maker.

An interesting finding in the study was the desired frequency of testing as perceived by student pharmacists both with and without grades associated with their performance. Whereas over 80% of students agreed that they would rather have more tests that accounted for a lesser percentage of their course grade as opposed to fewer tests that accounted for a larger percentage of the course grade, a minority of students wished to be tested once monthly or more frequently when grades were associated with the tests. Without grades associated with the tests, students did increase the desired frequency of testing. However, only 45.4% of students desired to be tested weekly or more frequently. From a learner perspective and taking into consideration the testing effect, more frequent testing would result in slower forgetting.²⁴ A meta-analysis conducted by Bangert-Downs and colleagues indicated that learning increased as the frequency of classroom testing was increased.²⁵ Ideally, students should want to be tested as often as is feasible to increase learning. However, a majority of students did not indicate this desire. Perhaps considering the current use of tests in the curriculum, a situation in which students are frequently tested, with or without associated grades, is difficult to comprehend.

Interestingly, the number of students who desired testing only once per semester increased sevenfold when grades were not associated with the tests as compared with when grades were associated with tests. This finding could be considered disappointing and is suggestive that these students do not perceive value in being tested, even with no stakes attached to the testing event. One might then question to what extent these students value learning as a whole, or how these students monitor their learning. Frequency of testing can have mediating effects on learning by allowing students to learn from feedback resulting from tests. Outcomes of tests also can guide future study tactics of students, a form of self-regulated learning. Research also suggests that increased testing frequency is associated with increased spaced practice as opposed to massed practice.²⁶ However, time constraints may serve as a barrier to implementing more frequent tests in the pharmacy curriculum.

An additional interesting finding in the study was the minority (38.3%) of students who felt that grades they received on tests in the pharmacy curriculum were a fair assessment of their content knowledge. Perhaps one method of positively impacting student attitudes toward testing events would be to make students aware of the learning that takes place as a result of testing. From a motivational belief perspective, some students who score poorly on a test are likely to attribute their poor performance to the inadequacy of the test itself. Attributions are only perceptions; they may or may not be accurate. However, the small number of students who believed tests were fair assessments of their content knowledge raises the issue of content validity of tests and warrants further research.

Student pharmacists' perceptions of study strategies indicated discrepancies in what students perceive regarding learning and how students actually prepare for examinations. While a majority of students agreed that they remember less if they wait until close to examination time to study as opposed to studying early, nearly two-thirds of students indicated that they cram for examinations. Supporting the evidence of cramming, a majority of students indicated that the main determinant in deciding what to study is related to time, ie, whatever material is due soonest or is overdue. However, nearly 90% of students agreed with the statement that distributing study time over several sessions generally leads to better memory of the information than conducting a single study session. Students seem to be aware that they should study earlier as opposed to later in terms of proximity to examination time, yet they still choose to cram for examinations, or cram out of perceived necessity. While most courses in the pharmacy curriculum at the study institution do not administer high-stakes tests more than 3 times per semester, when taking into consideration that students are enrolled in 4 or 5 courses per semester, the number of high-stakes tests taken could be considered substantial. High-stakes tests and additional assessments could lead students to develop a sense of trying to “keep their heads above water” regarding test preparation. However, student study strategies and/or time management skills also should be considered.

Only 11.8% of students indicated that they study the way they do because a teacher or teachers taught them to study that way. Kornell and Bjork posited that students who do not study in a manner taught them by teachers may simply improvise study strategies, making students more susceptible to the pitfalls of self-regulated study.⁴ P1 students based their study strategies on methods taught by teachers to a greater extent than their P2 and P3 colleagues. Postsurvey comments from P1 students indicated that a particular professor who teaches a required course for P1 students does take time to expose students to effective methods of learning and studying course material. Perhaps the increased ratings on this item were related to a single professor's suggestions. However, P2 and P3 students who had completed the same course with the same professor rated the item significantly lower. This finding could be related to a more holistic view of the pharmacy curriculum as compared with that of P1 students who have less experience in the curriculum.

There are several issues that present when considering the extent to which students read required course readings. If P1 students read significantly more than P2 or P3 students, perhaps it is because, as they progress through the curriculum, students develop a sense of what required reading material is actually necessary to “get by.” Additionally, this finding suggests that material presented in required readings may not be assessed by the instructor(s). If the materials were assessed, more students probably would read the required readings. Or, perhaps students can learn material presented in required readings by simply reading course handouts or other course material. The educator could perceive this finding in a disheartening manner; however, the appropriateness, quantity, quality, and applicability of required readings also must be considered when evaluating the behaviors of students. Again, further research is warranted regarding course readings.

Approximately 68% of students indicated they seldom or never reviewed previously completed course material. Reviewing course material could serve as a spaced practice technique for increasing the amount of material retained. Additionally, non-graded intermittent assessments that cover completed course material could be added to the curriculum to reinforce the importance of the lifelong learning perspective necessary to keep abreast of pharmacy knowledge. Whereas a majority of students do not review completed course material, an assessment and timely feedback would serve to increase learning regardless of performance on the assessment.

Student perceptions of their ability to recall examination material 1 week after the test were similar to results obtained by Karpicke and Roediger.¹¹ Students believed they would be able to recall on average 56.9% of examination material. Regardless of the study strategy employed, Karpicke and Roediger found that students believed they would be able to recall approximately 50% of material on a cued recall test after a 1-week delay. However, students who did not employ self-testing strategies tended to overestimate their ability to recall material and those who used retrieval practice underestimated their ability to recall material. Student pharmacists believed they would be able to recall 37.1% of examination material 5 weeks after the examination. Male students expressed more confidence in their ability to recall examination material. However, confidence is merely a perception and may not be indicative of actual ability to recall material. Male students may overestimate their ability to recall examination material or female students may underestimate their recall ability.

The finding that P3 students believed they would be able to recall significantly more examination material 5 weeks after the examination than P1 and P2 students may be explained by the increased applicability of course material during the P3 year. Students may feel more confident in their ability to recall information that they are able to relate more specifically to pharmacy practice. However, as previously mentioned, P3 students may overestimate their ability to recall examination material as compared with P1 and P2 students. Further research is warranted.

When asked the extent to which students felt confident in their ability to remember information they were currently learning when they become pharmacists, a majority of students did not express confidence. Surprisingly, P3 students, who are closer to graduating than P2 and P1 students, did not indicate the ability to remember significantly more information. The extent to which students actually consider this situation as they learn material in the curriculum is interesting. Students are not expected to remember all information taught in the pharmacy curriculum. Likewise, assessing competence regarding all information taught is not feasible. However, the fact that most students do not express confidence in their ability to remember information learned in the curriculum could indicate that students are employing short-sighted learning strategies as opposed to learning strategies that will serve to increase long-term retention of material.

Two items were included in the instrument that assessed the extent to which student pharmacists would employ testing or restudy techniques after reading a chapter for an upcoming examination. In the item that did not give students the opportunity to restudy material in the chapter, only 11.6% of students indicated they would try to recall material. Therefore, 88.4% of students indicated they would use some technique other than self-testing. Results from a study conducted by Karpicke and colleagues¹⁷ indicated that 18% of students chose to recall material given the same scenario. In a second item, students were presented the same scenario, but were allowed to restudy chapter material after attempting to recall material. We hypothesized that more students would choose to employ recall techniques since they would then be able to restudy chapter material. The percentage of students who indicated they would employ recall techniques mirrored the results of Karpicke and colleagues (40% vs. 42%).¹⁷ Results of these 2 items indicate a lack of awareness related to benefits of self-testing, or attempting to recall material, as it relates to increasing learning.

In support of a lack of awareness of the benefits of self-testing, a majority of students indicated they employed self-testing strategies primarily to see how well they had learned the material. Only 27% of respondents indicated that they self-tested because they learned more through employing this technique as compared with rereading material. These results were similar to research conducted by Kornell and Bjork⁴ in which 18% of students indicated they self-tested because they learned more as opposed to rereading. Despite so few pharmacy students indicating that self-testing techniques are employed as learning events, a majority of students did indicate that self-testing can be used as a metacognitive tool to learn about their learning. Therefore, students do see value in self-testing; however, based on this study, students may not perceive all of the value that exists in self-testing and testing in general.

Considering again the scenario involving 50 brand name/generic name anti-cancer medication name pairs, students were asked to indicate how they would react in given learning situations. Research indicates that when given the opportunity to test oneself on material or restudy material, even if one believes he or she knows the material, dropping material from testing or study leads to decreased retention.¹⁹ When presented the opportunity to drop perceived learned material, 63.4% of student pharmacists chose to drop material from study in an effort to focus on material yet to be learned. This finding could be related to the manner in which students approach material encountered in the pharmacy curriculum. Kornell and Bjork argue that short-term performance and “making study decisions by triage” is a common manner in which students approach learning as opposed to examining learning from a long-term perspective.⁴

In the brand name/generic name recall scenario, after 40 minutes of studying the name pairs, 25.7% of students indicated they would choose to keep restudying the name pairs as opposed to employing testing strategies. Even after 40 minutes of studying the name pairs, many students believed that restudying was the optimal study technique. One wonders how long students would choose to keep restudying the name pairs as opposed to being tested on their ability to recall them if given the opportunity. Questionable in this scenario is the extent to which students self-tested during studying yet did not indicate that they would employ self-testing techniques.

Students were given the opportunity to indicate a preferred pattern of alternating between study and test periods while learning the 50 brand/generic name pairs. Most students did choose patterns that included more than 1 testing period. However, only 46.1% of students indicated they would use the pattern that maximized opportunities to self-test. Interestingly, in a follow-up question, 65% of students indicated that the pattern that would allow them to retrieve the most name pairs from memory after 1 week was the pattern that maximized opportunities to self-test. One must question why a nearly 20% difference exists between the perceived best pattern for learning and the chosen pattern for learning. Perhaps this finding is related to a negative view of testing. Perhaps, too, students perceive the middle approach to be easier than the alternating study/test approach to learning. Or, perhaps students believe the middle approach will allow them to obtain sufficient knowledge to “get by” in terms of performance criteria. Of interest is how student pharmacists would approach this task in an experimental setting.

The current study has several limitations that may reduce the validity of the findings. Students' perceptions of testing and study strategies were assessed using single instrument items. Reliability and validity of the items is not discernable. The manner in which the survey instrument was constructed could also have biased the manner in which students responded to instrument items. Only 1 version of the survey instrument was administered. An inherent limitation of self-report questionnaires is the reliance on respondents to answer items in an honest manner. Students may have been biased to respond in a manner they believed was acceptable to the investigators. Considering the inclusion of student pharmacists at only 1 institution, results cannot be generalized to the student pharmacist population. Additionally, this study attempted to gather holistic student perceptions of the pharmacy curriculum. There are likely differences in student perceptions across courses in the pharmacy curriculum.

CONCLUSION

Student pharmacists' perceptions of testing and the manner in which they engage in learning activities may not be optimal for developing self-directed lifelong learners. Moreover, a majority of student pharmacists do not base their study strategies on methods taught to them by educators. Pharmacy educators must be willing to teach students how to study and to use/administer tests that serve to improve learning and not simply generate grades. Also, the frequency of testing may need to be increased because the process of testing itself, which requires retrieval, may be a better method of learning than rereading or memorizing material. Research on a national level regarding student pharmacists' use of retrieval practices, perceptions of testing, and engagement in lifelong learning is needed.