Abstract
Objective. To determine whether there is a correlation between pharmacy students’ scores on the Health Science Reasoning Test (HSRT) and their grade on a package insert assignment designed to assess critical thinking.
Methods. The HSRT was administered to first-year pharmacy students during a critical-thinking course in the spring semester. In the same semester, a required package insert assignment was completed in a pharmacokinetics course. To determine whether there was a relationship between HSRT scores and grades on the assignment, a Spearman’s rho correlation test was performed.
Results. A very weak but significant positive correlation was found between students’ grades on the assignment and their overall HSRT score (r=0.19, p<0.05), as well as deduction (a scale score of the HSRT; r=0.26, p<0.01).
Conclusion. Based on a very weak but significant correlation to HSRT scores, this study demonstrated the potential of a package insert assignment to be used as one of the components to measure critical-thinking skills in pharmacy students.
INTRODUCTION
A pharmacist’s role as a medication therapy manager requires the use of problem-solving and critical-thinking skills. Pharmacy education is therefore transitioning from knowledge-based to competency-based curricula that emphasize the development of critical-thinking skills. Inherent with this shift in androgogy is the need for teaching methods and assessment tools that effectively promote and evaluate these skills. Standards for the professional program in pharmacy issued in 2016 by the Accreditation Council for Pharmacy Education (ACPE)1 stress the importance of implementing critical-thinking skills in pharmacy school curricula. Faculty members are encouraged to employ teaching methods that stimulate higher-order thinking, problem-solving, and clinical-reasoning skills. According to the American Association of Colleges of Pharmacy (AACP), critical thinking and problem-solving are two essential skills all pharmacy graduates should possess.2
Critical thinking is defined as “the process of purposeful, self-regulatory judgment.3 The critical-thinking process requires interpretation, analysis, evaluation, inference, explanation, and self-regulation.2 Validated instruments used to measure critical-thinking skills include the Watson-Glaser Critical Thinking Appraisal (WGCTA),4 California Critical Thinking Skills Test (CCTST),5 California Critical Thinking Disposition Index (CCTDI),5 and the Health Science Reasoning Test (HSRT).6 The HSRT, a specialized form of the CCTST, is designed specifically for health science professionals and students in health science programs. Results from the HSRT provide an overall critical-thinking score, a percentile ranking, and the HSRT scale-scores of analysis, inference, evaluation induction and deduction.
The usefulness of these commercially available instruments in assessing critical thinking in health science students has been explored in several studies. These studies include candidate selection relating to admissions and/or progression,7-11 curricula assessment,12 and evaluating improvement in critical-thinking skills after a particular intervention or longitudinally.13-16 Two studies to date have used these instruments to validate custom assessments to measure critical thinking.17,18 Adamcik and colleagues showed that student performance on only one component (puzzles) of a custom computer assessment program was significantly correlated to WGCTA scores (r=0.31, p<0.05).17 Similarly, Buur and colleagues demonstrated a weak positive correlation (r=0.22, p<0.05) between a custom assessment and deduction (a scale score of the CCTST).18 Neither of these studies used the HSRT in validating their assessments. In light of these findings, effective assessment measures of critical-thinking skills validated by the HSRT still need to be developed. In this study, HSRT scores were correlated to package insert assignment grades in a pharmacokinetics course. We postulated that a student’s grade on this package insert assignment would show a significant positive correlation with the student’s HSRT score. A positive correlation may support the use of this type of assignment as one of the components used to assess critical-thinking skills longitudinally in a PharmD program.
METHODS
Student categories, HSRT score, and assignment grade were retrospectively collected from 234 students enrolled in the PharmD program at the Gregory School of Pharmacy, Palm Beach Atlantic University. Overall and scale scores on the HSRT and grades on the package insert assignment were collected for each P1 student in the PharmD program. In addition, the following student information was collected: age, gender, ethnicity, and number of college credits at admission. Prior to statistical analysis, all data were de-identified. The HSRT is routinely administered to students in the Applications of Critical Thinking to Patient Care course in the second week of the spring semester in the P1 year. In the same semester, in the pharmacokinetics course, the package insert assignment was administered twice, once in the seventh week and again in the 13th week of the 15-week semester. Students who did not take the pharmacokinetics course for whatever reason were excluded from the study. The use of human subjects was approved by the Palm Beach Atlantic University Institutional Review Board.
The Applications of Critical Thinking to Patient Care course introduces students to critical-thinking skills for use in analyzing patient data to be able to formulate an assessment and treatment regimen. Students are taught a systemic process of clinical decision-making applied to pharmacy practice, data interpretation, and medical documentation. Thirty percent of the course is in lecture format and 70% is process-oriented guided-inquiry learning (POGIL). The POGIL component is conducted via small group problem-solving and discussion. Requirements of the course include the completion of the HSRT, participation in discussions, quizzes, a SOAP note, a pharmacist interview report, a written analysis of HSRT results, and a final examination. Each of these assignments is assessed using an applicable rubric.
In the pharmacokinetics course, students receive required textbook reading assignments along with reading guides to facilitate in-class discussion of the topics covered. Each reading guide on a topic includes several problems for the student to solve on their own or in groups during class. Assessment is both formative and summative. Assessment tools include the package insert assignment, quizzes (mostly formative) using both Turning Point® (Turning Technologies, Youngstown, OH) and cue cards, graphing data in class and examinations (all problem based and graded by the professor). In addition, weekly structured tutorial sessions are offered by upper classmen selected by the professor.
The package insert assignment addresses one of the five learning outcomes in the pharmacokinetics course. This learning outcome determines the students’ ability to interpret the physiological meaning of pertinent pharmacokinetic parameters found in the clinical pharmacology section of a drug package insert. This assignment assesses the students’ ability to apply pharmacokinetic information to patient care. This course is mapped to one of the school’s program outcomes, which states that graduates will be able to “design drug dosing regimens for patients utilizing population or patient-specific pharmacokinetic and pharmacodynamics parameters.” Each student was assigned a drug package insert at the beginning of the semester. Twice during the semester, the students were required to answer a series of questions regarding their package insert. The question format included calculations and short-answer questions (Appendix 1). Students had 48 hours to submit their answers. Assignments were graded using a grading form created in GradeMark® (Turnitin, Oakland, CA).
The median with interquartile ranges (IQR) of assignment grades and HSRT scores in each student category are reported. Differences in assignment grades or HSRT scores within the student categories (gender, ethnicity and college credits) were determined using one-way analysis of variance (ANOVA) at a p<0.05 level of significance (SPSS ver. 17, IBM Analytics, Armonk, NY). The correlation between the package insert assignment grade and HSRT scores (overall as well as scale scores) was determined using a Spearman’s rho correlation at a p<0.01 level of significance. A guide developed by Evans19 was used to classify the magnitude of correlation (r) as follows: 0.00-0.19, very weak; 0.20-0.39, weak; 0.40-0.59, moderate, 0.60-0.79, strong; and 0.80-1.0,very strong.
RESULTS
Two-hundred twenty-nine of 234 students (98%) were identified for inclusion in this retrospective study based on the exclusion criteria. Approximately 56% of the participants in this study were female, age 28±5 (mean±SD) years and 44% were male, age 27±6 years. Approximately 47% of the student population was white, and 53% was African-American, Hispanic, or Asian. Most students had from two to four years of college upon admission into the PharmD program. No significant differences in HSRT scores or package insert assignment grades were noted within these student categories (Table 1).
Demographic Comparisons of Pharmacy Students’ HSRT Scores and Grades on a Package Insert Assignment
The HSRT results can be compared to standard “cut scores” that have been identified for four levels of performance using the 33-point version of the form (Table 2). An overall score of ≥26 is designated as superior, defining a level of performance that is far above others and corresponds to the ability to participate in more advanced learning. A strong score (21-25) reflects the ability for career development and educational achievement. A moderate score (15-20) suggests the student may have some difficulties with problem-solving or decision-making skills. Lastly, not manifested (0-14) implies suboptimal effort when completing the test or a comprehension deficit.6 According to these cut scores, the median HSRT scores of the students in this study were strong (Table 1, 2, 3). The scale scores on the HSRT are similarly categorized into strong, moderate, and not manifested cut scores (Table 2). Median student evaluation scores fell within the strong range, whereas the scale scores for analysis, inference, induction, and deduction were all within the moderate range. The median package insert assignment grade of 91.7 qualified as a grade of A according to the course grading scale. Significant but very weak positive correlations were found between students’ assignment grades and their overall HSRT scores (r=0.19, p<0.01); notably, weak correlations were observed between the assignment grade and deduction (r=0.23, p<0.01, Table 3).
Recommended Performance Assessments for Overall HSRT Scores and HSRT Scale Scores6
Correlation of the HSRT Scorea with Package Insert Assignment Grade (N=229)
DISCUSSION
The findings in this study show a significant positive correlation (r=0.19, p<0.01) of overall HSRT scores (albeit very weak) to assignment grades. In contrast, Buur and colleagues17 reported a correlation between a custom assessment and deduction (r=0.22, p<0.05), but no correlation was found between the custom assessment and the overall CCTST score. Another study18 showed that student performance on only one component (puzzles) of a custom computer assessment program was significantly correlated to WGCTA scores (r=0.31, p<0.05). In contrast to these results, student performance on the package insert assignment was significantly correlated to overall HSRT score and deduction (a scale score of the HSRT). Additionally, this study used HSRT scores, which are a more relevant measure of critical thinking for the health sciences than either the CCTST or WGCTA.
The significant correlation between deduction and the package insert assignment could be due to the nature of the assignment in the first year of the PharmD curriculum, as a package insert contains several known facts about the drug including the population pharmacokinetic information. Deductive reasoning moves from the assumed truth of a set of beliefs to a conclusion that cannot be false if those beliefs are true. Inductive reasoning, on the other hand, draws inferences about what we think is probably true based on analogies, case studies, prior experience, statistical analyses, simulations, hypotheticals, and patterns recognized in familiar objects, events, experiences, and behaviors.6 This assignment required a student to take known facts about the pharmacokinetic properties of a drug and deduce consequences and applications. For example, if a drug is mostly metabolized by CYP3A4 enzymes in the liver, one could deduce that this would predispose this drug to interactions with other drugs that are also metabolized by CYP3A4 enzymes.
Student performance on the HSRT suggests that a majority of the cohort had strong overall critical-thinking skills, strong deductive-reasoning skills, and at least moderate skills in inductive reasoning, analysis, evaluation, and inference. Likewise, a majority of the cohort performed effectively on the package insert assignment. Nevertheless, the range of scores indicates that, although most students demonstrated proficiency in critical thinking, some students appeared to lack basic problem-solving and decision-making skills.
The package insert (prescription drug information) provides health care professionals and patients with clear and concise information about prescriptions to ensure safe and effective use of medications, which translates into optimal health outcomes for patients and more efficient delivery of health care. This package insert assignment allows students to be able to correctly answer a patient-related question by applying their scientific and clinical knowledge. As one of the components used to assess critical-thinking skills, such an assignment could be administered, independent from any course, longitudinally across the first three years in a PharmD curriculum. In the first year, students would be required to apply knowledge from the pharmaceutical sciences courses to answer questions about the drug they have been assigned (Appendix 1). As an example, a question for a P1 student might be, “What is the major route of elimination for this drug?” In the second and third professional years (P2 and P3), questions could focus more on the integration of pharmaceutical and clinical sciences. A more complex question for the second- and third-year students might be, “Would the elimination of this drug be affected in a patient with renal impairment?” (Appendix 2).
Study limitations may include variability in the rubric that was used to grade the package insert assignment, student motivation while taking the HSRT, and student proficiency with English. All package insert assignments were graded by the same evaluator; however, generalized rubrics later became more specific, being able to better distinguish correct from incorrect answers. To prove this point, correlations between the assignment grade and HSRT scores were stratified according to professional year (ie, P2, P3 and P4). Weak significant correlations were observed among P2 students’ scores (overall HSRT score, r=0.26, p<0.05; Deduction, r=0.34, p<0.01). Much lower insignificant correlations coefficients were observed among P3 and P4 students’ scores. Perhaps a stronger correlation would have resulted if the same rubric had been used for all students. Another limitation is the possibility that some students were not motivated to put forth optimal effort when taking the HSRT. Although completing the HSRT was a course requirement, the score obtained did not impact a student’s course grade. This limitation could be mitigated in the future by monitoring the time required by each participant to complete the HSRT as well as the number of questions answered. Lastly, a language deficiency might have impacted the HSRT results of students for whom English is not their primary language.
CONCLUSION
Pharmacy schools must produce graduates who possess strong critical-thinking skills that support sound clinical reasoning and decision-making. The ability to rapidly and reliably assess critical thinking as a longitudinal process throughout a PharmD program would greatly facilitate student development in these areas. Based on a very weak but significant correlation to HSRT scores, this study demonstrated the potential of a package insert assignment to be used as one of the components to measure critical-thinking skills in pharmacy students. Although further study is warranted, these results offer promise that critical-thinking assignments could be designed to serve the dual purpose of facilitating and monitoring the ongoing development of pharmacy students’ critical-thinking abilities.
ACKNOWLEDGMENTS
The authors thank Daniel Brown, PharmD, director of faculty development, Palm Beach Atlantic University, for providing the HSRT data and reviewing the manuscript.
Appendix 1. Package Insert Assignment
Instructions: Answer the following questions using the package insert assigned to you. Upload your completed assignment by the due date and time.
Do NOT use other sources besides the package insert when answering the questions. Failure to use the package insert as your only resource will result in NO CREDIT for that question.
Cut and paste (use the snapshot tool in Adobe) the section of the insert pertinent to the question you are answering. Failure to include the section will result in HALF CREDIT per question.
This assignment should be worked on individually; you may NOT ask the tutors, APPE student, or professor for assistance.
If the insert does not have the information, then state ‘information is not provided’. Some answers may not be readily apparent which may require you to calculate the answer based on equations that we have covered in this course.
Name: _________________________________________
I. Multiple Dosing
1. State a typical multiple-dosage regimen for the drug. [10 points]
2. Calculate the accumulation index at steady state for the regimen above. [30 points]
II. Clearance and Volume of Distribution
3. What is the major route of elimination? Provide a rationale. [10 points]
4. Is the drug restrictively or non-restrictively eliminated? If not enough information is available, provide steps on how you would determine if the drug was restrictively or non-restrictively eliminated. [30 points]
5. Is the drug tissue bound? How do you know? [20 points]
Total Points: /100
Appendix 2. Examples of Longitudinal Critical-Thinking Questions
- Received January 19, 2016.
- Accepted July 30, 2016.
- © 2017 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.