An Environmental Scan on the Status of Critical Thinking and Problem Solving Skills in Colleges/Schools of Pharmacy: Report of the 2009–2010 Academic Affairs Standing Committee

The Changing Learner

The ability to educate students is an increasingly complicated task that is impacted by a number of factors including the changing learner. It has become apparent that today's students communicate and may even learn in ways unlike previous generations.⁶ These students are more comfortable with technology, having used computers, played videogames, and utilized other interactive technologies throughout most of their lives. They are more adept than previous generations at multi-tasking with media and technology; they listen to music, talk or text and use the computer simultaneously.⁷ Some categorize these students as the Net generation, while others refer to them as the Millennial generation.⁸ This group, typically born between 1982 and 1991, is the first generation to be raised with Internet availability.⁸ Prensky refers to these learners as digital natives, as they are “native speakers” of the digital language of computers and the Internet.⁷ Computers, the Internet, online resources, and the desire for instantaneous access are fundamental to these students' lives.⁸ They are constantly connected and in communication with others through the use of email, social network sites, instant messages (IM), and text messages.⁹

Socialization is very important to this new generation of learners. They are especially attracted to activities that include interaction with their peer group.⁶ They like to learn and work in teams, however, their interaction need not be face-to-face. For them, technologies such as email, blogs, and online social networking sites can be just as effective as face-to-face interaction.⁸ Despite the fact that this generation frequently uses technology, they want their learning to include more than just technology. They value their teachers' knowledge and expertise and want to have the opportunity to communicate with faculty when assistance is needed.⁶

This new generation of learners has less tolerance for passive instruction and prefer to learn by doing.⁹ Just as they rarely read newspapers, they also tend not to read lengthy instructions or assignments.⁸ They like learning environments that are rich in multimedia, especially those that are visual and/or audio in nature. Those individuals raised in an environment with ready access to computers have less linear thought processes and struggle with traditional approaches that are substantially more linear. These students prefer random access.⁷

These new learners are achievement oriented.^8,10 They prefer structure and want clear learning outcomes.⁸ They seem to work well under pressure¹¹ and flourish with immediate gratification and frequent rewards.⁷

What we are currently experiencing in higher education with these new learners is just the tip of the iceberg. A 2009 study by the Kaiser Family Foundation surveyed over 2,000 young people between the ages of 8 and 18 years and reported that this population spends on average approximately 7.6 hours each day using media. This represents an increase from 6.5 hours in a similar survey conducted in 2005. On a typical day, an average 8- to 18-year-old spends approximately three hours evenly divided between playing video games and other recreational computer use. Compare this to the average amount of computer time daily (16 minutes) devoted to homework. Comparison of the 2009 survey results with those from the 2005 study shows an increase in overall time that young people spend using more than one media at a time (29% versus 26%).¹² It is interesting to note that heavy media users were more likely to earn fair or poor grades (respondent self-identified categorization: 47% heavy users, 31% moderate users, 23% light users), to be bored, to get into trouble, or to be often sad or unhappy, and are less likely to get along with their parents.¹² Another important change is the increase in access to and use of computers and the Internet. The number of young people with computers in the home increased from 86% in 2005 to 93% in 2009, and access to the Internet in the home increased from 74% to 84% during this five-year period.¹²

The extensive exposure of the millennial generation to video games led to a movement toward incorporation of video games in education and training.¹³ PriceWaterhouseCoopers estimates that the global video game market will grow to $48.9 billion in 2011 and that many corporations use games to recruit and train employees from the digital native workforce.^9,14 Oblinger suggests that games have attributes associated with learning including problem solving and the use of research skills (e.g., testing hypotheses).^8,15

Given the unique characteristics of the changing learner, what are the implications for pharmacy educators? There are at least 4 key suggestions to consider: 1) lectures may not provide an optimal learning environment.^8,9 This generation has different expectations of faculty and the delivery of course content. The teacher needs to become a facilitator of learning and less of a provider of information.⁶ 2) The relevance of assignments must be transparent to the learner. These students detest busy work.¹⁶ 3) Interaction should be an important component of the educational process, as these students value group work and collaborative learning.^6,9 4) The new generation of learner appreciates faculty who respond quickly, provide constructive feedback, and are familiar with and effectively use digital tools in the classroom.¹⁷

The characteristics of the changing learner have curricular implications as it relates to the structure, as well as the methods used in both the pre-professional and professional curricula. The purpose of this Report was to provide an environmental scan on the curricular considerations associated with both critical thinking and problem solving. While this is certainly not a new skill set, it behooves us to re-evaluate the methods and tools that we use going forward for skill development and application given the characteristics of the changing learner.

Critical Thinking and Problem Solving

The AACP Commission to Implement Change in Pharmaceutical Education identified six general outcomes/competencies that serve as the foundation of the education of a professional. The ability to think, including logical thinking, analytical thinking, problem solving and decision making, was identified as an important outcome for pharmacy education.¹⁸ The use of critical thinking, identified by the commission as a competency of the thinking ability outcome, is assumed to result in pharmacy practitioners that are better able to solve problems and think as experts.¹⁹ In describing their rationale, the commission stated:

Entry-level graduates must be able to examine issues rationally, logically and coherently. Although critical thinking is a universally desired educational outcome, professionals particularly need a repertoire of thinking strategies that will enable them to acquire, evaluate and synthesize information and knowledge. Since much of professional practice is problem solving, students need to develop analytical skills to make decisions in both familiar and unfamiliar circumstances. Critical thinking fosters a questioning attitude among professionals; and it is a prerequisite skill in making judgments.¹⁸

Critical thinking and problem solving, as indicated above, are often mentioned together. However, multiple authors have expressed the relationship between critical thinking and problem solving in different ways. Some authors state that critical thinking encompasses problem solving.^20,21 Others suggest that problem solving may be one of the purposes for which we engage in critical thinking.²² Taking this purpose to another level of specificity, some authors suggest that while problem solving is an often performed action, it may not always be performed by thinking in a critical manner.²³ Based on the work of the AACP Commission to Implement Change in Pharmaceutical Education, this final view appears to be most applicable to goals in pharmacy education.

While it is important to consider that critical thinking and problem solving are linked, situations arise where one skill may exist separately from the other. Individuals may engage in critical thinking, but not use the resulting thought to solve a problem. Thus, consideration of these terms separately is warranted in order to proceed with instructional design and assessment. Thus, a clear operational definition of each is required.²⁴

Critical Thinking Debates

The definition of critical thinking has opened the door to multiple debates of the term. The broadest debate among both experts and popular culture involves the separation of critical thinking from good thinking. Critical thinking may be assumed to be good thinking that solves problems, which is the opposite of illogical or irrational thinking that is more likely to cause problems.²³ The majority of experts agree that critical thinking is not the only kind of good thinking. Broad in scope, good thinking may include creative or innovative thinking. Creative or innovative thinking results in new insights, novel approaches, fresh perspectives, and whole new ways of understanding and conceiving concepts or processes. Experts have concluded that what critical thinking means, why it is of value, and ethics of its use are 3 separate concerns.²⁵

Three debates exist with regard to the definition of critical thinking as it relates to teaching strategies. The first debate centers on the view that critical thinking is either a general skill set that spans disciplines or is a specific skill set that varies across disciplines. Most experts suggest that general principles apply across disciplines and support learning critical thinking skills in concert with specific subject matter. The second debate addresses the exclusion of evidence such as experience, emotion, feeling, care and commitment in definitions of critical thinking. It is suggested that relativism may be stimulated by focusing only on valid arguments developed through rational and logical reasoning. Experts argue that dispositions related to critical thinking must balance reason.²⁶ This is an especially important debate for health care professionals because critical thinking is often employed in the care of individual patients. The final debate considers the degree to which social context is a part of critical thinking. Experts make a case that the political and social justice impacts of argumentation and reasoning should be considered in teaching critical thinking.²⁶

Critical Thinking Defined

The American Philosophical Association (APA), through an expert consensus statement, supports the value of critical thinking by stating, “Critical thinking is essential as a tool of inquiry. As such, critical thinking is a liberating force in education and a powerful resource in one's personal and civic life.”²⁵ Such emphasis can lead to ambiguity and the proliferation of a variety of definitions. Multiple articles in the pharmacy education literature have articulated the wide variety of definitions of critical thinking.^19,27–29 Despite the diversity there are clear trends that are identifiable in the definitions.^19,27–29 In Table 1 there are a variety of critical thinking definitions that speak to the different sides of the previously mentioned debates. It should be noted that the most rigorously defined version of critical thinking comes from the APA Expert Consensus Statement for Purposes of Educational Assessment and Instruction.²⁵ The definitions in Table 1 support Halpern's claim that there is enough overlap in the definitions of critical thinking to take the next steps in preparing learners to think critically.²⁴ Because the definitions consist of similar skills, actions, and dispositions, a reasonable next step is to extrapolate and explain these skills and dispositions. It is important in this step to describe examples from pharmacy that help to identify when and where these skills and dispositions can be assessed. Table 2 summarizes the skills associated with critical thinking that have been identified by the APA, associated definitions and selected pharmacy examples. Table 3 summarizes those dispositions mentioned by the APA that encompass approaches to life and living as well as specific issues, questions or problems.

• Download figure
• Open in new tab
• Download powerpoint

Table 1.

Definitions of Critical Thinking

• Download figure
• Open in new tab
• Download powerpoint

Table 2.

Critical Thinking Skills²⁷

• Download figure
• Open in new tab
• Download powerpoint

Table 3.

Critical Thinking Dispositions²⁵

Problem Solving Defined

Problem solving is considered a high-order thinking skill that represents the ability achieved by mastery of each level of Bloom's taxonomy.³⁷ Though problem solving is a high-order thinking skill with a close connection to critical thinking, it has a definition distinct from critical thinking. While also a highly debated topic, most debate related to problem solving revolves around the techniques and methods of problem solving rather than the definition. Problem solving can be broadly defined as the process of designing, evaluating and implementing a strategy to answer an open-ended question or achieve a desired goal.³⁸

The Association of American Colleges and Universities (AAC&U) VALUE (Valid Assessment of Learning in Undergraduate Education) program was an attempt to create and disseminate rubrics to assess learning through the measurement of outcomes.^36,38 The rubrics delineate expected criteria and information to determine individual standards to match growth. The Critical Thinking VALUE Rubric and Problem Solving VALUE Rubric are excellent instruments that can be molded to meet the needs of pharmacy education. Defining the expectation of mastery of each area at benchmark, milestone, and capstone levels allows the evaluator to measure student attainment. The fact that separate rubrics were developed to allow for independent measurement of these skills provides ample support for separate consideration of these terms.

The Critical Thinking VALUE rubric³⁶ provides the expectation that mastery of critical thinking includes explanation of an issue, selection and use of evidence to investigate a point of view or a drawn conclusion, the influence of context and assumptions, the position on the issue taken by the student/learner, and the conclusions drawn by the student/learner. The Problem Solving VALUE rubric³⁸ provides the expectation that mastery of problem solving includes definition of the problem, identification of strategies to solve the problem, solutions or hypotheses proposed, evaluation of potential solutions, solution implementation, and outcome evaluation.

Implementation of these rubrics requires evaluator training and education of the student/learner. The evaluator must understand expectations and terms and be able to use them to “rank” the student/learner's performance. The student/learner should be exposed to the rubric to gain an understanding of the expectations for mastery at each level.

If the academy faculty agree that critical thinking and problem solving skills are fundamental to the successful practice of pharmacy and that our graduates should achieve an appropriate level of competency prior to graduation, then every effort should be made to establish consistent goals and definitions related to this skill set. Though not the focus of this Report, faculty development opportunities should be available to establish and/or supplement the faculties' ability to teach and apply principles of critical thinking and problem solving.

Recommendation 1: Pharmacy education should support the development and use of clear and distinct definitions for critical thinking and problem solving to guide instructional development and assessment of each.

Suggestion 1: Colleges/schools of pharmacy must recognize critical thinking as a general skill set that can be further developed for use in the practice of pharmacy through instruction and assessment within the professional pharmacy curriculum.

Suggestion 2: Colleges/schools of pharmacy must link learning experiences to specific critical thinking skills that can be assessed.

Suggestion 3: Colleges/schools of pharmacy must foster critical thinking skills in all areas of a student's present and future life, including, but not limited to clinical issues specific to pharmacy.

Curricular Considerations

Accreditation Council for Pharmacy Education (ACPE) Standards and Guidelines 2007 require that colleges and schools of pharmacy ensure that graduates possess critical thinking and problem solving skills.³⁹ Although the Standards and Guidelines go on to describe that the development of these skills should be supported through the application of computerized and other instructional technologies, laboratory experiences, and experiential education, ACPE does not provide reasons for this requirement nor do they provide specific examples of the way in which this outcome should be achieved and demonstrated.

As noted elsewhere in this report, we believe that this requirement is in place, in part, because all health care professionals (e.g., pharmacists, physicians, nurses, dentists) need to make numerous decisions that affect patient care on a daily basis. Frequently, the information needed to make a therapeutic decision is incomplete and therefore requires the health care professional to look beyond established data to identify a course of action. The ability to evaluate the data available, identify missing information, think outside established norms, contemplate logical scenarios should different strategies be implemented, and deal with inferences, may be termed critical thinking.

As just a couple of the many components of the practice of pharmacy, patient safety and treatment effectiveness, should certainly be a significant component of the rationale for teaching and developing critical thinking skills in student pharmacists. Pharmacists must instinctively and consistently question prescribers to ensure reconciliation of medications across the continuum of care. Similarly, discharged patients and/or caregivers must be knowledgeable of the therapy plan, including the role of the pharmacist. It is the pharmacist that will evaluate relevant cultural, educational and cognitive traits and their impact on development of a learning plan for a specific patient.

It is certainly a challenge to teach critical thinking²³ or to enhance a student's ability to think critically, nevertheless we do so in a variety of ways across pharmacy and other health care profession curricula. Table 4 provides some examples of the ways that this issue has been addressed by colleges/schools of pharmacy. Even a cursory review of these (and other) references suggests that little has been published on teaching critical thinking throughout the curriculum. It is difficult to know whether this is simply an indication that little has actually been published or a reflection of the fact that few institutions ensure that critical thinking (and perhaps problem solving) is a component of the professional curriculum.

• Download figure
• Open in new tab
• Download powerpoint

Table 4.

Examples of Publications Dealing with Teaching/Evaluating Critical Thinking in the Pharmacy Curriculum

One issue that faces colleges/schools of pharmacy is how and when development of critical thinking and problem solving skills should start from a curricular perspective. Should this be a component of the pre-pharmacy curriculum or limited to the entry-level PharmD curriculum? Should this skill set be addressed in a specific course or two in the professional curriculum or does this skill set need to be disseminated across the curriculum – similar to the way in which experiential education is distributed? Both approaches have been addressed to some extent in the literature, however, it appears to the Committee that the latter approach is optimal to ensure that today's student pharmacist develops the critical thinking skills needed to be a successful professional.

Suggestion 4: Colleges/schools of pharmacy should ensure that the development of critical thinking skills becomes the backbone of the curriculum, and that courses throughout the curriculum are designed specifically to provide students with the opportunity to obtain and use these skills, that the skills are assessed appropriately, and that the attainment of these skills are documented before the student is permitted to graduate. Assessment of this skill should be a formal component of preceptor evaluation of P4 students.

Recommendation 2: AACP should issue a call for school posters to be presented at the AACP Annual Meeting that describe the methods by which critical thinking is taught and/or evaluated in the didactic and experiential curricula at colleges/schools of pharmacy.

Recommendation 3: AACP should issue a call for submissions to Pharmacy Education Assessment Services (PEAS) for assessment methodologies including rubrics that are used by colleges and schools of pharmacy to assess the critical thinking and problem solving skills of their students.

Recommendation 4: AACP should plan an Institute that provides hands-on application of evidence-based active learning strategies for the new generation of learners. (prior to final publication of this report this recommendation was accomplished at the May 2010 AACP Institute in Lansdowne, Virginia.)

When to Teach Critical Thinking and Problem Solving Skills

As mentioned in the previous section, the first question to reflect on is whether critical thinking skills must be taught by content area or if they can be taught as a skill set that can be transferred across multiple disciplines. In other words, can these skills be taught in general pre-pharmacy courses or should this type of development be delayed until students are enrolled in professional program courses that are specifically focused on pharmacy topics? In cognitive psychology, there is support for the idea that an explicit effort to teach critical thinking skills that are transferable across several disciplines can be successful. If the value of this skill set across disciplines is not made obvious, then it is likely that the learner will limit his/her skill development to the course(s) in which it is used.⁴³ This supports the idea of development of critical thinking and problem solving skills in pre-pharmacy curricula, although the variation among pre-pharmacy curricula is notable. The second question to consider is how to influence students to apply critical thinking once they have the skills. The students' disposition toward critical thinking includes their ability to recognize when that skill is needed, as well as their willingness to put forth the effort to use it. It may be easiest to influence students to apply those skills while enrolled in the professional program when the relevance of applying these skills to patient care-based issues is the most obvious. In order to facilitate teaching critical thinking skills, Halpern proposes a four-part instructional model: skill instruction, disposition development, problem recognition, and observation of the process used in critical thinking (metacognitive monitoring).⁴³

Assessment of Critical Thinking and Problem Solving Skills of Incoming Students

If the development of critical thinking and problem solving skills occurred as an element of the pre-pharmacy curriculum, then assessment of these skills as a component of an admissions application may assist colleges/schools to predict candidate success in the professional program. Over the past several decades, studies were conducted to determine whether pre-pharmacy or admissions variables could predict which students would succeed in a professional pharmacy program. The focus of most of these studies was on “success” in pharmacy programs, with grade point average (GPA) used as the measurement of success.^44–54 It is of note that critical thinking and problem solving skills were not specifically addressed in these studies or considered as a specific measure of success.

Many studies found positive associations between success in a pharmacy program and prior completion of a four-year degree, but none looked closely at the characteristics of applicants with four-year degrees to determine the extent to which factors such as age, motivation, or intellectual ability contributed to the students' completion of the degree and subsequent performance in a professional program.

One study by Cline et al. did shift the focus to success in practice (and not academic program) and used 4 predictors of success not utilized in other studies: career commitment, innovativeness, materialism, and other-directedness.⁵⁵ The innovativeness dimension of practice success included measures of willingness to adopt new ideas and included how a person would be inclined to bring new ideas to problem solving efforts.⁵⁵

Other studies included an explicit measure of critical thinking skills. Allen and Bond found a link between performance on the California Critical Thinking Skills Test (CCTST) and subsequent performance in practice-based courses.⁵⁶ In this study, the Pharmacy College Admission Test (PCAT) was also a good predictor of performance.⁵⁶ Kidd and Latif later found the CCTST and not the PCAT to be a predictor of performance in advanced pharmacy practice experiences (APPEs). A second measure of critical thinking, the California Critical Thinking Disposition Inventory (CCTDI), was a predictor of success in earlier coursework.⁵⁷ Lobb and colleagues looked at the results from the Learning and Study Strategies Inventory (LASSI), Defining Issues Test (DIT), and Watson-Glaser Critical Thinking Appraisal (WGCTA) as predictors, however, found no correlation with subsequent academic performance.⁵⁸ In all of these studies, the subsequent academic success was measured by grades and not critical thinking per se.

Not directly studied was the impact of age on critical thinking and problem solving skills. Non-Piagetian theorists suggest that there is a fifth stage of cognitive development that occurs in young adults; this fifth stage is called post formal thinking and it focuses on problem finding. At this stage of development, the young adult begins to identify the problem before him/her and determines ways to solve it even if faced with contradictory information and awareness of the relativistic nature of their knowledge.⁵⁹ Given the age of a typical student pharmacist, it is certainly possible that this last stage of cognitive development may still be “percolating” and that assessment of this skill set may reflect, at least in part, the extent to which this stage has matured.

Teaching critical thinking and problem solving skills in the professional curriculum

Once enrolled in the professional program, students participate in both didactic and experiential learning experiences. There are specific instructional methods considered effective in the development of critical thinking and problem solving skills. These methods tend to be learner-focused, inquiry-based learning approaches that employ reflective methods, writing to learn, and paired or group processes. Regardless of whether the method focuses on discipline-specific or general critical thinking and problem solving skills, it is important to employ good instructional design. Two recommended resources are Chickering's “How People Learn” and Gamson's “Seven Principles of Good Practice,” both of which were listed in the Curriculum Summit white paper by Jungnickel et al.³

A key to success in the development of this skill set is to incorporate those active learning instructional methods into both the classroom and experiential settings that are known to develop critical thinking including, inquiry-based learning, working with others, and writing to promote reflective thinking.³ As expected, an effort should be made to explicitly teach and then assess critical thinking and problem solving skills. Several instructional approaches used to promote the development of critical thinking and problem solving skills are described in the following sections.

Inquiry-Based Learning

Inquiry-based learning revolves around the learner and what he/she already knows and needs to find out in order to complete a task such as solving a problem, case review, answering a complex query, or preparation of a care plan. The use of questions by the instructor as well as the student is a key part of this method. Students should learn through a discovery process where they ask the questions and seek the answers. Instead of providing answers, the teacher serves as a guide or facilitator for the process. Through the question and answer process, students should develop new knowledge while applying critical thinking and problem solving skills.³

One form of inquiry-based learning that is used in pharmacy programs is problem-based learning (PBL). This approach to inquiry-based learning focuses the learner on a problem that needs to be resolved. It involves presentation of a problem followed by group discussion to identify what is known, what is unknown, as well as to formulate hypotheses. Group members then conduct research on their portion of the investigation and bring findings back to the group to review and reach a solution. For example, students may be given the task of designing a pharmacy service or a new pharmaceutical agent.⁶⁰

Another inquiry-based approach frequently seen in pharmacy programs is case-based learning. This differs from problem-based learning in that it focuses on building knowledge and resolution of problems that may not have a clear-cut answer. Ethical dilemmas and patient-specific cases are often used in a case-based approach. Many of the methods used in problem-based learning such as group discussion and independent research are also viable in case-based learning. Examples of inquiry-based learning methods used in pharmacy programs are listed in Table 5.

Team or Group Learning

Another type of instructional method designed to promote critical thinking and problem solving skills is team or group learning; this is often paired with an inquiry-based learning experience. This approach includes discussion and communication among group members, a valuable tool in the development of critical thinking skills. Because group or team learning assumes all students in the group have team skills, it is important to ensure that they do. As a result, in addition to explicit efforts to develop critical thinking and problem solving skills, the instructor should also promote the development of communication and team or group skills.^42,62

Writing to Learn and Reflect

Writing is often overlooked as a tool for learning. While the development of formal writing skills is essential, the development of informal writing skills that can be used to facilitate concept mastery and reflection is also important. In the development of critical thinking skills, informal writing exercises can help the student identify and organize issues and ideas, uncover missing information, and find misconceptions or faulty reasoning. Such writing is not graded for its literary value, but rather for its demonstration of thoughtfulness and idea development. Writing to learn may be accomplished in a variety of formats including journals, one-minute papers, as well as wikis and blogs.^29,61

Technology-Based Tools

As mentioned earlier, technology alone does not ensure a good learning experience. As with other methods, it must be used in conjunction with good instructional design. That said, technology is thought to promote critical thinking and problem solving skill development in a support role. Specifically, technology facilitates the ability to provide immediate feedback; it promotes communication of ideas, serves as an aid in reflection, and enhances access to information needed for a particular task. In addition, for younger learners, it may be a more familiar, thus less daunting, means for communication and documentation of their thoughts and reactions. Learning management systems now provide methods to document thoughts via discussion or chat rooms, blogs, and tweets, as well as Web-based conference capability for visual and audio connections, and support for group work. Numerous software programs are available that offer interactive methods to learn specific topics such as anatomy. Virtual patient care settings that employ avatars have been developed to permit more realistic simulations of patient care. When used in conjunction with a well-designed learning experience, technology can enrich the experience. Selected examples of pharmacy programs that incorporate technology into coursework that is designed to develop critical thinking and problem solving skills can be found in Table 5.^63,67

Experiential Learning Component

Many articles in pharmacy education focused on critical thinking and problem solving skills concentrate on didactic courses. It should be noted that many of the classroom and lab methods for the development of critical thinking and problem solving skills can be applied in the experiential setting as well. Patient care is certainly the ultimate example of inquiry-based learning. The use of informal writing and activities that include reflection may not be as frequently employed in this component of the professional curriculum. This type of educational setting is a logical place to insert technology, such as in the use of an electronic portfolio, as a means to enhance development of critical thinking and problem solving skills.

Need for Research on Instructional Effectiveness

Part of the process of curricular incorporation of those instructional methods designed to promote the development of the critical thinking and problem solving is the assessment of their effectiveness. Such assessments require attention to study design, analysis methods, and measurement of student learning outcomes. Cisneros and colleagues outlined specific gaps in our knowledge about the effectiveness of various instructional methods for the development of problem solving skills and called for additional research in these areas.⁷³ Their suggestions, which focused primarily on study design, included conducting more longitudinal studies of inquiry-based instructional methods, making comparisons to control groups, comparing across different schools, exploring the role of group learning, and using both quantitative and qualitative research methods. Because problem solving and critical thinking are complex behaviors, consideration of many factors in these studies will be needed if a complete understanding is to emerge. The authors also called for improved methods for measurement of student learning outcomes when inquiry-based instructional methods are employed despite the fact that several measurement instruments currently exist. These instruments are described in detail in the next section.

Methods to Assess Critical Thinking Skills

As mentioned in an earlier section, one mechanism by which academic pharmacy measures critical thinking skills in their applicants is through the use of formal instruments, including the Watson-Glaser Critical Thinking Appraisal (WGCTA, Pearson), California Critical Thinking Skills Test (CCTST, California Academic Press), California Critical Thinking Dispositions Inventory (CCTDI, California Academic Press), and Health Sciences Reasoning Test (HSRT, California Academic Press). These instruments have been studied for application in admissions,^74–79 assessment of student pharmacists,^{19,27,28,57,80} as well as in program assessment.⁸¹ Other specialized assessment instruments have been developed and used to evaluate student pharmacists' performance in specific tasks. These instruments include a computer assessment program to evaluate problem solving and critical thinking skills in clinical decision-making,⁸⁰ an instrument based on Rasch modeling to evaluate critical thinking skills in scientific literature evaluation,⁸² as well as an instrument designed to determine if self-assessment exercises improve critical thinking skills.³¹

The WGCTA, CCTST, and HSRT are multiple-choice question exams. The WGCTA assesses the respondent's ability to make accurate inferences, recognize assumptions, properly deduce, interpret information and evaluate arguments.⁸³ The CCTST provides a total score and reports five subscales: analysis, evaluation, inference, deductive reasoning, and inductive reasoning.⁸⁴ The CCTDI evaluates personal disposition to utilize critical thinking. The instrument is a list of reflective statements with the application of a Likert scale for statement evaluation by the respondent. This instrument reports seven subscales: truth seeking, open-mindedness, analyticity, systematicity, self-confidence, inquisitiveness, and cognitive maturity.⁸⁵ The HSRT is designed to assess critical thinking skills of health science students and professionals.⁸⁶ Test items frame professional and clinical practice without the requirement of prior content knowledge of the topic areas. The HSRT reports the same subscales (Health Sciences Reasoning Test Scale Descriptions)⁸⁷ as the CCTST.

• Download figure
• Open in new tab
• Download powerpoint

Table 5.

A Sample of Instructional Methods for Teaching Critical Thinking and Problem Solving Skills in Pharmacy

The CCTST, utilized as part of the admissions process, was shown to be an effective predictor of student success.⁷⁵ Correlation was highest for student success in management, behavioral science, pharmacy practice, and experiential coursework. Total CCTST scores and subscores tested during the admissions process were found to be weakly but significantly predictive of P1 GPA.⁷⁴ The HSRT was studied in P1 student pharmacists with results compared to admission data.^76–79 Basak et al. employed both HSRT and WCGTA in two cohorts of students. While there was significant correlation between scores of the two exams, neither exam was found to be predictive of P1 GPA.⁷⁶ Smith et al. found that HSRT results were significantly correlated with Pharmacy College Admission Test data.⁷⁸

Use of the WGCTA in P4 student pharmacists showed strong correlation with grade point average and performance in therapeutics coursework, but not with clinical problem solving skills.⁸⁰ Miller employed both the CCTST and CCTDI in assessment of the growth of critical thinking skills in student pharmacists through their academic career.²⁸ There was a significant increase in performance on the CCTST that demonstrates an increase in critical thinking ability, while there was no change in CCTDI performance that indicates no change in student disposition toward critical thinking. Further study of student pharmacists in a drug literature evaluation course showed correlation between their CCTST score and course grade and final examination grade.¹⁹ Kidd and Latif found a strong correlation between CCTST score and P4 experiential course performance while CCTDI score correlated to classroom course performance.⁵⁷ Phillips et al. used both CCTST and CCTDI given to student pharmacists periodically during their academic careers.⁸⁴ The results compared well to national norms. A significant increase was seen for CCTST scores and performance on the CCTDI was shown to improve over time. The authors concluded that specific areas of performance strength and weakness could be used as a guide to curricular change in program assessment. A study of CCTST and CCTDI given to P1, P2, P3, and P4 student pharmacists showed no incremental change in one year.²⁷ However, significant changes were seen in the subscales inference (CCTST, decrease), open-mindedness (CCTDI, decrease), and self-confidence (CCTDI, increase).