Abstract
Objective. To identify and describe validated assessment tools that measure cultural competence and are relevant to pharmacy education.
Methods. A systematic approach was used to identify quantitative cultural competence assessment tools relevant to pharmacy education. A systematic search of the literature was conducted using the OVID and EBSCO databases and a manual search of journals deemed likely to include tools relevant to pharmacy education. To be eligible for the review, the tools had to be developed using a study sample from the United States, have at least one peer-reviewed validated publication, be applicable to the pharmacy profession, and be published since 2010.
Results. The systematic literature and manual search identified 27 tools. Twelve assessment tools met the criteria to be included in the summary and their relevancy to pharmacy education is discussed.
Conclusion. A review of literature demonstrates that assessment tools vary widely and there is no one tool that can effectively assess all aspects of cultural competence in pharmacy students or the Doctor of Pharmacy curriculum. As cultural competence is a priority within the accreditation standards for pharmacy education, PharmD programs are encouraged to develop additional tools that measure observed performance.
INTRODUCTION
Cultural competence is an important program outcome in pharmacy education because pharmacists who understand how cultural beliefs influence patients’ health, illness, medication taking behaviors, and treatment decisions, and consider these beliefs when providing care, are thought to practice in a culturally competent manner.1 Culture includes race, ethnicity, socioeconomic status, religion, age, gender, sexual orientation, disability, and health beliefs.1 Cultural competency is an ongoing process that does not have a fixed endpoint. Cultural competence refers to an ability to understand and effectively communicate with people across cultures and it includes: awareness of one’s own cultural beliefs, attitude towards different cultures, knowledge of different cultural practices, and cross-cultural skills.1 ⇓⇓-4 In health care, cultural competence includes a set of congruent behaviors, knowledge, and attitudes among professionals that allows for the provision of effective care to meet patients’ culturally unique needs.1 Relatedly, health disparities are preventable differences that are experienced by socially disadvantaged populations and create barriers to achieving optimal health outcomes.5 Negative health outcomes may result when sociocultural differences between patients and providers are incompatible.6 Research demonstrates that there is a clear link between cultural competent care and eliminating racial/ethnic disparities in health care.6 Sometimes the terms cultural sensitivity or cultural humility is used in lieu of cultural competency in pharmacy education as it is impossible for anyone to be competent in all cultures and it is easier to capture the level of knowledge, skills, and attitudes that students possess upon graduatiton.7 ⇓-9 However, cultural competency adds in a level of operability by encouraging providers to reflect on their beliefs, engage patients, and tailor health care delivery to individual needs.1 ⇓⇓-4 For consistency, cultural competency is the most common term and it will be used in this review.
There is growing emphasis on teaching and assessing cultural competency in pharmacy education and integrating it across the entire curriculum (didactic, experiential, and co-curriculum) as documented in the release of the 2013 Center for the Advancement of Pharmacy Education (CAPE) Outcomes that lists it as one of the 15 outcomes expected of pharmacy students upon graduation.7 The 2016 Accreditation Council for Pharmacy Education (ACPE) Standards require pharmacy programs to teach and assess cultural competency and cultural sensitivity in both the didactic and experimental curricula so that students have the ability to practice in culturally diverse environments.9 Pharmacy students need specific training in their curriculum so they can better understand their own health care beliefs and those of their patients, recognize when health disparities exist, and deliver patient care in a culturally sensitive manner so they can provide the best care for their patients.1
While pharmacy programs are including more cultural competency content into their curriculum, the assessment of these outcomes lags behind and no best practices exist.1,10 Multiple articles urge integration of these topics in pharmacy, but there is uncertainty because there has not been a comprehensive national assessment of how this content should be incorporated.11 Integration of cultural competency within the curriculum is a concern among other health professions programs, such as nursing and medicine, where there is also a lack of consensus on how this content should be structured, organized, and facilitated.12 A 2013 white paper by O’Connel and colleagues outlined individual and programmatic assessment tools that were used in health care education (ie, colleges and schools of pharmacy, medicine, occupational therapy, physical therapy, and nursing) as well as in health care practice areas such as hospitals and general counseling settings; however, this work is more than eight years old.2 Therefore, the goal of this review was to update the existing literature by identifying and evaluating the available validated tools for measuring cultural competency in pharmacy students.
METHODS
Systematic literature review was used to identify cultural competency assessment tools with published psychometric information published since 2010 that would be relevant for use in pharmacy education. As a first step in the systematic review, to identify studies, a full text publication search was conducted using the OVID and EBSCO databases. The relevant terms used in this search were: cultural competence, multiculturalism, cross-cultural, intercultural, cultural sensitivity, cultural empathy, health disparities, and diversity education. Next, an electronic search of the website of pharmacy education journals (ie, American Journal of Pharmaceutical Education, Currents in Pharmacy Teaching and Learning, Pharmacy Education, Journal of the American College of Clinical Pharmacy, American Journal of Health-System Pharmacy) was conducted using the same relevant terms.
In order to select studies for the next step of the systematic review, the inclusion and exclusion criteria of article selection were discussed and determined by the authors prior to the initial literature search. Articles were collected if they: were empirical and peer-reviewed; developed, validated, or applied measures of cultural competence using a study sample from the United States; focused on health care professions (eg, pharmacy, nursing, medicine, dental, physical therapy, etc.); and were recently published (2010 and after). Two rounds of literature searches were conducted. In the first search, the abstracts of the articles retrieved from the search were reviewed to determine if the authors had developed and/or applied a cultural competency assessment tool, and only articles meeting this criterion were retained. Next, all the databases were searched again to ensure that no relevant articles were missed. Some studies had used a cultural competency assessment tool and the article cited relevant psychometric articles or research articles that used the tool; these articles were retrieved in order to gather all relevant information on the assessment tool and its use. That is, if an assessment tool was applied in a research study conducted since 2010, the previous studies that developed and validated this assessment tool were also collected during the second round of searches. These two steps were conducted by one researcher, and if inclusion of an article was questionable at these steps, a second researcher was asked to conduct a second review. The two researchers then engaged in discussion about the article until consensus was reached.
For the third step of the systematic review, evaluating the identified articles, all of the researchers reviewed the publications related to the remaining tools and conducted a critical appraisal, which included a discussion of the strengths and weaknesses of the measures and their applicability to pharmacy education. The following criteria were used to assess each tool: whether the tool measured cultural competency, whether the tool’s development and reliability had been evaluated in at least one peer-reviewed article, whether the tool questions were available for review, and whether the tool was applicable to pharmacists and/or pharmacy students (ie, the tool language or questions did not preclude the instrument’s use with pharmacists/student pharmacists, the tool did not present a practice situation irrelevant or unfamiliar to pharmacists, and/or the tool did not require significant question revision to be applicable to pharmacy).
A table with a list of the tools that met all criteria was created. Descriptive information entered in the table included the name of the tool, overview, description of the tool, and relevant validation and research studies conducted with the tool.
RESULTS
The initial literature search for articles describing a developed tool or evaluation of a relevant tool identified 72 papers covering 36 tools. Some of the 72 articles described revisions of the same tool; in such cases, only articles describing the most recent version of the tool were retained. Nine articles did not provide a name for the measure, did not contain information about the measure, and/or had not been peer reviewed, and therefore were removed from review. This reduced the number of articles to 63 and the number of tools to 27. Of the 27 tools described in the remaining articles, six were excluded because the study in which they were tested was not conducted in the United States, leaving 21 tools for critical appraisal. Of the remaining 21 tools, two were excluded because they did not measure cultural competency (one measured empathy and the other hospital systems); three were excluded because there was insufficient information on their development and reliability (ie, the tool was developed by a researcher for a specific study); and four were excluded because they were not applicable to pharmacists and/or pharmacy students. In the end, 30 articles examining 12 different assessment tools met the inclusion criteria. The literature search and selection schematic for this review is presented in Figure 1. The key information for the 12 tools included in this study is presented in Appendix 1 and Appendix 2.
Search and selection process used in conducting a literature review to identify tools to assess cultural competence in pharmacy students and the Doctor of Pharmacy curriculum.
The authors identified three main categories of tools based on their evaluation of the article, purpose of the measure, the background of the students the tool was used to assess (eg, first year or APPE students), and the tool questions. This resulted in three tool categories. The first category covered tools designed to be used for students with health care practice experience. There were seven tools grouped into this area. These tools generally contained at least some questions that asked about knowledge or skills used while providing patient care. For example, the Clinical Cultural Competency Questionnaire (CCCQ) assesses the skills domain with item wording like “providing culturally competent patient education and counseling,” to which incoming pharmacy students with no practice experience would not be able to respond.13 However, more advanced students would be able to assess this as they have clinical experience.
The second category covered tools designed for broad student use. There were four tools grouped into this area. These tools are not specific to pharmacy or health care, yet many have been used to assess health care or pharmacy students, particularly those students at an early point in their respective programs (ie, first year pharmacy students). An example of this is the Everyday Multicultural Competencies/Revised SEE (EMC/RSEE) instrument, which was developed from four existing measures (see Appendix 1).14 It assesses cultural competency areas and items including cultural openness and desire to learn (example: “A truly good education requires knowing how to communicate with someone from another culture”) and awareness of contemporary racism and privilege (example: “I am aware of how society differentially treats racial and ethnic groups other than my own”).
The final category in Appendix 2 covers tools designed for curriculum assessment and included only one measure, the Revised Tool for Assessing Cultural Competence Training (TACCT-R).15 This measure was initially designed to help medical colleges integrate cultural competency content in their curriculum and used a panel of experts to identify relevant areas for training. The instrument covers topics, including knowledge, skills, and attitudes impacting cultural competency training on six domains. The instrument has been adapted for use in nursing and dental education, and the domains were recently used as a framework in a pharmacy education research study.16 ⇓-18 This type of instrument can be useful to identify under-addressed curriculum areas, measure faculty and student perception of the curriculum, and to monitor new curriculum innovations.
DISCUSSION
This review identified 12 cultural competency tools that met the inclusion criteria for the study. These 12 tools were categorized into three main areas of pharmacy education: seven tools for evaluating students completing health care practice experiences, four tools for broad use with students, and one tool for evaluating the curriculum. The number of questions on the tools ranged from 15 to 75, and most used subscales or domains within the tools (range of two to six subscales or domains). Most of the tools were free, and the scales and items were available for review by accessing the published article that reported on the tool. However, a fee was charged for use of one tool, the IAPCC-R.19 Some tools included pharmacy students as part of the study population to determine validity and reliability of the instrument.19 Some of the other tools had been used in research with pharmacy students to establish the instrument’s applicability.18,20 ⇓⇓⇓⇓⇓⇓⇓⇓⇓-30 Additionally, some tools had been used to assess HDCC in health care professionals or health care professional students.15,19,27,31 ⇓⇓⇓⇓⇓-37 Only one tool was identified that was intended to assess cultural competence content within the curriculum rather than cultural competence in students.15,18,30 However, this study was included as it met the criteria for study inclusion. Also, this tool may become more relevant if health care programs are required to fully delineate cultural competence outcomes and map those outcomes in their overall assessment plan for their didactic and experiential curriculum and co-curriculum.38 Others have stressed that it is important for programs to develop an integrated program of assessment and adhere to best practices for assessment in general, and this is also true regarding cultural competency.39,40 The cultural competency integrated program of assessment should consider the who, what, where, when, why as answers to these questions will help in selecting an appropriate tool.
The first consideration is who are the students being assessed (ie, first professional year students vs students with practice experience). The measures reviewed in the current study are grouped according to who would be able to respond to most questions (ie, students with health care experience vs tools for broad student use). Addressing who is completing the assessment will assist pharmacy schools in finding the most relevant and appropriate tool(s). Next, what is being measured. Most of the tools identified in this review are readily available and items can be reviewed within the reference’s publications. Programs should evaluate the tool items and subscales to ensure that they align with their intended cultural competency learning objectives and/or outcomes. The decision of where in the program to assess students is important and may be driven by where in the student experience self-awareness or growth in cultural competence may be triggered. Based on this information, programs must then decide when to assess students so they can determine whether students’ cultural competency perceptions improve over time. Programs should consider assessing students at baseline, or early in the curriculum, so they can track changes over time. Decisions will need to be made regarding the exact times to conduct follow-up. For example, programs may want to reassess students after they have been exposed to or completed specific experiences so the effectiveness of a specific event(s) (ie, instruction on topic, experience) can be evaluated. Alternatively, a program may want to examine change or growth across multiple years (ie, cultural competency instruction intentionally layered within several courses) and therefore should administer a baseline assessment with annual follow-up assessments.
There are three limitations to our study. First, the tools identified in this review measured students’ cultural competency, but tools for use in assessing other areas related to cultural competency, such as empathy and cultural humility, were not identified for review. This was to keep the focus on a specific area of tools that are most relevant to programs developing a cultural competency assessment plan. Second, the review process may not have located all cultural competence tools relevant to pharmacy education. We conducted an extensive search and review; however, we may have failed to identify tools that may have been relevant, particularly if the publication(s) were outside of pharmacy. Similarly, we attempted to identify relevant research publications that used the tools as this provides additional information on how the tools have been used; however, we may have failed to identify some publications in our search. A strength of our study is that the authors strictly adhered to the inclusion criteria of selecting articles and tools and vetted all of the tools identified in the review as a team, including engaging in robust discussions until consensus was reached. Future research should repeat this review process as the field will continue to expand as existing tools are modified and new tools are developed to improve instruction and assessment of cultural competency.
There are five additional considerations when selecting tools to include in a cultural competency assessment plan. First, all of the tools identified in our review were quantitative, self-report measures designed to assess the perceptions of the person completing the tool vs their authentic behaviors and actions with people of different cultures. Adding assessments of observed performance is important as perceptions of competence do not guarantee that people will use their cultural competency skills appropriately. Therefore, using a mixture of both quantitative methods and observation, reflective writing and/or qualitative tools may provide a more complete picture of the cultural competency of those being assessed.38 For example, tools such as culture clinical case vignettes or objective structured clinical examinations (OSCEs) could be used alone or with quantitative tools to assess patient-centered culturally sensitive health care knowledge and skills.41 Those pursuing this mixture of assessments should consult the literature for recommendations.6,42 Future tool development and research may include developing tools that evaluate culturally congruent clinical practice and/or include assessments of how students and/or practitioners actually act, not just of their self-assessed knowledge, skills, and behaviors.43
Second, when selecting tools to include in a cultural competency assessment plan, programs should examine the tool’s subscales and domains as well as individual items for relevancy. For example, the CCCQ, CCHPA-67, and HPCCI tools contained some questions that were focused on physicians and not applicable to pharmacy practice.31,33 These questions would need to be modified if the tool was selected for use with pharmacy students, which was done with the CCCQ.13 In addition, the number of questions in the selected tool(s) should be assessed for the possibility of survey fatigue as some of the tools found in this review contained up to 75 items. Also, some of the instrument’s subscales/domains may be relevant to the assessment a particular program wants to conduct, while other subscales may be less relevant. None of the studies mentioned whether selected subscales or items from a tool could be used rather than the entire tool without compromising the tool’s integrity. Yet gleaning the most applicable content may be one way to reduce the number of items while accomplishing a robust assessment of relevant areas, particularly if different assessments are combined (eg, a broad use tool to assess a student’s perspectives combined with a tool for students with health care practice experience to assess a student’s practice behavior). Another approach would be to select a tool for broad student use to administer to students as they enter the program or early in the program, then when students enter their IPPE or APPE years, supplement or replace the first assessment with a tool specifically for use with students with health care experience (Appendix 2). It is important for programs to remember that if they modify a tool to fit their needs by adding or deleting questions, doing so may change the psychometric properties of the tool.
Next, most of the studies identified in the review assessed participants at one timepoint, while only a few assessed students pre/post (eg, CCCQ, CCHPA, EMC/RSEE, IAPCC, SEE).13,18,20,24 ⇓-26,32,34,36 None of the tools reported on in this review had been used longitudinally, beyond pre/post administration, to assess participants’ growth. Although measures may be effective at assessing cultural competency at one point in time, evidence of whether the tools included in this review are sensitive enough to detect change or growth in cultural competency over time requires additional study. Programs should consider this when selecting a tool.
Fourth, when determining whether a tool should be included in a cultural competency assessment plan, attention should be placed on the number, recency, and generalizability of research publications pertaining to the tool. For example, the EMC/RSEE, the ISS, and the MEQ may be appealing tools to use to assess pharmacy students early in the curriculum, and the CCA later with students when they gain health care experience, but only one study had been published for each of these tools at the time the literature review was conducted.29,34,35,37 Reviewing the published literature can also provide insights on how a programs is modifying a tool(s).44 The IAPCC is a tool that has been used frequently in the literature and a student version has been created, but the fee for use may prohibit some programs from adopting it if financial resources are not available.18,24 ⇓-26
Finally, while the quantitative tools identified in this review assessed the student or practitioner’s cultural competence or sensitivity, none assessed how they respond to receiving culturally inappropriate messages or behaviors. Students also were not allowed to explain their answer choices or the context behind them. Future tools and studies should include assessment of health care providers and students responses to cultural incompetence, prejudice, and bias.45 ⇓-47
CONCLUSION
Standards 2016 emphasize the importance of cultural competency instruction and assessment. This study used a systematic process to identify 12 cultural competency quantitative assessment tools relevant to pharmacy education. The tools were grouped by categories according to how they could be used, ie, tools for students with health care practice experience, tools for broad student use, and tools for curriculum assessment. There are a number of considerations when choosing a cultural competency assessment tool and developing a cultural competency assessment plan, including the use of multiple tools at different points. The 12 tools in this review all measure self-assessment of cultural competency. Health care professions and education in general and pharmacy educators in particular would benefit from the development of cultural competency assessment tools that measure observed performance.
Appendix
Instruments Identified in a Systematic Review of Assessment Tools Measuring Cultural Competence Outcomes Relevant to Pharmacy Education in Students
Instruments Identified in a Systematic Review of Assessment Tools Measuring Cultural Competence Outcomes
- Received April 1, 2021.
- Accepted January 2, 2022.
- © 2022 American Association of Colleges of Pharmacy