Abstract
Objective. To initiate mapping of the curriculum to the core competencies of our program and to develop a novel and visually accessible method for communicating data to all stakeholders.
Design. An online survey was developed to gauge the extent to which instructors and students perceive that courses within the curriculum and assess students’ ability to achieve each of the 10 core competencies. Response data were collected and processed as radar plots in Microsoft Excel.
Assessment. Response rate for faculty members was 100% and 86%for students. Radar plots provided images of the relative extent to which each course surveyed was perceived to contribute to core competencies.
Conclusion. Using radar plots to present perception data is visually descriptive, makes patterns/trends readily apparent, and facilitates the development of an ongoing culture of assessment.
INTRODUCTION
Curricular mapping is the process of making connections between the curriculum and other components of a program, such as its mission and competencies.1 It is a way of making the curriculum more transparent to stakeholders and may be used to inform the development of a comprehensive assessment plan.2 Many innovative approaches to the process of curricular mapping are reported in the literature, including a graphical technique by Plaza and colleagues.3
As a new program, and in anticipation of the Accreditation Council for Pharmacy Education’s renewed emphasis on assessment (ACPE 2016 Standards) and in the wake of the launch of the Center for the Advancement of Pharmacy Education (CAPE) 2013 Learning Outcomes,4 we recognized an opportunity to identify and incorporate existing best practices of curricular mapping into the development of a comprehensive assessment plan, while putting our own spin on the mapping process.
The doctor of pharmacy (PharmD) program of Western New England University is built on a mission to prepare entry-level practitioners to provide pharmacy care to a diverse population in a variety of settings. The mission is supported by a set of institutionally defined core competencies that represent the knowledge, attitudes, and abilities graduates are expected to demonstrate upon completing the program. There are 10 core competencies, 5 of which are considered general, and 5 that are considered professional. Each competency is defined by learning outcomes (87 in all; see Appendix 1).5 The program is delivered, in part, through the curriculum (Appendix 2), and mapping the curriculum to the competencies through the learning outcomes is a way to assess alignment between the curriculum and the mission (Figure 1).6
Concept map of program design and alignment between the mission and curriculum.
Assessment and mapping are important components of curricular validation. Arguably, all faculty members are stakeholders regarding content and delivery of a curriculum. However, functionally, not all faculty members are engaged in the idea and process. Radar plots provide clear visual maps of where faculty members’ materials lie within the curriculum, which allows for quick gap analyses. Thus, the primary objectives of this project were to initiate the process of mapping the curriculum to the core competencies of our new program and to develop a novel and visually accessible method for communicating the data to stakeholders by using radar plots.7
DESIGN
Inspired by the term “mapping,” which implies a graphical representation of information using spatial relationships to represent relationships within the data, we developed a visual approach to the curriculum mapping process through the use of radar plots.8 Radar plots are 2-dimensional graphs designed to plot one or more series of values over multiple common quantitative variables by providing an axis for each variable, arranged radially as equiangular spokes around a central point. Radar plots are circular rather than linear, and when the plotted variables are connected with a line, an enclosed shape results, making the data more visually accessible than when displayed in a tabular format. While not appropriate for displaying all types of quantitative data, radar plots are well-suited for efficiently displaying a wide variety of data (and patterns within the data) in a single image.9 With regard to curricular mapping, radar plots provide a convenient way to visualize how individual courses contribute to the programmatic “big picture.” For example, consider the radar plots shown in Figure 2 that map 3 hypothetical courses (A, B, and C) to 10 core competencies (I-X). The radial axes correspond to a 4-point Likert scale representing the average level at which the courses are perceived by instructors to assess learning outcomes associated with a given competency as follows: 0=outcome is not assessed; 1=outcome is assessed at an introductory/foundational level as part of a classroom session, assignment, or examination; 2=outcome is assessed at a higher level as part of a classroom session, assignment, or examination; 3=outcome is assessed in an experiential or simulated setting. The resulting enclosed shapes provide a convenient visual representation of how each course maps to the core competencies. One can see at a glance that course B is perceived to target fewer competencies than course A, and that course C appears to target all 10 competencies to the fullest extent. The same data displayed in a tabular format (Table 1) are not nearly as visually descriptive as the corresponding radar plots. Accordingly, as a way of initiating the curricular mapping process and determining where within the curriculum each of the 10 competencies are thought to be assessed, we surveyed the faculty members.
Sample rader plot of hypothetical courses A,B, and C mapped to the 10 Core Competencies.
Mapping Data for Three Hypothetical Courses
EVALUATION AND ASSESSMENT
An online survey instrument was developed to gauge instructors’ perceptions of the extent to which all required courses within the curriculum assess the learners’ ability to perform the 87 learning outcomes associated with each of the program’s 10 core competencies.10 The survey instrument consisted of 87 learning outcome statements and a 4-point Likert scale indicating the extent to which each outcome was thought to be assessed in a given course. The 4-point Likert scale was developed simply to help us get a “lay of the land” at this phase of our assessment plan.
The survey was administered electronically to instructors of record for all required courses in the curriculum through SurveyMonkey (SurveyMonkey, Palo Alto, CA), and it had a 100% response rate (48 surveys administered with 48 responses). Response data were exported to Microsoft Excel, where they were condensed by averaging the results for the learning outcomes associated with each of the 10 competencies. The calculated average scores for each competency then were used to generate radar plots in Microsoft Excel for each course similar to those shown in Figure 3.
Rader plots of two representative courses (PHAR 510 and 612) mapped to the 10 Core Competencies.
The resulting plots were analyzed qualitatively to identify trends, gaps, and overlaps. We administered a similar survey to a sample of learners to ascertain alignment between instructors’ and students’ perceptions. Volunteers were solicited from the classes of 2016 and 2017. The first 10 respondents from each cohort were sent a link to the survey for 3 different courses and given one week to complete it. The data were worked up and analyzed as described above. We chose to survey only a sample of students for a few selected courses simply to gauge alignment. Administering an 87-question survey to all students for all courses would have been not only prohibitively demanding, but also beyond the scope of this project. This work received exempt status from the Western New England University Institutional Review Board.
DISCUSSION
While it is not the objective of this article to present a rigorous analysis of our curriculum, as perception data alone are not sufficient to directly assess achievement of competencies, what follows is a brief discussion of some of the trends that we were able to spot with our radar plots.
Sample radar plots are shown in Figure 3 for 2 representative courses based on instructors’ perceptions of the learning outcomes assessed in those courses. Different courses target different core competencies and, therefore, generate their own shapes on the radar plot. For example, Introduction to Pharmacy is perceived to most significantly assess the core competency Active Citizenship and Leadership, while Principles of Medicinal Chemistry targets core competencies Thinking and Learning and Knowledge Base.
We observed that similar courses, such as those in the physical sciences, target similar competencies and generate similar-looking shapes on the radar plot (Figure 4). The maps in Figure 5 show that core competencies Thinking and Learning and Knowledge Base are perceived to be emphasized in these courses and that the course Genetics and Genomics also is perceived to target the core competencies Social and Cultural Awareness, Ethical and Legal Judgment, and Communication, as might be expected.
Radar plots of physical science courses mapped to the 10 Core Competencies.
Radar plot of Core Competency I (Thinking and Learning) mapped to all courses (shown as course numbers around the perimeter of the plot).
In addition to mapping individual courses to all of the core competencies, we mapped all courses or groups of courses to a single competency. For example, in order to determine which courses within the curriculum are thought to emphasize Thinking and Learning, a radar plot such as the one shown in Figure 5 was generated. In this single image, we could pinpoint which courses were perceived to contribute most significantly to this competency.
Radar plots also are useful for assessing progression from year to year or for sequences of courses meant to build on one another, as we observed for first-year and second-year professional development courses. For these courses, 2 instructors completed the survey instrument separately, with results suggesting a perceived natural progression toward the same set of core competencies (I-V, Figure 6). Moreover, when a course series (eg, course 1 and subsequent course 2) is analyzed, an instructor can visualize the extent to which the continuation enhances learning outcomes in the second offering.
Radar plot showing perceived progression in sequenced courses.
Although we were primarily interested in instructors’ perceptions at this stage of our assessment project, we also surveyed a sample of learners. For most courses for which the students completed the survey, including Practice Management I (Figure 7), we observed reasonably good alignment (defined as gross visual overlap in the radar plot) between instructor and student perceptions with regard to which core competencies were assessed. Generally, students perceived they were being assessed at a higher level relative to instructors’ perceptions (Figure 7). The observed gap between the perception of learners and that of instructors was significant (p<0.05), which we reasoned was likely because of the learners’ limited perspective of the big picture.11 However, the visual cue of overlap is the key message when analyzing the radar plot.
Radar plot showing alignment of learner and instructor perception for the course Practice Management I.
Ultimately, this mapping process provided indirect evidence that all 87 learning outcomes were assessed in at least one course or practice experience within the curriculum. No gaps were detected, and no 2 courses were identically mapped to the learning outcomes and core competencies (ie, no unnecessary duplication was observed). Thus, the first 3 years of the curriculum (mainly didactic courses) provide learners with opportunities to establish a solid foundation in the core competencies, while the fourth-year curriculum (mainly experiential) is perceived to assess the learners at a higher level, especially for core competencies Patient-Centered Pharmaceutical Care, Populations-Based Pharmaceutical Care, Systems Management, and Public Health and Wellness, which more heavily involve the psychomotor domain of learning (“doing”). We will use this knowledge to guide the next phase of our assessment plan: direct assessment of student learning. For this, we also intend to use radar plots to assess alignment between what we perceive the students are learning and what they are actually learning.
CONCLUSION
Using radar plots to present perception data is visually descriptive and shows patterns/trends that may facilitate the development of an ongoing culture of assessment. As a result of using radar plots, we have a sense of where and to what extent the core competencies are being assessed within the curriculum.
Appendix
Core Comps and Learning Outcomes
Appendix
Western New England University College of Pharmacy Curriculum
- Received September 2, 2014.
- Accepted December 8, 2015.
- © 2015 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.