Assessment of Streams of Knowledge, Skill, and Attitude Development Across the Doctor of Pharmacy Curriculum

INTRODUCTION

Doctor of pharmacy programs are expected to engage in a process of continuous quality improvement, initially achieved through curriculum review and mapping, with subsequent action taken and modifications made to address any weaknesses identified. The results of a curricular review should identify areas for improvement and form the basis for future curricular revisions and reviews to provide ongoing programmatic improvements. The success of the review and mapping depends upon broad and meaningful involvement by faculty members and practicing pharmacists, and also requires structured processes and procedures.¹ The Accreditation Council for Pharmacy Education (ACPE) provides guidance to schools and colleges of pharmacy about this process in standards 3, 10, and 15 of the Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree.² Detailed descriptions of each of these standards as they relate to curricular review and improvement may be found at www.acpe-accredit.org.² Despite guidance from the literature and from ACPE, colleges that have completed curricular review and mapping have sought to determine the next steps in continuous quality improvement in their professional programs.

The University of Oklahoma College of Pharmacy has been involved in such inquiry and work continuously since 2003, learning and then evolving its approaches to curriculum management as the college gains greater understanding of its professional program. The college's first efforts, from 2003 through 2005, involved a peer-review process of reviewing, assessing, and mapping of each course within the professional curriculum and resulted in recommendations for changes in content and assessment strategies for some courses. This work was supported by the development of a searchable, electronic curriculum database, named the Pharmacy Curriculum Management System. The database included course materials as well as information from the curriculum and assessment committees (including meeting minutes and peer-review reports). Since the inception, the database has been updated each semester by course coordinators with course syllabi, objectives, content, assessment strategies, and maps of course objectives to the programmatic outcomes and levels of learning achieved, thereby providing faculty members access to information taught throughout the curriculum to aid in the revision and improvement of their individual courses and in the development of new electives.³ From 2006 to 2007, the college completed a second review of individual courses to evaluate compliance with and the impact of implementing the recommended changes and to determine appropriate sequencing of all courses within the professional curriculum. This work resulted in the movement of some core professional courses to better align the curriculum content and to further support the development of pharmacy practice skills.

During these 2 individual course reviews, the leadership of the curriculum committee and the college realized that many individual courses largely operated in silos, segregated from the rest of the curriculum, while at the same time having the expectation of producing well-rounded and competent pharmacists. The leadership recognized that further changes were needed in the organization, integration, delivery, and assessment of the professional program to fully achieve the stated goals and outcomes of the PharmD program. While individual course reviews had yielded much useful information and resulted in improvements in both content and assessment, a comprehensive review was needed to reevaluate courses longitudinally to determine how well each course provided either an adequate foundation or facilitated the development and progression of knowledge, skills, and attitudes expected for future courses in the curriculum.

During 2008 and 2009, the curriculum committee continued its efforts in continuous quality improvement by following a logical progression from curricular mapping to evaluation of content, integration, and assessment of “streams” of knowledge, skill, and attitude development across the 4 years of the professional curriculum. This allowed for a continued college dialogue and sustained the collaborative efforts among faculty members and other stakeholders regarding content of the professional curriculum, instructional strategies, and assessment methods, all in light of the expected outcomes for graduates. This paper reviews the college's most recent approach to reviewing the curricular structure, including course sequencing, reiteration, and integration across the 4 years of the professional program.

DESIGN

At the beginning of the fall 2008 semester, the dean charged the curriculum committee with organizing and overseeing a college-wide curricular assessment of streams of knowledge, skill, and attitude development across the professional curriculum. The initial planning occurred within the curriculum committee, and the process was carried out within ad hoc peer-review committees that involved the entire faculty and selected preceptors. The curricular stream assessment was completed within 1 academic year, and the timeline is described in Table 1.

Because individual courses had previously undergone 2 separate peer reviews, the goal of the curricular review was to evaluate how knowledge, skills, and attitudes were developed throughout the curriculum and to assess whether coordination was occurring among courses with similar topics to ensure these “streams” were continuous. The review encompassed all required didactic courses and advanced pharmacy practice experiences (APPEs) in the 4 years of the PharmD curriculum. The curriculum committee identified 18 topics to be reviewed. Smaller topics with similar content and skills and those involving the same courses were grouped together to reduce the number of ad hoc committees required. The topics taught within the pharmaceutical care modules also were combined, but given the large amount of content covered within this 10-course series, the work was divided and allocated to 2 teams. This resulted in 9 ad hoc committees (referred to as “stream teams”) charged with evaluating the following streams: clinical communications; drug information and literature evaluation; pharmaceutics, pharmacokinetics, and pharmacy math; pharmacy administration, law, ethics, and professionalism; decision making and problem solving; non-prescription products, preventative health, and special populations; pharmacy practice experiences; pathophysiology, pharmacology, medicinal chemistry, and therapeutics in integrated pharmaceutical care modules 1-5; and pathophysiology, pharmacology, medicinal chemistry, and therapeutics in integrated pharmaceutical care modules 6-10.

Prior to launching the stream teams, online curricular resources were developed and the Pharmacy Curriculum Management System database was updated to provide the necessary data for teams to complete the curricular review in an efficient manner.³ As a supplement to the information already housed in the Pharmacy Curriculum Management System, a stream survey was created and administered to all course coordinators of required didactic courses. This survey enabled course coordinators to self-identify if a particular stream topic was taught and/or assessed in their course and for the curriculum committee to advise each stream team on which courses required their review. The survey instrument was used to collect additional data such as how stream topics were assessed in a course and if communication regarding course content occurred with other course coordinators. The course coordinators completed the survey instrument electronically and the curriculum committee then reviewed the data and generated a report for each stream topic that listed which courses taught that topic and included the additional data collected. Each report was distributed to the corresponding stream team chair. Information for required APPEs was uploaded to the Pharmacy Curriculum Management System for easy access by all stream team members. The community and hospital APPE content and assessments are standardized and facilitated by the college's office of experiential education; thus, course syllabi and assessments were provided by this office. A survey report on ambulatory care and acute care APPE content that had been completed by the curriculum committee 2 years prior to the stream reviews was provided to the teams.

In addition to gathering the pertinent data for review, the curriculum committee developed a reporting tool to guide the stream teams’ work and ensure consistency among the data collected (this reporting tool is available upon request from the author). Teams that were assigned multiple topics were encouraged to complete a separate report for each topic for which they were responsible. The chairs of the 2 teams that were assigned the topics within the pharmaceutical care modules were encouraged to collaborate and were scheduled to present their findings in sequential order to allow for discussion of these findings in an integrated fashion.

To ensure leadership and administrative support to guide the work, each stream team was led by a chair and vice chair. Also, each 8-person team included at least 1 individual from the curriculum committee and 1 or more faculty members from each of the college's 3 departments. This broad representation of the college ensured that diverse perspectives were considered. All full-time faculty members participated on a stream team, and select adjunct preceptors were appointed to teams within their areas of expertise (eg, a community preceptor served on the nonprescription products team and a hospital clinical pharmacist served on the decision-making and problem-solving team). Faculty participation on the teams was an expectation and considered part of their service to the college. Charges to other college committees were reduced for the academic year in which the curricular review was conducted to compensate for the added work time the stream reviews required. Adjunct preceptors participated on a voluntary basis and agreed to participate prior to announcement of the formal team assignments. Additionally, an administrative support team consisting of the chair of the curriculum committee, the associate dean for academic affairs, the assistant dean of assessment and evaluation, and 2 administrative staff members with expertise in the Pharmacy Curriculum Management System database was available to attend stream team meetings and to consult with chairs and provide guidance as needed.

The stream teams received written communication of their appointment and charges, and the chairs and vice chairs were invited to a special meeting of the curriculum committee to review the timelines and process to help launch the teams. The stream teams were given 10 weeks to complete their curricular review.

Each stream team met 2 to 4 times for 1 to 2 hours over the 10-week period, with some preparatory work done prior to each meeting. Each team submitted their findings and recommendations in writing using the stream reporting tool. Stream team reports described time allotted to the stream topic, timing of topic introduction, topic repetition and advancement, topic assessments, and team recommendations. The stream team chairs met individually with the curriculum committee to discuss the findings of their stream review and recommendations for curricular improvements. After these meetings, each stream team chair wrote an executive summary report that was distributed to the entire faculty. The process culminated with the presentation and discussion of the stream team findings at a 2-day faculty retreat.

EVALUATION AND ASSESSMENT

Upon reviewing the 9 stream team reports, the curriculum committee found several common themes. There were some areas of inconsistency in the delivery, continuity, and integration of topics across the curriculum. For example, the college did not teach a consistent approach to problem solving and decision making, and therefore, each faculty member taught this process differently. Most stream teams found that faculty members conducted their courses independently and identified a need for improved coordination and communication among course coordinators. The stream teams recommended that a formal process of assigning “stream champions” to work with course coordinators be implemented to ensure specific topics were advanced and integrated within the curriculum. Additionally, some topics lacked repetitive assessment across the curriculum. For example, the pharmacy math course occurs in the first year, and several courses in subsequent years taught or applied math skills, but did not consistently assess students’ math skills. Finally, many teams expressed that the stream review process was difficult because they did not fully understand the outcomes and the desired level of competencies that the college had for each topic.

At a faculty retreat, the college's entire full-time faculty identified priorities for curricular improvement after reviewing and discussing the findings from the 9 stream teams. Each of the 45 faculty members in attendance was asked to write his/her top 2 priorities for curricular change on an index card (Table 2). The same 2 priority areas for curricular improvement were listed by over 25% of the faculty members polled. The first priority was to develop a consistent management system for stream topics to improve coordination of streams of knowledge, skills, and attitudes across the curriculum. The second was to redefine program outcomes and clarify and agree upon the proficiency level that students need to demonstrate for each outcome. These top 2 faculty priorities were consistent with the major themes that the curriculum committee had identified upon review and discussion of the stream reviews within curriculum committee meetings. Additional priorities that were ranked by 10% to 25% of the faculty polled applied to specific topics or areas within the curriculum. These included repetition and assessment of pharmacy math skills beyond the first-year math course, teaching nonprescription products and self-care concepts in a separate course, and improvement and consistency in teaching problem solving and critical thinking. These top priorities areas were consistent with needs identified in previous curriculum reviews and discussions.

DISCUSSION

During the 5 years of curricular mapping and course peer review that preceded the committee review, the University of Oklahoma College of Pharmacy worked to make the curriculum more transparent and encouraged dialogue within and among courses to focus instructional efforts. While these reviews offered insight into individual courses, the need to understand how courses related to each other remained. Based on common themes and recommendations from the previous reviews, the college focused its next step on evaluating curricular streams of knowledge, skills, and attitudes, which would provide a unique cross-sectional overview of the curriculum. This review evaluated where repetition was needed and beneficial for student learning, taking on a different emphasis than previous reviews that focused on individual courses and elimination of redundancy.¹ Overall, the evaluation of the streams of knowledge, skills, and attitudes across the curriculum was an internal peer review that had broad faculty involvement, was completed in one year, and resulted in outcomes that have led to additional next steps in the curricular review and improvement process.

Keeping the stream review manageable required the curriculum committee to merge 18 knowledge-, skill-, and attitude-based topics together in a coherent fashion. While this consolidation helped reduce the number of teams needed, it also resulted in grouping some topics that were not directly related. For example, combining pharmacy administration, law and ethics, and professionalism as a stream was atypical. Even though the 3 topics can be categorized under social and administrative sciences, they are not directly linked. The primary reason for combining topics was there were not enough faculty teams available to assign only 1 topic to each team. This consolidation required some debriefing of the stream teams and led members of each team to “divide and conquer” the topic areas among themselves, which ultimately led the team to complete a thoughtful review of each assigned stream topic. Similarly, there also was debate among members of the curriculum committee about whether pharmacy math should be an independent stream or considered a component of pharmacokinetics and pharmaceutics. Ultimately, math was merged with pharmacokinetics and pharmaceutics because the committee agreed that there would be an overlap among the topics and combining them would further limit the number of stream teams required. Colleges that have a larger faculty and/or a preceptor base that is more involved in the didactic curriculum may be able to increase the number of stream teams and divide some of the combined topics into separate topics assigned to separate stream teams, such as reviewing pharmacy administration, pharmacy law and ethics, and professionalism separately.

There were 3 perceived strengths associated with the stream curricular review. First, the review process was facilitated by an electronic curricular database, the Pharmacy Curriculum Management System, which made the most current course information immediately accessible to stream teams. This database contributed to the completion of the project within the 1-year timeframe. However, a limitation of a retrospective database review is the assumption that the intended curriculum is the same as the delivered curriculum.³ Live delivery of the associated streams of knowledge, skills, and attitudes was not evaluated, although activities, assessments and student evaluations were reviewed.

Another strength of the review was the culture of assessment at the college and the acceptance of curricular review among the faculty members. As reported previously, establishing buy-in with the process of review can be time consuming and a potential roadblock.¹ The longstanding history of curricular review at the college allowed the stream teams to progress more easily into the review process, allowing the reviews to be completed in a timely manner.

The stream reviews used a standardized, structured, and systematic process for evaluating the curriculum that was based on the outcomes of the college's previous reviews. This structure allowed all stream teams to remain focused on the same elements of the review.

Due to a lack of literature reporting on next steps of curricular review beyond curriculum mapping and a lack of validated tools to assist with the process, the stream review process was a novel approach that used tools that lacked psychometric data. Aside from the evaluation tools used, this was our third iteration of curricular review and the refinement of the process over time has been successful as evidenced by our timely completion and detailed outcomes that provide a foundation for future work. Understanding faculty priorities for future curricular change and review was instrumental in the next steps to be undertaken. The college is now using the insights gained over the years from individual course and broad stream reviews to refine, and in some cases, redefine the outcomes of our professional program. The outcomes that the doctor of pharmacy program was established with in 1998 were based on the Center for the Advancement of Pharmaceutical Education (CAPE) outcomes. The stream evaluation allowed the faculty to evaluate its success with and understanding with the CAPE outcomes. The greatest gains from the review process have been increased faculty awareness of the contributions of their course to program outcomes; the relationship of individual courses within streams of knowledge, skill, and/or attitude development; and most importantly, greater insight into stated program outcomes and a more realistic view of how to attain them.

SUMMARY

Having completed extensive curriculum mapping and individual course review over a 5-year period, the University of Oklahoma College of Pharmacy undertook a curricular review to assess streams of knowledge, skills, and attitudes across the 4-year professional curriculum. Keys to success for this process were the existence of an updated electronic curriculum database, and faculty members accepting their role as reviewers and having past experience in conducting curricular reviews. This review enabled the college to identify areas for improvement within the curriculum that were not readily apparent to the faculty from prior reviews of individual courses. The college agreed that clarification of programmatic outcomes was a top priority for curricular improvement, and an ad hoc committee was appointed to address this over the next 2 years. The stream review outlined above is a process that could be replicated or adapted by other colleges or schools of pharmacy seeking to move beyond their initial curriculum mapping efforts and continue to refine their program.