Co-Curriculum Implementation and Assessment in Accredited Doctor of Pharmacy Programs

INTRODUCTION

Standards 2016 from the Accreditation Council for Pharmacy Education (ACPE) introduced a co-curriculum requirement for Doctor of Pharmacy (PharmD) programs.¹ The “Guidance for Standards 2016” defines co-curricular activities as experiences that complement and advance what is taught in the formal didactic and experiential curricula.² These experiences can encompass diverse activities and should be deliberate, intentional, and linked to the curriculum, particularly the educational outcomes of Standards 1-4, which are also the 2013 Center for the Advancement of Pharmacy Education (CAPE) Outcomes.^1-3 While not all students have to complete the same activities, all students must engage in the co-curriculum to comply with the co-curriculum accreditation expectation. Furthermore, there is an expectation that programs demonstrate how co-curricular experiences advance learning.

Embracing ACPE’s expectation that the co-curriculum complements the formal curriculum, recent AACP reports have highlighted the various challenges in developing, implementing, and assessing curriculum models that embrace effective educational models. The 2011-2012 Argus Commission commented on the “lock step curricula” that existed and outlined the need for disruptive innovation in pharmacy education.⁴ The 2012-2013 Argus Commission report stated that students want learning experiences that meet their educational and personal needs, including both on- and off-campus offerings.⁵ The co-curriculum is one opportunity to provide such learning opportunities to students that may be more flexible, customizable, and individualized than the formal curriculum while augmenting their didactic and experiential learning. This type of flexible learning can provide “students choice in the pace, place, and mode of learning,” thus empowering them to exercise self-awareness, reflection, and life-long learning skills.⁶

While the co-curriculum may offer benefits to the student learning experience, the requirement has prompted significant conversation within the Academy regarding how to structure co-curriculum models, including strategies for implementation, types of activities to include, and number of activities or hours needed. Articles outside of pharmacy education have attempted to define the term “co-curricular,” usually by clarifying the definition of extracurricular.^7-9 The Glossary of Education Reform states that co-curricular activities are an “…extension of the formal learning experiences in a course or academic program, while extracurricular activities may be offered or coordinated by a school, but may not be explicitly connected to academic learning.”⁸ While the ACPE co-curriculum requirement may be relatively new, the broader idea of experiences outside the traditional didactic and experiential components that contribute to overall student learning is not. Many programs have traditionally contained a combination of classroom (didactic) teaching, experiential learning, and unstructured activities, including participation in local health fairs, national organization meetings, and student-led extracurricular events. These unstructured activities can be important in the learning process as they can develop and reinforce student learning.^10-14 However, because the ACPE requirements were not prescriptive in nature, it remains to be seen whether the definitions available from higher education literature (and inclusion of previously termed “extracurricular activities”) align in a broader sense with the intent of ACPE for co-curriculum to be incorporated into pharmacy programs.

While existing pharmacy curricula likely have offered and continue to offer numerous extracurricular activities, programs must now decide how to address the co-curriculum requirement set forth by ACPE. Patel and colleagues outlined two potential ways to incorporate co-curricular activities into the existing curricula.¹⁵ The first is to modify existing extracurricular structures, while the second is to develop a new model that meets the program’s specific needs. Several studies have described and evaluated how existing curricula were adapted as co-curriculum models.^11-13 Regardless of the approach, there are common concerns that need to be addressed. Patel and colleagues suggest that one way to address the lack of concrete guidance on a definition for co-curriculum is that programs embed components of the co-curriculum, including its purpose, personnel involved, and activities, into the definition itself.¹⁵ Studies in pharmacy to date have explored a number of areas related to co-curriculum. Some studies have described co-curriculum mapping in the context of curricular outcomes.^14,16 Other studies have described and assessed an institution’s co-curriculum model.^11-13 As part of a national survey examining leadership development, Ross and colleagues included a question asking respondents to document the types of co-curricular activities they offered related to leadership.¹⁷

The purpose of this study was to assess how co-curriculum models are being implemented, including types of activities, structure, learning outcomes, oversight, and assessment in ACPE-accredited pharmacy programs.

METHODS

A survey was developed by a working group from the American Association of Colleges of Pharmacy (AACP) Assessment Special Interest Group (SIG) to collect information related to how the co-curriculum requirement was being addressed by pharmacy programs accredited by ACPE. The survey was administered to programs from February through May 2018. The survey instrument was constructed by a working group of assessment leaders from 13 pharmacy programs, and went through several iterations before being finalized. Items were included to gather information about the co-curriculum model used, including activities, structural characteristics, learning outcomes, oversight, and assessment. The survey was administered electronically through Qualtrics, Version 2015 (Provo, UT). The survey instrument is available upon request from the corresponding author. The Cedarville University Institutional Review Board deemed this study exempt in February 2018.

A list of prospective participants was obtained from the AACP Roster of Faculty and Professional Staff.¹⁸ All those in the directory who indicated having responsibility over assessment and were AACP members were identified. If more than one person was listed, the individual with the highest rank was selected. An e-mail was sent to these prospective participants that provided a brief overview of the research and an invitation to complete the survey. The primary contact was encouraged to solicit input from others at the school with knowledge of the co-curriculum model. To allow feedback from others, a document version of the survey was attached to the e-mail, along with the link to complete the survey online. Reminder emails were sent to nonrespondents at two, four, and eight weeks. Prospective participants who had not responded after eight weeks were contacted by phone by one of the investigators and asked to complete the survey using the online link.

Institutional demographic data (ie, year founded, institution type, cohort size for academic year 2017-2018, and whether the most recent ACPE accreditation review was focused or comprehensive) were obtained from AACP and ACPE.^19,20 These data were merged with the co-curriculum survey responses. The survey respondents’ programs were compared to the target population of all ACPE-accredited programs. The main purpose of the institutional demographic data was to demonstrate that the sample of respondents was similar to the population. Trends were examined between programs of different backgrounds (ie, ACPE accreditation review before 2016 vs after, cohort size). However, there were no clear patterns or differences. Furthermore, the analyses were complex for a number of reasons (eg, select all that apply responses). We determined that the data could be best used to describe the current landscape of co-curriculum models across all accredited programs.

Respondents were not required to complete all survey questions for their information to be included in the dataset. Thus, all valid responses were included in the analysis. Some items were “select all that apply” to capture potential diversity in approach to addressing the co-curriculum. The survey used general terminology (ie, activities that fit a category or “bucket”) and did not provide examples of possible responses. This was done to ensure we captured a broad understanding of how models were set up. Some item responses were open-ended or partially open-ended, and respondents were asked to elaborate when they selected “other.” For these items, we created codes based on the common responses. These codes have been distinguished from the response options available to respondents using superscripts in the results tables. Data analysis consisted of descriptive statistics using SPSS, Version 21 (IBM, Armonk, NY).

RESULTS

Of the 143 pharmacy programs contacted, 107 completed the survey for a response rate of 74.8%. A comparison of the background information provided by the programs and the data available for all ACPE-accredited programs is presented in Table 1. The responding programs were similar to other ACPE-accredited programs with regard to year founded, institution type, program cohort size, and timing of their most recent focused or comprehensive ACPE review.

The data collected on the patient care and non-patient care activities included in co-curriculum models are presented in Table 2. The vast majority of programs indicated that the patient care activities listed were included in their co-curricular model. The most common patient care co-curricular activities included public health outreach (99.1%) and public education events (94.4%). There was more variation (22.4%-92.5%) in the non-patient care activities included as part of the co-curriculum model. The most common non-patient care co-curricular activities were legislative advocacy (92.5%), leadership/professional service (91.6%), and professional education or meeting attendance (90.7%).

The structural characteristics of co-curriculum models reported by respondents are presented in Table 3. Most programs reported co-curriculum models as optional activities outside of the formal curriculum (65.4%) and/or programmatic requirements distinct from coursework (57.0%). Other programs have adopted co-curriculum models as part of the advising process (36.4%), the experiential program (35.5%), didactic courses (23.4%), and/or a standalone co-curriculum course (21.5%). Many programs described the model used as a combination of pre-specified activities, a list of activities that fit into a category or “bucket,” or individual plans based on educational outcomes. A substantial portion of programs (40.2%) responded that co-curricular activities counted toward course credit. Most programs categorized activities by experience setting or type (26.2%) or by desired competencies (58.9%).

The learning outcomes or competencies, if any, to which co-curricular activities were mapped by the program and students are presented in Table 4. While a minority of respondents (20.6%) reported that the program did not map co-curricular activities to learning outcomes, the majority (79.4%) reported that their program did map co-curricular activities to at least one set of learning outcomes. Also, 55.1% reported that their program did not require students to map activities to outcomes. In both groups, the most common outcomes used for mapping were ACPE Standards 1-4 (ie, 2013 CAPE Outcomes) followed by institutional or program learning outcomes.

The data collected regarding co-curriculum oversight, including the committee and administrative structure for co-curriculum models are presented in Table 5. Most programs (63.5%) reported having a co-curriculum committee, subcommittee, or task force (ad-hoc or standing), or both. Most respondents (71.0%) identified a primary office or department as being responsible for the co-curriculum at their school. However, 18.7% reported “other” as an option and were recoded to having shared responsibility based on their text response stating this. The most common offices or departments listed as having primary responsibility and supporting offices can be found in Table 5. Of the 91 programs that reported categorizing co-curricular activities, the final categorical determination of an activity was most commonly performed by a committee (36.4%), the director of co-curriculum (20.6%), and/or another administrator (20.6%).

Documentation and assessment methods used in co-curriculum models are presented in Table 6. The survey asked how activity completion was being tracked, with the option to select all that apply. Although seven (6.5%) programs responded that they were not tracking co-curriculum activity, most schools used a combination of at least two of the following: self-assessment or reflection, an accounting of the activities completed, number or hours of activities, and/or development in learning outcomes. Manny programs reported using technology to assist with compiling or assessing student work in the co-curriculum model and providing feedback to students.

In addition to the data presented in the tables, the survey also asked in which professional year students had co-curricular requirements, how completion of the requirements was tracked, and the number of times the requirements were assessed within each professional year. As most programs (73.8%) reported having a curriculum consisting of three years of didactic instruction and one year of experiential learning, only programs with this configuration were included in the following analysis. Most of these programs reported having a co-curricular requirement in the first professional year (91.1%), second professional year (89.9%), and third professional year (88.6%). However, only 50.6% reported having a co-curricular requirement in the fourth professional year. Consistent across all four years, most programs monitored student completion of co-curricular requirements by tracking activities (60%-67.5%) or tracking hours or activities (22.5%-28.6%). A smaller proportion of programs (7.5%-10.0%) reported tracking completion of requirements using hours only. The assessment used in co-curriculum models was most commonly reported as occurring once per semester (approximately 40%), though multiple times per semester and once per year were each reported by nearly 25% of respondents.

The survey asked respondents to select all of the measures used in the co-curriculum model at the time of the survey as well as those measures that the program planned to use in the next 12 months (Table 7). Five programs (4.7%) reported that no assessment was used or planned.

DISCUSSION

Interestingly, a universally accepted definition of co-curriculum and an optimal model for the implementation of the co-curriculum requirement from ACPE are currently not available within pharmacy education. In this study, we have described how co-curriculum models are characterized and being implemented by pharmacy programs, which provides further information about how programs are defining co-curriculum. This is the first study within pharmacy education to evaluate co-curriculum model characteristics among ACPE-accredited programs.

The types of co-curricular activities reported by programs were relatively consistent and aligned with the list of examples of co-curricular experiences provided in the Guidance to Standards 2016 from ACPE.¹ A number of studies from a variety of institutions have reported similar activities as a part of their co-curriculum models.^11-13 In our study, more than 90% of programs reported public health outreach, public education events, legislative advocacy, and professional meeting attendance as co-curricular activities. These types of activities, which align closely with expected components of didactic and experiential curricula, were likely already taking place prior to the co-curriculum requirement as extracurricular or course activities. Thus, when developing the co-curriculum model, these activities were probably easily incorporated. Interprofessional education or collaboration was reported as a co-curricular patient care activity by over 75% of the programs and as a non-patient-care activity by nearly 60% of the programs in our study. The prevalence of programs opting to include these as co-curricular activities reflect ACPE Standards 2016, which have placed significant emphasis on interprofessional education. Also, the Guidance to Standards 2016 even states that “interprofessional co-curricular programs and experiences are also encouraged.”^1,2

Several questions in our survey assessed how programs structured their co-curriculum model. Over half of the programs (57%) reported having co-curricular requirements which students needed to be in order to progress and an additional one-third of programs reported implementing co-curriculum as a part of the experiential program or a stand-alone or capstone course. We believe these program or course requirements provide a way for programs to provide evidence that all students are meeting the co-curriculum requirement. If some co-curricular activities were a part of the curriculum or specific course(s) prior to the accreditation requirement (such as an introduction to practice or experiential readiness course), this area or course(s) may have taken on the responsibility for ensuring co-curriculum requirements were met. The available literature on co-curriculum in pharmacy education provides descriptions of non-graded requirements in an experiential program or course that evolved into a way to meet the co-curriculum requirement.^11-13,21

Most programs in our study reported that their students were able to choose co-curricular activities from a list of pre-specified activities. Interestingly, several programs (18.7%) reported that their co-curriculum model required students to develop an individual plan based on individual learning outcomes. In theory, co-curricular activities should allow students to pursue activities of interest, rather than to simply meet programmatic requirements.²² Models allowing students to develop their own plans based on learning outcomes grant students flexibility to pursue areas of interest. Furthermore, these various models suggest that programs are following the ACPE guidance that all students meet the requirement without mandating that all students complete the same activities. However, as stated previously, the ACPE co-curriculum requirement was not prescriptive in nature. Thus, whether the co-curriculum is within the required curriculum, outside the required curriculum, or a combination of the two may not be of great significance and may reflect how the program incorporated similar activities prior to the requirement.

An important consideration in co-curriculum models is establishing intentionality of co-curricular activities to advance knowledge, skills, and abilities taught and assessed in the curriculum.²³ By establishing categories into which activities are grouped, as nearly 80% of responding institutions reported doing, programs would be able to ensure that students have a minimum exposure to certain activities at certain points and meet specific learning outcomes related to the curriculum. Vos and colleagues described a model that established a threshold for the number of co-curricular hours required from three categories: professional leadership, professional service, and community engagement.¹³ Hoffman and colleagues described a model in which students were required to complete a specified number of co-curricular activities, which varied by professional year, from five categories: professional education, patient care services, legislative advocacy, professional service/leadership, and health care-related community service/philanthropy.¹¹

Another way of providing possible evidence of the intentionality of co-curricular activities could be mapping them to learning outcomes. Providing students with a map of how the co-curriculum model is linked with course work may help students to identify which outcomes are being met. The learning outcomes to which activities were most frequently mapped (by 75.7% of responding institutions) were ACPE Standards 1-4, which could be expected given that the Guidance to Standards 2016 includes that co-curricular experiences linked to outcomes in Standards 1-4 are most useful to students.² As previously described, Hoffman and colleagues provided an example of a co-curriculum model including a table demonstrating categories, examples of activities for each category, and ACPE accreditation standard alignment for each category.¹¹ Another study by Ramia and colleagues demonstrated the mapping of personal and professional development using both faculty members (enacted curriculum) and students (learned curriculum) in curricular, co-curricular, and extracurricular experiences.¹⁶ However, a student’s learning can vary based on the setting and experience.²⁴ Therefore, it may be useful to have students report what learning outcomes were attained during a particular experience. In our study, 13.1% of responding programs reported that students determined the final categorization of co-curricular activities. Fjortoft reported having students in leadership complete a survey following the implementation of student organization events. Students were asked to identify the domains, based on ACPE Standards 1-4, that characterized the activity.²⁵ Perhaps this approach could be used by students for reporting co-curricular activities as well.

In our study, 40% of programs reported that co-curricular activities counted as course credit, which may challenge the notion of co-curriculum. However, given the open-ended nature of our survey instrument, it is unclear how respondents interpreted this question. Because ACPE does not require a prescriptive co-curriculum design requirement, our intent with open-ended questions on the survey was to capture as much data as possible from programs for subsequent thematic analysis. A review by Bartkus and colleagues concluded that varying definitions exist but co-curricular activities appear to require “a student’s participation outside of normal classroom time as a condition for meeting a curricular requirement.”⁷ Arguably, activities within the curriculum that require outside experiences should count as co-curricular. Some programs may have experiences embedded in courses which require activities outside of classroom time, and they could count these as co-curricular activities, such as creating a business plan, making an advocacy visit to a legislator’s office, or attending a support group meeting for substance abuse. Over half (54.2%) of responding institutions reported that the co-curriculum was implemented as a part of a standalone co-curriculum course, a capstone course, or another didactic course that was not standalone or capstone in nature. Given the close association between the curriculum and co-curriculum mandated by ACPE and evidence of overlap found in our survey results, any changes made to the curriculum could have a direct and tangible impact on the co-curriculum and vice versa. Thus, institutions engaged in curricular revision, whether prospective in nature or part of normal quality assurance, should ensure that appropriate considerations are made so that the full intent of the program’s curriculum and co-curriculum are carried out.

Several questions were also posed regarding the structure and personnel involved in the program’s co-curriculum model. While two-thirds of programs reported having a standing or ad hoc committee, subcommittee, or task force to oversee the co-curriculum, almost 20% of programs reported that co-curriculum was a shared responsibility and most programs reported having additional offices supporting the co-curriculum. This may reflect that activities now categorized as co-curricular were previously administered by multiple offices and subsequently remained under their oversight. However, with the ACPE co-curriculum requirement, additional tasks may be required, and it is unclear whether programs have allocated additional resources for these offices to accommodate the increase in workload. Interestingly, over 20% of programs reported that a director of co-curriculum (or similarly titled individual) determined the final categorization of activities. Given the relatively new emphasis on co-curriculum as an accreditation requirement, these position titles may either be new or have evolved from previous oversight roles under either assessment or student affairs offices as noted previously.

Our research demonstrated a high use of reflections (nearly 90% of responding programs) for tracking and assessing the co-curriculum, which would be expected given that ACPE encourages student self-reflection.² We also showed that a large majority (83.2% of responding programs) were using either self-assessments or reflection for tracking student completion of co-curricular activities. Researchers of co-curriculum in higher education recommend the use of self-reflection for assessment as they have found that reflection helps students to internalize their experiences.^26,27 Reflection, as defined in higher education literature, supports completion of learning objectives by promoting critical thinking in the student, demonstrating the student’s inductive or deductive reasoning skills, intentionally engaging the student in community experiences, and cultivating the student’s awareness of herself as an active participant in the activity.²⁸ Other common measures of assessment employed by institutions responding to our survey included student self-assessment surveys (63.6% of programs), checklists (37.4% of programs), and introductory pharmacy practice experience (IPPE) evaluations (26.2% of programs). Within pharmacy education, studies evaluating a program’s co-curriculum model offer some insight into how assessment can be completed. Hoffman and colleagues reported the frequency of student participation in co-curricular activities and use of a satisfaction survey.¹¹ Students were required to submit a personal reflection, but the reflections were not analyzed. The researchers recognized a need to reevaluate the reflection process to include more structured reflection items. Vos and colleagues reported the number of hours pharmacy students spent engaged in co-curricular activities. Additionally, at the beginning of the year, the students wrote professional development goals.¹³ More research is needed regarding the effectiveness of assessment measures, including reflections, in tracking student development. Also, pharmacy faculty members may require training in how to properly use rubrics and accurately evaluate reflections to ensure consistency, and more information is needed on how this is being done in co-curriculum models.

Despite the high response rate of this study, there are several important limitations to note. First, the study was limited to the responses obtained from the respondent for each institution and may not reflect all elements of the co-curriculum model accurately, particularly those co-curricular aspects that the respondent was not aware of. To prevent this, the survey was sent to respondents as a document and recommended that they collect answers from all individuals able to provide input. However, some respondents may have completed the survey online without consulting others or may have excluded individuals who had relevant knowledge. Second, the exact definition of co-curriculum in pharmacy education is still in development, and an ACPE-endorsed model satisfactory for accreditation requirements is not available. Given the lack of a widely accepted definition for co-curriculum, respondents may have had different interpretations or used different terminology when considering their school’s co-curriculum model (ie, model description, categorization), which may have influenced their survey responses. We considered these possible limitations when developing the survey instrument; thus, almost every item had a response option of “other” where respondents were able to write-in their response. This feature proved useful as it gave us insight into the characteristics of programs and led us to identify themes that would have otherwise remained unrecognized.

Continued studies of individual co-curriculum models are important to help define co-curriculum and to determine what works well.² Such studies will help to define key features of co-curriculum models and eventually establish best practices once an ACPE-endorsed model is determined. Another area that should be addressed is methods for the assessment of co-curricular experiences, particularly the use of reflections as this has not been thoroughly evaluated in the current literature. Finally, the ACPE co-curriculum requirement is still relatively new and definitive measures for adequacy and appropriateness of co-curriculum activities have yet to be outlined. While this study attempted to present the current landscape of co-curriculum across US pharmacy institutions, readers should recognize that what is currently being implemented may not be the best or most appropriate way to fulfill the co-curriculum requirement per ACPE accreditation requirements. Future studies should also assess the progression and maturation of co-curriculum models. Some programs may be able to restructure their co-curriculum model, particularly as they receive feedback from the ACPE during regular reporting and accreditation site visits. Alternatively, programs may find that certain co-curriculum models are more resource-intensive and may reallocate resources or create additional positions to support the model.

CONCLUSION

This study described how co-curriculum models are being implemented in pharmacy programs including purpose, types of activities, personnel involved, and assessment strategies. We found significant consensus among pharmacy schools regarding the structure and types of activities included in their co-curriculum model and in the collaborative oversight of the co-curriculum. Notable variation in assessment of co-curriculum was evident. Further research is needed to track development and changes in co-curriculum models over time and to determine what additional resources are needed to meet this requirement. The study results may be useful to members of the Academy when evaluating the current state of co-curriculum implementation across institutions and potentially areas of program development or resources that may be needed across the Academy.