Implementation and Initial Evaluation of a Research and Scholarship Training Pathway in a Doctor of Pharmacy Curriculum

INTRODUCTION

Creating and disseminating new knowledge to guide clinical decision-making and optimize medication use are critical to advancing pharmacy practice.¹ Professional pharmacy organizations have emphasized the need for pharmacists to develop skills in research and scholarship.^2-4

Ideally, pharmacists in-training should learn such skills while enrolled in a Doctor of Pharmacy (PharmD) program and then continue to develop these skills through postgraduate training, continuing education, and professional organization involvement. The Accreditation Council for Pharmacy Education (ACPE) recognizes the importance of research and encourages student engagement in faculty research programs.⁵ However, neither the 2016 ACPE standards nor the Center for the Advancement of Pharmacy Education (CAPE) 2013 Educational Outcomes provide specific recommendations on how research training should be delivered to students.^6,7 Competing priorities in PharmD curricula and lack of coordinated research opportunities can create barriers to students developing research and scholarship skills prior to graduation.

Many schools of pharmacy offer formal and informal research opportunities for students. A select few programs have required students to engage in faculty-mentored research projects.^5,8-10 Students in these programs reported an improved ability to work independently, evaluate literature, distinguish themselves from peers, and build faculty-student mentoring relationships.^8-10 Graduates have also reported improved problem-solving skills, better marketability, and more effective functioning in their careers.⁵ However, most PharmD student research training programs are elective experiences.^11-15

Despite several examples of successful pharmacy research training opportunities, there is often substantial variability in the quality and depth of student projects and high rates of attrition.¹⁶ Previous studies have reported publication rates from PharmD student research projects of less than 10%.^16,17 Challenges such as time constraints for the student and mentor, limited resources for statistical support, and students graduating prior to project completion diminish the depth of training received and the quality of the research.¹ While most pharmacy curricula include foundational instruction in literature evaluation and study design, numerous challenges can impede opportunities for student engagement in research. These challenges can be curricular or institutional in nature (eg, limited time available in the student schedule for research, limited availability of research infrastructure and resources to support student projects) and/or related to the research project itself (eg, length of time required for institutional review board submission and approval, insufficient project depth for presentation or publication).¹ Thus, there is a tremendous need for schools of pharmacy to increase the availability and enhance the quality of opportunities for student engagement and training in research and scholarship.

The University of North Carolina at Chapel Hill (UNC) Eshelman School of Pharmacy launched a new PharmD curriculum in 2015.¹⁸ One of the four defining elements of the new curriculum was fostering scientific inquiry and innovation. To meet this goal, an elective pathway built around a longitudinal, faculty-mentored scholarly project was developed and implemented within the curriculum. The purpose of this study was to describe the design, implementation, and initial impact of the Research and Scholarship in Pharmacy pathway.

METHODS

In designing the Research and Scholarship in Pharmacy pathway a reverse engineering process was employed.¹⁸ A guiding principle was that faculty members would instill habits of inquiry, curiosity, and critical thinking in students and inspire students to be lifelong learners. The Research and Scholarship in Pharmacy pathway was designed through a series of subcommittee meetings with faculty representation from all five academic divisions and both campuses (Chapel Hill, NC, and Asheville, NC), as well as student representation. The pathway aimed to address previously described barriers for meaningful student research experiences in PharmD curricula, including lack of student knowledge of the publication process, limited student mentorship, attrition without completing data analysis or manuscript development, and time constraints.¹ The guiding principles of a quality student research and scholarship experience (Appendix 1) were defined by the subcommittee, endorsed by the school’s Curricular Transformation Steering Committee, and used to design the pathway.

We created a longitudinal, elective pathway within the PharmD curriculum built around a mentored, in-depth, scholarly project where each student would frame a research question guided by a faculty mentor, generate and interpret relevant data, and communicate their findings in an oral and written form. This was accomplished by integrating students into the existing scholarly programs of the school’s faculty in each of the five academic divisions and on both campuses. The RASP projects spanned multiple domains (eg, preclinical, translational, clinical, epidemiologic, health services, educational, quality improvement, method development) and research designs (eg, experimental, quasi-experimental, observational), and included hypothesis-driven and non-hypothesis driven research.

Over the course of this longitudinal experience in research and scholarship, students gained skills and achieved learning outcomes (Appendix 2) through development and execution of their project, participation in weekly in-class sessions, completion of four pathway deliverables, self-directed learning about specific topics and methods relevant to their unique research project, and guidance and instruction from their faculty mentor. Although project advancement varied by student and project, a general framework and timeline is outlined in Figure 1. Students enrolled in a course entitled Research and Scholarship in Pharmacy (RASP) over three semesters, which included a 1.5-credit eight-week course in the spring semester of year two (RASP 1), a 1.5-credit eight-week course in the fall semester of year three (RASP 2), and a 3-credit full-semester course in the spring semester of year three (RASP 3). These elective courses followed completion of a required first-year Evidence-Based Practice course, which taught students how to identify, critically evaluate, and interpret scientific literature to make evidence-based patient care decisions.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Framework and Timeline of Key Events for Research and Scholarship in Pharmacy (RASP) Pathway

The three courses and 6 credit hours earned through the program counted towards fulfillment of the 8-credit hour minimum elective requirement for PharmD students. The pathway deliverables in the three courses included: a written project proposal (RASP 1), an oral project proposal presentation and revised written proposal (RASP 2), a poster presentation (RASP 3), and an original research manuscript (RASP 3). In addition to providing instruction and feedback on the deliverables, courses included in-class instruction and discussion on topics such as qualitative and quantitative data management and analysis, proposal, abstract, poster, and manuscript development, research ethics, and career pathway exploration. Each semester, students defined goals and expectations in collaboration with their mentor and self-evaluated their performance and skill development, and mentors evaluated the student’s performance and skill development. Faculty mentors were recognized as instructors in each course, and reported time spent mentoring as part of their annual teaching activities.

An oversight committee served as the pathway governance structure. The committee was chaired by a faculty director and included representation from all five academic divisions, both campuses, and a staff program coordinator from the curricular affairs office. The committee solicited project opportunities from faculty members during the summer. Each committee member provided oversight and guidance to approximately three to five student-mentor pairs in each cohort. Notably, each course included multiple roundtable sessions for students to meet with their assigned oversight committee member, received feedback on deliverable assignment drafts (eg, proposal, poster, manuscript), and discussed project specific milestones (eg, IRB approval) and challenges encountered. The oversight committee was one of the school’s standing committees and contributed to each faculty member’s service effort. The committee members were also recognized as instructors in the pathway courses and reported instructional and assessment time as part of their annual teaching activities.

The pathway courses were preceded by an exploration and enrollment period (Figure 1). During the first year, students interested in research were encouraged to independently explore and initiate research opportunities with a faculty member. During fall semester of their second professional year, students indicated their interest in the pathway via a school-wide survey regarding elective interests, were sent a summary of available project opportunities, met with prospective faculty mentors to learn about project opportunities, and enrolled in the pathway by providing a rank-ordered list of projects they wished to pursue. Students who had already initiated work on a research project were encouraged to expand their work on that project. The oversight committee coordinated the student-faculty connections based on student preference and consultation with the faculty mentor. In the event that multiple students requested the same mentor, the faculty mentor was granted the flexibility to connect with the student that best fits their research program. The program director worked with the remaining unconnected students to determine whether another suitable mentor was available. Students connected with a faculty mentor entered the pathway in November of their second academic year and were encouraged to initiate work on their project as soon as possible. Students and faculty mentors also completed a student-mentor agreement form outlining the expectations of the course series and any additional expectations from the mentor, such as attending research group meetings and/or completing project-specific training.

During the elective courses, students were expected to dedicate an average of 16 hours per week toward course activities and their project. In addition to working on their project during the semester, the pathway facilitated a longitudinal research experience by strongly encouraging students to advance work on their project during the summer between their second and third year, as well as after completion of the final Research and Scholarship in Pharmacy course in spring of their third year. The expectation was that students would proactively communicate with their faculty mentor, advance and complete work on their project, and pursue presentation at a national meeting and/or publication in a peer-reviewed journal during fourth-year advanced pharmacy practice experiences (APPEs). Students wishing to pursue an Honors designation at graduation in accordance with university requirements were required to submit an advanced version of their RASP 3 manuscript as a thesis by March of their fourth year. The manuscript was reviewed and approved by their faculty mentor and the oversight committee prior to graduation.

A retrospective multi-methods analysis was used to evaluate the initial impact and perceived value of the Research and Scholarship in Pharmacy pathway. Data collection and analysis focused on the first two student cohorts who completed the course sequence in May 2018 and May 2019. For both cohorts, data describing the number of students, faculty mentors, and project types were obtained from programmatic data. The number of student-led abstracts presented at national meetings was self-reported by students and mentors via email. Before graduation in May 2019, the first cohort was anonymously surveyed regarding their satisfaction with the Research and Scholarship in Pharmacy pathway and its perceived value on their skill development. Because of the timing of this analysis, survey data were not available from the second cohort.

The survey included 26 items: nine items were rated using a Likert scale ranging from 1=strongly agree to 5=strongly disagree and four items were rated using a Likert scale ranging from 1=very satisfied to 5=very dissatisfied. Additional items asked whether the faculty mentor wrote a letter of recommendation or served as a reference, the student’s plans to engage in future research activities, the student’s perceptions on the value of the experience, and whether the student would recommend the pathway to other students. The survey also included the following open-response items: “What is the primary value that the RASP pathway adds to the curriculum?” “What worked well in the pathway that you would not change?” “What would have made RASP a better experience for you as a student?” “Please provide any additional thoughts you want to share.” The survey was administered using Qualtrics (Provo, UT).

Data were analyzed using descriptive statistics and reported as counts (percentages) unless otherwise indicated. Qualitative comments provided by students on the exit survey were analyzed by one coder using one round of thematic analysis. The UNC institutional review board designated this project exempt from full review.

RESULTS

Fifty students (25 students or 18% of the overall graduating class from each cohort) completed the three-course sequence. As described in Figure 2, 96% (53 of 55) of students requesting to enroll were connected with a faculty mentor, 94% (50 of 53) of enrolled students completed all three courses, and 86% (43 of 50) completed the requirements for an honors designation (84% and 88% in the first and second cohorts, respectively).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Research and Scholarship in Pharmacy (RASP) Enrollment and Project Domains for First Two Cohorts

Students were supported by 33 distinct faculty mentors. This included 19 unique mentors in the first cohort, 20 unique mentors in the second cohort, and six mentors that supported students in both cohorts. Mentors represented all five academic divisions and both campuses. The projects spanned a breadth of project types (Figure 2). Projects classified by the faculty mentors as health services oriented (n=12; 24%), clinical (n=12; 24%), translational (n=9; 18%), and basic/preclinical (n=8; 16%) were most common, with numerous projects classified by multiple domains.

Thirty-eight (76%) students presented an abstract derived from their project at a national pharmacy or biomedical specialty meeting; eight students presented research results at two distinct meetings. In the first cohort, 10 (40%) students published an original research manuscript within one year of graduation. Six additional students (24%) had either submitted a manuscript or were preparing a manuscript with an intent to submit within the calendar year. Overall, this included 10 (40%) students as first author (eight papers published, two submitted), with three students also publishing a second manuscript derived from their project as a coauthor.^19-30 An additional six (24%) students were co-author on a published (n=1), submitted (n=2), or in preparation (n=3) manuscript.³¹

Feedback from the inaugural cohort exit survey revealed very positive student perceptions regarding the value and satisfaction of the overall Research and Scholarship in Pharmacy pathway (Table 1). Of the 24 students who completed the exit survey (96% response rate), 23 (96%) were satisfied or very satisfied with their research experience and 21 (88%) were satisfied or very satisfied with their faculty mentor. In addition, 96%-100% agreed or strongly agreed that this experience helped them gain a deeper understanding of how to conduct research, develop project management skills, and increase their confidence to contribute to a research project team. Twenty-three (96%) agreed or strongly agreed that the deliverable assignments helped develop written and oral communication skills, and 92%-100% agreed or strongly agreed that the assignment accomplished its intended learning objective.

Additionally, 20 (83%) believed their experience helped differentiate them from other candidates during postgraduation interviews and 20 (83%) had their faculty mentor serve as a reference. Twenty-one (88%) students indicated plans to engage in research and scholarship activities in the future, and only one (4%) student stated they would not recommend the Research and Scholarship in Pharmacy pathway to future students.

In response to the open-response question, “What is the primary value that the Research and Scholarship in Pharmacy pathway adds to the curriculum?” several themes emerged. These included that the pathway: provided a mechanism for students to understand and execute the research process in a structured and logical manner, allowed students to get involved with research at an early point in the curriculum, helped students develop close relationships with faculty members, and provided students with an opportunity to differentiate themselves. Themes found from the open-response question, “What worked well in the RASP pathway that you would not change?” included: useful and purposeful deliverable assignments, the structure and format of the pathway, and the faculty mentorship and support provided. The open-response question, “What would have made RASP a better experience for you as a student?” yielded themes pertaining to starting the pathway earlier in the curriculum, time constraints, and incorporating more data analysis focused sessions into the courses.

DISCUSSION

Research skills play a critical role in helping the pharmacy profession keep pace with a dynamic health care environment. However, numerous barriers prevent the availability of quality research training opportunities in PharmD curricula.^1,32 We described the design and initial impact of a newly implemented research and scholarship training pathway for PharmD students. Our initial results demonstrated high rates of student satisfaction with their experience, project and mentor, and skill development; low rates of student attrition; and high rates of student scholarly output in the form of national meeting abstracts and original research manuscripts. Taken together, these early findings suggest that the Research and Scholarship in Pharmacy pathway feasibly and effectively integrated a longitudinal, in-depth, and mentored research and scholarship training experience within a PharmD curriculum that fosters skill development, facilitates scholarly productivity, and provides a differentiating experience for students.

Numerous benefits have been associated with pharmacy student participation in research, including collaboration, teamwork, and communication skill development.^{1,8-10,16,33-35} As evidenced by student perceptions, the Research and Scholarship in Pharmacy pathway appeared to promote student learning and skill development in these areas. In particular, the written project proposal, project proposal presentation, poster presentation, and original research manuscript assignments collectively facilitated development of written and oral communication skills. This included helping students achieve a deeper understanding of their project by fostering their ability to communicate the project’s rationale, design, and limitations, and to ultimately present and interpret the results. Moreover, these scholarly deliverables provided a strong foundation for students, in collaboration with their faculty mentor, to pursue submission of abstracts for presentation at national meetings and original research manuscripts for publication in peer-reviewed journals. These findings align with previous reports of communication skill development outcomes within a research curriculum.^8-10,33,34

Dissemination of research results in the form of presentations and publications was common. Seventy-six percent of students in the first two cohorts presented their results at a national meeting. Forty percent of students in the first cohort had an original research manuscript published^19-26,30 within one year of graduation, and an additional 24% had submitted a manuscript or prepared a manuscript for submission at the time of the study. This publication rate is higher than that found in a systematic review of 13 other studies of pharmacy student research experiences, which reported average publication rates at less than 10%.¹⁶ Further, students described their project and scholarly skills development as a differentiator during interviews, with most students securing a recommendation from their mentor. Data collection over multiple cohorts will be needed to assess sustainability and impact on postgraduation plans and determine whether the project type influences student satisfaction and publication rates.

Long-term benefits may be derived from incorporating a meaningful research experience into students’ training. Research experiences provide students with opportunities for professional growth and differentiation as candidates for increasingly competitive postgraduate training and employment positions. Participation may also foster students’ commitment to incorporating research activities into their career and ability to serve as a research mentor for future trainees.^1,36 Research training pathways provide an opportunity to realize these benefits.

There are several reasons why the pathway may have been effective at our institution. First, Research and Scholarship in Pharmacy was specifically designed to address barriers noted in previous studies about student research experiences.^1,37 Notably, the pathway was created as a longitudinal, immersion experience built around a faculty-mentored research project that provided elective course credit to protect student time, establish structure and expectations, and provide teaching credit for faculty. Accordingly, students and mentors opted into participating, mutually elected to connect on a project, and mutually agreed to continue after each semester. Students selected projects of interest to them from a breadth of research areas and had time for research and scholarship activities. Finally, students were incentivized by the offer of a path to graduate with an honors designation and participate in a differentiating experience. Collectively, these attributes likely increased the probability of a valuable and productive student-faculty experience and decreased attrition. Considering lessons learned from other programs, sustainability will require careful attention to faculty workload and student time.¹ Additionally, the impact of the honors designation on student desire to enroll in the pathway remains unclear.

Effective execution of a longitudinal research and scholarship pathway required support from leadership and a substantial time commitment from faculty members. As a research-intensive university, the pathway at our institution seeks to integrate students into existing faculty research programs. Thus, faculty resources are necessary to support the availability and execution of research projects. While some in-class instruction on general research methodology and scholarly dissemination are provided to all students, the breadth of project types and limited time precludes formal instruction on research methodology. As a result, it is the responsibility of the faculty mentor and their research team to provide project-specific training, guidance, and supervision. In addition to the benefits conferred by receiving student assistance to advance their research agenda, teaching credit is provided to incentive faculty mentor participation. Programs must provide leadership, infrastructure, and incentives for faculty members and students to ensure that research experiences embody appropriate quality, depth, and focus.

A pathway governance committee, with staff program coordination support, has proven to be critical to the success of the program by providing administrative oversight and serving as an objective third party to assess the course deliverables. Teaching and service credit is also provided to committee members to recognize the value of their contributions and serve as an incentive to engage in the pathway. Studies have repeatedly demonstrated that an oversight committee is critical to the success of trainee research programs.^8,36-38 In addition to providing structure, oversight committees provide independent accountability monitoring for the trainee and input to the research process.^8,36-38 Along with providing the above features, the oversight committee also offers support to faculty mentors. The pathway was designed to provide faculty mentors with autonomy to mentor students on their research project while communicating programmatic requirements and timelines and delivering independent feedback to students on their scholarly assignments. The balance between programmatic oversight and faculty mentor autonomy to promote student skill development and productivity is a challenge that remains a work in progress.

Despite providing course credit, the constraints that participation in the pathway places on student time has remained a challenge. In our school’s new curriculum, there are two semesters (spring year two and fall year three) where students are split into two cohorts and spend eight weeks in the classroom and eight weeks in clinical immersion experiences. These two semesters overlap with the first and second semester of the pathway and span a 12-week gap during the summer between years two and three in the curriculum (Figure 1). These disruptions in the schedule have created new barriers to students dedicating time to research activities. While students are strongly encouraged to stay connected and continue working on their project during these off-campus periods, and to use this time to obtain IRB approvals, without protected research time, other priorities inevitably take precedent. These challenges can slow or disrupt students’ ability to advance their research project. Certain project types, most notably laboratory-based projects, are particularly susceptible to these scheduling challenges. Indeed, both faculty members and students have reported the block schedule makes it very challenging to effectively develop experimental skills and complete laboratory-based projects. While offering a wide breadth of research project opportunities has been central to the initial success of the pathway, most students have pursued health services and clinical research projects. Given the large number of exceptional laboratory-based research mentors at our institution, identifying mechanisms to integrate students more efficiently into laboratory-based projects will be a critical focus in the future.

While the data presented herein provides insight into a recently implemented research and scholarship training pathway, there are several limitations worth noting. Although the high degree of student satisfaction and publication rates in the first cohort are very promising, whether these positive results can be sustained in future cohorts or maintained if the program increases in size remains unknown. Additionally, the impact of student contributions to co-authored abstracts and spin-off projects were not quantified. Further, our survey and analysis focused on student self-reported perceptions and opinions. A formal assessment of faculty mentor satisfaction was not completed, which is a limitation. Finally, the data presented represent an experience at a single institution. Whether the program could be replicated with similar success at other institutions is unknown. Despite these limitations, the initial results suggest that the pathway feasibly and effectively integrates a research and scholarship training experience into a PharmD curriculum. Future research will include conducting a faculty mentor satisfaction assessment at the completion of each student cohort and performing focus group sessions with graduating students and alumni to assess their experience and its impact on their career path. Future research is also needed to determine whether the early impacts and value of student research training programs translates to longer term scholarly success for participants.

CONCLUSION

The Research and Scholarship in Pharmacy pathway at the UNC Eshelman School of Pharmacy provided a pragmatic and structured mechanism to integrate a longitudinal, faculty-mentored research and scholarship training experience within a PharmD curriculum that promotes skill development, creates scholarship opportunities, and provides a differentiating experience for students. Initial implementation has demonstrated early success in the completion of student research projects with low attrition, high rates of student scholarship in the form of abstracts and publications, and high perceived value of the experience on skill development. Although the sustainability of these results and the potential to replicate this program at peer institutions and within other health professional training programs remains unknown, our results offer the potential to inform program development efforts and stimulate future research.