Scholarly Contributions of Required Senior Research Projects in a Doctor of Pharmacy Curriculum

INTRODUCTION

To successfully practice in diverse and complex environments, pharmacists must be able to identify problems and to develop, test, and implement viable solutions. Research-related knowledge, skills, and experiences foster the development of critical-thinking and problem-solving skills and facilitate the application of science and evidence-based medicine to patient care and health care environments.¹ The American Association of Colleges of Pharmacy’s 2011-2012 Argus Commission noted that student research skills help develop inquisitive pharmacists with attributes required for scholarly clinical practice.² Current and future accreditation standards for the doctor of pharmacy (PharmD) program and the Center for Advancement of Pharmacy Education (CAPE) Educational Outcomes reflect the need for research-related knowledge and skills development for professional practice.^3-5 Professional organizations also advocate for including research-related coursework and experiences in the PharmD curriculum.^6-8 Recognition of the value of research-related knowledge and skills for pharmacists extends beyond the United States to international educational institutions and professional organizations.⁹

Multiple studies explore the prevalence of and perspectives associated with pharmacy and other health profession students’ participation in research.^9-19 In general, students and faculty members agree that student participation in a required or elective research project is valuable and enhances students’ academic experiences. Kim and colleagues surveyed students completing a required senior research project and found that 86% of those pursuing postgraduate training agreed or strongly agreed that research experience in pharmacy school made them more competitive for these opportunities.¹¹ Internal data from more than 300 alumni of our institution found that graduates from the more research-intensive curricular pathways reported significantly higher rates of engaging in scholarly activities since graduation.²⁰ Nevertheless, more than 65% of graduates from the least research-intensive pathway had also engaged in some sort of scholarly or research-related activity since graduation. Results from Sheaffer and colleagues’ study of graduating students, postgraduate trainees, board-certified pharmacists, and newer faculty members suggested student participation in research may stimulate and promote interest in academic pharmacy.²¹ Barriers to including research experience in the curriculum, however, are negative faculty perceptions of the time and resources necessary to implement and provide the required coursework and experiences.^9,10,13 Few studies have documented dissemination, including peer-reviewed publications, of pharmacy student research projects.^10,12,22

Since 2002, students at the University of California, San Francisco (UCSF) School of Pharmacy have selected from 1 of 3 curricular areas of emphasis, or pathways: pharmaceutical care, health services and policy research, and pharmaceutical sciences. While the pathways vary in the required amount of direct patient care vs research-related experience, all 3 require completion of a capstone senior research project. The project takes place over the last 12 months of the program under the supervision of a faculty preceptor. Previous studies examined preceptor perceptions of and dissemination-related outcomes associated with pharmaceutical care pathway projects completed from 2002-2007¹⁰ and health services and pharmaceutical sciences pathway projects completed between 2002-2011.²² The purpose of this study was to investigate dissemination-related outcomes for and preceptor perceptions of pharmaceutical care projects conducted during 2008-2011. When combined with previous research, these results allow for a complete classification of pharmaceutical care projects spanning a 10-year period and for comparison of project outcomes across all 3 pathways since the inception of the required senior research project.

METHODS

The pharmaceutical care pathway project provides 3 experiential units (approximately 120 hours). Students work alone or in groups of up to 4 (depending on the project scope).¹⁰ The majority of projects are group-based.¹¹ Group projects are typically precepted by more than one faculty member (eg, one paid and one or more volunteer faculty members), with one preceptor assuming a primary or lead role. Projects are completed concurrently with advanced pharmacy practice experiences (APPEs) over the final 12 months in the PharmD curriculum. In contrast, health services policy research and pharmaceutical sciences pathway projects account for 12-20 experiential units (480-800 hours). Students in these pathways work alone and have assigned, protected blocks of time devoted solely to conducting research. Students present their research to fellow students and faculty members and submit a final written manuscript. While dissemination of results is not a course requirement for any pathway, it is highly encouraged.

A modified survey was designed based on ones used in previous studies and administered electronically using Qualtrics online survey software (Qualtrics, LLC; Provo, UT) in September 2013.^10,22 Subjects were eligible for this study if they were identified as having served as the primary preceptor for one or more pharmaceutical care pathway student research projects for the graduating classes of 2008-2011 and if they had a valid e-mail address. Preceptors were categorized as paid or volunteer faculty members based on their status in the school at the time the project was completed. Participation was solicited via an e-mail invitation containing information about the study and a survey link. Preceptors received one e-mail message for each unique student project they had precepted. Prior to survey launch, experiential program directors from the school’s 6 geographical APPE regions contacted their local preceptors to inform them of the study and encourage participation. E-mail reminders were sent out to participants who had not completed their survey 2 and 4 weeks later. Data collection ceased 8 weeks after initial survey invitations were sent.

The extent of dissemination-related outcomes for project results were classified as follows: (1) presented at an institutional forum (eg, a pharmacy and therapeutics committee meeting, departmental quality assurance meeting); (2) delivered as a poster at a professional meeting; (3) delivered as an oral presentation (eg, panel, platform) at a professional meeting; or (4) submitted for publication. Response options for these 4 items were yes or no. For the purposes of this study, one project could have been disseminated via 1 to 4 different channels. Results for each of the 4 channels were tabulated individually.

Poster and oral presentations were sub-categorized by professional meeting venue as local, state, national, and international. For each response option, respondents could select all that applied. A project could therefore have been presented as a poster and/or oral presentation more than once (eg, an encore poster presentation at a state professional meeting for a poster originally disseminated at a national meeting). For purposes of this study, a yes response to poster or oral presentation was counted only once for that category of dissemination, regardless of the number of dissemination venues indicated.

Studies submitted for publication were sub-categorized as: (1) submitted but not accepted, no plans to resubmit; (2) submitted but not accepted, plans to resubmit; (3) submitted, reviewed, returned for revision, currently under revision or revised and resubmitted; (4) accepted, manuscript in press; or (5) accepted and published. Journal name and citations were requested for submitted and accepted or published papers. Dissemination for projects reported as accepted for publication was validated through a search of published manuscripts (Appendix 1).

Preceptors provided feedback on their precepting experience and perceptions of the project requirement by responding to a series of statements. Three statements focused on preceptor perceptions of the value of the project to themselves and their institution. Two statements solicited perceptions related to student preparedness to conduct research and the value of the project as a student learning experience. The final statement explored the respondent’s willingness to serve as a project preceptor in the future. Response options included 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree, and not applicable.

Statistical analyses were conducted using Predictive Analytics Software 17.0 (IBM, Armonk, NY). Data from previous studies of pharmaceutical care pathway projects conducted between 2002-2007 and projects from health services policy research and pharmaceutical sciences pathways completed between 2002-2011 were used for additional analyses.^10,22 Dissemination-related outcomes between different time periods and pathways were compared using the Fisher exact test. For available data, differences in Likert-type scale responses between different time periods and pathways were compared using the nonparametric Mann-Whitney U test.

All study measures and procedures were approved by the UCSF Committee on Human Research.

RESULTS

From 2008 to 2011, 129 research projects were completed by 396 pharmaceutical care pathway graduates. Of these, 123 projects were linked to 59 research preceptors with a valid e-mail address. Survey responses were received for 111 projects for an overall response rate of 90.2%. Twenty-eight full-time (paid) faculty members responded for 81 projects and 23 volunteer faculty members responded for 30 projects. The volunteer faculty respondents represented 14 different educational affiliated organizations or sites across California. Table 1 summarizes the types of projects completed. Similar to pharmaceutical care projects described previously,¹⁰ the majority of projects from 2008-2011 (n=103, 92.8%) were nonexperimental or observational in nature.

Of the 111 projects, 28 (25.2%) were not disseminated. Of these, 6 projects (5.4%) had been submitted but not accepted for publication at the time of the study. Twenty-seven projects (24.3%) were only internally disseminated (eg, within an institutional forum such as at a school, university, or host institutional meeting). Fifty-six projects (50.4%) were disseminated externally. Twelve (10.8%) projects went straight to publication. Twenty (18.0%) projects presented as a poster were also published. Table 2 summarizes the external dissemination outcomes of pharmaceutical care projects completed within the study timeframe.

Table 3 summarizes the dissemination outcomes of pharmaceutical care projects completed from 2002 to 2011. Results of projects completed between 2008 and 2011 were disseminated as a presentation at an institutional venue more often than those completed in the first 6 years (2002-2007) of the pathway (61.3% vs 47.3%, p=0.02). Compared to earlier projects, those completed from 2008 to 2011 were also significantly more likely to be externally disseminated as posters at a national or international meeting (30.6% vs 19.6%, respectively, p=0.03). No significant difference in the number of oral presentations at a national or international professional meeting between the 2 cohorts was observed.

For 2008-2011 projects, 32 (28.8%) were published in professional, peer-reviewed journals. Three projects were combined into a single publication, while one project yielded 2 unique publications, resulting in a total of 31 publications (Appendix 1). A student served as lead author on 11 publications while students were not listed as authors on 7 publications. Of these latter publications, student investigators were acknowledged for their contributions in 5 papers. More projects completed between 2008 and 2011 than those completed between 2002 and 2007 were submitted for publication (42.3% vs 10.7%, respectively, p<0.001) and published (28.8% vs 5.3%, respectively, p<0.001).

Dissemination-related outcomes for 10 years of projects from the less research-intensive pathway (pharmaceutical care) and more research-intensive pathways (health services policy research and pharmaceutical sciences combined) are summarized in Table 4. When compared to projects from the research-intensive pathways, pharmaceutical care project results were less likely than research-intensive pathway projects to be presented as a poster (23.3% vs 37.7%, p=0.003) or platform presentation (1.8% vs 18.9%, p=0.002) at a national or international meeting.

The majority (88.3%) of preceptors agreed that precepting the project was valuable to them professionally, and more than 3 quarters agreed that the results of the project were valuable to their own or another institution (Table 5). All but one preceptor (50 of 51 unique respondents) felt the research project provided a valuable learning experience for students. The majority (73.8%) of respondents indicated students were adequately prepared to conduct the research project. Nearly 95% would continue to precept student research projects.

When results for identical items were compared across the 2 pharmaceutical care pathway cohorts, significantly more 2008-2011 respondents than 2002-2007 respondents agreed that the project provided a valuable learning experience for students (mean rank 192.8 vs 155.7, p<0.001). No significant differences were noted in preceptors’ views on student preparedness for the research project for earlier vs later pharmaceutical care projects or across the different curricular pathways.

DISCUSSION

This study determined dissemination outcomes and preceptor perceptions for required senior research projects from the least research-intensive track of the UCSF PharmD curriculum. In the initial 10 years of experience with a required student research project, approximately 1 in 3 projects was disseminated as a poster or oral presentation at a professional meeting and 1 in 10 was published in a peer-reviewed professional journal. These findings are comparable to those from other dissemination outcomes-related studies of student and postgraduate research experiences in health professions.^22-24 Our findings suggest research conducted by student pharmacists can transcend practical learning experience into meaningful contribution to the knowledge shaping science and practice.

A significant difference in dissemination rates between the pharmaceutical care and health services policy research and pharmaceutical sciences curricular pathways is not surprising. The latter 2 research-intensive pathways provide additional coursework specific to designing and conducting research. Almost all of these projects are conducted with a one-to-one student to preceptor ratio. Unlike their pharmaceutical care pathway classmates, students in the research-intensive pathways have large blocks of time devoted solely to research-related experience, during which they are not concurrently scheduled for APPEs. The additional time and its protected nature may also foster one-on-one faculty mentorship to support student interest in dissemination and meaningful contributions towards authorship of papers presented at professional meetings and submitted to peer-reviewed professional publications.

Despite the lower dissemination rates compared to the research-intensive pathways, the number of posters and publications resulting from pharmaceutical care projects increased significantly over time. This finding may reflect ongoing preceptor experience in the supervision of capstone research projects and a greater appreciation for the types and scope of projects that can be completed and disseminated by a busy senior pharmacy student. Increased research experience itself, derived from precepting student projects over the years, may also positively influence successful dissemination of project results by both paid and volunteer faculty members.

Based on our results and others, perceptions of the value of student research and the precepting experience itself have been generally favorable among faculty members.^10,13,22,25 Despite a difference among pathway curricula, there was no significant difference in preceptor perceptions of student preparedness to conduct research. Increased agreement over time in preceptor perceptions of the value of the learning experience may reflect growing faculty sentiment of the importance of experiential research experience for graduates and/or their satisfaction with participating in this process. Preceptors may also benefit both professionally and personally from working with students in such a capacity, from extending their own research activities to assisting students in their transformation into scholarly practitioners. Similar to Kim and colleagues’ report of student project perceptions,¹¹ preceptors also expressed that the projects provided a mechanism for graduates to differentiate themselves from other applicants for postgraduate training and employment opportunities.

The results of our combined studies provide objective data useful for curricular assessment, quality assurance, and improvement initiatives. In spring 2014, the school launched a new annual survey of project preceptors for outcomes of pathway projects completed in the previous calendar year. In combination with related literature to date, our findings are also informing current redesign of our PharmD curriculum. Moving forward, education in the health professions must prepare practitioners to practice in complex systems and to problem solve by working successfully in teams and across disciplines. The pathway research project experiences, which essentially provide a structured APPE in research, provide our PharmD program with a mechanism for instilling attitudes, applying previously learned didactic knowledge, and gaining practical hands-on experience conducting research.

Our findings, which compared results across different curricular pathways within our own program, also highlight how different models for the scope and design of a research requirement can achieve the intended/desired outcomes. While dissemination rates were significantly higher for the 2 research-intensive pathways, dissemination was not designed to be a course requirement nor an outcome measurement for student performance. As our school embarks on curricular redesign, faculty members may reflect on concerns for professional and personal time constraints, shrinking resources in education, and successes to date based on experiences in order to best determine how the research-related curriculum will evolve over the next decade.

No published studies to date have explored postgraduate outcomes associated with student participation in research. Future research efforts may want to focus on the impact of research experiences on career choices and whether any correlations may exist between engagement in scholarly activities within a position or practice setting and past educational research experiences.

Our results are not without limitations. The retrospective design and the fact that some faculty members precepted more than one project over the years may increase the risk for respondent recall bias regarding dissemination outcomes and their perceptions of and experiences with individual projects. Based on the response rates for paid and volunteer faculty members, paid faculty perspectives regarding the precepting experience could be overrepresented. While poster and oral presentation results were not verified, publication citations were validated by searching for and obtaining the articles. Actual publication rates may have been underreported, as some respondents indicated project manuscripts were under review during the study period. Because faculty members solicited for this study may have known the investigators, social desirability bias may have skewed preceptor perception responses and results. Nevertheless, our findings remain consistent with those from other studies conducted to date on students and postgraduate trainees in pharmacy and other health professions.

CONCLUSION

This study summarized dissemination outcomes related to and faculty perceptions of senior research projects conducted by fourth-year students who graduated from the pharmaceutical care pathway between 2008-2011. The results of this study compared different cohorts within the same curricular pathway and across 3 different curricular pathways over a 10-year time frame. While these projects lacked a dissemination requirement, external dissemination was observed and rates increased significantly over time. Faculty perceptions of the projects themselves and the precepting experience were positive. The results of this study are informing current curricular revision of the PharmD program. They may also provide a curricular example of an experiential research course and related outcomes assessment for other programs seeking to increase student knowledge, skills, and actual experiences conducting research. Future studies may want to explore the impact of student research experience on graduate choice of career path and/or practice setting.