The Current Landscape of College-Sponsored Postgraduate Teaching and Learning Curriculum Programs

INTRODUCTION

The availability of postgraduate teaching and learning curriculum (TLC) programs (also known as teaching certificate programs) for pharmacists pursuing postgraduate training has significantly increased over the last 15 years.^1-11 The increase in availability of TLC programs has been fueled by the need for highly trained clinical pharmacy practitioners who can also serve as academicians.^12,13 The American Society of Health-System Pharmacists (ASHP) accreditation standards for postgraduate year one (PGY1) pharmacy residency programs require that residents achieve competencies in teaching, education, and dissemination of knowledge.¹⁴ In addition, some residency programs may offer an additional and more detailed elective competency in teaching and learning.¹⁵ Teaching and learning curriculum programs can be used by residency program directors to create resident learning experiences that help residents achieve these teaching-related competencies.

Although many postgraduate residency programs partner with schools of pharmacy to offer TLC programs, there are currently no specific curriculum standardization guidelines or external validation processes to evaluate the rigor or success of these programs. In 2014, the American Association of Colleges of Pharmacy (AACP) and ASHP published recommendations and a list of 12 best practices for the development, conduct, and evaluation of TLC programs.¹⁶ The American College of Clinical Pharmacy (ACCP) has also published similar recommendations on how to develop an effective teaching program.¹⁷

Although recommendations from national pharmacy organizations are available, the extent to which these recommendations have been implemented by TLC programs remains unknown. The Student and Resident Engagement Task Force of the AACP’s Pharmacy Practice Section was charged with exploring the TLC program landscape to determine the degree to which these programs have implemented best practice recommendations. Therefore, the research aims of this project were: to describe the current availability of college-sponsored TLC programs in the United States and to evaluate the extent of their adoption of published best practice recommendations.

METHODS

A survey instrument was drafted by the investigators using the 12 recommendations for TLC programs published by AACP and ASHP and the six guidelines for teaching certificate programs recommended by ACCP.^16,17 The draft survey instrument was reviewed and pretested by four faculty members who had experience with TLC programs, but were not affiliated with the study. The final survey instrument consisted of 28 items, including demographic questions about the program (eg, public/private, years the program has been offered, size of program) and specific details regarding the content, delivery, and evaluation of the program. The first item asked if the college or school offered a TLC program. Respondents without a TLC program were routed directly to the end of the survey. Respondents with a TLC program were directed to complete the remaining 27 questions. An electronic invitation to complete the survey instrument was sent to one representative of the AACP House of Delegates from each of the 137 accredited US colleges and schools of pharmacy listed on the AACP website in January 2018. Delegate names and contact information was obtained from a publicly available list posted on the AACP website. The electronic invitation requested that the survey be forwarded to the most appropriate representative with the necessary knowledge regarding the school’s TLC program. Qualtrics (Provo, UT) was used to electronically distribute the survey instrument. As an incentive, respondents were sent a $10 Starbucks gift card for their participation. To maximize the response rate, multiple reminders were sent to nonrespondents over a period of three months. Finally, for any programs that had not responded, we identified potential contacts via an online search of the college or school’s website and contacted the person(s) directly via email. The project was reviewed by investigational review boards at each investigator’s institution and determined to be exempt from oversight.

RESULTS

Of 137 institutions contacted, 88 responded, resulting in an overall response rate of 64%. Of the 88 institutions that responded, 68 (77%) reported having a TLC program. Four of these institutions did not complete the entire survey, so they were removed. Five of the institutions offered TLC programs in collaboration with one or more other institutions. Therefore, the three institutions reporting duplicate programs due to collaboration with other schools were removed. After seven schools were removed, 61 unique TLC programs were left and included in the final analysis.

The general characteristics of the TLC programs for the responding institutions are shown in Table 1. Use of published best practice recommendations was reported by 75% of TLC programs, with 10% indicating that they were not aware that there were published recommendations. Approximately 75% of programs reported that faculty members received some type of incentive from their institution for participating in the delivery of the TLC program. However, the majority of institutions reported that they did not receive compensation from residency programs for providing the TLC program.

The content, delivery, and evaluation of the responding TLC programs are described in Table 2, stratified by each of the published recommendations to guide best practices for TLC programs. More than 80% of programs expose residents to a variety of core topics, deliver content using various teaching styles, require participants to gain teaching experience and develop and maintain a teaching portfolio, evaluate participants’ performance, and require that participants have a teaching mentor. The teaching activities required by the TLC programs are listed in Table 3. The most common requirements were attending and participating in TLC seminars, facilitating small-group student discussions, providing didactic lectures, and developing multiple-choice questions. Approximately 61% of TLC programs required participants to either serve as a preceptor or co-preceptor for advanced pharmacy practice experience (APPE) students or develop learning assessments.

The various methods used for participant evaluation and the frequency of these evaluations are presented in Table 4. Although formal evaluation of resident teaching was routinely done by preceptors, faculty members, and mentors, fewer than half of the programs reported formal evaluation of resident teaching being done by students or peers. The majority of programs required the residents to develop and maintain a teaching portfolio; however, only 44% formally evaluated these portfolios. In addition, only 40% of programs provided summative evaluations at regularly scheduled intervals. About 28% of TLC programs did not undergo programmatic assessment and continuous quality improvement on at least a semi-regular basis. In addition, only half of the programs formally evaluated the faculty and mentors involved in delivery of the TLC program.

DISCUSSION

Our analysis evaluated TLC program content at US schools and colleges of pharmacy relative to recommendations set forward by the AACP/ASHP and the ACCP. While 90% of responding schools participating in the delivery of TLC programs were aware of published best practice guidelines and were following the majority of recommendations, we found considerable variability in the delivery and conduct of TLC programs across the country. One potential reason for the variability among programs is the lack of a formal external validation process for TLC programs. Until the implementation of such a formal external validation or accreditation process, variability and inconsistencies across programs are likely to persist.

Our findings are generally consistent with a similar evaluation conducted by Islam and colleagues¹⁸ who surveyed the pharmacy practice department chairs of 40 colleges and schools of pharmacy and conducted a search of 132 pharmacy school websites in the United States. The authors identified 69 TLC programs in the United States and found similar compliance with best practice recommendations for discussion of minimum core topics (100%), use of different teaching styles (100%), experience with a variety of educational modalities (100%), evaluation at regular intervals (74%), required teaching portfolio (74%), and incorporation of a system for ongoing programmatic improvement (74%). While we found that 84% of TLC programs required participants to have a specific teaching mentor, Islam and colleagues reported that only 34% of TLC programs had this requirement. The reason for this discrepancy (ie, 84% vs 34%) could be because of differences in data collection methods as we relied on completed survey data and did not search for the information on program websites. The teaching mentor requirement may not have been explicitly stated on the program websites that Islam and colleagues searched or may have been a recent addition to the program. Because one of our primary research aims was to determine the extent to which colleges are adopting published best practices for TLC programs, our survey instrument captured additional information that was not reported by Islam and colleagues. Some examples included information on whether programs offered the delivery of standardized vs customized content commensurate to the learning level of the participant, and information regarding whether the college had adequate personnel and institutional resources to support the TLC program.

Although pharmacy schools’ adoption of published best practices was high in most areas, we identified notable areas for improvement in TLC programs, including incorporation of a system of assessment to enhance ongoing programmatic improvement. More than a quarter of the TLC programs surveyed did not undergo at least semi-annual programmatic assessment. The first step towards continuous quality improvement of any program is to periodically conduct self-assessments. Thus, TLC programs should develop structured programmatic assessments to fuel ongoing quality improvement and to ensure their goals are being achieved. Data from these assessments could also be used to document the value of the program. Another area for continued improvement is conducting formal evaluation of participants’ performance at regular intervals. Although 82% of programs reported conducting some type of formal evaluation of participants, only 40% conducted summative evaluations more than one time prior to residents completing the program. Although 90% of programs required residents to prepare a teaching portfolio, only 44% of programs reported formally evaluating these portfolios. Robust evaluation and feedback on the teaching portfolio is highly important as this tends to be the culmination of the learner’s TLC program journey.

Our survey also identified inconsistencies in the teaching activities required of participants. Approximately 80% or more of surveyed programs required participants to deliver lectures, facilitate small-group student learning, and attend TLC seminars. However, only 61% of programs reported requiring residents to precept APPE students on rotations. A similar finding was noted by Islam and colleagues as they reported that only 74% of programs required residents to have precepting experiences. From this data, it appears that a quarter to a third of TLC programs do not require their residents to gain precepting experience. Precepting is an experience specifically recommended by both the ACCP and AACP/ASHP best practice documents.^16,17 Although many programs required residents to have lecture experiences, only a small subset of residency graduates actually pursue a career in academia. A survey of pharmacy residents found that only 7% accepted a faculty position upon completion of residency training.¹⁹ Pharmacy residents are therefore more likely to pursue careers as clinical practitioners who primarily serve as preceptors. Accordingly, institutions offering TLC programs should place more emphasis on exposing residents to precepting opportunities as this is the most likely teaching experience they will pursue in their careers after postgraduate training. Finally, because a formal validation process for TLC programs does not yet exist, the need to provide detailed descriptions of the breadth and depth of TLC program requirements online and/or in printed recruitment materials is paramount. Without having publicly available detailed descriptions, the Academy and employers have no way of judging the robustness of TLC programs. While about 75% of respondents reported that they are transparent about their TLC program expectations, that leaves a quarter of programs that are not as transparent. In the absence of official standards for TLC programs, the Academy should strive for 100% transparency regarding the depth and breadth of TLC programs.

Making improvements to TLC programs may prove challenging for many programs. The majority of institutions surveyed were not compensated for providing a TLC program. Most schools and colleges of pharmacy acknowledged faculty members who participate in TLC programs by noting these efforts during the faculty member’s annual review and viewing this work favorably in terms of the faculty member’s eligibility for promotion and/or tenure. However, there was minimal, if any, financial incentive for faculty members to participate. Although a culture shift toward providing financial compensation is unlikely, schools and colleges of pharmacy should develop clear criteria, incentives, and recognition for faculty members to serve as mentors in TLC programs. Also, there was a substantial range in the number of residency programs and participants included within each TLC program. As these numbers continue to grow, adequate resources may not be available to change current program offerings.

Our analysis was limited to only college-sponsored TLC programs and data from TLC programs not affiliated with a pharmacy school were not included. Limiting the sample to programs facilitated through a school or college of pharmacy was intentional as the published recommendations for TLC programs encourage this as best practice. The low response rate to our survey limits the generalizability of our findings to all schools and colleges of pharmacy. However, we attempted to combat the low response rate by sending multiple e-mail reminders and providing a small incentive for participation. Nevertheless, there is a possibility that the survey instrument was not forwarded to the most appropriate person, ie, the faculty member most familiar with the institution’s TLC program. While this survey was designed to assess adoption of published best practices, we recognize that TLC programs may be conducting other activities of value that are not captured by best practice recommendations. Also, some residency programs may include additional teaching requirements and activities that were not collected by our survey instrument. Future research should determine what value academic institutions place on an applicant’s completion of a TLC program during the faculty recruitment process.

CONCLUSION

Most institutions offering TLC programs are aware of published best practice guidelines and are following the majority of recommendations. However, considerable variability exists in the delivery and conduct of these programs across the country. The Academy should consider providing resources and tools for institutions to create robust programmatic offerings as part of existing and developing TLC programs. Finally, the inconsistencies we identified reaffirm the need for a formal external validation process for TLC programs as previously recommended in best practices published by AACP and ASHP.