Using the Pharmacist Interaction Tracking Tool for Capturing Student-Patient Interactions in Direct and Simulated Patient Care Activities

INTRODUCTION

The 2013 Center for the Advancement of Pharmacy Education (CAPE) Educational Outcomes outline as part of the “essential premise” that pharmacists must function collaboratively as part of an interprofessional team, advocate for patients, provide care for diverse populations, contribute to the health and wellness of individuals, and provide patient education.¹ Domain 2 focuses on “essentials for practice and care,” which include collecting and interpreting patient-related information, prioritizing patient needs, formulating and implementing evidence-based care plans, and documenting patient care activities.¹ Embedded throughout the Accreditation Council for Pharmacy Education (ACPE) Standards 2016 are specific expectations for assessment and evaluation.² For example, Standard 24 states that “colleges and schools should develop resources and implement a plan to assess attainment of educational outcomes to ensure that graduates are prepared to enter practice.”

At the University of Pittsburgh School of Pharmacy, several curricular outcomes focus on development of student skills for direct patient care, including patient assessment, medication therapy management, and communication. However, assessment of these skills occurs within therapeutic courses or experiential learning activities and are not able to be collated to gain an understanding of students overall exposure to patient care activities through their curricular career. Developing a method to collect data from student patient care experiences in a format that allows for assessment and evaluation would benefit the documentation of meeting curricular outcomes and Domain 2 of the CAPE Outcomes.

Paper and electronic methods for documentation of patient care activities within pharmacy and medical schools are described in the literature, including using handheld devices or scanable paper forms, extracting data from existing medical center documentation systems, and using commercially available products.^3-9 Several studies show the impact or justification of student contributions to patient care in pharmacy student clerkships.^3,6,7,9 However, these students only collected data during clerkships and did not capture other learning activities within the curriculum. Two studies evaluated types of systems, commercial or custom, for collecting patient care outcomes and found that medical record systems are often difficult to extract data from in a reportable format and commercially available products may be cost prohibitive.^8,9

Two studies evaluated documentation tools to enhance student learning and assess the degree to which curricular outcomes were met.^4,5 Kurth and colleagues described the medical school process of using a “student log” to collect information regarding the type of patient care experiences on ambulatory clerkships.⁴ The data were used to ensure students were gaining the appropriate diversity of patient experiences and reassess curricular needs. The authors concluded use of an electronic format allowed for a higher number of documented encounters and ease of analyzing data for use in curricular assessment. Mackinnon described a scannable paper tool to collect the type of interventions and recommendations pharmacy students made during practice expereinces.⁵ A check box design for the form was employed to promote ease of use. Although it was a standardized way to collect data to assess student activities, the paper format proved cumbersome. An electronic format to collect patient care activities in all aspects of the curriculum and specific for pharmacy students would be beneficial.

The ACPE recommends a variety of teaching methods be incorporated across the curriculum, including “active learning strategies … laboratory experiences, case studies, guided group discussions, simulations and other practice-based exercises.”² Having a way to document patient care activities across curricular years for simulated classroom activities, as well as during experiential learning placements, would provide data across the curriculum for quality assurance and continued assessment of whether student learning activities were meeting curricular outcomes. Such a tracking tool would provide objective data for key elements in ACPE Standards 2016, such as experiential quality assurance (10.15), care across the lifespan (12.4), introductory pharmacy practice experience (IPPE) expectations (12.5), simulation for IPPE (12.7), patient care emphasis (13.1), diverse populations (13.2), and interprofessional experiences (13.3).²

This paper describes the development of a tracking tool to collect data on student patient-care related experiences in experiential and simulated classroom activities. The process of the creation of the tool, implementation, and use is discussed.

DESIGN

In fall 2010, faculty and student representatives from the Curriculum, Experiential Learning, and Curricular Assessment Committees, as well as selected clinical practice faculty members, and information technology staff were engaged to determine key documentation elements and a format for a tool that could be used progressively across the curriculum by students to document numbers and types of patient interactions they experience.

Agreed upon principles and goals included ease of use and ability to complete at the point of care, ability to track actual and simulated patient care activities across the curriculum, data management in a centralized database, and generation of reports at the individual student level and in aggregate for curricular assessment. As patient care documentation is a specific process taught to students within the curriculum, the term “tracking tool” was chosen for the tool that would mimic a workload or billing process in practice. The goal was not to duplicate a clinical documentation system to provide patient care, but rather to facilitate collection of quantitative data for tracking student progression toward achievement of curricular outcomes, specifically patient assessment, medication therapy management, and communication. The Pharmacist Interaction Tracking Tool (PITT Form) was the name selected for this tool.

Initial discussion focused on what data were to be collected: types of patients encountered, drug-related problems identified, and interventions provided.¹⁰ Other items discussed were how to link interactions to experiential learning sites and courses within the curriculum. The data domain included nonidentifying patient demographics, type of interactions (patient or provider), reason for the encounter, disease states encountered, drug-related problems identified, drug therapy recommendations provided, patient education provided, referral to health care practitioners, method of communication with the patient and provider (verbal and/or written), acceptance of recommendations, and time of interaction, including total time and billable time. Discrete data fields (check box or prefilled drop down lists) were used rather than free text fields to facilitate data analysis. Institutional review board approval was not required as it was process improvement and no protected health information was being collected.

Figure 1 outlines the time course of and progression from paper to electronic format. Students were introduced to the electronic PITT Form through an orientation session within the Professions of Pharmacy course as part of learning the patient care process. Data is from the 2013-2014 academic year.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Timeline and Summary of the Implementation of the PITT Form.

EVALUATION AND ASSESSMENT

The PITT Form data allowed for evaluation from various perspectives, including quantification of student learning experiences, student impact on patient care, and curricular assessment. During the 2013-2014 academic year, 27 778 entries were completed across all professional years. Students completing IPPEs reported 7272 encounters and simulated classroom activities generated 2739 encounters, 26% and 10% of the total entries, respectively. Students on advanced pharmacy practice experiences (APPEs) in patient care reported 17 767 encounters (64%). Patient demographic information for direct patient care and classroom simulation revealed a relatively even exposure to male (46%) and female (54%) patients. Students had the greatest exposure to patients aged 61-80 (31%) and 61-79 (30%). Students gained experience across other age ranges: <18 (6%), 18-25 (7%), 26-40 (13%), and >80 (8.5%). Analysis of ethnicity data revealed exposure to primarily White (68%) and African American (15.5%) patients, with American Indian, Latino, and Asian patients each at less than 5%, which is representative of the surrounding area and Commonwealth of Pennsylvania.¹¹

Simulated learning activities (SIM) occur in a variety of ways and include standardized patients and standardized colleagues, simulation laboratory, case-based practica, virtual patient cases, and comprehensive capstone cases. Entries reflecting SIM activities were most prevalent in the first professional year, with an average of 10 per student, and were similar within the second and third professional years, with and average per student of eight and seven, respectively.

Direct patient care (DPC) experiences included all activities in the curriculum during which students interact with patients and providers in a variety of care settings. These experiences begin in the first professional year with an average of seven documented DPC experiences per student. The average number of documented experiences per student in the second professional year was 39 and, in the third professional year, the average was 15. The average number of DPC experiences per student in the fourth professional year was 156. The documented reasons for the encounters were similar between SIM and DPC activities and included drug therapy monitoring, medication assessment, patient counseling, self-care recommendations, and drug information inquiries. Simulated learning activities revealed student exposure to a variety of disease states reflective of concurrent curricular content. The most common disease states documented during DPC activities were hypertension, diabetes, infectious disease, pain, gastrointestinal disorders, and hyperlipidemia, and accounted for half of all diseases encountered. The number and type of drug-related problems and drug therapy recommendations for both SIM and DPC activities are outlined in Tables 1 and 2.

Of the documented patient education provided in SIM and DPC activities, prescription medication education was the most common (37%), followed by education on nonprescription medication use (14%), disease state (14%), and adherence (10%).

Communication, both written and verbal, occurred with patients and providers, in both SIM and DPC activities, as shown in Tables 3 and 4, respectively. The data reveal that students gained exposure to communication forms with patients and health care providers.

Aggregate data across all academic years showed a balance between patient and health care provider as the initial source of the interaction [11 090 (54%) and 10 983 (46%), respectively]. However, 20% of DPC entries did not identify the initial source of contact. Many student interactions occurred with patient and provider, and this data is reflected Tables 3 and 4. Analysis by academic year showed a predominance of patient interactions early in the curriculum and increasing engagement with health care providers across the curriculum. In the first professional year, 730 patient and 103 health care provider interactions were documented for the class and increased to 5020 and 9921, respectively, in the fourth (P4) year.

Introductory pharmacy practice experiences occur during each semester across the first (P1), second (P2), and third (P3) years. Across all IPPEs, 3149 drug-related problems were identified, the majority of which were identified in the P2 year (53%). The most common drug-related problem identified was “needs drug therapy” (1196; 44%), with 75% of those related to an untreated problem. Students identified 645 adverse drug reactions: 241 actual reactions and 388 potentially avoided reactions (Table 1). The most common drug therapy recommendations were “addition of medication” (37%), “change in medication” (16%), and “discontinuation of medication” (12%) (Table 2).

Regarding APPEs, students identified 13 507 drug-related problems in 63% of patients and provided 14 218 drug therapy recommendations (Tables 1 and 2). “Needs drug therapy” was the most common problem identified (34%), with 44% of those indicated as an untreated problem. Students in APPEs identified 1343 adverse drug reactions in 7.5% of patients of which 641 were actual and 702 were potentially avoided reactions. The most common recommendation was “addition of medication” (4728; 42%), with “increase in dose” (1745), “change in medication” (1741), and “discontinuation of medication” (1711) each at 15%.

The PITT Form data were used by students, instructors, and administrators. Students were required to maintain an assessment portfolio throughout their academic career. The portfolio included student-specific evidence and reflections to track progress in meeting curricular outcomes. The data from the PITT Form were included in the student portfolio as evidence of patient care and interprofessional skill development.

Instructors also had access to and used the information in the PITT Form. During the didactic professional years, the most common use was to verify completion of assignments. For example, students were required to complete the PITT Form as part of practica assignments. A report could then be generated to verify completion. If the form was not submitted points were deducted. Recognizing the need to verify that appropriate data was being documented, instructors also incorporated this into the assignments. Within the IPPEs during the P2 year, completion of the PITT Form was implemented as a process similar to billing an encounter, and the students used the form as such at their community sites to model medication therapy management practice. The PITT Form data was then verified with the patient care documentation for accuracy. During APPEs, the number of submissions was reviewed after each rotation block by the experiential learning coordinator, and students not meeting the required number for patient care rotations were brought to the attention of the experiential learning director to be addressed.

The Curriculum Assessment Committee oversaw the evaluation of curricular outcomes including the student portfolio and use of the PITT Form. In addition to the student portfolio and tracking individual student patient care experiences, aggregate data provided objective evidence of the types of patient care activities to assure curricular quality and that accreditation standards were met.

DISCUSSION

The objective was to create and implement a standardized data collection tool for capturing student-patient interactions in SIM and DPC environments. The PITT Form demonstrated the ability to collect various types of data from patient care activities both in the classroom and during experiential learning activities. The standardized nature of the data collection allowed for curricular assessment of activities across years and individual student use.

The data was most useful for the student portfolio and individual assessment of progress toward achieving curricular outcomes. The student portfolio was sectioned by curricular outcomes. The PITT Form allowed the students to provide objective data supporting the curricular outcomes in patient assessment, medication therapy management, and communication. Students were able to provide this objective data on the breadth of patient care and health care provider experiences to support their perceptions of their capabilities. Individualized student reports of their interactions fostered self-evaluation and “personalized education” and informed students on opportunities for continued growth. Students also were able to see the direct impact they had on patient care through the number and type of drug-related problems identified and drug therapy recommendations provided. Cumulative reports allowed the students to see individual contributions to patient care over their academic career.

As students gained experience with the functionality of the PITT Form, it was also used for tracking of extracurricular activities for student organization such as patient care screenings for the National Association of Chain Drug Stores Million Hearts Campaign and the American Pharmacist Association-Academy of Student Pharmacists Operation Immunization. Using the tool meant students did not need to create their own format for collecting data to present for these projects, and it was easily accessible to students as part of their usual patient care activities. The PITT Form can be utilized to collect cocurricular patient care activities as the data entered does not have to be tied to a specific course or curricular activity. This may be useful for students who are seeking out projects and other activities to personalize their education.

The Curriculum Assessment Committee uses the PITT Form data to identify areas of strength and further need. Students across the curriculum documented nearly 28 000 patient interactions during which they provided more than 16 000 drug therapy recommendations in one academic year. Data from the PITT Form quantify the number of DPC interactions and acuity of disease states as outlined in ACPE Standards 2016’s key element patient care emphasis (13.1).² This process also allows the school to provide quality assurance of the experiential sites through review of the amount and type of patient interactions [Standards 2016 key element experiential quality assurance (10.15), and IPPE expectations (12.5)].² In addition, this data quantifies types of assessment and recommendations that support the school’s curricular outcomes of patient care activities of mediation therapy management.

Another strength is the ability to evaluate experiences within the classroom separately from direct patient care, which provides data to support ACPE Standards 2016 key element 12.7 simulation for IPPE.² Students in APPEs had more than 11 500 interactions with providers, which lends support for Standards 2016 key elements 13.3 interprofessional experiences, as well as supporting the school’s curricular outcome on communication.² The PITT Form also allows for capture of the patient population types that students encounter, specifically age and ethnicity, providing data to assess meeting ACPE Standards 2016 key elements care across the lifespan (12.4) and diverse populations (13.2)². These data were helpful in identifying this as an area for improvement and in developing strategies, through simulation and direct patient care, to increase student exposure to diverse populations.

Limitations of the data analysis specifically caution interpretation of the raw data. As with all self-reported data, there may be inaccuracies or missing elements in the data that reflect what students interpret and verify as drug related problems and recommendations. Although students and faculty members were provided instruction on the PITT Form use, the training did not include ensuring consistency in entering data across users. This issue was addressed through streamlining data choices and removing ambiguous questions. Another limitation of the data presented is that few assignments and learning sites verified the accuracy of what the student was entering. This has been recognized and assignments will be incorporated in which the data is reviewed with the individual student and inaccuracies addressed. The PITT Form was not mandatory and was used at the discretion of instructors for classroom activities. Therefore, possible underreporting of SIM activities was possible.

It is also possible that the number of DPC activities students actually participated in vs reported in the PITT Form were different as students may only have entered the minimum requirement. Also, there was a difference in expectation in the number of entries between IPPE in the P2 and P3 years. The P2 year IPPE expectation was for all interactions at the practice site to be submitted in the PITT Form, while the P3 year IPPE set a minimum of 10 per student per practice experience. When evaluating aggregate data of patient interactions, it appears to be a substantial decrease, however, when looking at individual student data, the greatest number of P2 entries was 68, while P3 entries totaled 145. Finally, an important limitation of the process described is that the data focused on volume and type of experiences, but not the details of the experience, thus quality of the experiences cannot be extrapolated.

The future use of the PITT Form includes formation of a standing subgroup under the Curriculum Assessment Committee to provide continued quality improvement of the tool itself and the data output. Limitations, such as providing clear direction to the students and instructions to faculty members on proper use of the form, and appropriate expectations for the minimum numbers of encounters to be reported, will be reassessed. Additionally, opportunities for expanded use to provide data directly to preceptors is being explored. Preceptors have found value in sharing the data captured by students at their practice site with supervisors for practice enhancement.

SUMMARY

The Pharmacist Interaction Tracking Tool (PITT) Form helps capture student-patient care activities in a standardized format in simulated and direct patient care environments. A robust reporting structure allows data use by individual students to track activity and progress, and by faculty members in aggregate for curricular assessment.