Validity Evidence for a Pharmacists’ Patient Care Process Self-Efficacy Scale Among Pharmacy Students

INTRODUCTION

The Pharmacists’ Patient Care Process (PPCP) is a framework established by the Joint Commission of Pharmacy Practitioners (JCPP) for pharmacists to follow in delivering patient-centered care.¹ It is rooted in the pharmaceutical care model developed by Hepler and Strand and serves as a structured guide to ensure the delivery of “high quality, cost-effective and accessible health care for patients” in any pharmacy practice setting.^1,2 The PPCP has been adopted by the Accreditation Council for Pharmacy Education (ACPE) in the 2016 Accreditation Standards, Standard 10.8, which states “the curriculum prepares students to provide patient-centered collaborative care as described in the Pharmacists’ Patient Care Process.”³ Doctor of Pharmacy (PharmD) programs across the United States continue to implement the PPCP throughout their didactic and experiential curricula.^4,5 Assessment is an essential component of ensuring the proper implementation of the PPCP in curricula and measuring its impact on student learning.⁵ As such, the PPCP has been mapped to the Center for the Advancement of Pharmacy Education (CAPE) Educational Outcomes as well as the American Association of Colleges of Pharmacy (AACP) Core Entrustable Professional Activities (EPAs) for New Pharmacy Graduates framework.^6,7 Both the CAPE outcomes and the AACP EPAs are essential elements for schools and colleges of pharmacy in their ongoing curricular and student outcome assessments.^8,9

Several studies have described or examined implementation of the PPCP in specific courses.^4,5,10-13 Rivkin described how the PPCP was incorporated in an introductory pharmacotherapy course to introduce first year pharmacy (P1) students to PPCP components and how to apply the process when evaluating a patient case.⁴ The author reported that although students lacked significant clinical knowledge at that stage in their curriculum, their competency in collecting a medication history and identifying drug-related problems during subsequent introductory pharmacy practice experiences (IPPEs) was higher compared to that of students in previous cohorts.⁴ Gonyeau and colleagues reported on a similar approach to incorporate the PPCP into a disease management course series for second and third year pharmacy (P2 and P3) students.¹⁰ The PPCP concepts were introduced to students and incorporated via concept mapping and case-based assignments, and mapped to assessment questions. The authors evaluated both faculty members’ incorporation of PPCP into their teaching materials as well as students’ performance on summative assessments and assignments. Students varied in terms of their performance on specific PPCP domains, with higher performance on collect and implement domain-related questions.¹⁰ Another example of purposeful incorporation and evaluation of the PPCP was described by Phillips and colleagues in the context of a capstone course.¹¹ In this third-year skills-based course, the PPCP domains were infused into patient cases and assignments. Authors reported improvement in students’ confidence in their ability to apply the PPCP as a whole.¹¹ Additional studies have described how the PPCP was implemented in an entire pharmacy program⁵ and how students’ knowledge of the PPCP steps was assessed in a medicinal chemistry course¹² and an IPPE course.¹³

There continues to be a need for instruments with demonstrated validity evidence that assess students’ ability to implement PPCP components, especially as part of advanced pharmacy practice experiences (APPE) and practice readiness assessments.⁵ Evaluating student confidence in PPCP is one important component of a global approach to assess students’ competency with applying the PPCP. Self-assessment is a critical skill for continuous professional development in pharmacy education and can provide students with insight into potential gaps in their learning.^14,15 It can be implemented alongside knowledge-based or skill-based assessments to provide a more comprehensive picture of students’ learning and skill development.

The purpose of this study was to further refine and examine initial validity evidence of an instrument for assessing PharmD program students’ self-efficacy in implementing PPCP components when provided with a patient case. Such an instrument could serve as a valuable resource to assess pharmacy students’ readiness for both APPEs and pharmacy practice.

METHODS

A self-report instrument for pharmacy students that addressed the first three domains (collect, assess, plan) was initially developed and administered as part of a pilot evaluation project at Manchester University in spring 2018. Items were developed with the JCPP’s descriptions of PPCP domains serving as a guide or framework.¹ Bandura’s theory of self-efficacy was used to structure the scale.¹⁶ Self-efficacy is defined as an individual’s “belief in their capabilities to produce given attainments.”¹⁶ As such, it was determined that a self-efficacy scale is appropriate for examining confidence in implementing PPCP components among pharmacy students. Prior to administration, the instrument was reviewed by subject matter experts who were pharmacy practice faculty working in a variety of settings. In addition, the instrument was distributed to several fourth professional year (P4) pharmacy students who were asked to comment on instrument clarity and length. The feedback obtained was used to further revise the instrument.

According to Bandura’s guidelines for constructing self-efficacy scales (SESs), a unipolar continuous scale is recommended.¹⁶ Such scales are better predictors of efficacy than interval scales, and Likert-type scales are considered inappropriate for assessing self-efficacy.¹⁶ Students had to rate themselves on a 100-point unipolar continuous scale ranging from (0=not at all confident to 100=highly confident). Additional items were included on the instrument to gather demographic information, including age, gender, race/ethnicity, previous work experience, and college/school.

With the release of operational definitions for the PPCP’s domains in July 2018,¹⁷ the instrument was revised and expanded to include items addressing each domain. The final instrument included multiple items addressing each of the domains: five items for the collect domain, six items for the assess domain, 10 items for the plan domain, three items for the implement domain, and four items for the follow-up domain. While it is difficult for students in the didactic curriculum to participate in activities that are directly aligned to implementation and follow-up of care plans, these items were included to allow for a more comprehensive instrument and potential use by pharmacists in future research.

This study was approved, deemed exempt, or granted a waiver by all participating institutions, with Manchester University’s institutional review board (IRB) being the IRB of record. Faculty members who were engaged in curricular or assessment activities were approached and invited to participate in distributing the instrument. In addition, an announcement was posted in a PPCP-related learning community on the AACP website to recruit interested pharmacy schools. Both public and private institutions were recruited. The 34-item instrument was distributed electronically via Qualtrics (Qualtrics) to P3 students across a convenience sample of seven pharmacy schools. Upon approval of each university’s IRB, a contact person at each institution sent out an email with the link to the Qualtrics survey to their P3 students. Two follow-up emails were sent in one-week increments. Start and end dates varied between institutions; however, all data collection was completed during April and May 2019, prior to students starting their APPEs.

Demographics were assessed using descriptive statistics. Reliability of the PPCP-SES was measured using Cronbach alpha, with a coefficient of 0.70 or greater indicating good internal consistency.^18,19 The Cronbach alpha was calculated for the entire instrument as well as for each of the domains. Possible inclusion of specific items in the final instrument were initially assessed through Cronbach alpha. Items were removed if the item was: deemed complex or multi-faceted, and/or the Cronbach alpha for the construct was increased after item deletion, and/or inclusion of the item did not improve the model fit. Items retained and final domain Cronbach alpha coefficients are presented in Table 2. Given that the items were theoretically developed based on the PPCP descriptions, a confirmatory factor analysis (CFA) structural model was performed to test the model and determine if the relationships between items aligned with their respective domains.²⁰

The model chi-square, root mean squared error of approximation (RMSEA), standardized root mean squared residual (SRMR), and comparative fit index (CFI) were used to assess model fit. The model Χ² assesses overall fit of the model to the data, with a significant p value (<.05) indicating lack of fit, although it is sensitive to large sample sizes.^21,22 The RMSEA is an absolute fit index that corrects for model complexity and is not as sensitive to sample sizes, with values <0.10 indicating acceptable fit. The SRMR is another absolute fit index, with values <0.10 indicating acceptable fit. ^21,22 The CFI is an incremental fit index that measures improvement in fit, with values ≥0.90 indicating good fit, and is generally not sensitive to sample sizes.^21,22 Several models were tested and the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), two commonly used model selection criteria, were used to compare models and choose the model with best fit.²³ Demographics and Cronbach alpha were evaluated using SPSS, version 25 (IBM), and confirmatory factor analysis was conducted using Stata Statistical Software, version 15 (StataCorp, LLC).

RESULTS

Three hundred P3 students across seven institutions completed the PPCP-SES. Participants were from both public (n=3) and private (n=4) institutions from the Midwest (n=4) and South (n=3) regions of the United States. All participating institutions were four-year pharmacy programs. Characteristics of pharmacy student participants are presented in Table 1. The majority (85.6%) of student participants were attending private institutions. Among all respondents, 70% were female, 47.3% were White, and 90% had worked in a pharmacy setting for at least one year. Two items from the plan domain were removed based on Cronbach alpha coefficients and because they did not improve model fit. One item from the implement domain was removed for being a compounded statement, which asked for multiple evaluations within the statement, and for the subsequent improvement of the Cronbach alpha. The remaining 25 items were loaded onto the five constructs of the PPCP: five items on collect, six items on assess, eight items on plan, two items on implement, and four items on follow-up. The overall Cronbach alpha coefficient for the instrument was 0.98, and the individual Cronbach alpha coefficients for each domain were as follows: collect, α=0.89; assess, α=0.92; plan, α=0.95; implement, α=0.96; and follow-up, α=0.95. All the items, along with the Cronbach alpha for their respective domain, are shown in Table 2. Several potential models were assessed to identify the best fit based on the AIC and BIC. Models that were tested included models with all the original items as well as models examining whether some items loaded onto different or multiple constructs. The resulting model produced the following fit statistics: Χ² =1482.595 (p<.05), RMSEA=0.13, CFI=0.86, and SRMR=0.06.

DISCUSSION

To the authors’ knowledge, this is the first study reporting on the validity of an instrument aimed at assessing pharmacy students’ self-efficacy in implementing the PPCP framework. The PPCP Self-Efficacy Scale (PPCP-SES) demonstrated evidence of reliability, with an overall Cronbach alpha of 0.98 and alphas in the 0.89-0.96 range for each of the domains. A Cronbach alpha of greater than 0.7 is considered good, while an alpha greater than 0.95 usually indicates a redundancy of items.¹⁹ However, based on feedback from subject matter experts early on, we deemed it was necessary to retain specific items in the instrument given the types of activities they measure.¹⁹ Because of the nature of the PPCP domains, many of the activities in the assess, plan, implement, and follow up domains can overlap and be similar. However, they each relate to a different part of the patient-care process and involve different skills. Results from the confirmatory factor analysis for the PPCP-SES were not consistent. A significant chi-square for the overall model, a RMSEA value (0.13) slightly greater than 0.10, and a CFI value (0.86) slightly below 0.90 do not meet the standards for good model fit. The only value that indicated good fit was the SRMR. Although the Χ² statistic is generally sensitive to large sample sizes, the RMSEA, CFI, and SRMR are not.²¹ With factor analysis, acceptable fit levels tend to vary, and there is some disagreement among statisticians and researchers on the most appropriate cutoff criteria.²¹ Although the RMSEA and CFI did not meet the good fit threshold previously established for this study, the values are near the borderline cutoffs. For conducting confirmatory factor analysis, a respondent to item ratio of 5:1 or 10:1, or a sample size of 300, is considered optimal.^18,20 Both of these criteria were met in this study. As there is no other published instrument, to the authors’ knowledge, with which to compare this instrument, other sources of validity evidence could not be assessed.

One potential explanation for the lack of good fit is the study population (ie, P3 students), given the nature of the items related to the implement and follow-up domains. Most students at this stage will have had experience, via standardized patients, simulations, and assigned cases, to practice their skills to collect, assess, and develop a plan based on a patient case. However, they may not have had experience implementing a plan and following up with patients or health care providers. As seen from other studies, assessments related to the PPCP tend to focus on the collect, assess, and plan domains.

As the pharmacy profession continues to implement principles of the PPCP within practice settings, pharmacy schools are working on integrating and assessing PPCP components within didactic and experiential curricula. Pharmacy students need to be adequately prepared to deliver patient-centered, evidence-based care in accordance with the PPCP framework to ensure optimal patient outcomes. Previous studies have examined how to integrate the PPCP within specific pharmacy didactic coursework. Phillips and colleagues evaluated students’ performance and confidence in applying the PPCP components to patient cases within one college of pharmacy’s capstone course.¹¹ Students completed case-based worksheets mapped to three PPCP components (collect, assess, plan) and evaluated their overall confidence in applying the PPCP to patient cases. The authors reported that students rated their confidence in applying the PPCP significantly higher after completing the course.¹¹ Additional studies have described incorporation of PPCP elements within specific coursework or integration within overall curricula.^4,5,10,12,13 Despite the presence of some literature on PPCP incorporation within pharmacy education, studies have been descriptive in nature, focused within one course at one school, and have not assessed self-efficacy for individual components of the PPCP.^4,5,10,11 An instrument assessing pharmacy students’ self-efficacy in application of the PPCP in practice can be very valuable, especially when used with other assessment tools. It can be used to evaluate students’ readiness for APPEs and practice and reveal any gaps in knowledge or skills. Although student self-assessment can sometimes have limitations, they are still important for both students’ own professional development and can provide students and faculty with insight into opportunities for improvement in knowledge and skills.^14,15

This study has several limitations. The mixed results with the model’s fit statistics suggest that survey revisions and/or administration in other populations is necessary. The participating institutions were selected based on a convenience sample and did not have representation from the Northeast and West regions of the United States. Although there was a mix of public and private institutions, there was much higher participation among students from private institutions compared to public institutions. The level of exposure to the PPCP within the didactic curriculum can also vary based on the institution. In addition, ideally in factor analysis, each construct has at least three items. However, the implement domain was a two-item factor because the third item was removed. All of these might have had an impact on model fit. Despite these limitations, the PPCP-SES demonstrated initial evidence of reliability and validity and might provide schools with some initial information on students’ confidence in applying the PPCP.

Future research could examine validity evidence with the PPCP-SES among practicing pharmacists in various settings. Bradley and colleagues reported on the creation and initial validity evidence of an instrument to assess the scope of ambulatory care pharmacists’ practice activities centered around the PPCP.²⁴ The Tool for Assessing Ambulatory Care Pharmacy Practice included statements from each of the PPCP domains, with the implement and follow-up domains combined, and ambulatory care pharmacists were asked to rate how often in a typical month they were engaged in specific PPCP-related activities during direct patient care interactions.²⁴ Examining practicing pharmacists’ confidence with implementing the PPCP can provide valuable insight into the uptake of the PPCP framework across the profession and offer opportunities for improving pharmacists’ patient-centered care skillset.

CONCLUSION

This is the first study to describe initial validity evidence of an instrument aimed at assessing self-efficacy in implementing the Pharmacists’ Patient Care Process components, the PPCP Self-Efficacy Scale (PPCP-SES). Although some studies have described and or examined PPCP incorporation within pharmacy education curricula, there is a critical need for assessing students’ ability to apply the PPCP through both self-assessment and knowledge and skill-based assessments. Results indicate that the PPCP-SES demonstrates preliminary evidence of reliability and validity for use by pharmacy faculty members to provide information on students’ self-efficacy in relation to implementing the PPCP. It can also be used to identify learning opportunities for students to improve their application of the PPCP to patient cases. More work is needed to provide additional validity evidence in other pharmacy student populations and among practicing pharmacists in various settings.