Using Best-Worst Choice Methodology in a Survey of Pharmacists Regarding Pharmacy Practice Skills Teaching

INTRODUCTION

The expanding roles for pharmacists, increasing use of digital technology in health care, empowerment of pharmacy technicians, and increasing collaboration as part of interprofessional teams are positive professional developments for pharmacists.^1,2 There is a directly associated increased demand, however, for pharmacist training programs to adapt and modernize curricula to equip future pharmacy graduates with the necessary skills and abilities to provide competent care in these changing settings.³ Especially in the digital age, this need for curricular reform can come suddenly, unexpectedly, and frequently. Programs must therefore be ready to recognize rapidly changing professional landscapes and be ready to action processes and procedures to develop, implement, and evaluate curricular modifications.^4,5 Programs must also be prepared to remove curricular components that may be deemed to be outdated or irrelevant to modern pharmacy practice. One of the curricular components most likely to require frequent reform is the teaching and assessment of pharmacy practice skills.⁶ As skills represent the most practical nature of professional practice, the teaching of skills may be especially vulnerable to changing professional landscapes.⁶ During times of reform, programs must prioritize which skills to include, exclude, or refine when developing new content.

Decisions regarding curricular reform should be informed by input from key professional stakeholders.⁷ However, seeking feedback that accurately reflects practice priorities, especially from pharmacists in large practice settings and/or in diverse types of practice settings, can be difficult. With respect to informing pharmacy practice skills, the input of practicing pharmacists across regions and practice settings is beneficial.⁸ However, conducting widespread surveys can be time consuming and difficult, and results collected from Likert-scale survey items or items requesting the rank ordering of a list may not offer enough discrimination between the items of interest (in this case, pharmacy practice skills).⁹ The use of interviews and focus groups (including advisory boards) can provide useful information but may not be representative of the current state of practice and general population of interest.¹⁰ A novel type of survey methodology, stated preference methods, or choice experiments may account for these challenges by measuring and quantifying stakeholders’ priorities and preferences. Best-worst scaling is one stated preference method that has gained momentum in the health literature.¹¹ The advantages to this approach include the ability to measure a participant’s preferences for all skills using a single scale and the ability to collect more data from each participant. As such, it may provide an efficient and discriminatory way to capture pharmacy stakeholders’ perceptions in terms of prioritizing pharmacy practice skills for curricular reform.

Aside from using stakeholder data to inform inclusion/exclusion of learning outcomes and practice skills during curricular reform, capturing priorities may also help to guide development of quality assurance and assessment procedures.¹² If there is a mismatch between stakeholder and university priorities, for example, findings from a best-worst choice survey may inform the need for preceptor education regarding a program’s direction and/or emerging competencies required for practice. Given the importance of obtaining stakeholder input for curricular reform and in consideration of the challenges for obtaining accurate and representative feedback from these stakeholders, the purpose of this study was to employ best-worst scaling to prioritize pharmacy practice skills for inclusion in a new curriculum for pharmacy students in New Zealand.

METHODS

To determine the potential skills that could be taught in a new Bachelor of Pharmacy (BPharm) curriculum, a search of the literature was completed to get an overview of the skills typically taught in modern undergraduate pharmacy curricula. The following databases were searched: PubMed, Ketu, Google Scholar, and Ovid. Combinations of the following keywords were used in each database: skills, intern pharmacist, undergraduate, and pharmacy. A web-based search of pharmacy curricula in Australian and North American entry-level pharmacy programs was also conducted. From these sources, a list and description of 16 potential skills was compiled for evaluation in the best-worst choice analysis. A description of each skill was written to explain the skill in context where necessary. An overview of each skill and the matching description shown to participants during data collection is provided in Table 1.

Registered pharmacists and intern pharmacists (those who have completed a BPharm degree but are not yet licensed to practice until completion of a one-year internship) in New Zealand were contacted by email through the Pharmaceutical Society of New Zealand, which maintains a database of registrants. Each participant received a link to access the survey. The participants could access the survey at any one time during a two-month period. The total number of pharmacists invited to complete the survey was 3836.

Survey software (Qualtrics, Provo, UT) was used to design the survey and respondents were able to complete the questionnaire online. The survey consisted of demographic questions and a best-worst choice survey regarding a pharmacist’s preferences for skills to be taught in the undergraduate pharmacy program in the new curriculum at the School of Pharmacy, University of Otago. An example of a best-worst choice question is provided in Figure 1. Sawtooth software (Sawtooth Software, Inc, Sequim,WA) was used to create the experimental design for the survey. In this algorithm, one-way frequency (how many times each item appears across the entire design), two-way frequencies (how many times each pair of items appears within the same set across the entire design), positional frequencies (report how many times each item appears in the first, second, third, and fourth position) and connectivity (all items are linked directly) were considered.^13-15 The 16 skills selected for inclusion in the survey were presented in 20 sets with four skills listed per set (Figure 1). Each participant was asked to select the most preferred (“best”) skill and least preferred (“worst”) skill from the four skills in each choice set. Participants were presented with repeated varying choice sets which, upong completion of the survey, allowed the application of mathematical modeling to determine the quantitative score and ranking of items.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Example Survey Question Using Best Worst Choice Scaling

The demographics and professional backgrounds of the respondents were characterized using frequencies and means. Best minus worst scores (the frequency of choosing an item as “the most important” minus the frequency of choosing it as “the least important” one) were used to initially characterise preferences which have been shown to be a good approximation for results obtained from the logit models. A conditional logit model was used to estimate coefficients for the various skills. In both cases, higher scores represented higher preferences for the skill. For the conditional logit models, the scores (utility values) and p values were relative to a reference score (the skill with the lowest preference). The influence of covariants (gender, ethnicity, location, age, and experience) was examined by using interaction terms. Age was categorized as less than 31 years, 31 to 40 years, 41 to 50 years, and greater than 50 years. Locations were categorized as a city (>50,000 inhabitants), a town (1000 to 49,999 inhabitants), or rural (<1000 inhabitants).

Approval for this study was granted through the University of Otago Human Ethics Committee, and this study was discussed with the Ngai Tahu Consultation Committee. Participants provided informed consent and were welcome to withdraw from the survey at any time. All responses were made anonymously.

RESULTS

Of the 3836 pharmacists invited to participate in the survey, 396 started it, and 388 fully completed it (RR=10%). Demographics of the participants matched those typical of New Zealand pharmacists. The majority of participants were female (69%), which mirrored the pharmacist population of New Zealand (65% female). The average age of respondents was 42.4 years (range, 20 to 99 years) and most identified as European (68%), Asian (9%), or Māori (4%). Most (65%) pharmacists’ highest level of pharmacy education attained was an entry-level qualification (diploma, BPharm, or PharmD) with others having a postgraduate certificate, diploma, or master’s degree (33%), or a PhD (2%). The largest category of respondents was employee pharmacists in community practice (41%), followed by pharmacist owners (24%), dispensary managers or team leaders (14%), and interns (4%). Most participants worked mainly in a community pharmacy (67%), a hospital (20%), or a primary health organization (3%), with others working in academia, government, or industry.

The results of the conditional logit model based on best-worst choice responses in terms of coefficient means are presented in Table 2. All of the coefficient means were significantly higher than the reference category coefficient mean (specialty compounding). On average, the importance of comprehensive chronic disease management for respondents was higher than that of other items. Respondents also placed high value on the provision of specialty medications that only pharmacists can provide (eg, trimethoprim for urinary tract infections, oral contraceptives, etc.) and Medicines Use Review (MUR). Conversely, independent prescribing and injections including vaccinations were considered relatively less important for respondents.

When examining the results by covariants, gender, age, location, and ethnicity had significant impacts on pharmacists’ preferences for the school to teach various skills. For example, gender had a significant influence on participants’ preferences for students receiving education in interprofessional health care (mean for male participants was 0.48 lower, p<.001), chronic disease management (mean for male participants was 0.38 lower, p=.002), specialty medicines (mean for male particpants was 0.39 lower, p=.002, business skills (mean for female participants was 0.44 lower, p<.001), and ordering and interpreting laboratory tests (mean for male participants was 0.33 lower, p=.009.

Age had a significant impact on participants’ preferences for students learning to conduct Medicines Use Reviews (mean increased by 0.17 for each of the age categories [p<.01]), specialty medicines (mean increased by 0.14 [p<.01] for each age category), ordering and interpreting laboratory tests (mean increased by 0.14 [p<.05] for each age category) and technology and informatics for pharmacy (mean increased by 0.29 [p<.01] for each age category). Similarly, location of practice had a significant influence on preference for specialty medicines (mean increased by 0.28 for cities, p=0.03), medicines therapy assessment (MTA) (mean decreased by 0.36 for cities, p=.005, independent prescribing (mean increased by 0.25 for cities, p=.037), ordering and interprheting laboratory tests (mean increased by 0.38 for cities, p=.003, and complementary medicines (mean decreased by 0.29 for cities, p=.02). Finally, ethnicity of respondents also appeared to influence preferences, with those of European descent placing higher preferences for skills in interprofessional health education (mean 0.09 higher, p=.013), comprehensive disease management (mean 0.37 higher, p=.003, dispensing (mean 0.44 higher, p=.003, medicines use review (mean 0.37 higher, p=.004), specialty medicines (mean 0.39 higher, p=.002), complementary medicines (mean 0.28 higher, p=.03), and technology and informatics education (mean 0.44 higher, p=.001.

DISCUSSION

The purpose of this study was to use best-worst choice scaling to determine practicing pharmacists’ preferences for prioritization of pharmacy practice skills during curricular reform. Using best-worst choice scaling, we were able to differentiate between prioritized skills. As shown in Table 2, the methodology was able to identify the priorities of practicing pharmacists with respect to pharmacy practice skills. Favored skills were mixed between cognitive-based skills (such as comprehensive chronic disease management) and technical skills (such as dispensing). Skills that were not favoured included those that are perhaps more specialized and not necessarily generalizable to the larger population of practicing pharmacists. Examples of these types of skills were specialty compounding, independent prescribing, physical assessment, and business. As the largest group of respondents were employee pharmacists in community settings, we were not surprised that these specialized skills were not deemed of great importance. However, as staff community pharmacists make up the greatest percentage of pharmacists in New Zealand, these results are likely reflective of the current priorities of the largest practitioner group and, as discussed below, may have implications for curricular reform.

A notable finding from the best-worst choice analysis was that practicing pharmacists tend to prioritize duties/skills currently required of the profession, such as chronic disease management and dispensing, rather than emerging opportunities. New responsibilities of pharmacists in global settings, including injection training and prescribing, were not prioritized by the pharmacists in this sample. There are many potential explanations for these findings, including the current scope of practice of the majority of pharmacists in New Zealand (eg, limited injection and prescribing roles), lack of training opportunities for practicing pharmacists to upscale their own skills, limited reimbursement for expanded pharmacists services, or expectations that students are not yet “practice ready” and will develop professional-oriented skills during the internship year.¹⁶ The implications of these findings are interesting, as universities must balance stakeholder input with the need to train pharmacists for success in future practice. These results stress the need for universities to develop close ties with all stakeholders, including government and regulatory bodies, in order to better understand how the landscape of pharmacy practice may change and require different skills of pharmacists within the next five to 10 years.

The differences found with respect to participant demographics may also be important for educators, employers, and policymakers to consider. Results show that older participants preferred more practical skills, such as speciality medicines and technology and informatics. This finding may reflect older participants’ awareness of their need to learn skills required to enhance their practice, or it may be a function of younger participants’ experiences with newer curricula that focus less on specific aspects such as specialty medicines. The differences in skill preferences by gender may suggest the need to consider how specific student populations are presented with the applicability of certain skills to practice. For example, female respondents ranked interprofessional activities as being more important than did male respondents. Educators should be aware that these gender-based differences may apply to students in their undergraduate and postgraduate pharmacy programs and be ready to further investigate the reasons why this might be so. The same reasoning may be applied to ethnicity, where pharmacists of European descent found skills such as interprofessional education, chronic disease management, and medicines use review more important. The reasons offered for the differences between these subgroups warrant further investigation because of the possible impacts these findings could have on experiential training and preceptor selection for assessment of program-required skill-based activities.

The results of this study have implications for curriculum reform. Teaching of pharmacy practice skills must be dynamic and respond to the emerging needs of society and the profession yet also ensure that graduates are equipped with the skills to satisfy expectations from preceptors, colleagues, and new employers when entering practice.⁶ This study showed that some emerging skills are not yet preferred above some more traditional skills in the undergraduate curriculum by practicing pharmacists, and that training programs still need to ensure that graduates can function in traditional roles. Careful integration of new skills with traditional skills may therefore be required by programs to produce graduates that meet the expectations of the profession but who are also prepared for expanded responsibilities and practice niches. While the results of this study informed development of the skills program at the University of Otago, administrators also prioritized some “newer” skills to ensure that the future workforce is adequately prepared to provide services that are likely to be expected of pharmacists by the time current students graduate and begin practice. Results also call for strengthening quality assurance mechanisms related to curricular reform in order to determine how changes to learning outcomes and pharmacy practice skills influence the perceived success of graduates in practice. Practice-based preceptors for the BPharm program should be informed of curricular changes and expectations to ensure that students are fairly assessed across their full skill set. For countries that require internships or further training upon graduation from the university-based program, close relationships developed with these providers will help to ensure that the continuum of skill development occurs until graduates are qualified to be deemed “practice ready.” Lastly, this type of analysis may be considered for obtaining stakeholder input across different areas of curricular reform for programs seeking informed feedback.

The decision to survey pharmacists via an electronic questionnaire resulted in feedback obtained from a small sample of pharmacists mainly employed in community pharmacies, which reflects the practice setting of the majority of pharmacists in New Zealand. The small number of respondents may have reflected only those pharmacists interested in curriculum reform and/or those associated with the program (eg, preceptors), which may have introduced selection bias. The group responding to the survey may have prioritized duties that they encounter in their daily practice. Future work could explore the perferences of pharmacists outside of the community setting by stratifying preferences of prescribing pharmacists and those working in primary care and hospital settings. Likewise, the sample of pharmacists was roughly representative of pharmacists in New Zealand; respondents were predominantly female, middle-aged, and identified as “New Zealand European”; therefore, the views of male pharmacists, older pharmacists, and those identifying as “non-New Zealand European” merit further investigation. There were some differences in demographics of our sample and the profession including under representation of Asian ethnicity, which accounts for 27% of the pharmacist workforce in New Zealand but only 9% of our cohort. While the use of a best-worst choice scaling survey was an efficient way to collect data from a large number of pharmacists, the instrument may not have included all of the skills that pharmacists might want to prioritize. We also assumed that, when making their choices, pharmacists would be familiar with the labels and skill descriptions that were used in the survey . However, participants may have conceptualized skills (such as screening, technology, or infomatics) differently and this may have impacted their responses to survey items.

CONCLUSION

Engagement of stakeholders is an essential component for curricular planning and reform. The use of best-worst choice scaling can efficiently capture input from large groups of people and can be used to discriminate between items intended for rank ordering. When employed in this study, the use of this methodology found that practicing pharmacists in New Zealand emphasize skills required for current practice and deemphasize skills that may be required for emerging professional responsibilities. Results support the notion that if curricular reform is to include new skills that are largely unfamiliar (or deemed less important) by practicing pharmacists, quality assurance of students’ practice experiences paired with preceptor education about the expectation of students’ skill sets must be adequately developed and addressed.