Abstract
Objective. To evaluate the effectiveness of a distance-learning course in the training of pharmacists in drug dispensing.
Methods. This interventional study was carried out with community pharmacists throughout Brazil. A 12-week distance-learning course was conducted. Pharmacists’ knowledge and performance during drug dispensing were evaluated before and after completion of the course.
Results. Community pharmacists’ knowledge of drug dispensing was greater after completing the distance-learning course than before (mean=5.7, SD=2.8 points vs mean=3.8, SD=1.9 points, respectively). However, no improvement was observed in pharmacists’ drug-dispensing practice. Almost all participants (96.5%) considered that they were more qualified to perform the drug dispensing after the course. There were 1492 pharmacists enrolled in the course, of which 1061 were included in the study, and 472 completed the surveys. Only pharmacists from Ribeirão Preto, Brazil, were invited to take the mytery shopper evaluation (n=26). However, 10 pharmacists completed the mystery shopper encounter.
Conclusion. A distance-learning course was effective in improving community pharmacists’ knowledge. However, the course did not seem to improve overall drug-dispensing performance. Additional educational strategies should be adopted in future studies to address this gap.
INTRODUCTION
In recent decades, the role of community pharmacists in Brazil has shifted from simply dispensing medications to providing clinical pharmacy services. Examples of such services according to the Brazilian Board of Pharmacy include pharmacotherapeutic follow-up, drug reconciliation, and drug dispensing, which are clinical services most commonly performed in community pharmacies worldwide. Prior to taking the drug, the patient’s last point of contact with a health professional is during drug dispensing (for either nonprescription or prescription drugs). As part of drug dispensing, community pharmacists can screen patients for contraindications and counsel patients regarding the correct use of drugs.1-4
However, in several countries, the process of drug dispensing is inadequately performed.5-7 For example, the mean time to dispense a drug in Brazilian community pharmacies is less than one minute.8-10 Furthermore, studies have demonstrated that dispensing of nonprescription and prescription drugs was commonly done in an unsatisfactory way and that community pharmacists’ knowledge regarding drug dispensing was rated as “low.”10-14
Some of these findings may be attributed to the lack of focus on clinical activities in associated curricula, and to the low quality of pharmacy courses in several countries.5-7 In Brazil, few courses in pharmacy school focus on developing communication skills and pharmacotherapy knowledge. According to the Brazilian Board of Pharmacy (CFF), 77.6% of pharmacy courses in Brazil have medium- or low-quality standards.15,16
Faced with this reality, the need to establish innovative ways of training community pharmacists in drug dispensing to promote the rational use of drugs, the health recovery of patients, and the prevention of comorbidities has become evident. In this context, distance-learning courses seem to be a worthwhile strategy for training pharmacists. Online education is an option for providing continuing education programs because it offers practicing professionals the opportunity to continue working while engaging in an active-learning environment that allows them to proceed at their own pace and at their convenience.17-19 This study aimed to evaluate the effectiveness of a distance-learning course on drug dispensing practice formulated to train community pharmacists in this subject. The study hypothesis was that participating in a distance-learning course would improve community pharmacists’ knowledge and better qualify them to dispense drugs.
METHODS
This was an open, non-randomized, single-grouped, diagnostic interventional study, in which a free distance-learning course was developed to train pharmacists in drug dispensing. The administrative, didactic, and academic structure of the course was previously described elsewhere.20 The course, titled PharmDisp (Pharmacists in Dispensing Drugs), was developed in Modular Object-Oriented Dynamic Learning Environment (MOODLE) (Perth, Australia) because the platform is internationally recognized for promoting new strategies for learning and has functional tools to manage the teaching process. Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) was used to plan this research.21
The study was performed with pharmacists who worked in Brazilian community pharmacies. The recruitment of participants was done through social networks, emails, websites, instant messaging applications for mobile devices, and scientific meetings. The established criteria for participation in the course were to have a bachelor’s degree in pharmacy, to be properly registered with the State Board of Pharmacy, and to be an employee or owner of a community pharmacy.
The course content focused on the dispensing of pharmacological and nonpharmacological treatments of high-prevalence noncommunicable chronic diseases, antimicrobials, nonprescription drugs, drug fractionation, and health law, as recommended by the American College of Clinical Pharmacy for clinical training.22 Collaborative learning was explored to encourage critical thinking, exchange experiences, and enable the collective construction of knowledge. Discussions were mediated by faculty members who began the sessions by asking guiding questions concerning a clinical situation previously presented in the theoretical course supplies. For each unit, the pharmacists were evaluated for their participation in discussions and completed exercises involving practical issues in drug dispensing.
Assessment of course effectiveness was performed based on Kirkpatrick's precepts for continuing education programs: learner satisfaction=reaction (level 1), learner outcomes=learning (level 2), performance improvement=behavior (level 3), and patient or health outcomes=results (level 4).23 Achieving each of these levels increased the clinical significance of the results obtained. Therefore, the primary outcome considered in this study was pharmacists’ increased knowledge about drug dispensing (ie, learning). Secondary outcomes were pharmacists’ degree of satisfaction with the course (learner satisfaction – reaction) and changes in pharmacists’ professional behavior during drug dispensing (performance improvement – behavior).
Enrolled participants completed a sociodemographic questionnaire prior to beginning the course. The questionnaire included questions on a participant’s age, gender, place of practice (city and type of establishment), working conditions (weekly workload, time of employment, income, benefits, commission on sales, and incentives for training), academic education (institution attended, year of graduation, and continuing education activities completed after earning a bachelor’s degree), and length of time they had been dispensing drugs.
Increase in knowledge was measured by comparing participants’ performance on the same evaluation administered before and after the course. A previously validated questionnaire24 that had been conceptually structured according to the contents of the PharmDisp modules was used to assess participants’ knowledge of drug dispensing and was scored from 0 to 10 points. Although identical, the evaluations were referred to as the diagnostic evaluation and final evaluation to avoid bias.
Participants’ satisfaction with the course was measured using a questionnaire completed at the end of each course unit. The questionnaire was composed of items addressing the following variables: participants’ self-perception of their achievement in the unit; self-perception of the knowledge they had obtained about drug dispensing; satisfaction with the unit load; satisfaction with the language used in course materials; satisfaction with the use of a virtual platform; coherence of evaluation activities with the content studied; satisfaction with the available theoretical and complementary supplies; satisfaction with the course tutor; self-perception of degree of preparation for drug dispensing after having studied the unit. Response options for each item were rated on a Likert scale that ranged from fully agree=1, agree=2, neither agree nor disagree=3, disagree=4 to strongly disagree=5.
Changes in participants’ conduct were evaluated using the mystery shopper technique, which was performed before and after the course. The process of performing this type of evaluation has been described elsewhere.20 The mystery shopper technique involved a trained actor posing as a shopper entering the pharmacy and presenting a valid prescription to the pharmacist to be filled. The actors were properly trained to avoid bias and only the researchers knew the identity of the collaborators who acted as mystery shopper. The entire evaluation process was recorded by a hidden camera. Weeks prior to the mystery shopper evaluation, researchers went to the pharmacies where the pharmacists worked, explained the evaluation and invited them to participate. Those who accepted signed an informed consent form. A jury composed of university professors (PhD) and researchers with experience in dispensing medicines subsequently analyzed the images and scored the pharmacists in the following categories: provided information (posology, dosage to be used, potential adverse effects, precautions, and drug interactions); counseling (used appropriate language and confirmed that the patient understood the instructions); communication skills (the order of facts, concepts presented, and nonverbal communication). Pharmacists could receive up to 260 points on the evaluation. The highest possible score in each category was 20 points for provided information, 90 points for counseling, and 105 points for communication skills. In addition, jury members could award up to 45 extra points for their perception about the pharmacists’ dispensing in general (for a total of 260 points). For logistical and financial reasons, only pharmacists from the host city of the study (Ribeirão Preto, Brazil) were evaluated using the mystery shopper technique.
The fourth step of the evaluation process, results (ie, patient or health outcomes=results) was not carried out because the mystery shoppers were not actual patients. Participants who completed the training course with a final score greater than or equal to 70% were considered approved. Participants who did not complete the course were contacted by email and asked to explain their reason for dropping out. Participants who did not take part in at least two units (25%) of the course were considered dropouts.
Sample size was preliminarily estimated using a specific formula that provide desired power for testing the significance of the difference between baseline-to-endpoint change scores in two treatment groups.25 In this way, we established that a minimum sample size of 10 participants considering the difference among endpoint means (x1–x2=40%), within-groups error variance (SD=1.12) at cross-section in time, and baseline-to-endpoint correlation (r=.1).9
Analysis of the baseline data (obtained by diagnostic evaluation), performance in each course unit, endpoint data (obtained by final evaluation), and characterization of dropouts (participants who did not complete the course) were performed. Secondary outcomes were obtained through analyses of course satisfaction and the performance of those participants whose drug dispensing encounter with a mystery shopper was evaluated.
Sample normality was analyzed using the Kolmogorov-Smirnov test. In all other analyses, measures of central tendency were used to characterize the data set. Normal distributed data were presented as mean and standard deviation (SD) and nonparametric distributed data were presented as median and interquartile range (IQR).
The Mann-Whitney (nonparametric distributed) and paired Student t tests (normal distributed) were used to analyze the performance of participants at baseline and end of the course. The Spearman test was used to correlate the final score obtained by participants and the knowledge measured in the final evaluation. Skills and abilities of participants evaluated in the mystery shopper were analyzed by Student t and Mann-Whitney tests. Cronbach's alpha coefficient was used to verify the internal consistency between the evaluations performed by the five jurors during the mystery shopper analysis.
All statistical analyses were performed using SPSS, version 23 for Mac (IBM, Armonk, NY). Probability value of p<.05 was considered statistically significant. The study was performed in accordance with the Declaration of Helsinki and was approved by the local ethics committee. The study was registered in The Brazilian Clinical Trials Registry on January 15, 2015.
Following Brazilian law, participants were fully assured of data confidentiality and privacy during the completion of the satisfaction questionnaires. Participants were only enrolled after they signed an informed consent form. Personal information collected through the sociodemographic questionnaire could only be accessed by a single researcher. Video recorded for the mystery shopper evaluation was destroyed two months after the end of study.
RESULTS
Of the 1492 pharmacists who registered to complete the course, 28.9% did not meet the eligibility criteria to be included in the study because they did not work in community pharmacies (as an employee or owner). Thus, 1061 participants were selected. Most of the participants were female (n=832; 78.4%) and had a mean age of 31 (SD=6) years.
The majority (70.6%) of participants had only a pharmacy degree and 44.9% of them had a specialist degree. About 74.5% of participants graduated from private universities. Approximately 80% of participants had completed their undergraduate pharmacy degree as of 2009, and 58.7% had attended further education and/or other short courses. For 57% of the respondents, PharmDisp was their first experience with taking a distance-learning course. Fifty-three percent of participants stated that they had not received any incentive from the pharmacy they worked for to pursue additional professional training.
The diagnostic evaluation was completed by 99.2% of participants. The median score was 3.8 (SD=1.9) points (minimum and maximum scores = 0 and 8.1 points, respectively). Increases in knowledge were significant in all units in comparison to baseline knowledge as measured by the diagnostic assessment (Table 1).
Comparative Analysis Between the Acquired Knowledge in Each Unit of the Course and the Baseline Knowledge (Diagnostic Evaluation)
Of the total number, 472 participants completed the final evaluation. The median score was 5.7 (2.8) points (minimum and maximum scores = 0 and 10.0 points, respectively). Achieved scores after the course were higher than at the beginning [5.7 (SD=2.8) vs 3.8 (SD=1.9) points, p<.05, respectively]. Analysis of the data showed that participants’ knowledge was greater after the course than before (1.7, SD=1.7, p<.05). All participants who completed the final evaluation also completed the diagnostic evaluation.
There were 580 withdrawals, of which 558 (52.6%) participants dropped out of the course and 22 (2.1%) took the entire course but did not take the final evaluation. Those participants who did not finish the course were asked why they had not done so, and 141 provided their reasons for withdrawal. No differences were seen between the sociodemographic profiles of those participants who started the course and those who withdrew from the study.
Satisfaction scores of participants remained steady throughout the course and were close to the maximum possible score (weight 1). The satisfaction questionnaire was made available at the end of PharmDisp, and was completed by 677 participants (including those who dropped out). In this survey, the free course was assessed with the highest degree of satisfaction by 71.8% of the participants. Similarly, flexibility for the study was assessed with maximum satisfaction by 56.4% of pharmacists. Also, 639 (94.4%) participants indicated that the training fulfilled their expectations, and 653 (96.5%) considered themselves more qualified to perform drug dispensing after completing the course.
Twenty-six participants from Ribeirão Preto, Brazil enrolled in the course and were the only ones eligible to be included in the mystery shopper evaluations. Ten (40.0%) of the 26 were evaluated using the mystery shopper technique in two different visits to the pharmacy where they worked (one before and one after the course) (Figure 1). Fourteen pharmacists were not included because they either did not agree to participate in the mystery shopper evaluation, the pharmacy manager did not authorize their participation, or they were not located in the pharmacy where they had indicated they worked. Additionally, two pharmacists were dismissed or resigned from the pharmacy where they had been working at the beginning of the course and so were excluded from the mystery shopper evaluation.
Flowchart of Participant Distributions in the Evaluation by Mystery Shopper Technique
Pharmacists who participated in the mystery shopper encounter achieved average scores of 18.7 (7.1) points and 17.4 (11.6) points (p=.47) before and after completing the course, respectively. Of the 10 pharmacists who participated in the mystery shopper evaluation, only four (25%) participants performed better in drug-dispensing practices after the course than before. Participants performed counseling and communication skills better during the second visit by the mystery shopper than during the first. However, in terms of providing information to the “patient,” the participants’ post-course performance was rated lower (Table 2). Variations in performance before and after the course were not significant.
Paired Student t Test of Final Evaluation versus Diagnostic Evaluation (n=472)
Pharmacists’ drug dispensing skills improved overall when baseline was compared to endpoint (mean score of 3.7 vs 4.5 out of 5, p>.05). Evaluations made by the five judges had internal consistency (α=1.0), both in the categoric analysis and in the general perception. Of the 10 participants in the mystery shopper evaluation, eight (80%) completed the course. Among them, the median scores obtained on the diagnostic evaluation were 5.7 (SD=4.3) points (minimum and maximum scores = 3.3 and 9.1, respectively) and in the final evaluation were 3.3 (SD=3.1) points (minimum and maximum grades = 1.0 and 7.1, respectively) (Table 3). No correlation was observed between the performance in the mystery shopper encounter and in the theoretical evaluations (diagnostic evaluation and final evaluation; ρ=.6, p>.05, respectively).
Performance of Pharmacists in the Evaluation by the Mystery Shopper Technique, by Analyzed Category (n=10)
DISCUSSION
One of the main innovations of this study was the attempt to change the practice of community pharmacists in Brazil by improving drug-dispensing skills. We found that a distance-learning course was effective in increasing pharmacists’ knowledge about drug dispensing, but it did not have a significant impact on their conduct while providing these services. According to Bloom, the educational process must be structured in the cognitive, affective, and psychomotor domains, which has not been explored sufficiently in training in the health sciences.26,27 PharmDisp contemplated the cognitive and the affective domain more broadly (through collaborative learning structured in discussion and sharing of experiences activities). The quality of the didactic resource and the interactive asynchronous forums mediated by tutors are also plausible variables to justify this result.28,29 Nevertheless, physical limitations inherent in distance teaching did not allow the psychomotor domain to be satisfactorily explored in our course.
Participants’ mean performance in each of the course modules was lower in modules III (dispensing drugs in hypertension) and IV (dispensing drugs in diabetes mellitus) (Figure 2). Although dispensing methods were discussed in module I (introduction to dispensing drugs), the discussion was theoretical and did not impact the long-term knowledge of pharmacists and proved the difficulty that they experience in dispensing.10,12,14,30-32 The discussion was purely theoretical and did not explore skills and attitudes (practice). As we advance to modules III and IV, where it was necessary for pharmacists to know how to dispense drugs in the context of hypertension and diabetes mellitus, their performance dropped. This showed that the theoretical discussion of module I was not sufficient to prepare them for a more applied discussion of dispensation, which happened in modules III and IV. Conversely, successive improvement in pharmacists’ performance in modules V (dispensing drugs in dyslipidemia) and VI (dispensing dugs in asthma) and the supplementary material (pharmaceutical care) proved a greater theoretical appropriation of the proposed method and an increase in knowledge about drug dispensing. This directly reflected on the performance of pharmacists in the course, who achieved their best scores in these modules. However, the possibility of a "natural selection" process occurred in the last modules must be considered. The improvement of pharmacists' performance in the final modules of the course may not be related to the actual gain in knowledge, but rather to the fact that the best pharmacists were able to go to the end of the course ("natural selection") while those with more difficulty left the course. The professionals got more involved with the studies, since the trend to quit decreased from module V.
Total Number of Participants Per Module and Mean Performance at Each Stage (n=1061)
These results would not be different if PharmDisp had been developed for another platform (digital resources such as CD/DVD, web conferencing, audio or audio/video lessons, printed materials). A review of the literature found no difference in learning effectiveness between these various learning platforms, either by the use of one or a combination of more than one tool.33 However, the option of exclusively classroom or long-term training courses (with more than eight modules or more than 20 hours) compromises the learning effectiveness at Kirkpatrick evaluative levels.23 Also, this phenomenon was corroborated by other authors,31,33 and may partially explain the decline of performance observed in the middle of the course.
The dropout rate during the course was significant. It has been reported that values above 40% are common in distance-learning courses.31 The minimum study time required per day (one hour) and duration of the course (90 days) may have influenced the number of dropouts. Lack of time has been reported in other studies as a barrier to effective participation in distance courses.34-38 Dropout rates tend to be significantly lower when the course lasts for no more than three days,31 which is much shorter than that established for PharmDisp.
Charging fees for the course could have been a way to curb the high level of dropouts. Conversely, this decision would have certainly discouraged many of the pharmacists from participating in the training course in the first place. Not having to pay any fees is a key factor for adherence in distance courses,39 which was confirmed by the satisfaction for this type of education.28,29
Despite the positive results observed in Kirkpatrick's levels 1 (satisfaction) and 2 (knowledge) analyses,23 no improvement occurred in the conduct (level 3) of the community pharmacists in terms of the skills and abilities assessed in dispensing practice. The American College of Clinical Pharmacy considers performance improvement legitimate when the professional acquires knowledge, skills, and attitudes to promote the rational use of drugs, thereby providing patients with a better quality of health care.40,41 Poorer performance by pharmacists in the simulation performed after the course may be a reflection of the results obtained in the final evaluation. However, it is also important to consider that the market pressures on sales in pharmacies (eg, sales commissions) may represent a factor that interferes with patients' access to drugs and instruction in their correct use.13,14,47,48 The Brazilian law that characterized pharmacies as health establishments and ensured the pharmacist greater professional autonomy was aimed to minimize this type of interference.44 However, pharmacists must assume their role as drug dispensers and they must not allow market influences to direct their conduct to the detriment of the knowledge they have for the proper professional practice.
The use of the mystery shopper technique minimized bias caused by the Hawthorne effect, which is characterized by a momentary change in the behavior of professionals because they knew they were being evaluated, and reinforced that effectiveness of health education should encompass the four steps proposed by Miller (knows, knows how, shows how, does).45,46 Nevertheless, the importance of using distance education as a way to qualify pharmacy professionals for dispensing drugs should not be minimized as the increase in pharmacists’ theoretical knowledge obtained was significant. However, our results show that for satisfactory preparation of students and professionals to perform clinical activities, the course should assess their skills and abilities in order to promote clinical thinking, search for scientific evidence for correct decision making, communication with patients and interrelationship with health team.39,47
Although there was no control group in the study, the design was appropriate to test the effectiveness of the proposed intervention. Also, the sample size included in the mystery shopper evaluation was limited for logistical reasons but necessary to assess the participants’ performance improvement after completing the course. Nevertheless, the uniqueness of the data and its consistency with previous reports in the literature make it relevant.
CONCLUSION
The results of this study confirmed that a distance course may be effective in training pharmacists in relation to drug dispensing, but there is no evidence that this strategy is sufficient to make more satisfactory the realization of the service when used exclusively in the distance modality. Practical activities that develop the performance and conduct of professionals should be explored throughout the training. In addition, pharmacists need to engage in more conducive behaviors with patients during drug dispensing, not allowing marketing pressures to impact the quality of their service. Educational strategies should be adopted in future studies to address the lack of practical activities in distance learning educational strategies.
ACKNOWLEDGMENTS
The authors thank the staff at Ribeirão Preto Information Technology Center (CeTI-RP) for offering technical support to the online course; the School of Pharmaceutical Sciences of Ribeirão Preto, University of São Paulo for providing conditions for the course to be conducted within the institution; the National Council for Scientific and Technological Development (CNPq) for funding; and to the tutors for collaborating in the mentoring process of the course.
- Received November 7, 2017.
- Accepted September 18, 2018.
- © 2019 American Association of Colleges of Pharmacy
REFERENCES
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.