Use of a General Level Framework to Facilitate Performance Improvement in Hospital Pharmacists in Singapore

INTRODUCTION

Hospital pharmacists are essential members of the multidisciplinary team who promote rational and cost-effective use of medicines and improve patient outcomes by reducing morbidity, mortality, adverse drug events, and hospital length of stay.^1-8 The role of pharmacists using evidence-based practice to ensure patient safety and the best use of medicines has been endorsed by the World Health Organization (WHO) and government bodies at a global level.^9-11 As the burden of disease increases because of an aging population with multiple comorbidities, there is growing pressure on pharmacists to improve patient outcomes in all developed and developing countries. To maximize improved patient outcomes, it is essential to have an adequate supply of appropriately educated and clinically skilled pharmacists.

The Singapore government has responded to this changing landscape by increasing staffing levels of healthcare professionals by 40%, including funding extra pharmacists whose clinical role in Singapore is being increasingly recognized and developed.^12,13 Approximately 65% of hospital pharmacists in Singapore have less than 3 years of experience. Considering the relative inexperience of most pharmacists in Singapore, a regular structured system of clinical mentoring and practitioner development, including standards for practice, self-reflection, peer evaluation, feedback, and directed learning would be an added benefit to the academic postgraduate opportunities already available. In a survey of the Australian Hospital Pharmacy workforce, a third of pharmacists indicated that they would leave the department within 2 years if support for staff development were not available.¹⁴ A structured, robust, evidence-based tool for practitioner development could assist in ensuring a competent workforce and act as a primary motivator to increase job satisfaction and, hence, retention. Such a tool could also identify common development areas for an entire cohort, direct training programs, and enable departments to set and monitor service standards.

While no single model may be appropriate for all cultures and contexts, there are significant global health and labor market drivers to suggest that a competency-based approach to professional development is sensible and sustainable.^15-17 The Competency and Education Development Group (CoDEG) in the United Kingdom used the Whiddett and Holyforde model as a basis for developing the General Level Framework, a competency-based performance-development tool for general- or foundational-level pharmacists (those with less than 3 years post-registration hospital experience).^18-21 The general pharmacist practitioner is the second of 4 levels of practice previously described by the CoDEG: registered pharmacist, general pharmacist practitioner, advanced pharmacist practitioner, and a consultant pharmacist.

The GLF was developed as a ward-based tool to facilitate the development and evaluation of the general pharmacist practitioner by means of direct observation of their practice.²² This process was subsequently validated in general pharmacist practitioners in UK hospitals by Antoniou and colleagues. This study demonstrated that practitioners who received feedback on their performance and agreed upon a development plan using the GLF up to 3 times in a 12-month period reached and maintained a defined level of competence faster than did those who were observed without this intervention.²³

The GLF provides a structure for the development of pharmacists in their professional capacity. It is not meant to replace formal qualifications but rather to complement them. The GLF is a useful tool for engendering awareness of practice standards, facilitating self-reflection, providing a platform for feedback, and planning needs-based learning under the guidance and accountability of a more experienced practitioner. Further, it enables the identification of strengths and setting of individualized learning objectives to target weaknesses.

Learning objectives may be achieved in several different ways. In some cases, directed reading and discussion may be sufficient (eg, looking up common dosing regimens or antibiotic susceptibilities). However, because demonstration of the ability to apply learning is as essential as the learning itself, if not more so, more creative approaches to achieving objectives may also be considered. These include case presentations on an unfamiliar topic, literature reviews and discussion of findings, and presentations to other healthcare professionals (eg, presentations on crushing of solid dosage forms to nursing staff members or on the use of intravenous lines to pharmacists). Achieving objectives could even entail developing guidelines or conducting audits and research. In some cases, the GLF may identify areas for formal training, although it largely prompts self-reflection, promotes a culture of on-the-job, needs-based, lifelong learning, and directs continual professional development.

By setting objectives that are achievable in an assigned tangible timeframe and that are useful to both the organization and the individual, pharmacists gain confidence and recognition, which improves job satisfaction, build on their knowledge and skills base, and in many cases, educate others in the process, which further improves overall patient care.

Since the introduction of the GLF, similar frameworks have been developed globally that have also contributed to achieving and measuring improvements in performance over time in both hospital and community pharmacy settings.^24-29 In fact, the current global interest in efficient programs for development and evaluation of competencies for pharmacists resulted in the International Pharmaceutical Federation forming an ongoing partnership with WHO and the United Nations Educational, Scientific and Cultural Organization in 2008 (Global Pharmacy Education Taskforce www.fip.org/education). This partnership included an objective to develop a globally acceptable competency framework.^16,30,31

Because of the similarities between the role of a hospital pharmacist in Singapore and that in the United Kingdom and Australia, versions of the GLF from the United Kingdom and Australia were reviewed and components from each adapted for use in a large tertiary hospital in Singapore by selecting and mapping relevant GLF competencies to the applicable Singapore Pharmacy Council competencies.^19,25,32 A working group of junior and senior pharmacists who would be using the tool were consulted on the draft, a pilot was conducted, and revisions were made based on the feedback. The final Singapore-adapted GLF consisted of 63 behavioral statements grouped within 14 competencies and distributed into 3 competency clusters: delivery of patient care, problem solving, and professional (Tables 1, 2, and 3, respectively). The performance-level rating for each behavior, which was based on how frequently it was demonstrated in practice, ranged from rarely to consistently. The version of the GLF adapted for use in Singapore was initially introduced at 1 large government hospital in June 2009, with the expectation that it would be expanded to other SingHealth Institutions prior to island-wide implementation.³³ The aim of this prospective cohort study was to evaluate the acceptability and validity of the Singapore-adapted GLF as a tool for the facilitation and evaluation of performance development for a group of general pharmacist practitioners in a large tertiary hospital in Singapore.

METHODS

The 8 GLF facilitators at the study site attended 2 in-house training sessions introducing the concepts of the GLF as a tool for performance development, evaluation, and feedback. Practical application was addressed further by selecting 2 “super trainers” from each institution within the SingHealth cluster, including 2 from the study site. Along with several other GLF facilitators at the study site, super trainers attended 2 half-day training seminars led by CoDEG and Medication Services Queensland (MSQ) on principles of adult learning, effective feedback, and practical use of the GLF and its associated tools. Following these seminars, each super trainer conducted a self-evaluation using the GLF before observing a member of CoDEG or MSQ completing a GLF evaluation and providing feedback to himself/herself and a peer. Super trainers were subsequently responsible for providing training to others at their site(s) of practice. Each super trainer completed a feedback questionnaire.

All general pharmacist practitioners at the study site attended an in-house training session on the principles of the GLF and associated professional-development tools. Additional seminars on workplace education and work-based learning were provided by CoDEG and MSQ and attended by most GLF pharmacists at the study site. Feedback was sought from the facilitators and the general pharmacist practitioners at regular intervals.

Thirty-five general pharmacist practitioners working in the inpatient setting at 1 large tertiary hospital in Singapore were enrolled in the study. GLF facilitators required general pharmacist practitioners on their clinical team to conduct a self-reflective assessment of their current level of performance using the GLF. This instrument was subsequently used to conduct baseline and repeat observational evaluations during daily clinical activities and subsequent feedback from which individualized learning plans were formulated. Learning objectives were set and progress reviewed by a designated GLF facilitator in accordance with an agreed-upon timeframe. All pharmacists underwent a minimum of 2 evaluations over a median of 9 months. At each visit, the performance of the individual for each behavior was rated from 1 to 4 (1 = rarely; 2 = sometimes; 3 = usually; 4 = consistently). If a behavior was not observed, it was categorized as “not assessed.” Baseline scores were calculated from the first evaluation in the study period and repeat scores were calculated from the last.

A password-protected database was created in Microsoft Excel and the facilitators entered results for each GLF evaluation. For each behavior, median scores and ranges were calculated and the difference between individual baseline and repeat observed performance was analyzed using the Wilcoxon paired signed-rank test. Mean scores for competency clusters were compared to illustrate the change in performance from baseline to repeat visit. Analysis was carried out using the Statistical Package for Social Sciences (SPSS version 17; IBM, New York). Ethics approval was applied for and waived by the study site’s institutional review board.

RESULTS

Thirty-five general pharmacist practitioners working in the inpatient setting at a large tertiary hospital in Singapore underwent baseline and repeat observational evaluations between June 2009 and December 2010. All but 1 of the pharmacists enrolled in the study were female (97%). At baseline, pharmacists had a median of 2 years of post-registration experience (range, 1 to 4). The median time to repeat observation was 9 months (range, 4 to 10), and pharmacists had a median of 3 observation visits (range, 2 to 4).

An improvement in the mean competency cluster score was demonstrated for all 3 clusters from baseline to repeat evaluations (Figure 1). Of the 63 behaviors analyzed, all but 8 showed significant improvement (p ≤ 0.05) between baseline and repeat observations (Tables 1, 2, and 3). In all cases, nonsignificant improvements were the result of pharmacists being perceived to already be practicing at the highest performance level at the time of initial baseline evaluation. No decrease in performance was demonstrated in any behavior.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Changes in mean individual scores by competency clusters.

Significant improvements in performance were seen in all behaviors in the problem-solving cluster (Table 2), although baseline and repeat median scores were generally lower in this cluster compared with the others. Problem-solving competencies included behaviors relating to problem identification, knowledge, analysis and recommendations, and follow up.

Six of the behaviors that failed to demonstrate significant improvement were in the patient-care competency cluster. Five of these related to the provision of medication competency and 1 to the patient-education competency (Table 1). All other patient-care competencies (patient consultation, information gathering, drug-specific issues, and risk management and service improvement) showed significant improvement.

The 2 other behaviors that failed to demonstrate significant improvement were in the professional cluster, relating to confidentiality and recognizing the value of team members, which were categorized within the professionalism and teamwork competencies (Table 3). The remaining 2 professional competencies (organization and communication skills) did show significant improvement.

Learning objectives were achieved in a variety of ways: directed reading and discussion of findings, case presentations or presentations of literature, attending specific training courses or undergoing certification, conducting audits, reviewing intervention records or maintaining and discussing care plans with facilitator, and observing improved practice.

Of the 21 first- and second-year pharmacists at the study site who underwent the first round of evaluation with the GLF, 81% agreed that it “added value to their learning experience.” Comments from the super trainers highlighted that the GLF process was an overall positive experience that prompted reflection on practice, and culminated in needs-based learning and ultimately improved patient care, although some reported that they found the process to be taxing (Figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Feedback from super trainers following General Level Framework evaluation (N=14).

DISCUSSION

The Singapore-adapted GLF was well-received and proved a useful educational and development tool that demonstrated improvement in performance over time. The majority of pharmacists who undertook an evaluation using the tool at the study site felt it made a positive contribution to their learning experience. Feedback from the super trainers was encouraging and indicated that the whole GLF process was well-received as a way to evaluate many different aspects of practice (ie, attitudes, knowledge, and skills), provide inspiring and practical feedback, prompt reflection on practice and thus provide a platform for needs-based lifelong adult learning (Figure 2). The few perceived negative aspects included feeling uncomfortable being observed and providing feedback to a peer. Others found the process taxing, a sentiment echoed by colleagues in the Queensland study.²⁴ Both of these aspects are expected to become easier with time as the GLF is integrated into daily practice and a more open learning culture develops.

Many pharmacists found the process of self-reflection on their own practice almost as important as the observed evaluation and feedback process. The process of self-reflection, which is a significant component of the GLF, is important to facilitate a greater understanding of defined and accepted expectations and is consistent with the adult learning principles of self-reflection, feedback, and needs-based learning. Once learning objectives were identified using a combination of self-reflection, evaluation, and feedback, different strategies were used to meet these objectives. Some simply involved reading and discussion of findings (where possible in relation to a live case), maintaining and discussing care plans, or reviewing intervention records. Others consisted of giving case presentations, conducting literature reviews and presenting the findings, writing summary documents on new research findings pertinent to clinical area, or demonstrating improvements in practice by training others in a task and being observed performing activities such as taking medication histories. Some cases involved more formal training, such as an in-house intensive-care training course or certification using outside providers (eg, the fundamentals-of-critical-care support course offered by the Society of Critical Care Medicine).

Of the 63 behaviors analyzed, all but 8 demonstrated a significant improvement between baseline and repeat observations. The 8 cases in which no significant improvement was observed were a result of pharmacists being perceived to already be practicing at the highest performance level during the baseline evaluation. The competencies that this applied to related to tasks that were widely accepted as comprising fundamental roles of a hospital pharmacist in Singapore, an observation that has also been made in other studies.^23,24

The behaviors in the problem-solving cluster all demonstrated significant improvement as well as the largest change in performance over time out of the 3 clusters (Table 2). This finding reflects research results from other countries using this tool and emphasizes the large growth in problem-solving skills that takes place in the first years of practice.^23,24 This cluster measures knowledge-based behaviors, such as describing the pharmacology of drugs, pathophysiology of disease, and mechanisms of interactions, which require a more complex understanding and continual learning, compared with the process-based behaviors that comprise the majority of the other 2 clusters. Process-based behaviors that involve the learning and refining of processes related to daily activities appear to be mastered more quickly than knowledge-based behaviors, which are subject to continual improvement and can be fully refined only over time in conjunction with appropriate mentoring and continued professional development.¹⁵

The lower median scores demonstrated in the problem-solving cluster, compared with those in the other 2, suggest that this cluster may show the greatest improvement from continual mentoring and guidance. They also highlight the gap between theory learned at university and during the preregistration year and the application of knowledge and skills in practice. Parts of the United Kingdom have addressed this gap with the introduction of a Diploma in General Level Pharmacy Practice, which incorporates the GLF to support the development of junior pharmacists using work-based self-directed learning and case-based assessments under the mentorship of a more experienced practitioner.³⁴ In response to the learning needs identified by the GLF in this study, pharmacists at the study site were divided into clinical teams, each of which was led by a designated senior pharmacist. Teams met each week to discuss cases, share learning experiences, and review GLF training needs.

Eighteen of the 24 behaviors in the patient-care cluster demonstrated significant improvement in performance, including the behaviors around medication history-taking and allergy documentation (Table 1). When carried out by pharmacists, these activities have been demonstrated to be more complete and to result in reduced mortality.^7,35-37 Therefore, they are essential components of a clinical pharmacy service that must be performed at the highest level to ensure optimal patient outcomes.

Six of the behaviors that failed to demonstrate significant improvement were in the patient-care competency cluster. Five of these related to the provision of medication and included ensuring that prescriptions were unambiguous and legal, medication labels contained the required information, the correct drugs, patients, and labels were provided, and supplies of medication were documented. These supply-related behaviors are traditionally seen as some of the most well-established roles of the pharmacist and were being performed at the highest level at the baseline evaluation. Interestingly, “ensuring medication availability,” which was 1 of the 5 behaviors within the provision of medication competency, did show significant improvement, suggesting that this was perhaps not as obvious a role among general pharmacist practitioners as the other 4 supply-related behaviors, and highlighting how the GLF can be a useful tool to set standards and ensure uniform provision of services.

A similar trend was seen in the behaviors relating to the patient-education competency. In this competency, provision of appropriate oral/written information did not show significant improvement over time because of the high level of baseline performance, but the advice on non-pharmacotherapy treatments and assessing the patients’ understanding of the information they had been given did show improvement. “Advice on non-drug therapy” scored a median of 3 at both time points; although significant improvement was demonstrated, it was the lowest-scoring behavior in the patient-education competency. This finding raises the issue of whether it is necessary for pharmacists to aim for the top performance level in all behaviors or if it might be more advantageous for a department to prioritize by setting minimum standards of performance based on staffing, expectations, and targets.

The professional competency cluster was the highest scoring of the 3 on repeat evaluation and contained the final 2 behaviors that failed to demonstrate significant improvement. These behaviors related to confidentiality and recognizing the value of team members (Table 3). Again, performance was already considered to be at the highest level at the time of the baseline evaluation, suggesting that these values were instilled early in the pharmacists’ careers. The 2 behaviors in this cluster that ranked lowest at both time points were “demonstrates confidence” and “active in educating and training healthcare professionals.” It is possible that these 2 behaviors are linked, and pharmacists will become more involved in education and training as they gain experience and, therefore, confidence.

The GLF demonstrated improvements in performance over a median of 9 months, demonstrating that it is a valid tool for measuring performance over this timeframe. There was also potential for further performance improvement after 9 months, suggesting that this tool would be a valid development aid beyond this time period.

This study is the first analysis of the use of the GLF in an Asian setting. The introduction of this tool was received with generally positive feelings in a culture where knowledge-based assessment is traditionally favored over competency-based programs. However, a shift in thinking is occurring, in which the value of individualized competency-based development in addition to (rather than instead of) academic merit is being recognized by many of the most influential clinical pharmacy leaders in Singapore. The GLF has now been adopted by a significant number of public healthcare institutions, with others expected to follow in the near future.

Pharmacists were initially nervous at the thought of being observed in practice but also quite excited at the prospect of having help from a more experienced mentor to develop their practice. Pharmacists are traditionally accustomed to working alone, and it is rare to have someone observe their practice after the initial training period. Although hands-on training and observation are key components to the postgraduate development and training of medical staff members and other healthcare professionals in Singapore, clinical pharmacists have largely been left to develop and, in many cases, pioneer their own practices. Some thrive on this challenge, but many others require more support, empowerment, and instruction.

Another argument in support of observing professional practice is that all practitioners should be open to having their practice reviewed by peers. Such interaction provides the opportunity to ensure practitioners are providing safe and effective care to patients and to congratulate those who achieve identified goals and perform tasks according to standards. The GLF plays a pivotal role in clinical governance in that it can be used as an effective tool by heads of pharmacy to measure how pharmacists are performing according to defined and accepted standards.

The current study demonstrated significant improvements in 87% of all behaviors (n=63) evaluated over a median of 9 months. This compares with 95% (n=58; p < 0.05) at 6 months (sustained at 12 months) in the original London study, and 57% (n=61; p < 0.05) over a median of 14 months in the Queensland study.^23,24 Performance improvements are comparable between the London and Singapore cohorts, but there is an obvious disparity in the Queensland results. Although the competency frameworks used in the 3 studies contained different behaviors so that a direct comparison is not feasible, some observations can be made.

The weighting of behaviors in the London and Singapore frameworks were comparable, whereas the Queensland framework had a larger focus on the patient-care cluster (Table 4). The patient-care behaviors in the Queensland framework were more detailed and included extra behaviors, such as relevant patient background, patient’s understanding of illness, and patient’s experience of medication use. These behaviors generally had lower baseline and repeat scores, perhaps indicating that other behaviors were prioritized over these.

Nine behaviors in the Queensland study (compared to 8 in the Singapore study) failed to show significant improvement due to pharmacists already performing at the maximum level upon initial evaluation. These behaviors mainly related to the professional cluster and the discharge-facilitation competency. The remaining 17 behaviors, which did not significantly improve (although most demonstrated a trend toward improved performance) were primarily in the patient-care cluster. The explanation provided for this finding was that these behaviors were associated with a deeper understanding of medication-related consultation. If the Singapore GLF had been as detailed, perhaps a similar result would have been demonstrated.

Two of the behaviors that failed to demonstrate improved performance in the Queensland study which were also included in the Singapore GLF were “medication reconciliation” and “mechanisms of interactions.” These showed a significant improvement in 1 country but not in the other, a difference that could be explained by variations in expectations and accepted standards of practice between the 2 countries. For example, the Singapore GLF states that medication reconciliation should be done “when appropriate,” whereas in Queensland, it is a standard procedure for all patients admitted to hospital. Perhaps this disparity is attributable to time pressures, staffing levels, an understanding of the importance of this process, or the level of development of clinical pharmacy practice.

Only the Singapore GLF contained a competency related to provision of medication and, on initial evaluation, performance of most of the associated behaviors was maximal. Now that technicians have largely assumed the supply role in Singapore (as in London and Queensland), perhaps these behaviors could be transferred to a technician-level framework.^38,39

The validity, sensitivity, and reliability of the GLF evaluation process has previously been evaluated.^22,23 However, using such a tool is always open to variations in the expectations of individual assessors. To reduce inter-rater variability, the intent was that the same facilitator should complete all evaluations for an individual pharmacist throughout the study. However, in some instances, this was not possible; 20% of pharmacists had 2 facilitators. The GLF can be used as a developmental tool for individual pharmacists, independent of such variations, but the comparison of scores between individuals should be done loosely, taking this potential limitation into account. In response to feedback from this study, a handbook was produced as a reference for trainers and trainees to provide more detailed descriptions of the competencies to facilitate standardization of the process.³³

Feedback was obtained at regular intervals from the general pharmacist practitioners, and 81% of those surveyed indicated that the GLF added value to their leaning experience. Ideally, general pharmacist practitioners will echo the positive feedback received from the super trainers, but this outcome cannot be confirmed without completion of similar feedback questionnaires.

CONCLUSIONS

The General Level Framework as adapted for the Singapore hospital setting was an acceptable educational tool for the facilitation and evaluation of performance development in general pharmacist practitioners. Although a single framework or target level of performance may not be appropriate for all contexts, the GLF is a useful development and evaluation tool that can be tailored to address local cultural needs and meet the expectations of individual institutions.