Pharmacy Students’ Perceptions of Reflective Portfolios and their Effect on Students’ Deep Information-Processing Skills

INTRODUCTION

Learning involves recalling prior knowledge and connecting it to existing cognitive or metacognitive networks of ideas.¹ Three learning styles have been identified: surface, deep, and strategic.² The deep-learning approach enables learners to actively engage with learning materials and relate new ideas to their previous knowledge and experience.² It is preferred among the different learning styles as it promotes complete understanding and long-term knowledge retention and encourages the development of higher intellectual skills, such as reasoning and problem solving.^3,4 For example, medical students who adopted a deep-learning approach performed better on their final examinations than surface learners did.⁵

One way to promote deep learning is through reflection.^6,7 A positive correlation between a deep-learning approach and reflection has been reported.^8-10 The concept of reflection has evolved from the early model of reflective thought by Dewey, to Schon’s theory of reflection-in-action and reflection-on-action, Kolb’s experiential learning theory, Boud’s seven elements of reflective processes, and Mezirow’s transformative learning.^11-15 These form the basis for the reflective concepts framework, which includes three main processes; acquiring new knowledge through exposure to an experience, transformational learning that links academic knowledge with practice, and integrating new knowledge and understanding, which leads to new perspectives and solutions.⁶

According to Tsingos and colleagues,⁶ reflective practice helps pharmacy practitioners integrate theories with practice by promoting critical thinking, problem-solving, and self-directed and lifelong learning, and enhancing communication and collaboration. These outcomes lead to improved clinical decision-making capabilities.^16,17 Also, these skills are essential for pharmacy graduates who need to be able to ask questions, critically appraise new information, identify their own knowledge and skill gaps, reflect on their learning process and outcomes, and work with other health professionals as part of a multi-disciplinary team.

Most adult learners are independent, self-directed, and motivated to learn to satisfy their needs and interest.¹⁸ In order to help students assume responsibility for their own professional development, the reflective portfolio has been used increasingly in health profession education, including in medical and nursing education.^19-21 However, limited research has been conducting regarding the use of portfolios in pharmacy education.⁶ A reflective portfolio is a collection of assignments and self-reflective writings for personal and professional development purposes. It provides evidence of the insights and experiences that a student has gained during a course of study. Through reflection, students develop critical awareness of their strengths and weaknesses and make improvement plans for their skill development. The portfolio enables learners to bridge theory and practice based on real-life experience.²² The Accreditation Council for Pharmacy Education recommends the portfolio as a tool “to document self-assessment of, and reflection on, learning needs, plans and achievements, and professional growth and development.”²³ Other benefits of the portfolio include improvement in knowledge and understanding, enhancement of self-awareness, engagement in reflection, and an improved student-mentor relationship.²⁰

A systematic review of the literature revealed there were relatively few reports on the impact of reflective portfolios on student knowledge, skills, and attitudes.²⁰ In view of the resource intensiveness of a portfolio program, especially in terms of the time commitment of both students and faculty members, identifying evidence regarding the effectiveness of different portfolio types and implementation methods, as well as the optimum academic level for their use, was important. The objectives of this study were to evaluate the perception of pharmacy students on the educational value of the reflective portfolio as they progressed through their academic years, and to gain an understanding of the factors that may influence their perceptions. In addition, the effect of the reflective portfolio on students’ deep information-processing (DIP) skills was investigated.

METHODS

Ethical approval for conducting the research study was obtained from the International Medical University’s Joint Committee of Research and Ethics. Informed consent was obtained from all participants.

The study was conducted with Bachelor of Pharmacy (Honours) (BPharm) students at the International Medical University (IMU), Malaysia. These students enrolled in the BPharm program after completing a pre-university program (ie, A-levels, which is a UK pre-university entrance qualification commonly undertaken by students for post-secondary education in Commonwealth countries that adopt the British education system, or equivalent). The four-year curriculum covered the disciplines of pharmacy practice, pharmaceutical technology, pharmaceutical chemistry, and life sciences, which integrated body system modules. The program prepares graduates to work in the pharmaceutical industry, hospitals, and community pharmacies. The program was fully accredited by the Malaysian Qualifications Agency and Malaysian Pharmacy Board. An outcome-based approach was adopted in the curriculum design, with an increasing expectation on student-centered learning activities as students progressed through the academic years. Besides classroom learning, students were required to participate in a range of formal experiential learning activities. These included practical work, community service, industrial visit, and internship.

The reflective portfolio was introduced for all pharmacy students beginning with year 1 in 2013. The students were briefed on the objectives and guidelines for the reflective portfolio at the beginning of the year. The briefing notes remained available in the e-learning portal for students’ reference. Implementation of the reflective portfolio was guided by the literature review findings of Plaza and colleagues,²⁴ and took into consideration student and faculty buy-in, student-faculty participation, types of learning evidence, student workload, use for learning vs assessment, assessment methods (qualitative vs quantitative) and the availability of a suitable technology platform.

The reflective portfolio was an integrated component of the curriculum whereby, at the end of every semester, students were required to self-assess their competencies in the identified competency domains (including knowledge and skills of the profession, social responsibility/service orientation, communication skills, ethical practice, lifelong learning, and a commitment to self-improvement, evidence-based practice, and teamwork). They were required to submit reflective essays through the Mahara electronic platform (Catalyst IT Limited, New Zealand), an open source digital portfolio tool that allows students to record evidence of learning. The essays were based on their learning experience in the previous semester. These experiences had to be supported by photographs and other evidence of their participation in the activities. Students also had to include an improvement plan in the portfolio.

The portfolios were assessed formatively by mentors, who were faculty members that acted as the students’ personal tutors. The tutors used a grading rubric for reflective writing to assess the portfolio entries and also provided written feedback to each student. Prior to launching the portfolio requirement, the University’s education office trained faculty members on how to give feedback to students. Submission of the reflective portfolio was compulsory, and students were required to resubmit it if the quality was unsatisfactory.

The questionnaire for assessing students’ perceptions of the educational value of the reflective portfolio (Appendix 1) was adapted from that created by McMullan.²⁵ A pilot test of the questionnaire was carried out with 127 third-year pharmacy students from the cohort that commenced the program as first-year students in 2013. Reliability analysis showed the questionnaire had acceptable internal consistency, with Cronbach alpha values lying in the range 0.6 to 0.9 for all domains. Students of the cohort entering the program in 2014 were invited to complete the questionnaire voluntarily at the beginning of years 2, 3, and 4 of the BPharm program.

The fourth-year students were also invited to participate in one-to-one, semi-structured interviews conducted by one of the researchers (MKJM) who was not a faculty member. The interview was based on a set of predetermined open-ended questions intended to explore the key factors associated with the interviewees’ experience with the reflective portfolio, including their expectations, perceptions about assessment, and mentor’s feedback and support. The interviews were recorded with the consent of the participants and transcribed verbatim. Interviews were conducted with 12 female and eight male students.

To evaluate the impact of the reflective portfolio on students’ DIP skills, fourth-year students who had entered the pharmacy program in 2012 (before the introduction of the reflective portfolio) and 2014 (after the introduction of reflective portfolio) were invited to complete the DIP questionnaire reported in Appendix 2 (adapted from Schouwenburg and colleagues²⁶). This questionnaire had been pilot tested in a previous study with pharmacy students in the same institution.²⁷ The academic admission requirements for the comparison cohort and study cohort of students were the same and both cohorts completed an identical curriculum apart from the implementation of the reflective portfolio for the study cohort. The study design is shown in Figure 1. Statistical package SPSS version 18.0 (SPSS Inc., Chicago, IL) was used for the data analysis. Non-parametric tests were used as the data were not normally distributed. The students’ perceptions scores were analyzed using the Kruskal-Wallis test (non-parametric ANOVA) to determine whether there were any significant differences between the scores of students in different academic years. The Mann-Whitney U test was used to determine differences between the scores of two academic years.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Study Design for Pharmacy Students’ Perceptions of Reflective Portfolios and the Effect of the Portfolio on Students’ DIP Skills

The DIP scores of fourth-year students before and after the implementation of the reflective portfolio were compared using the Mann-Whitney U test. The correlation between the DIP scores of the fourth-year students and their total perception scores on the reflective portfolios was determined using the Pearson correlation coefficient.

Based on the transcribed interview data, a thematic analysis approach was used to identify the factors associated with the positive and negative student experiences with the reflective portfolio. Two researchers thoroughly familiarized themselves with the data by going over it repeatedly. They then identified and coded important features of the data (keywords) that might be relevant to the positive and negative experiences of creating the reflective portfolio. Themes were identified by examining the codes and further refined to ensure reliability.

RESULTS

Students from the cohort who began the BPharm program in 2014 completed the perception questionnaire at the beginning of years 2, 3, and 4. Across the academic years of the study, students’ ages ranged from 20 to 24 years. There were between 81% and 84% females and between 16% and 19% males. The response rate for the questionnaire was 81% (157) for second-year students, 66% (119) for third-year students, and 75% (135) for fourth-year students. For the DIP skills, 100 fourth-year students from the cohort commencing the program in 2012 (response rate 56%; 76% female students and 24% male students; aged from 20 to 24 years) and 135 fourth-year students from the cohort commencing the program in 2014 (response rate 75%; 82% female students and 18% male students; aged from 20 to 24 years) completed the questionnaire at the beginning of year 4.

Internal reliability of the perception questionnaire was indicated by Cronbach’s alpha values in the range of 0.46 to 0.93 (Table 1). The domains on personal development (A), professional development (B), portfolio support and guidance received (D), and total perception showed good reliability scores, with an alpha greater than 0.7.

The students’ perception questionnaire data are provided in Table 2. The Kruskal-Wallis test revealed a significant difference in the scores of all domains and total scores among the academic years (p<.01). Students’ perceptions of the educational value of the reflective portfolio increased in all domains from year 2 to 3, and from year 2 to 4. A significant difference (p<.01) with medium to large effect size was observed between the total perception scores between years 2 and 3 (Cohen d=0.95), and between years 2 and 4 (Cohen d=0.44). Nevertheless, the students’ perceptions decreased consistently in all domains from year 3 to 4, and the total perception score for year 4 was significant lower (p<.01) than that for year 3 (Cohen d=0.45).

The interviewees gave their perspectives and views on the reflective portfolio based on their personal experience. Five main themes were identified: understanding of objectives and guidelines for the reflective portfolio; development of personal and professional skills; mentor’s feedback; learning attitude and motivation; and technical accessibility. A summary of the themes and quotations from the interviewees is illustrated in Appendix 4.

Half of the interviewees could not remember the objectives and guidelines of the reflective portfolio. As a result, they prepared it without relating to the objectives and guidelines that were uploaded on the e-learning portal. Meanwhile, those who understood the objectives reflected on what they had done in the previous semester and prepared according to the guidelines. Twelve interviewees expressed that they had a better understanding of what they had learned in the past semester as a result of completing the reflection assignment and could use these insights to improve themselves. On the other hand, two of the interviewees reported that reflection was challenging for them as they had difficulty recalling the activities that they had carried out in the previous semester.

Twelve of the 20 interviewees conveyed that the reflective portfolio contributed to their development in personal and professional skills, such as communication, reflective skills, and self-improvement. Twelve of the interviewees did not think that their mentors’ written feedback was useful. Also, some felt that the effort they spent on preparing the reflective portfolio was not appreciated by their mentor. On the other hand, those who were satisfied with their mentor’s feedback appreciated the suggestions given on areas for improvement. Also, the students expressed a preference for face-to-face feedback sessions rather than written comments

The students’ learning attitude and motivation may have influenced their perceptions of the reflective portfolio. The interviewees who had positive perceptions appreciated its educational value and put effort into preparing their portfolio. However, some felt that it created an extra workload for them and was not beneficial. When the interviewees were asked about their views on changing the portfolio assessment to summative, all of them expressed that they would put more effort into preparing the portfolio if it was summatively assessed (ie, contributed to the cumulative grade point average score [cGPA]). Those who preferred that portfolios undergo a summative assessment felt that clearer instructions and guidelines should be provided. On the other hand, some students did not believe there should be a summative assessment of the reflective portfolio because students might not be honest in their reflective writing in an attempt to receive a higher score.

In addition to the factors discussed above, there was an association between students’ perception of the reflective portfolio and the technical accessibility of the electronic platform for portfolio submission. Four interviewees highlighted problems in the submission of their reflective portfolios using the Mahara electronic platform.

The internal reliability of the DIP scales indicated by the Cronbach alpha values are presented in Table 3. High internal reliability values (alpha greater than 0.8) were achieved for the total DIP scores and the domain scores for “broaden one’s context.” There was a significant difference between the total DIP scores of both groups, with mean rank scores of 106.9 and 126.2, respectively, for the cohorts who did and did not complete a reflective portfolio. The effect size was medium (Cohen d=0.28). In particular, the domain scores for “critical reading” and “broaden one’s context” were significantly higher among the cohort of students who completed the reflective portfolio (p<.05). A strong positive correlation was found for fourth-year students between the total perception score on the use of the reflective portfolio and the DIP scores (Pearson r=0.507, N=134, p=.01).

DISCUSSION

Both the quantitative and qualitative findings from our study indicate the potential of the reflective portfolio to be a useful tool for promoting personal and professional development in pharmacy students. Overall, the percentage of students who agreed with the perception statements in the personal and professional development and support and guidance domains was higher than those who disagreed, and the trend increased from year 2 to year 4 of the curriculum. However, this study found no evidence of improvement in students’ self-esteem and confidence as a result of completing the reflective portfolio. While this finding was similar to findings among one group of nursing students,²⁵ it was contrary to findings among another group of preregistration nursing students.²⁸ The decrease in students’ total perception score from year 3 to year 4 was an unexpected finding and could be attributed to the general student perception that using the portfolio was time-consuming and stressful. In the current study setting, the learning activities in year 4 were mainly self-directed, and thus demanding on the students’ learning time, with the reflective portfolio possibly seen as additional work. Besides, as students’ progress through the later years of the program, they might have higher expectations of themselves as well as the learning environment.

The students’ responses on the general statements in the questionnaire were mostly neutral and consistent with the findings among nursing students reported by McMullan.²⁵ The students felt that completion of the reflective portfolio occupied a large amount of time and caused a lot of anxiety, especially in year 2. The perception of using it as a learning tool became increasingly positive towards the final year, and when given a choice, the students generally preferred assessment of the portfolio to be formative rather than summative.

The students seemed to better understand the objectives of the reflective portfolio as they progressed through the course, although scores for the quality of the feedback they received remained low. This was further confirmed by the interviewees, whose opinions regarding the usefulness of their mentor’s feedback varied. Students perceived useful feedback as that which helped them to identify areas in which improvement was needed as well as strategies to achieve the changes. The concerns raised by students relating to feedback suggests that faculty training in how to assess reflective writing and give effective feedback should be continuous. For feedback processes to contribute to learning and professional development, the standard of performance required of students must be clearly communicated, so that they can work towards closing the gap.²⁹ Moreover, the feedback must be provided in a timely manner and suggest alternative behaviors that will help them achieve their goals.³⁰ A prognostic feedback model should be used which would provide information to students on how a task can be performed differently in the future in order to improve.³¹ Additionally, sufficient time must be allocated for effective student feedback and grading.¹⁶ The students’ self-assessment skills need to be strengthened as well.^32,33 Nicol and Macfarlane-Dick argue that a wider framework extending beyond cognition to self-regulation of motivation and behavior is necessary for good feedback practice.³⁴ To overcome the limitations of written feedback in ensuring understanding and internalization by students, feedback dialogue has been suggested as a way to engage both the teacher and student in a constructive manner.³⁴

From the qualitative analysis of the interview data, the themes affecting the student experience in using the reflective portfolio included understanding of objectives and guidelines, usefulness of mentor’s feedback, students’ attitude toward and motivation for learning, and technical accessibility. Although studies have shown that portfolios can enhance students’ reflective skills and awareness about their strengths and weaknesses,^35,36 the students’ buy-in and commitment to a portfolio greatly depends on the clarity of the objectives and guidelines. The findings are consistent with previous reports that successful use of a reflective portfolio relies on clear guidelines on its purpose, content, and structure.^25,37

The interview findings confirmed that students were engaged in reflection during the preparation of their reflective portfolios; this was consistent with the quantitative data showing that the students perceived their self-awareness, critical thinking, and reflective skills were enhanced as they progressed from year 2 to year 4. Critical reflection is essential for perspective transformation.³⁸ This study also confirmed that using the reflective portfolio can promote deep information-processing skills in students, especially in critical reading (understanding) and broadening their context (elaboration). The engagement of students with the learning process, and their subsequent reflection on actions based on their experiential learning activities, collectively encourage learners to adopt a deep rather than a surface learning approach.²² Moreover, their development of deep information-processing skills was strongly correlated with the students’ perception of the use of the reflective portfolio. This finding is consistent with a previous report that students’ reflective ability is positively correlated with their perception of reflective portfolios.²¹

In addition to a user-friendly technology platform for submission of the reflective portfolio, another determining factor for students’ buy-in was their learning attitude and motivation. Motivation and self-esteem play an important role in learning and assessment.³⁴ Students with a positive perception of the reflective portfolio appreciated its educational value, while those who lacked self-regulation were likely to neglect it. The students indicated that they would put more effort into the preparation of the reflective portfolio if it underwent summative assessment. Driessen and colleagues reported assessment as a motivating factor for completion of the portfolio.³⁹ However, reports by McMullan and colleagues and Gallagher revealed that using portfolios for both assessment and learning can create conflicts.^40,41 These findings were consistent with some students’ views that summative assessment of the portfolio might lead to a lack of honesty in their reflective writing in order to achieve higher scores. This view is further supported by Schaffer and colleagues who suggest that summative assessment may decrease the effectiveness of the reflective portfolio by stifling honesty and reflection.⁴² In particular, the portfolio should not be used for summative assessment if the priority is to support reflection and encourage deep learning.⁴³ Additionally, high-stake summative assessments discourage motivation and self-esteem.³⁴

Future studies should examine faculty members’ perceptions on the use of the reflective portfolio as an educational tool. Moreover, further insights into the factors associated with the student experience of using the reflective portfolio should be explored. For example, students’ reflective thinking skills could be assessed using a validated instrument to determine the correlation with their perception. Because of the nature of the present study, the findings might be limited by the students’ perceptions, which could have been influenced by many factors, including their own perception of the relevance of the portfolio and of their competence in critical thinking and reflective writing. Additionally, the lack of buy-in from some faculty members as well as their heavy workloads may have affected the quality of the feedback they provided. These factors may have limited the full educational benefit of the reflective portfolio.

CONCLUSION

This study provided evidence that requiring pharmacy students to develop a reflective portfolio could aid them in developing deep information-processing skills. The students perceived that the portfolio helped with their personal and professional development. Factors that could enhance their perception of the value of the portfolio include clear understanding of the objectives and guidelines, useful feedback from a mentor, motivation and positive attitudes about learning, and a user-friendly technology platform for submission of the reflective portfolio. The findings support the use of reflective portfolios in the training of pharmacists to become reflective practitioners.