Abstract
Objective. To conduct a review of the pharmacy literature on the use of portfolios in US pharmacy schools.
Findings. This study provides examples of how pharmacy schools are using portfolios in various parts and across their curricula, however, assessment/outcome data is lacking. These examples can be used as a starting point for schools as they begin to design their own use of portfolios. Overall, students indicated that the use of portfolios is important in their professional development, but significant time is needed to complete.
Summary. Things to consider when implementing a portfolio system include how it will be used, who will review it, and what resources will be needed to sustain the project. It is important for schools to consider these items at the start of the process to ensure the portfolio process that is created is useful for assessing the overall programmatic or course outcomes being proposed by their use. More scholarly work needs to be published on the use of portfolios.
INTRODUCTION
Academic performance is measured in a myriad of ways. One that has been considered useful as a way to collect evidence of learning and encourage reflection is the academic portfolio.1 The portfolio in a university setting typically consists of documents that demonstrate a student has engaged in personal and professional development.2 Over the past decade, the use of portfolios has expanded from the arts, humanities, and education to health care disciplines such as medicine, nursing, and pharmacy. Within these health care disciplines, portfolios fill the need to enhance student learning and provide the evidence that student learning has indeed taken place.3
There have been multiple definitions used for portfolios over the years. McMullan and colleagues offer a definition that synthesizes the definitions found in earlier literature.2 They define a portfolio as a collection of evidence demonstrating personal and professional development through critical analysis and reflection of the portfolio’s content by the student. There are many reasons that portfolios are recommended as the best way to assess the skills mentioned in the pharmacy accreditation standards. First, portfolios are suggested to be a more “authentic” method of assessing student learning, especially if the student is required to define for themselves what evidence will be placed in the portfolio. Second, depending on the type of portfolio (reflective vs showcase) that is used, portfolios can provide both formative and summative assessment to display development of a student across a curriculum.4
Of the two portfolio types, reflective portfolios are used to demonstrate student growth both personally and professionally as students select artifacts that show how their proficiency in an area has grown over time.2 Reflective portfolios are discussed as being a more “constructivist” approach to assessing learning due to the need for students to select and assess the evidence of their learning.5 Learning that occurs with reflective portfolios is most effective when used as a formative assessment. In these instances, students are more comfortable including personal and more accurate reflections of their learning rather than feeling the pressure to write and include what they think the assessor wants.2 Conversely, showcase portfolios are commonly used for summative assessments, whereby students select their best work.5 These types of portfolios can limit the potential for learning from the portfolio process as students may limit inclusion based on assumptions of what artifacts will appeal to the assessors.2 This review focuses, unless otherwise stated, on reflective portfolios as they help bridge the divide between practice and theory, allowing students to take a deeper approach to learning and improve the students’ ability to develop critical thinking and problem-solving skills.5,6
The use of student portfolios as a way to illuminate student achievement of professional development competencies has been detailed by the Accreditation Council for Pharmacy Education in both Standards 2007 and 2016. Arguing for the use of reflective portfolios, the 2007 Standards suggest that portfolios include both student self-assessment as well as faculty and preceptor assessment of student achievement of the school’s educational outcomes.7 The 2016 Standards not only include the development of student self-awareness, but also call on pharmacy schools to develop other skills in students including problem solving, educating different audiences, patient advocacy, interprofessional collaboration, cultural sensitivity, communication, leadership, innovation/entrepreneurship, and professionalism.8,9 Both sets of standards strongly recommend the use of reflective portfolios as the best method to assess these competencies in students.7-9
With the call in 2012 for portfolio use in the accreditation standards, 70 pharmacy schools, out of the 80 that responded to an electronic survey (73% response rate) reported using portfolios in their curriculum. Results from this survey show despite such high usage, most of the pharmacy schools were not using portfolios to fulfill the accreditation requirements of assessment of competency and programmatic outcomes. Furthermore, there was a wide variation in the content and the methods of portfolio assessment among the programs.10 Since Plaza and colleagues’ seminal review of portfolio usage in 2007, there has been no synthesis of the studies published in the literature on portfolio usage, assessment, or best practices. Therefore, the purpose of this paper is to review the pharmacy literature on the use of portfolios in US pharmacy schools that has occurred over the past 10 years. This work will be helpful as pharmacy schools continue to work on implementing portfolios in their student assessment plans.
This paper begins by describing the methods used for conducting this literature review. Next, a discussion on and how portfolios are used in general in pharmacy education will be discussed. This is followed by a review of specific curricular uses of portfolios such as uses in specific didactic courses, Advance Pharmacy Practice Experiences (APPEs), and the full curriculum. Next, student perspectives on the use of portfolios will be discussed, followed by barriers to portfolio adoption. Finally, this review concludes with a set of questions for pharmacy schools to consider as they start to adopt portfolios as an option for calibrating student learning.
METHODS
A systematic literature search was conducted using the PubMed, CINAHL, MEDLINE, and ProQUEST Dissertation/Thesis databases. Search terms used included “pharmacy” and “portfolios.” Currents in Teaching and Learning was also searched outside the databases as it is a newer journal and is not yet indexed. The original search produced 98 articles and 5256 dissertations. Articles and dissertations were included in this study if they were published in the last 10 years (2006-2016), were written in English, and discussed the use of portfolios in pharmacy schools. Articles prior to 2006 were excluded due to a literature review on this subject that had been published in 2007. Reference lists for all included articles were reviewed to determine other papers that met the inclusion criteria. The inclusion criteria led to the review of one dissertation, 11 journal articles, and one poster abstract. A complete summary of included articles can be found in Appendix 1.
RESULTS
Trends in Portfolio Usage in Schools and Colleges of Pharmacy
When Plaza and colleagues conducted their seminal literature review, there was little scholarly work in the pharmacy literature on the use of portfolios.5 In fact, much of what was published was only available in the form of poster abstracts at academic conferences. In her 2006 dissertation, Plaza described three uses for reflective portfolios in pharmacy education: within specific courses in the didactic curriculum, during the experiential parts of the curriculum, and integrated throughout the entire curriculum.4 Each is described below.
Plaza’s dissertation described three specific didactic course examples that used portfolios prior to 2006.4 One course used a web-based portfolio in a five-quarter pharmacy practice skills laboratory associated with the college’s pharmacotherapeutics courses. This portfolio documented student growth throughout the laboratory course, but there was no mention on whether reflection was part of the course. Another pharmacy program used a portfolio in a compounding course to document individual student activities, assessments, reflections, and to document group activities. The individual portfolio assessed the achievement of course objectives. The group portfolio contained assessments on a student’s ability to collaborate on the group assignment as well as their communication skills. The last course Plaza documented involved a problem-based learning course in human resources management.4 Students in this course were required to keep a reflective portfolio, which was then used to assess student learning and effectiveness of the course but not much detail was provided in the abstract.
Plaza’s review of the use of portfolios prior to 2006 in the experiential parts of the curriculum indicates there is ambiguity on whether reflections were used as part of these portfolios. Most of these portfolios did, however, show accomplishment of objectives and competencies using both formative and summative assessments. Northeastern University School of Pharmacy was one of the few to document student use of portfolios during APPEs in a reflective manner. Students not only set goals at the beginning of each experience, they charted achievement of competencies during the experience, and wrote reflections using a template provided to assist them in writing their reflections. Only completion data was provided in the poster abstract presented at the AACP 2003 Annual Meeting, and no data related to assessment of the activities themselves were included.11
Plaza’s review noted only a handful of reflective portfolios integrated across the entire curriculum prior to 2006. Furthermore, most did not contain enough detail to determine the nature of the portfolio and whether it contained a reflective component or not. The University of Arizona College of Pharmacy offered one example of a portfolio integrated across the entire curriculum. In this example, student portfolios consisted of a reflective report on each domain of the college’s educational outcomes and included evidence that they have achieved the competencies and domains covered in that particular year.4 The student turns in the portfolio at the end of each spring quarter.
Since Plaza’s review and dissertation, Skrabel and colleagues conducted a cross-sectional survey of 109 US pharmacy schools to examine the usage of portfolios, content, method, training, resource support, and the benefits and challenges encountered when portfolios are inserted in the curriculum.10 The response rate for the survey was 73% (80/109 schools). Eight-eight percent of schools (70 schools) used a portfolio with 91% providing institution-specific purpose statements for the use of the portfolio and its assessment. Most schools used a behavioral and rubric-based approach for the portfolios while only a few used a reflective approach. Twenty-three schools did not assess their portfolios and of the 47 that did, 16 had one specific person who reviewed the assessment. Most schools used a committee or group consensus approach to assessment. Other ways to assess included use of a combination of non-graded and graded process using standardized rubrics. The study also found that despite 80% of schools favoring comprehensive portfolio usage, most schools (62%) report usage limited to the experiential parts of the curriculum.10
The following sections of this review will discuss the ways that pharmacy schools have published using portfolios since 2006. Interestingly, usage can still be divided into the same three categories that Plaza found in her dissertation: specific courses in the didactic curriculum, during the experiential parts of the curriculum (APPEs), and integration throughout an entire curriculum.4
Two studies were reviewed that focused on the usage of portfolios in individual courses. These works offer different approaches to using portfolios in specific classes. Ashcroft and Hall’s survey study of undergraduate pharmacy students focused on the use of a portfolio in a prescribing course taught in the final year of a pharmacy program with an emphasis on the development of the skill of prescribing.12 While the Ashcroft and Hall article is not based on experience in a US pharmacy school, it provides an example of one way that a portfolio system can be incorporated into a school’s curriculum.12 During a student’s final year of an undergraduate pharmacy degree, students participate in a course module that is designed to teach them the principles of prescribing, such as selecting medicines that are effective, safe, and cost effective and how to respect patient choices. As part of this course, students were asked to submit a portfolio in which they gathered evidence and reflected on how modules such as pharmaceutical care, therapeutics, and their clinical rotations link with the concept of prescribing. Ashcroft and Hall found that the use of a portfolio was accepted by most pharmacy students and students found portfolios a good assessment approach along with their more traditional assessments. Students in the study indicated that the portfolio helped them develop skills that would be useful later in their continual professional development.12
An alternative approach, used within an individual class, is described in Hobson and colleagues’ study.13 These scholars examined the use of a portfolio in a capstone course, whereby students reflected on their professional development, assessed their achievement of the college’s outcomes, and established a continuing professional development plan.13
In their case study, Hobson and colleagues detailed how students in their fourth year of a doctor of pharmacy curriculum took a 1-credit hour graduate capstone course that required them to complete a “professional portfolio” using an electronic portfolio system provided by the college.13 The purpose of the portfolio was to have students demonstrate their development from a student to a practitioner. Students were mentored by faculty members on what to put into their portfolios with emphasis placed on items that employers and residency directors might want to see. Required elements of the portfolio include a general biography, education history, courses taken, experiential rotations, employment history, professional activities (association memberships, community service, references, pharmacist’s oath), technical skill set, training and testing, licensing and certifications, liability insurance, immunization history, and professional goals. Recommended sections included honors/awards, professional activities (conferences attended, honor societies, fraternities/sororities, publications, research experience, grants, cultures, committees/boards), and other accomplishments. The portfolios were then graded by the course instructors using a three performance-level rubric (excellent, acceptable, and unacceptable). Portfolios were considered excellent (pass) if all required and selected optional items were included with equal depth, detail, and quality. The portfolio may not contain any inappropriate or inaccurate content. The portfolio also had to show that it had been thoroughly reviewed with limited errors. Portfolios were considered acceptable (pass) if they included all required and some selected optional elements but the elements may have needed some further development in depth, detail, or quality. There may also have been some inappropriate or inaccurate content or the portfolio may have needed some further editing. An unacceptable portfolio (fail) did not have all required elements or did not establish that the author had a commitment to developing a quality portfolio.
Data from two of the classes who completed the projects showed that at least 100% (103 students excellent and 36 acceptable) of enrolled students passed the portfolio project. According to course instructors, the portfolios only took about 5 minutes to grade and most student portfolios showed they were excelling in the areas of service of others, leadership, completion of core courses, and documentation that they had the skills needed for career mobility. Hobson and colleagues concluded that one problem with the portfolio was that it did not require a lot of higher-order thinking to complete as students were given a restrictive presentation template to use to develop the portfolio.13
These two studies suggest pharmacy schools can have different approaches for how portfolios can be instituted into course work. A common finding, however, was that instructors were specific on the information they wanted to be included in the portfolio, thus limiting student creativity and critical thinking.12,13
An alternative approach for instituting portfolios is within APPEs. Portfolios can be used and submitted (via email or through use of a patient care portfolio system) in different ways during a student’s APPEs. One usage is to document the patient care interventions that students are conducting while on rotations. This information could be used to document student achievement of the patient care outcomes in the ACPE standards. Another method includes allowing students to pick artifacts and write reflections that demonstrate achievement of multiple outcomes in the ACPE standards included in Domain 4.
The University of Georgia College of Pharmacy portfolio is directed to the skill of pharmaceutical care as students are required to upload information on six patients related to analyzing patient data, developing plans for medication therapy, and implementing the plans. Patients discussed are required to have certain disease states as determined by the school.14
The purpose of the patient care portfolio is to help students learn how to document and develop patient-centered pharmaceutical care plans. The portfolios are also assessed to see if there was continuity of patient-care activities across the state during APPEs. Most of the disease states chosen to be included in the portfolios represented core disease states recommended by the American Association of Colleges of Pharmacy’s professional affairs committee (diabetes mellitus, oncology, hypertension, urinary tract infection and pneumonia). The most commonly documented therapeutic interventions were therapeutic duplicates (332 times) and drug interactions (250 times). An average of 1.8 therapeutic interventions per patient case was documented with interventions occurring in all geographical regions across the state. Approximately half (48%) of all submitted cases were completed during students’ acute medicine APPE.14
In contrast to the University of Georgia’s use of portfolios to focus on patient care, Albany College of Pharmacy and Health Sciences uses APPE portfolios to demonstrate student achievement of multiple outcomes in the ACPE standards, including many of the outcomes listed in Domain 4. Students select artifacts from any of their APPEs that demonstrate achievement of ability-based outcomes specified in the college’s APPE course manual. Outcomes assessed include the ability to provide pharmaceutical care, drug information/literature evaluation skills, managing the medication use system, and self-learning abilities and habits (assessed using a reflective paper).15
Albany’s portfolio is assessed in multiple ways. The first three outcomes are assessed by the director of experiential education. The fourth outcome is evaluated by two reviewers using a 4-point scale rubric. If the evaluators differ in their rankings, then the rankings are averaged. The portfolios are required to be completed prior to graduation. In Briceland and Hamilton’s study of Albany’s portfolio usage, all student portfolios that were available for review (138/141) contained work product that demonstrated achievement of the three ability-based outcomes. Work product submitted in the portfolio included: provision of pharmaceutical care outcome (patient case studies, patient care notes, formal consultations, and medication reconciliation reports), drug information/literature evaluation skills outcome (patient education materials, journal article reviews, formulary reviews, during information responses, newsletters, case studies), and managing medication use system outcome (drug use reviews, formulary management reviews, and clinical practice guidelines). Data on how many times a portfolio had to be sent back for further information was not collected. For the fourth outcome, 135/138 students were assessed. Three of the reflections were lost electronically. The reflections were assessed for two domains: domain one assessed what was learned and applicability to future practice and domain two assessed continuous professional development plans. On a scale of 1 (not proficient) to 4 (exemplary), students scored an average of 2.9 on Domain 1 and 2.3 on Domain 2.15
These two studies illustrate different specific uses of portfolios during APPEs, with pharmacy schools using the portfolios to show student achievement of specific outcomes that are listed in the ACPE accreditation standards. The studies found the portfolios were a useful vehicle to demonstrate student achievement of programmatic outcomes and development of skills.14,15
Portfolios can also be used to display student ongoing development across the entire curriculum. The University of the Incarnate Word Feik School of Pharmacy, West Virginia University School of Pharmacy, and University of Arizona College of Pharmacy have implemented portfolios across their curriculums.
Feik School of Pharmacy developed a proprietary electronic portfolio system.16 This system has three components. One component allows students and faculty members to use the system to develop an electronic resume and/or curriculum vitae. Information included in this section may not be useful for students while they are in school but will be useful later in their careers (continuing professional development, licensure, liability insurance, preceptor training). The second section is related to the school’s ability-based outcomes. Certain student assessments and year-end tests are mapped to the school’s outcomes and students are required to upload this information into the portfolio. Students are then asked to write a reflection on the course-embedded assessments that are uploaded. To help students know which items to upload each semester, they are given a checklist of items to upload. Upon review of the student portfolios, 100% of the students uploaded items to the portfolio, but no student uploaded every one of the required elements. Reasons for not uploading include misplacing the assignment, faculty members forgetting to return the assignment, and faculty members forgetting to conduct the assignment.16
West Virginia School of Pharmacy implemented a purchased electronic portfolio to implement across the curriculum. The portfolio is not graded but is considered a curriculum requirement for all students. Students are provided with a mentor (faculty members, alumni, or preceptors) to help them in developing their portfolio. Students and mentors are asked to meet (via phone or in person) at the beginning and end of the semester. As part of their portfolio, students are asked to write an autobiographical sketch for their mentor. Students are instructed on what assignments should be included in the portfolio. Assignments include three self-assessments (provided rubric) of course-related assignments per semester (two pre-chosen and one chosen by the student from a list) and one professionalism submission per semester. Mentors do not grade the student course assessment, but do grade the student self-assessment responses on a 3-point scale (3=outstanding, 2=satisfactory, 1=unacceptable). Self-assessment questions include the following: “Question 1: Describe those aspects of the assignment you were most satisfied with, Question 2: Briefly describe how this assignment helped develop your knowledge and skills for two educational outcomes or abilities that were listed, Question 3: Describe the aspects of this assignment that you need to further develop or improve, and Question 4: Give specific examples for how you will achieve the development or improvement you indicated in question 3.” 17
Results from a review of the West Virginia student portfolios showed first professional year (P1) students had significant improvements in the self-assessment question scores of question 1 (p=.002) and question 3 between fall and spring semester whereas the second professional year (P2) students did not have any significant improvements in these questions. Scores on the self-assessment question 2 and question 4 were not statistically different for either P1 or P2 students between fall and spring semester. At the end of the spring semester, the scores of P1 students were higher on three of the four questions (questions 1, 3, and 4) compared to the scores of P2 students.17
In both the Feik School of Pharmacy and West Virginia University School of Pharmacy, students were asked to upload course-embedded assessments that are reflective of the outcomes provided by the school. Students were also asked to reflect on the items that were added. West Virginia University School of Pharmacy takes the process one step further by including mentors into the system to help students through the process.16,17 The University of Arizona College of Pharmacy takes a different approach in its use of its portfolio.
The University of Arizona College of Pharmacy requires all students to complete a paper-based portfolio each year they are enrolled in the program. The goals of the portfolio are to have students reflect on their professional growth throughout the program by using the school’s outcomes statements as the backbone of the portfolio. Guidance is given to students on what to include in the reflections and portfolio by the course coordinators. Portfolios are graded by one professor using a 50-point grading rubric. The rubric is provided to the students as a tool to help them in the development of their portfolios. Advisors meet with the individual students to discuss their grades as well as any thoughts/concerns that were discussed in the student’s reflections. The advisors collect all de-identified student responses and share this information with the college’s assessment committee annually. Areas for improvement are discussed with individual faculty members or department heads. No specific outcomes information on the portfolios was provided in the study.18
Whether a pharmacy school decides to use an electronic or paper-based portfolio system, a portfolio can be a useful tool in assessing student achievement of programmatic outcomes across a curriculum. While all three schools used a slightly different approach, all three schools found the portfolios to be a useful tool in assessing student achievement of their programmatic outcomes with only one of the schools providing outcome results other than completion rates.
Student Perceptions
Multiple studies have been conducted on student perceptions of the use of portfolios in pharmacy schools.1,12,14,18 All of the examined studies used convenience sample surveys of students in the programs using the portfolio systems. Response rates on the student surveys varied by study, but ranged from 67% to 90%.1,12,14,18 Three of the studies reviewed student perceptions on the design of the portfolio and implementation of the portfolio process. The fourth reviewed student buy-in to the portfolio process.
There were several commonalities across the three studies on perceptions on the design and implementation of the portfolio process. Overall, students believed that the portfolio process helped them develop knowledge in the areas that were assessed, encouraged them to develop insights into their learning, and allowed them to identify their strengths and weaknesses.12,14,18 Students also believed that the portfolios would be useful to them later in their careers.12 Table 1 provides details from each of the studies in these areas.
Student Perceptions of Portfolio Use
These three studies also identified areas for improvement in either development or implementation of the portfolio. In these studies, students indicated that they would prefer to have more information on how to construct the portfolio (80%) and that creating the portfolio took too much time or required completion of too much paperwork (40%).12,18 Murphy and colleagues asked students to rate on a 5-point Likert scale how much time they spent working on their portfolio (1=very little, 3=moderate amount, and 5=a great deal).18 P3 students indicated they spent more time (4.4 out of 5) on their portfolio than P1 or P2 students (2.1 and 3.1, respectively out of 5) and they perceived the amount of time spent increased incrementally from the first to the third years. Students discussed that the portfolios should be worth more credit (7%) and that more portfolio examples would have been helpful (34%).
Despite these commonalities, however, portfolio use by students varied by school. For example, Mercer University College of Pharmacy and University of Arizona College of Pharmacy require all students to complete a paper-based or electronic portfolio each year they are enrolled in the program,1,18 the Centre for Innovation in Practice Schools of Pharmacy and Pharmaceutical Sciences (in the UK) requires the use of a portfolio in a particular class,12 and the University of Georgia College of Pharmacy requires the use of a portfolio in their APPEs only.14
Mercer University College of Pharmacy implemented Practitioner’s Roundtable Discussions into their PharmD curriculum to help students develop “buy-in” to the portfolio development process. These discussions are incorporated as part of a first-year course. Practitioners from multiple areas of pharmacy are invited to come and speak about their careers and the impact, value, and application of portfolios. Students are surveyed prior to these discussions and again at the end to determine if their perceptions on portfolio use had changed based on the practitioners’ comments. The survey asked students questions related to the impact on their learning, value of the portfolio process, and practical application. Survey responses were based on a 5-point Likert scale (1=not beneficial, 2=slightly, 3=moderately, 4=very, and 5=extremely beneficial). Results from the pre-survey found most students already found portfolios moderately beneficial in all four areas except for questions related to organizational skills and the portfolio process being informative which were rated at very beneficial. There was little change in the students’ perceptions on the use of portfolios from the pre- and post-surveys except on the manageability of the process which changed from 37% saying moderately beneficial and 25% saying very beneficial to 31% saying moderately beneficial to 40% saying very beneficial. Secondly, scores on the “likely to make an abbreviated portfolio for interviews” also changed from 61% saying moderately beneficial or lower (25% moderately and 36% slightly/not at all) to 80% saying moderately to extremely beneficial (29% moderately and 51% very/extremely). This indicates that students would consider using portfolios after graduation.1
Despite the differences in portfolio models used across the studies, overall student perceptions on the portfolio use was positive. Students believed that portfolios can be useful to enhance their learning in pharmacy curricula. Students also indicated that portfolios may be useful tools to use for interviews and continued professional development after graduation.
Barriers to Portfolio Adoption
While data on the use of portfolios from students overall is positive, there are still several barriers that need to be addressed when schools launch a portfolio program. The most cited barriers were workload and time associated with the implementation and maintenance of a portfolio system. Schools also reported that student buy-in and motivation hamper the implementation of these somewhat cumbersome projects.10
While not discussed in this review, faculty members also tend to have lots of questions when starting a portfolio system. These questions include how to use the system, where will the portfolio be used, how will it be incorporated into the full curriculum, how and by whom it will be assessed, and what type of new and useful information will it provide to the program.10 These questions, if not addressed, can also lead to barriers in starting a portfolio system.
The challenges mentioned previously must be considered when developing and implementing a new portfolio system. These barriers can be addressed by asking questions around two areas when developing a program: composition of the portfolio and the implementation process. The first question to ask regarding composition is what is the purpose of the portfolio and should it be used to assess just a few very select, but maybe difficult to measure, student outcomes or should it be used to measure all of the school programmatic outcomes.3 The medical education literature shows that portfolios can be useful for both activities.19,20 More studies need to be conducted to show the same benefits in pharmacy education. One thing to consider when making a decision is student time needed to document on multiple outcomes as well as the faculty time needed to review portfolios.3
The second question to ask regarding composition is whether the college wants to implement a portfolio process whereby the evidence used is dictated by the college or whether the evidence should be student driven. In 2007, a meta-analysis was conducted on portfolio use in medical education.21 This analysis included a review of 1939 empirical studies on portfolio use in all parts of medical education. The results of this analysis found that effective portfolios were ones that had flexible but clear structures that allowed learners opportunities to describe their own development. The studies also found that when portfolios were heavily prescribed, students felt they were bureaucratic instruments only. Students/learners appreciated more the learning from the portfolios when there was some freedom given to determine the content.21 This is echoed by McMullan and colleagues who stated that portfolios can be a “catalyst for growth” through the process of developing the portfolio as much as the documentation the portfolio provides.2 The development of the portfolios itself can lead to student self-awareness of their own skills, strengths, and limitations. When deciding content for the portfolio, care needs to be taken to balance a heavily prescriptive portfolio versus one in which students can have full freedom to determine all content so as to provide an easy-to-use and measurable system.3
The last composition question to consider is whether the portfolio should contain a reflective component.3 Studies on the use of portfolios indicate that the best student learning from portfolios comes from the reflective components of the process, not just when the student selects artifacts. However, students may sometimes be reluctant to engage in reflections due to fear of pointing out their weaknesses or uncertainty on how to reflect. Also, portfolio mentors are not necessarily the best at helping facilitate student self-reflection. Care needs to be taken that when reflections are added to a portfolio that training for both the student and facilitator/grader/mentor is provided.2
Once the composition has been decided, the college needs to focus on the implementation stage. There are several questions that need to be asked in this stage. Two questions that a college needs to review when implementing a portfolio system have to do with how the portfolios will be reviewed (who and how) and how to ensure the students complete the portfolio (ie, will it be graded). These questions are hard to answer as they require a college to review its curriculum to determine if there is a course to add the portfolio to if a grade is given and to review workload to determine who may be the best person to review the portfolios. The literature is clear that the best experience with portfolios occurs when they are reviewed and feedback is provided to the student.3
The last question needs to be: “What is the best way to introduce students to the portfolio process?” To get the most out of a portfolio system, the college needs to provide clear and thorough instructions as well as continual reinforcement of the process throughout the students’ time at the college.3
Even after considering all of these questions, there will be many roadblocks that a school will encounter when starting a portfolio process. One roadblock is the decision on whether to implement a paper or electronic portfolio. If electronic is chosen, then a college must decide which storage solution to use. Determining the best storage solution (cloud-based, portfolio vendor, other) will depend on cost, ease of use, how complicated the system is, and whether students need to maintain access after they graduate. A college must also be prepared for the fact that once a portfolio is started, the goals and system needs may change which can make selecting a storage solution difficult as further flexibility may be needed.3
Another roadblock to be aware of is the underestimation of time and resources that portfolios can consume. In the planning process, schools should consider doubling or tripling both the time and resource estimates. There is also the roadblock of information dissemination to consider. Students may not want to attend activities that are outside of class time so schools may want to consider carving time out of a class or a group of classes to make announcements regarding the portfolios. Schools also need to be aware that despite the “tech savvy” student most colleges are recruiting today, technology frustrations are still common and will be one of the first frustrations that will make students lose “buy-in” to the system. This may even require additional technology resources for the college depending on the software vendor chosen. Two last roadblocks to consider are who will oversee the portfolio process and how will reviewers be trained. These are very important as who oversees your portfolio needs to be able to champion both the process and the portfolio itself. The training is also important to ensure consistency in reviews and timely feedback.3
CONCLUSION
The use of portfolios is becoming an important part of the assessment plan for all pharmacy colleges given the current ACPE accreditation requirements. This study provides examples of how pharmacy schools are using portfolios in various parts and across their curricula, however, assessment/outcome data is lacking. These examples can be used as a starting point for schools as they begin to design their own use of portfolios. There are many factors that need to be considered when implementing a portfolio system, from how it will be used, and who will review it, to what resources will be needed to ensure ongoing usage. It is important for schools to consider these items at the start of the process to ensure the portfolio process that is created is useful for assessing the overall programmatic or course outcomes being proposed by their use.
Overall, students indicated that the use of portfolios is important in their professional development, but significant time is needed to complete. This may pose a problem when securing student buy-in. Student buy-in is one reason why planning for the use of portfolios is so important. Care needs to be taken to ensure the portfolios being implemented are useful for both the school and the student. The amount of student time involved in creating and documenting in the portfolios needs to be taken into account when starting a portfolio system. As the use of portfolios becomes more ingrained in the assessment work of schools, more scholarly work needs to be published on the use and outcomes to provide evidenced-based best practices for schools seeking to start these large assessments.
Appendix
Article Summaries
- Received December 22, 2016.
- Accepted April 24, 2017.
- © 2018 American Association of Colleges of Pharmacy
In this issue
Jump to section
Similar AJPE Articles
Cited By...
- No citing articles found.