Development and Implementation of a Curricular-wide Electronic Portfolio System in a School of Pharmacy

Suite of Software

An outside vendor, RXinsider (West Warwick, Rhode Island), created the Web-based e-portfolio program for the school. The final product was launched in 3 phases. The first phase, RXportfolio, was a Web-based program to document student-specific data for general profile information and for professional development. The second phase, RXoutcomes, was to track and report student achievement of curricular outcomes, IPPE requirements, and APPE requirements. The third phase, RXpreceptor, was to schedule IPPE/APPE sites and to evaluate sites, students, and preceptors.

RXportfolio. The software developer and the e-portfolio manager created accounts in RXportfolio for all pharmacy students and faculty members in the school. The e-portfolio program generated login and password information for each student and automatically e-mailed the information to them. After a brief training period, students and faculty members entered their data into RXportfolio (Table 1). The students and faculty members could use their RXportfolio as an electronic resume or curricula vitae. RXportfolio also was designed for practitioner use. While students might not need all of the program's functions while in pharmacy school, the sections on continuing professional development, licensure, liability insurance, immunization history, and preceptor training will simplify record keeping and make the software a valuable asset for them even after they enter the profession. Each RXportfolio user was able to control several aspects of their portfolio including template style, sharing of the portfolio, and management of files (Table 1).

RXoutcomes. The e-portfolio manager added school-specific data into RXoutcomes including the curricular outcomes. Table 2 presents the different sections of RXoutcomes. The RXoutcomes resource library contained the student outcomes document, the course-embedded assessments checklists, and the 2007 ACPE Standards. A subcommittee of the assessment committee mapped all the courses to the school's outcomes document, the Center for the Advancement of Pharmaceutical Education (CAPE) document,¹⁰ North American Pharmacist Licensure Examination (NAPLEX) competency statements,¹¹ and ACPE guidelines.¹ Determination of competency of knowledge based outcomes was measured by a year-end test called the Annual Student Assessment and Progression (ASAP) Examination. Competency in the skills and attitude outcomes were measured by course embedded assessments. These assessments were described in the syllabus and communicated to the students during the course as the assignment due dates approached. As the students completed their course-embedded assessments and ASAP examinations, the documents and results were uploaded to the corresponding outcome to show competency. Students could write comments for each course-embedded assessment uploaded to reflect on the assignment. The e-portfolio manager ran reports in RXoutcomes to determine which students were deficient in documenting their achievements.

RXpreceptor. Finally, the e-portfolio manager created accounts in RXpreceptor for each pharmacist preceptor. The individual accounts included experiential site information and preceptor information (Table 3). The RXpreceptor support library included a copy of the preceptor manual, the IPPE/APPE manual, and all the IPPE/APPE syllabi and forms. Pharmacist preceptors used RXpreceptor to create a profile for each of their IPPEs and APPEs, to view a list of students assigned to their IPPE/APPE, and to evaluate students after completion of the IPPE/APPE. Pharmacy students used RXpreceptor to preview IPPE/APPE sites, select their IPPEs/APPEs, and evaluate experiences and preceptors after completion of the IPPE/APPE. The students also could grant their preceptors access to view any part of their RXoutcomes and RXportfolio accounts. The preceptor could use this information to customize the learning specifically to the student. The experiential coordinator used RXpreceptor to organize pharmacist preceptors, assign students to IPPE and APPE sites, and view evaluations.

RXpreceptor also was used to communicate with all students and preceptors via the document library, the message board, and e-mails. The school was able to contact preceptors through a mass e-mail option in RXpreceptor, eg, the school's drug information center was able to advertise its services to all preceptors.

Using the profiles as a guide, students were able to prioritize and select their APPEs online. Students were instructed on the selection process by the IPPE and APPE coordinators. The IPPE and APPE coordinators adjusted assignments as needed. Once student selections were made, the RXpreceptor program's lottery system assigned students to sites. Final assignments could be adjusted at the discretion of the APPE coordinator, allowing for timely display of results online to the students and preceptors. Overall, RXpreceptor provided for a straightforward and efficient APPE selection and assignment process. As an added benefit, the RXpreceptor allowed for tracking of site visits, IPPE/APPE hours, and payments to preceptors.

Documentation of Assessments and Examinations in the E-portfolio

The students were given a course-embedded assessment checklist every semester to assist them in identifying which assignments to post to a corresponding curricular outcome. The checklist included the specific course, the curricular outcome, and a description of the course-embedded assessment assignment required to fulfill the outcome. The checklist was also stored in the e-portfolio library of documents. Students either created an image of their graded paper copy of the course-embedded assessments assignments using a scanner or used the graded electronic document to upload to the e-portfolio. To document the grade achieved for the course-embedded assessment, the student selected a rating from 1 to 5 when each assignment was uploaded. Although the Feik School of Pharmacy chose to document achievement on a pass/fail system where 1 meant the student was not competent and 5 meant the student was competent, the software designer provided a 1-5 scale to allow other schools/users to customize the rating system for their specific needs.

The results of the ASAP examination also were uploaded to the e-portfolio. It was more efficient for the school to upload the results of the examination into the student's portfolios instead of returning them to the students and waiting for them to upload the results.

Students were given a deadline to upload course-embedded assessment assignments. The e-portfolio manager reviewed the students’ portfolios each semester to monitor their progress in uploading the course-embedded assessments. Students were reminded that satisfactory completion of the e-portfolio was a requirement for graduation. If students failed to upload their course-embedded assessments to the system, the reasons were determined and students were given additional training on the system if needed.

The number of course-embedded assessments required to have documentation in the e-portfolio system for graduation varied for each class. The expectation for the class of 2010 (67 students) was for 50% of the course-embedded assessments be uploaded. The expectation for the class of 2011 (86 students) was for 70% of the assignments to be uploaded. The expectation for subsequent classes was increased to 90% of assignments uploaded by graduation. The e-portfolio system generated reports that summarized which students had successfully fulfilled the requirements. In addition, any course-embedded assessment uploaded for a specific outcome could be viewed for any student.

Only the administrator and the student could access the student's accounts in RXportfolio and RXoutcome. Other than administrators, the students controlled who was able to view their personal profile and granted access to preceptors and employers when appropriate. The students controlled which categories within their e-portfolio could be viewed by the preceptor or employer. In addition, the student could set an expiration date for the RXportfolio access link they sent to the preceptor or employer.

The e-portfolio system used a variety of elements to enter data including free text, number only fields, dropdown menus, and check boxes. For saving information, the e-portfolio allowed any format of document (eg, Adobe PDF, Microsoft Word, Microsoft Excel, QuickTime, Windows Media) to be uploaded. These documents were used to demonstrate a student's proficiency in assessments such as calculations.

The e-portfolio manager conducted 2 training sessions for the students during their first and second years. The 1-hour sessions consisted of showing the group how to navigate each screen/page of the Web-based program and to enter information or upload documents. Less than 10% of students required additional training on how to use the software program. The e-portfolio manager conducted any additional brief training sessions on an individual basis. The e-portfolio system generated reports for tracking of student competencies and professional development. Reports of RXoutcome included: (1) historical display of curricular outcomes for one or more users, (2) current curricular outcome displayed by user name and rating, and (3) historical curricular outcomes displayed by user name and ratings. Reports for RXportfolio included, but were not limited to student employment, organizational involvement, publications, presentations, and community service. Only the e-portfolio manager and persons with administrative rights could run reports. There were no reports that student users could run.

Integration Into the Experiential Curriculum

RXpreceptor provided an efficient documentation system for tracking the completion of hours by a large number of pharmacy students (85 students at the time of implementation). Students uploaded evaluations and completed assignments from each experience into the APPE folder in RXoutcomes. The APPE folder in the e-portfolio was divided into 6 subsections: (1) Acute Care – General Medicine APPE, (2) Ambulatory Care APPE, (3) Community Care APPE, (4) Hospital/Healthcare System APPE, (5) Patient Care Elective APPE, and (6) Non-Patient Care Elective APPE. The subsection for each individual APPE was further organized into 2 categories: (1) Evaluations and (2) Other Completed Assignments. Students were strongly encouraged to scan assignments that they completed during each APPE and upload them into the Other Completed Assignments category, eg, drug monographs, drug use evaluations, patient case presentations, journal clubs, or pamphlets. Students could grant access to their preceptor to view their documents or resume prior to starting the IPPE or APPE. The preceptor could use this information to customize the experience to the student.

Benefits of the e-Portfolio System

There was minimal cost for training students on using the software program. The program is intuitive with all the different tabs for storage of information. Minimal faculty time was needed to teach the students the process of uploading information. Storing assessments electronically had numerous cost benefits. Using a traditional system, there would have been the cost to hire someone to track assessment data. An individual would have to collect all student assessment data and then file it in the appropriate folders. Additional storage space for student folders would be required. The individual would also have to analyze the data and generate reports. All of these costs are reduced by using the e-portfolio. There was a cost incurred in developing the program; however, it was much lower than the cost of personnel and storage space that a paper system would have required.

Students can also use their e-portfolio when seeking employment as an intern or pharmacist by granting access to their e-portfolio to employers prior to or after interviews. After a long day of interviews at a pharmacy school career fair, employers sometimes find it hard to distinguish between several outstanding candidates. The e-portfolio can be used as a reminder because it reads like a resume and a photo is attached. In addition, other achievements that might not have been discussed in the interview may be shown in the e-portfolio. Employers use behavioral-type interviewing questions to assess how past behavior will be indicative of future performance. The e-portfolio also has a category to allow students to reflect on past experiences. As the job market tightens, the e-portfolio will become invaluable to the students. It will give students the opportunity to showcase themselves. In the future, offers of employment may come down to a student with a portfolio and one without. Portfolios are quite common in other professions such as art and teaching.^12-14

Software Modifications

Minor software modifications were made to the system over the years. New sections were added to RXportfolios and additional reporting capabilities were added to RXoutcomes and RXpreceptor. Space allotments for document storage have increased and support for additional file types has been added. Every year, the pharmacy school sends a Microsoft Excel spreadsheet of user information for the first-year class to the software team for creation of new accounts in RXportfolio, RXoutcomes, and RXpreceptor. The IPPE and APPE Coordinators enter or delete preceptor accounts into RXpreceptor as needed.

Since the inception of the e-portfolio system, curricular requirements have changed. Students are now required to complete 3 hours of professional organization participation in their first and third years as well as 3 hours of health fair participation in their third year. The students have a specific form related to the IPPE activity that is signed by a pharmacist or preceptor at a professional organization meeting or health fair event. The students then formulate a reflective summary addressing their experience and what they learned. The e-portfolio system easily allows for documentation and tracking of this requirement. Students also are required to upload course-embedded assessments for certain IPPE-related activities. For example, the GRACE Program (Growing Respect and Care for the Elderly) is an IPPE activity where the students meet with a nursing home patient, write a SOAP note for the patient, and upload the graded SOAP note to their e-portfolio. Documenting these accomplishments is a great way for faculty members and future employers to review the student's experiences and performance.