The Need for New Guidelines and Training for Remote/Online Testing and Proctoring

Online Testing and Proctoring Software and Technology Options

A variety of third-party online testing and proctoring technologies are available, including Respondus Lockdown Browser and Respondus Lockdown Monitor (Respondus, Inc), Exam Monitor (ExamSoft, Inc), Honorlock (Honorlock, Inc), and ProctorU (ProctorU, Inc). Online testing and proctoring software provide the ability to administer online tests while the student is observed or recorded using webcam video and audio. Some programs offer additional features, such as securing the user’s device to block access to outside resources (ie, websites, class notes, other programs) or recording the computer screen through active screen capture. The four main online proctoring technologies include live online, record and review, artificial intelligence (AI), and web-conferencing platforms (Table 1).

Live online proctoring allows a proctor to actively observe the student in real-time through a webcam. Record and review proctoring technology records the student via webcam during the examination, then the video is later reviewed by a faculty or staff proctor for any abnormalities. AI proctoring automatically monitors the recorded video and audio during the examination and flags any suspicious behaviors, such as persistent off-screen eye gaze or additional people in the room. Finally, programs with a smaller number of students may have faculty or staff proctor students through the web conferencing software while students use a secure online testing platform.

When evaluating the various online proctoring programs, pharmacy schools should consider the compatibility of the program with existing online testing software, available campus supported platforms, minimum computer system requirements, the need for an active internet connection during the examination, the explicit and implicit cost of the system, and the degree of faculty and staff support needed or the level of support provided by the proctoring program. For example, some institutions may have an existing contract for a web conferencing platform and this option may be a lower-cost alternative compared to the purchase of online proctoring software, although it may be more resource intensive with regards to the number of school personnel needed for proctoring.

Developing Guidelines for Optimal Testing Behavior

After the adoption of an online testing and proctoring software and prior to the administration of any remote examinations, pharmacy schools should outline optimal testing behavior guidelines and expectations and distribute these to faculty, staff, and students. These guidelines may be developed by appropriate administrators, offices, or committees involved in assessment and then approved by the faculty and administration. It is helpful for a school to employ the same testing procedures in their institution for all courses in order to promote testing consistency and reduce confusion. School should create these guidelines even if a proctoring technology program provides instructions. The new testing policies should give notice to students that video capture will occur (if applicable) and can be used in cases of academic misconduct. This notice should be placed in the course syllabus, on the course management system, and/or provided to students in an email. Schools may require students to sign the document acknowledging that they received the notice.

Guidelines should include recommendations for the testing environment (well-lit room, computer device on a flat clean surface, testing surface clear of any notes or resources) and appropriate dress during the assessment. Guidelines should also inform students of prohibited items such as cell phones, headphones, hats, or smart watches, and provide guidance on minimizing distractions that may generate a flag by the proctoring system (eg, looking away frequently, talking, or leaving the room). Schools need to address potential inequities in student access to the optimal resources and environment for remote testing. For example, some students may lack access to reliable internet service, preventing test access or completion. Schools may need to help students identify reliable WiFi access so they can complete examinations. Some students may lack a suitable environment for remote testing (eg, sitting on a bed or in a car with their laptop) or may have difficulty preventing interruptions (eg, other household members and/or pets that are noisy or distracting). Students facing these challenges should discuss them with appropriate administrators or faculty to explore solutions.

Schools and colleges of pharmacy should also include guidance on who to contact about examination issues, absences, bathroom breaks, and emergencies and/or interruptions. A process to support students during these challenges is recommended, such as establishing a virtual “help room” via Zoom that is maintained by the course coordinator and/or assessment staff that students can access on a secondary device (eg, cell phone) that is not being used for administration of the examination. Another support option is to have a faculty or staff member on call during the examination through email, an office phone, or cell phone. For externally delivered and proctored examinations, some proctoring software companies provide immediate technical support via a phone or chat feature, while others provide an email or ticket-based support system. Local faculty and staff support should be sure to have the software contact information handy in order to quickly refer students if the proctor is unable to resolve the issue. Additionally, schools should consider adjusting guidelines related to examination settings, such as extending test upload deadlines and test time limits to allow for slow internet speeds.

Scratch paper is another item that should be addressed in the guidelines because students may have been allowed to use it during previous in-person examinations to perform calculations, access formulas, organize personal ideas, or alert faculty to concerns about an item on the examination. Prior to COVID-19, proctors collected and reviewed scratch paper at the end of the examination and shredded or stored it to protect the integrity of the examination for students in future years. When testing remotely, the same level of security with scratch paper cannot be enforced and students may not be able to print the scratch paper at home. However, alternatives to scratch paper exist, such as embedding formulas into the test question, using a dry erase board, or using a notepad within the testing software, if available. Whichever option is selected, guidelines should instruct students on acceptable sizes of the paper or white board, how to display the paper or white board on camera at the start and end of the exam (a blank paper or clean or erased board), how to alert faculty to concerns or problematic examination questions, and how to upload it into the course management tool dropbox or assignment folder for faculty inspection following completion of the examination, even if the scratch paper is blank.

One last consideration is that the new guidelines should include specific consequences for student nonadherence to the school’s online testing policies and procedures. These consequences may include an email warning with corrective actions or a grade penalty (eg, 10% reduction) on the exam.

Training Faculty and Staff in the Use of Testing and Proctoring Software

Once the guidelines are communicated to all stakeholders, the roles and responsibilities of faculty, staff, or assessment personnel should be clearly outlined prior in advance to any administration of an examination in order to reduce mistakes and confusion. Faculty and staff who are assigned and will serve as proctors then need to receive training on the testing/proctoring software and the new procedures for administering and proctoring examinations. The training may be conducted by the school assessment or instructional design staff, the software company, or an academic technology department at the university, and should include a demonstration of the technology, a review of the examination recordings and reports generated, instructions for how to access them, and reaffirmation of the role of the proctor in upholding academic integrity.

One important piece of training that proctors should receive if their school uses record and review software is how to review the video recordings of the examination and/or flagged reports for academic misconduct. The training should help proctors determine whether there are false flags (eg, poor lighting) and whether they need to further evaluate suspicious behaviors for possible breaches in academic integrity. Reviewing video recordings is the most time-consuming step with online proctoring, especially in schools with large class sizes, as it may take several minutes per student to review videos and evaluate potential flags. The proctor training should include examples of potential academic misconduct, such as students reading test questions and answers aloud, and looking anywhere other than their screen (eg, their lap, wall, or next to or above screen) as they may be looking at notes. Training should also outline what parts of the recording proctors should review entirely vs spot check. Specifically, proctors should review the entire environment scan; the exam start (in case students forget to show something in the environment scan); and the last five minutes of the examination when students may be erasing a white board on camera or showing their scratch paper. The proctor should then be trained in how to spot check the video throughout the recording at a faster speed (maximum 4x speed) as the flags may not identify all areas of misconduct. Proctor training should conclude with a review of the process for reporting suspected academic misconduct, including consulting the school’s faculty handbook for specific guidance. For schools that use ProctorU or another live third-party proctor, faculty and staff should still be familiar with the system and process for proctoring an examination.

Student Training on the Testing Software

Students should receive training about how to conduct a thorough environment scan, including showing phone location, removing watches, displaying scratch paper and calculators, confirming their identity, accessing test passwords, and stating honor codes. Furthermore, students should be provided with a mock examination or quiz to familiarize themselves with the use of the software. A mock examination also allows faculty to correct any student behaviors that may be flagged as a breach of integrity by the software, including poor positioning of the webcam, insufficient lighting, excessive movement (ie, getting up/walking around), or reading questions aloud during the assessment.