A Mixed-Methods Exploration of the Effect of Disabling Backward Navigation on Calculations-Focused Computerized Examinations

Abstract

Objective. The aim of this mixed-methods study was to examine the effect of disabled backward navigation on computerized calculation examinations in multiple courses.

Methods. Student performance on comprehensive pharmacy calculation examinations before and after implementation of disabled backward navigation were compared. Deidentified data from ExamSoft were used to determine median examination scores, passing rates, and time to completion for all three attempts given on comprehensive calculation exams held in a pharmacy calculations course (PharDSci 504) and in three applied patient care laboratory courses (Pharm 531, 541, and 551). An anonymous, voluntary student survey gathered student perceptions of disabled backward navigation. Qualitative data were evaluated for thematic findings.

Results. The impact of disabled backward navigation on test scores and passing rates varied by course and test attempt. Students in Pharm 541 and 551 performed significantly worse on the initial test attempt after backward navigation was disabled compared to the previous year, with no significant differences in student performance seen on the retakes. Performance in PharDSci 504 and Pharm 531 followed the opposite pattern, with no significant difference in performance for the initial tests but significantly increased performance on the retakes. The amount of time spent on examinations either significantly decreased or remained the same. Student perceptions were generally consistent across all cohorts, with at least 74% agreeing that disabling backward navigation increased examination difficulty.

Conclusion. Disabling backward navigation had a mixed effect on student examination performance. This may highlight how student behaviors change as backward navigation is disabled.