Examining the Relationship Between Biometric Indicators and Pharmacy Students' Academic Performance

INTRODUCTION

Student wellness, specifically activity and sleep, and its potential relationship with student performance outcomes is an important topic to address in higher education. This may be especially important in rigorous and stressful health professions programs such as the Doctor of Pharmacy (PharmD) degree program.^1-5 While there are national initiatives to improve health on college campuses such as Healthy Campus 2020, individual institutions should consider the specific needs of their student population.² A growing body of literature suggests that physical activity and appropriate sleep duration positively influence brain function in adults.^6,7 Other research has posited that sleep quality is more important than duration as related to academic performance.^8-11 These same factors may impact a student’s academic performance in the classroom.^12,13 However, a limitation to these published studies is that they were conducted retrospectively and may not have included a direct measurement of either students’ activity or sleep.^3,13,14 Also, there has been only limited research in this area that was specific to PharmD programs.

The data in pharmacy and medical education literature is generally consistent with that for other types of students regarding the impact of sleep and exercise on academic performance. Higher course grades and semester grade point averages (GPAs) are associated with longer duration of sleep on nights prior to the examination or with higher sleep quality.^13-17 Slade and Kies used a regression model to identify the role of recreation before examinations on changes in academic performance in medical students and concluded that an increase in recreational activity before an examination was positively related to examination performance, and that a reduction in recreation was negatively related to examination performance.¹² Chung and colleagues also identified a significant association between physical activity level and academic achievement in health professions students.¹⁸

Based on findings from the literature review, there appears to be a connection between activity and sleep when examined individually and academic performance. However, there are no data that examined the relationship of the combination of both sleep and exercise to academic performance. Additionally, much of the data were based on participant recall rather than prospective and objective data collection. This led us to identify variables that could be objectively quantified on a consistent, prospective basis across a target population. We believed that having students use a Fitbit (Fitbit, Inc., San Francisco, CA) bracelet would increase data quality by ensuring objective measurement and real-time, automated capture. We did not seek to validate the use of the Fitbit as an academic performance indicator. Instead, we selected the Fitbit to track the variables of activity and sleep because of its accurate and valid measurement of both steps and sleep duration.^19,20 Thus, the purpose of this study was to evaluate the relationship between the biometrically tracked metrics of physical activity and sleep duration with pharmacy students’ examination performance and semester grades.

METHODS

Doctor of Pharmacy (PharmD) students at Western New England University were invited to attend an information session regarding this research project in the fall semester of their second professional year (P2). This session outlined the research plan, elements of informed consent, and answered questions related to the research. Students were informed that, if they chose to participate, they would receive a Fitbit Charge bracelet prior to starting the spring semester and that the investigators would have access to their Fitbit data as well as their individual examination and overall course grades in their required courses. To allow for a standardized comparison across the semester, course grades from elective courses were not included. Students were also informed that upon study completion, they could keep the Fitbit. Sixty-three of the 74 P2 students consented to participate in the study. The participants were instructed to wear their Fitbit for the entire semester and to remove it only to recharge while attending a designated class that one of the investigators taught.

Each participant’s Fitbit account was enrolled into Fitabase (Small Steps Labs, LLC, San Diego, CA), a cloud-based data aggregation platform that synchronized with the students’ individual Fitbit app accounts. The Fitbit collected the students’ total number of daily steps and total daily minutes in which the student was very active, fairly active, lightly active, sedentary, or asleep. The college instructional design specialist collected all individual and final grade percentages from the university’s learner management system. The study period for collecting biometric data and grades was from January 25, 2016, through May 13, 2016.

The outcome variables of interest were student GPA and grades for 17 examinations administered across eight required courses during the study period. The study GPA was calculated using the actual earned grades in required courses. Primary predictors of interest were total daily sleep and total daily steps. Secondary predictors of interest were total daily minutes in which the participant was very active, fairly active, lightly active, and sedentary as these measurements were recorded using a proprietary Fitbit algorithm. Authors analyzed these predictors overall for the entire semester, as well as in periods prior to examinations. After exploratory analyses, the primary research questions relating steps and sleep to performance were addressed with a series of linear regression models. Authors considered several models for the primary analysis. First, simple linear regression models, both controlling and not controlling for person with sleep only, steps only, and a combination of sleep and steps were used. Second, mixed effect models, which treated the person effect as a random effect to control for dependence between observations on the same person were used. In these models, examinations were considered as both a random and a fixed effect. Last, median regression models were used as robustness checks against the undue influence of a few influential points.

For these models, one student with extremely atypical examination results (score of 0) was excluded (this student later withdrew from the program). Regression analyses were performed using R (R Core Team) employing Quantreg, version 5.29, for quantile regression and LME4, version 1.1-12 for linear mixed-effects models.

Next, we sought to determine if there was a change in a particular type of activity (eg, number of steps, sedentary time, or sleep time) in the 48-hour period preceding examinations as compared to typical individual activity. For this, the investigators performed paired-sample t tests, comparing each student’s average activity on all days with their average activity level on days just before tests. Last, using linear regression, the authors explored the overall relationships between the other Fitbit variables (total number of daily steps, total daily sleep minutes, and total daily minutes during which they were very active, fairly active, lightly active, and sedentary) and academic performance.

This study was approved by the Western New England University Institutional Review Board. Funding for the Fitbit Devices and Fitabase subscriptions was provided by the institutional start-up research monies.

RESULTS

Sixty-three students consented to participate in the study and were issued a Fitbit bracelet to track their daily number of steps, activity levels, and sleep minutes. On average, students were 21 years of age, 63% were female and a majority of participants (61%) had completed pre-pharmacy coursework at the study institution (Table 1).

Investigators did not find any relationship between students’ course GPAs and the total daily steps or total minutes asleep they had each day, regardless of whether predictors were included individually or in combination in the model. Also, no significant short-term relationships were found between sleep and examination performance across all of the models. Several models suggested a possible negative relationship between steps taken in the three days prior to examinations and examination performance. The results for standard linear regression models based on 1 week, 72 hours, 48 hours, and 24 hours prior to the examination are shown in Table 2. The results obtained from the median regression models can be found in Table 3.

Testing at the .05 level, there was a significant negative relationship between the number of steps students took 72 hours before the examination and examination performance in several models, in particular, when controlling for examination effects (eg, differences in examination difficulty), either alone or along with sleep 72 hours prior to the examination, or along with average performance for the person (Table 2). To further investigate this effect, we binned (binning is a data pre-processing technique that can reduce the effects of minor observation errors) average daily activity in the three days prior to an examination into three categories: low activity (less than 5,000 steps per day), moderate activity (5,000-10,000 steps per day), and high activity (more than 10,000 steps per day). Subsequently, investigators conducted regression analyses to compare the performance of student examination in each category. There were no significant differences between the low- and moderate-activity groups; however, the low-activity group significantly out-performed the high-activity group by an average of 2 points. The moderate-activity group also significantly outperformed the high-activity group by an average 1.9 points when average person effects were not controlled for.

During the semester, students took an average 7,622 steps per day. There was a 20% decrease in students’ average daily steps during the 48 hours prior to examinations, equating to 1,466 steps less than normal activity (p<.001, 95% confidence interval for a decrease of 1164 to 1768 steps). There were no significant differences in students’ sedentary time or sleep time in the 48 hours prior to examinations. Additionally, there was no significant relationships between overall academic performance and overall levels of other secondary Fitbit variables.

DISCUSSION

This was the first study to biometrically track both PharmD students’ sleep and activity in an attempt to identify a “sweet spot” that is associated with academic achievement. Our underlying motivation was to identify students who lacked an appropriate wellness balance throughout the semester and surrounding the time examinations were administered, thereby providing an opportunity for the school to intervene. Contradictory to prior research findings, we identified that students with reduced activity before examinations performed better. Fitbit devices have been validated to accurately capture these metrics; however, prior to this study there had not been an attempt to link the metrics to academic outcomes.¹⁹ The importance of this novel study was the use of the Fitbit to track and evaluate population-based, biometric activity and sleep data to identify any correlation with examination and semester grades. There are some data in health professions education to suggest that providing students with an objective measurement of their activity will encourage them to be more active.²¹ These findings may not justify the cost of acquiring Fitbit devices for PharmD students to assist in identifying those who may need encouragement to ensure they are getting sufficient exercise and sleep around the time of examinations.

The significant findings of this investigation appear to make intuitive sense. One would expect that students prepare for exams and thus are less active during the 48 hours leading up to an examination; the converse would be that and those who do not exhibit a similar decrease in activity, presumably study patterns, would not perform as well academically. While authors hypothesized that the decrease in activity is likely due to students sitting and studying, it is not possible to say for certain that this occurred. The days of the week on which examinations were scheduled varied; thus, there was likely not a “weekend” effect that could have accounted for the decreased activity. The binned results suggest that a decrease in performance may be especially pronounced, with students who took over 10,000 steps per day in the three days prior to an examination scoring lower. Combined with the results regarding the tendency for steps to decrease in the days before the examinations, this would suggest that successful students may be taking more time to study during this period.

This was an observational study rather than a controlled experiment. Because subjects themselves decided how they would behave, it is not possible to draw causal conclusions with confidence from these data. Therefore, while the results of this study in no way contradict the wealth of information available regarding the importance of personal health, they do support the idea that students need to take time prior to examinations to study.

There are a number of limitations to this study. Primarily, the independent variables were activity and sleep; we did not account for any confounding variables that are known to be correlated with academic performance in pharmacy education. While we did find a significant relationship between the number of steps students took during the three days before an examination and examination performance, a small number of data points could be driving the results as this pilot study had a relatively small sample size that was evaluated for only one semester. However, we believe those points are credible and informative, so it is acceptable for them to influence the outcomes. Additionally, results were not significant across the different numbers of days before the examination (ie, in addition to three days, one, two and seven days were tried). This may also suggest that this finding is sensitive to a particular set of events corresponding to three days before examinations in these data and not a general finding about “exercise before examinations.”

CONCLUSION

Based upon data obtained from a fitness tracking device from one cohort, sleep and physical activity are not robust predictors of academic performance in a PharmD program. While a Fitbit or other fitness tracker may be an impetus for students to be more cognizant of their activity level, the capital investment (ie, cost of the devices) did not yield returns, ie, the ability to intervene regarding student wellness to facilitate improvements in academic performance.