Comments on the Impact of a Yoga and Meditation Intervention on Students

To the Editor: Lemay, Hoolahan, and Buchanan published the results of a pilot study in a 2019 issue of the American Journal of Pharmaceutical Education.¹ They developed a promising six-week dual intervention involving yoga and meditation for college students (one session per week). One group of students was recruited (n=17) and their levels of perceived stress, anxiety, and mindfulness skills were assessed before and after the six-week intervention. We commend the authors for creating a promising tool to tackle the often put aside issue of student psychological distress. Nevertheless, we question and comment on the pilot study methodology used by the authors, the procedure used to assess the stated objectives, and the interpretation of the results.

Pilot studies are feasibility studies, meaning that they are conducted to assess the quality of the studied intervention and the smooth running of the study procedures before conducting a large-scale study.² Therefore, pilot studies only require a small sample and the collected data should not be interpreted per se.² The strength of a pilot study rests in the rigorous description of the procedure employed to conduct the assessment of an intervention (ie, methodology, procedure, intervention, quality, and feasibility assessments). Such description allows other researchers to replicate the study to further support the assessed intervention as evidence-based. In their pilot study, Lemay, Hoolahan, and Buchanan aimed to assess the efficacy of their dual intervention on reducing perceived stress and anxiety, as well as increasing mindfulness skills in college students. As is required in a pilot study, the authors clearly reported their methodology, that is, their inclusion and exclusion criteria, how they recruited participants, and the measures used in their assessment. They also used robust and psychometrically sound self-report measures to assess the levels of stress, anxiety, and mindfulness skills among their participants.

A key element of pilot studies was missing in this article. The authors asserted that their dual intervention was of high quality. However, their intervention lacked detailing (ie, yoga postures, meditation scenarios, room disposition, instructions technique used) and the authors did not report how they assessed the quality of their intervention, nor did they state how they evaluated the feasibility of their study procedures. Indeed, they adequately indicated their main study objectives, but did not state their feasibility objectives. This is problematic as it cannot be assumed that both yoga instructors employed for this pilot study adequately disseminated the dual intervention every week. The lack of feasibility objectives, quality assessment, and details regarding the intervention renders it impossible for other researchers to reproduce the study procedures and affects the study’s external validity. To counteract this issue, the authors could have implemented a quality check procedure, for instance, by recording the dual intervention sessions and having two assessors watch, rate, and score their agreement levels over the quality of the sessions. In addition, half of the group recruited in the pilot study were pharmacy students, while the other half were students from programs with “less rigorous demands.” Due to this high proportion of pharmacy students recruited, hiring two pharmacy faculty members as the yoga instructors might have affected both the external and construct validities of the study. The presence of this social desirability bias potentially resulted in reactivity to the experimental arrangements of the study (ie, priming the alteration of the performance of the participants who knew they were being studied), affecting its external validity.² Moreover, both instructors did not receive the same yoga training, which might have biased the procedures and affected the internal validity, as it is unclear whether the instructors disseminated the same yoga routine (eg, postures, time holding the postures, etc). These biases regarding the efficacy of the intervention prevent us from determining which construct influenced the severity of the symptoms and threaten the generalizability of the results.² In the future, the authors should use the services of external yoga instructors from the same yoga school.

As part of their study, the authors hypothesized that their dual intervention would yield “more benefits to students in a shorter time-frame than either practice separately.” The study design described by the authors does not support the examination of such a hypothesis. To examine this hypothesis, the authors would have needed to recruit three groups (only yoga, only meditation, and dual intervention), as the interaction of yoga and meditation could also have had a negative effect on the symptoms, not just a positive one. They would also have needed to implement repeated assessment measures of the dependent variables at least at three different time points or ideally after each session for all three groups to adequately attribute the decrease of symptoms to their dual intervention and demonstrate that their dual intervention required less time to provide the intended benefits. Therefore, we consider the authors’ suggestion “that adopting a mindfulness practice for as little as once per week for six weeks may reduce stress and anxiety in college students” goes well beyond what the study design and intervention protocol could support. Moreover, the authors timed the end of their intervention with the end of classes and beginning of examinations, mentioning it to be a time of supposedly increased stress and anxiety. Again, the study design reported does not support such affirmation. To support this affirmation, a control group (ie, fourth group not receiving a yoga, meditation, or dual intervention) should have been recruited. Such a control group would have helped to conclude whether the observed changes over time were truly due to the intervention or to other factors, like the end of classes. The implementation of a procedure including four groups (ie, yoga group, meditation group, dual group, and control group without any intervention) in a future large-scale study would entail the recruitment of 76 participants, given an attrition rate of 11%, in order to have 66 completers at the end of the study to detect a medium effect size (f =.42) and have a statistical power of .80.³

Although the authors conducted a pilot study, conclusions were extrapolated from the pilot study design. Some nuances therefore need to be raised regarding the analyses carried out and the interpretation of the results. It was difficult to determine whether the results of the dual intervention were generalizable to the population of interest (ie, college students) due to a focus put on the recruitment of pharmacy students.² The authors compared the “pharmacy” subgroup to the “other program” subgroup and concluded that no differences were found between the groups in terms of stress and anxiety levels. Given the subgroup sizes, we calculated that the highest level of power that could be achieved when considering a large effect size of .8 was .34, while the level of power reached .16 for a moderate effect size of .5 and only reached .07 for a small effect size of .2. Such low levels of power increase the likelihood of a type II error, greatly hindering the possibility of interpreting the results.² Additionally, Lemay and colleagues’ mentioned in their article that they encouraged participants to practice at-home meditation. This confounding variable was not reported to be controlled for in the analyses, nor were students’ prior yoga, meditation, and/or mindfulness experience levels. Without these results, it is impossible to know if the face-to-face dual intervention was a truly active ingredient in the decrease of symptoms, if its effect was different from that of at-home meditation practice, or if the at-home meditation practice could have been a moderating factor. Therefore, the potential confounders mentioned could have biased the results Lemay and colleagues’ attributed to the dual intervention (ie, significant decrease in levels of anxiety and perceived stress). Lastly, although the results were statistically significant, the effect sizes were not reported, keeping us from knowing whether the decrease in symptoms was clinically significant. In order to better interpret their results, we recommend that Lemay and colleagues’ recruit a representative sample of their population of interest in the future, whether it is pharmacy students or college students in general; recruit the number of participants needed to obtain a standard statistical power level of .80; include all confounders in their statistical analyses; and report the detected effect sizes.²

Lemay and colleagues’ idea to develop a dual intervention involving both meditation and yoga components has great potential to help students cope with academic pressures. Still, the authors’ pilot study did not demonstrate the feasibility and quality of their protocol to assess their stated objective, as is expected of pilot studies.² Despite their promising intervention, the lack of details regarding the intervention and the study methods give way to multiple threats to internal, external, and construct validity. This pilot study is therefore irreproducible and not suitable for the stated objective and hypothesis. Before conducting a large-scale study, the authors should consider the comments and recommendations provided here, revise their procedure, and perform a new pilot study to assess the feasibility of the improved design. In light of these comments, the authors should restrain at the moment from encouraging the implementation of their dual intervention in university settings.

• Received August 1, 2020.
• Accepted January 17, 2021.

• © 2021 American Association of Colleges of Pharmacy