An International Comparison Study of Pharmacy Students’ Achievement Goals and their Relationship to Assessment Type and Scores

INTRODUCTION

The quality of and motivation behind student learning is of great interest to tertiary educators, and considerable effort is devoted to evaluating that quality and enhancing it. Research that seeks to understand the relationships between student motivation and their academic performance is essential to this endeavor.

Achievement goal theory has been an important framework used to study undergraduate students’ motivation. Achievement goal theorists posit that students pursue 1 of 2 broad types of goals when they face any academic activity. They either try to understand this activity as much as possible (mastery goal) or they try to compete with each other (performance goal).¹ Gaining competence is the main reason for pursuing either goal.² Competence is viewed differently by students depending on their goal orientations. Students who adopt the mastery goal believe that competence can be gained by understanding the task at hand as thoroughly as possible and seeking help when they need it.³ They use self-referential standards to differentiate between success and failure.³ Students who adopt a performance goal, on the other hand, believe competence is gained by outperforming their peers and appearing talented in front of their teachers.⁴ These students adopt their teachers’ standards of success and failure.⁵

These 2 types of achievement goals are further subdivided into 4 types: (1) mastery-approach, where the individual is motivated to learn or improve skills; (2) mastery-avoidance, where the individual is motivated to avoid learning failures or declines in skill; (3) performance-approach, where the individual is motivated to outperform others; and (4) performance-avoidance, where the individual is motivated to avoid doing worse than others.^{6, 7}

Research conducted with undergraduate students from disciplines such as psychology, sociology, business, biology, and art investigated the impact of achievement goals on students’ interest in academic activities, academic achievement (eg, scores), anxiety, surface learning (eg, memorizing), and help seeking.^4,8,9 The results for mastery-avoidance and performance-avoidance goals are consistent in terms of their negative effects on students, including poor scores, low interest in the subject, anxiety, and cheating.^10-12

In contrast, the mastery-approach goal is linked to positive attributes, such as deep learning strategies,¹³ high interest in the subject,¹⁴ and seeking help when needed.¹⁵ From the teacher’s perspective, this goal orientation is highly valued. Despite these positive effects, however, empirical research to date has found no significant relationships between this goal and academic achievement.⁴ Adopting a performance-approach goal is linked to mixed outcomes. For example, students who adopt this goal use surface learning strategies such as memorization and are more anxious.^16-18 Other studies find students who adopt this goal orientation achieve higher scores on their examinations.^3,19

What is currently missing from the literature on goals and academic performance is an exploration of the types of examination undertaken. In universities, different types of assessments, such as oral, essay, and multiple choice-question examinations, are intended to measure student knowledge. Each of these examination types has its own advantages and disadvantages. For example, an advantage of essay style examinations is their capacity to assess deep understanding and critical thinking, while a disadvantage is their relative subjectivity when grading.²⁰ Multiple-choice question examinations address this subjectivity by limiting the answer to “one correct” answer, yet this method promotes surface approaches to learning.²¹ Goal theory would suggest that students who are strongly performance-oriented (and thus more likely to use surface learning and memorization) are expected to perform better on multiple-choice questions, whereas students who are more mastery-oriented are more likely to demonstrate deeper understanding and thus perform better on essay style questions.^21,22

While research to date provides us with valuable knowledge about the relationships between students’ motivation and key outcome indicators of their learning, the unexplored counterpoint to studying student achievement motivation is examining this construct from a teacher-focused perspective. Doing so raises the following issues: (1) what we currently know about students’ preferred achievement goal orientation(s) and what we can learn from this; (2) the extent to which student achievement goal motivation is generalizable across comparable degree programs and educational settings; and (3) the extent to which theoretical underpinnings of achievement goal orientations are predictive of different types of academic assessments.

There is a dearth of published research conducted in higher education settings regarding these issues. Moreover, little is known about the achievement goal motivations of pharmacy students, their relationship to academic performance, or how they are expressed in the pharmacy education environment. To our knowledge, only two studies have been conducted to investigate pharmacy students’ achievement goals. Gavaza and colleagues found in their cross-sectional study that second year PharmD students adopted the performance-avoidance goal more than first year students in the same program.²³ In addition, Alrakaf and colleagues found that, among a sample of undergraduate pharmacy students at a single Australian university, adopting the performance-approach goal correlated positively with academic achievement and adopting mastery-avoidance and performance-avoidance goals correlated negatively with academic achievement.²⁴ To our knowledge, no studies have been undertaken to compare pharmacy students’ achievement goals across comparable degree programs in different countries.

Our study sought to investigate these issues by conducting an international comparative study across four universities from Australia, New Zealand, Wales, and England. Based on achievement goal theory and research to date, the following hypotheses were proposed: (1) In light of the performance-based learning environment characteristic of higher education settings, pharmacy students preferred achievement goal(s) would be performance-oriented rather than mastery-oriented; (2) In the absence of previous research, no differences would exist among comparable pharmacy degree programs (eg, similar degree structure and language) in terms of achievement goal orientations; (3) In light of achievement goal theory, examination format, academic performance, and goal orientation would be related; that is, students with high scores on multiple-choice question examinations will be more strongly performance-approach goal oriented, and students with high scores on essay-style examinations would be more strongly mastery-approach oriented.

METHODS

The study was initiated in August/September of 2012. All participants were second-year undergraduate pharmacy students enrolled in bachelor of pharmacy programs at universities in Australia (The University of Sydney) and New Zealand (the University of Otago), or master of pharmacy programs in England (Nottingham University) and Wales (Cardiff University). Researchers at each university invited students to participate in the study during normal lecture or tutorial periods. They were advised that participation was voluntary and, if they chose to participate, they could withdraw from the project at any time. In addition, students were advised that their decision to participate would not affect their academic results or influence their student-teacher relationships. Researchers approached the students in group and not individually. A validated achievement goal questionnaire was administrated to students in paper form by the researchers.²⁴ Completion of the questionnaire took approximately 10 minutes. The locations for data collection were selected by the first and last authors, who contacted researchers from the countries of interest at pharmaceutical conferences. The four locations were chosen for their comparable degree program structure and primary language (ie, all were 4-year programs and in English speaking countries).

At the end of the teaching period, students’ scores from second-year units of study were collated from the four participating universities: Pharmacy Practice in Sydney, Biopharmaceutical Chemistry in Otago, Clinical and Professional Pharmacy in Cardiff, and Professional Skills 2 at Nottingham. Every unit of study had a final examination, but with varying formats, enabling a comparison to be made between examination type. Multiple-choice question and short essay scores, and the total grade were compiled from New Zealand and Australia. Short essay scores and the total grade were compiled from all 4 participating universities.

Following an international validation study,²⁵ the Achievement Goal Questionnaire (AGQ)²⁶ was used to measure pharmacy students’ goal orientations (Appendix 1). The questionnaire contains 12 items intended to measure the 4 types of achievement goals on a 7-point Likert scale (ranging from 1=not at all true of me to 7=very true of me). Socio-demographic data included gender, age, and student identification numbers, which were used only for matching students’ scores with their achievement goal orientations. Individual participants could not be identified in the analysis.

To ensure participants’ anonymity and confidentiality, all data entry was carried out by the first author who had no contact with the participants, and each participant was allocated a unique identifying code, which was matched to the student identification number and stored in a password-protected file accessible only to the first author. Once each returned survey form was received, the first author wrote the matching code onto the survey form, then detached the page containing the student identification number and stored them separately from the questionnaires. All analyses were based on group data and not individual data.

For all statistical analyses, SPSS v21 (SPSS Inc., Chicago, IL) was used. Descriptive statistics regarding gender and age were reported. One-way repeated measures analysis of variance (ANOVA) was used to compare each type of achievement goal at each university. Mauchly’s test indicated that the assumption of sphericity had been violated in the Cardiff, Otago, and Sydney samples (χ2 (5)=19.37, p<0.05; χ2 (5)=16.35, p<0.05; χ2 (5)=14.80, p<0.05, respectively). Therefore, degrees of freedom were corrected using Huynh-Feldt estimates of sphericity (ε=0.93, ε=0.93, ε=0.96) for Cardiff, Otago and Sydney samples, respectively. One-way ANOVA was used for multiple comparisons of each type of achievement goal among universities. Mean scores of achievement goals were used in all analyses. All mean difference analyses were subjected to post hoc tests (Bonferroni and Tukey HSD tests). Multiple regression procedures were performed to determine the extent to which achievement goals contributed to total, short essay, and multiple-choice question scores at each university. As gender was found to be a predictor of achievement goal orientation and academic performance in a previous study,²⁰ this variable was included in the model. Forced entry method was used to examine the odds ratios of all variables, even if not significant. A p value of less than 0.05 was considered significant for all analyses.

Ethical approval was granted by human ethics committees at the 4 participating universities as follows: Human Research Ethics Committee at University of Sydney, Human Ethics Committee at University of Otago, Cardiff SPPS Research Ethics Committee at Cardiff University, and Faculty of Science Ethics Committee at Nottingham University.

RESULTS

Four hundred eighty-six students with a mean age of 20 participated in this study. Descriptive statistics for the countries’ participants are reported in Table 1. Contrary to expectations, hypothesis 1 was not supported—the overall preferred goal orientation across all 4 universities was the mastery-approach goal (Figure 1/Table 2). The ANOVA test and post hoc comparisons of the main effect using Bonferroni adjustment revealed that students at 3 of the 4 universities reported significantly higher scores for the mastery-approach goal than the other 3 goal orientations (Otago: F=17.35, p<0.01, eta2=0.16; Cardiff: F=42.47, p<0.01, eta2=0.34; Nottingham: F=37.12, p<0.01, eta2=0.22, respectively). At the fourth university, Sydney, a significant effect for achievement goals (F=56.80, p<0.01, eta2=0.25) and Bonferroni adjustment revealed that students in this sample reported significantly higher mastery-approach goal than performance-approach and mastery-avoidance goals, but no significant difference between mastery-approach and performance-avoidance goals.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Goal Orientation Mean Scores

Regarding hypothesis 2, an overall similarity in students’ goal orientations was evident (Figure 1), however some within group variations were apparent for each goal orientation. Differences were also identified between groups for each of the goal orientations: one-way ANOVA results revealed significant differences between groups in mastery-approach (F=8.98, p=0.000) and mastery-avoidance (F=3.44, p=0.017), but not performance-approach and performance-avoidance goals. Tukey post hoc comparisons of the 4 groups indicated that Cardiff students (mean (SD): 6.32 (0.80)) pursued the mastery-approach goal significantly more strongly than their peers in Sydney (5.67 (1.07)), Otago (5.93 (1.04)), and Nottingham (5.75 (0.99)). Tukey post hoc comparison outcomes revealed that Otago students pursued the mastery-avoidance goal significantly more strongly than other groups (4.81 (1.40)) (Table 2).

In order to test the third hypothesis, 3 multiple regression procedures were conducted to test the extent to which goal orientation and gender contribute to the variance in students’ total scores, multiple-choice question scores, and short essay scores. With respect to total scores, while the full model was significant (F=2.50, p=0.03), only gender made a contribution (beta=0.18; t=3.31; p=0.001; CI:1.57-6.15), indicating that females overall attained higher total scores than males.

With respect to predicting the relationship between goals and multiple-choice question scores, the full model containing all predictors was significant (F=4.04, p=0.002). As shown in Table 3, only 2 of the independent variables made a unique significant contribution to the model (mastery-avoidance and performance-avoidance goals). The strongest predictor of multiple-choice question was mastery-avoidance goal (beta=0.18, p=0.01). This indicated that students who strongly pursued the mastery-avoidance goal were significantly more likely to gain high scores than those who did not pursue this goal, controlling for all other factors in the model. The performance-avoidance goal also made a significant contribution, whereby students with a strong orientation toward this goal were likely to achieve lower multiple-choice question scores (beta=-0.14, p=0.02).

In predicting the relationship between goal orientations and short essay examinations, the full model containing all predictors was significant (F=4.20, p=0.001). As shown in Table 4, 3 of the independent variables made a unique, significant contribution to the model (performance-avoidance, mastery-approach goals, and gender). The strongest predictor of the written examination scores was performance-avoidance goal (beta=-0.14, p<0.01). This indicated that students who strongly pursued the performance-avoidance goal were likely to achieve lower scores than those who did not pursue this goal, controlling for all other factors in the model. The mastery-approach goal was a significant positive predictor of higher scores in the short answer essays (beta=0.13, p=0.01), meaning that the more strongly students pursued this goal, the higher their scores were. Gender also had a significant relationship to the written examination scores (beta=0.10, p=0.03), indicating that females achieved higher scores than males.

DISCUSSION

For more than 3 decades, achievement goal theory has been used to investigate student learning and academic achievements across a range of disciplines, but there remains a paucity of research on student motivation in pharmacy education. By adopting a teacher-focused lens, our study revealed important and useful information for pharmacy educators regarding student achievement goal motivation and provided pointers to future research. Comparative studies enable teaching academics to compare and contrast among different educational settings in order to borrow successful practices from each other.^27,28

The finding that the predominant goal adopted by pharmacy students across all 4 universities was mastery-approach was unexpected. Universities by their nature base student progression on successful demonstration of competence. This demonstration is usually examination-based, and the evidence to date suggests that a performance-approach goal orientation is associated with higher scores.^3,4,19 Furthermore, Western cultures are characterized as highly individualistic, competitive, and materialistic,^29-31 and there is evidence that students in such cultures are inclined to adopt the performance-approach goal.³² In addition, Smith and colleagues found that second-year students were more inclined toward the performance–approach goal and preferred external directions from their teachers.³³

This unexpected finding may please faculty members at these universities, as students across all universities, regardless of subject studied or place of learning, were mastery-approach oriented. This indicates a preference for deep learning and interest in the subject matter. Previous research indicated that mastery-approach developed not only competence but also confidence,^29,30 attributes which foster life-long learning. This finding should reassure faculty members that their teaching practices encourage students to adopt productive approaches to their learning. Students who are strongly mastery-approach oriented are taught by teachers who themselves adopt mastery-approach goal and encourage their students to adopt this type of achievement goal.³⁴ Shim and colleagues proposed that teachers who adopt the mastery-approach goal can inspire their students to pursue this goal as well.³⁵

The results of the second aim of our study showed little difference in the pattern of students’ goal orientations across the university degree programs. With one exception, a pattern indicating a strong preference for the mastery-approach goal orientation, followed by performance-avoidance, performance-approach and finally mastery-avoidance orientations was evident. The exception was the Otago cohort, which indicated a stronger preference for mastery-avoidance; however, this was not significantly different from the 2 performance goals. Of concern was the finding that a preference for the performance-avoidance goal among student cohorts was also consistently evident, and its negative influence on performance was demonstrated in this study. As this goal orientation is the most maladaptive and unproductive of the 4, teachers could explicitly focus on classroom practices that mitigate it, such as activities that foster confidence, reduce test anxiety, and encourage questions. These practices could include encouraging students to ask questions regardless of their simplicity, ensuring students’ learning tasks are incremental and achievable, encouraging team work, and giving regular feedback on student performance in terms of both mastery and achievement.

Identifying the relationships between achievement goals and academic achievement also revealed interesting results. Total scores can be an imprecise indicator of approaches students may take to learning, and this was borne out in our results. Apart from gender, there were no significant relationships between students’ achievement goals and their total scores. In contrast, when a more comprehensive analysis was undertaken, a more instructive picture emerged. Participants in our study indicating a preference for a performance-avoidance goal orientation were more likely to achieve lower scores on both multiple-choice question and short essay examinations. This is in line with theory, whereby the primary motivation behind the performance-avoidance goal is avoidance.^3-5 These students, lacking confidence, strive to avoid appearing incompetent to their teachers and peers and tend to experience test anxiety.⁴ These students view the performance-avoidance goal as a means of developing competence, but empirical testing of the theory shows that this approach is a recipe for attaining low scores—that is, performance-avoidance attributes are incompatible with acquiring and demonstrating competence. High scores in the multiple-choice question format, on the other hand, were positively associated with the mastery-avoidance goal orientation. Like the performance-avoidance goal, this goal orientation is characterized by “avoidance” motivations, but with mastery-avoidance, they manifest as striving to avoid a decline in skills or a failure to learn. It is possible that these unproductive attributes lend themselves to performance on examination formats such as multiple-choice question. This is a novel finding and further research is needed to understand the mechanisms behind the mastery-avoidance construct and academic achievement.

Significant positive relationships were also found between goal orientation and achievement in the short essay examination format, whereby students with a preference for the mastery-approach goal orientation were more likely to achieve high scores. Short essay examinations are mainly written to assess understanding, application, depth of knowledge, reasoning, and problem-solving skills of the examinees.³⁶ Thus, this finding is consistent with theory and confirms our hypothesis that high scores on essay-style examinations are strongly associated with the mastery-approach goal.²⁰ Students who adopt the mastery-approach goal demonstrate positive attributes such as deep learning, confidence, and usually have a low level of test anxiety.⁴

Although multiple-choice question examinations provide relatively fast feedback and freedom from grading bias,³⁷ their disadvantages include giving pointers to the correct answer³⁸ and testing memory rather than understanding.²⁰ We posited that adoption of the performance-approach goal, which is associated with the use of superficial strategies such as memorization,¹⁸ would have a significant relationship with high scores on this type of examination. However, our hypothesis was not supported. In fact, this achievement goal was not an orientation preferred by any of the cohorts in this study and did not emerge as a predictor of academic achievement.

Two challenges, therefore, present themselves to pharmacy educators: maximizing the benefits of multiple-choice question formats without compromising learning fidelity or promoting unproductive approaches to learning and fostering productive and adaptive approaches to learning while rewarding deep understanding with high scores.

Not conducting a parallel qualitative study with our samples was a limitation of the study. However, the study might open a door for qualitative studies that can clarify some of our results. Longitudinal analysis to track changes in achievement goals as students progress through their degree would be beneficial.

Future research in pharmacy education could focus on a deeper exploration of the impact of the mastery-avoidance goal on student learning, academic performance, and teacher practices. Investigating teachers’ goal orientations is also warranted. Interventions testing novel teacher practices that enhance the mastery-approach goal are recommended. Future research might also explore our preliminary findings that students with a preference for the mastery-approach goal are often taught by teachers with the same preference.³²

CONCLUSION

Pharmacy students representing a multi-national, multi-site population show a strong preference for the productive mastery-approach goal orientation over any other goal. The multiple-choice question examination format shows relationships with both avoidance goal orientations, whereas the essay-style format shows relationships with positive and productive approach goal orientations. To our knowledge, this is the first study to differentiate between examination formats and their relationship with achievement goals. This study demonstrates both the inadvisability of using a global measure of student academic performance, as well as the advantages of separating overall scores into individual components in order to assess the motivational mechanisms behind student learning.