A Mobile Learning Module to Support Interprofessional Knowledge Construction in the Health Professions

INTRODUCTION

Studies of physician-pharmacist collaborative care models have shown improvements in medication management for medically complex patients, such as those with hypertension, pediatric asthma, and diabetes.^1-3 Preparing health professions students to provide such care requires training opportunities within small collaborative teams.^4,5 Yet, implementing these training opportunities is challenging because of logistical problems associated with scheduling students across health professions educational programs.^6-8 By allowing asynchronous interactions between students, use of mobile learning modules might overcome many of these logistical barriers and, with appropriate levels of scaffolding, effectively support collaborative learning.

While the concept of mobile learning is still emerging and the debate regarding its definition continues, Crompton proposed a broad definition of mobile learning as “learning across multiple contexts, through social and content interactions, using personal devices.”^9,10 The process of knowledge construction, derived from social constructivist theory, is a goal of collaborative learning activities and practice. Knowledge construction occurs when two or more learners work together to actively build new knowledge or meaning.^11-13

According to Gunawardena’s knowledge construction framework, social construction of knowledge progresses across five phases that reflect increasingly collaborative interaction among participants.¹¹ Mental engagement increases with each successive phase, from phase I which is “sharing/comparing,” to phase V, which is reaching “agreement/applying newly constructed meaning” (Table 1). Higher phases of knowledge construction behaviors, associated with moderate to high levels of mental engagement, may result in “substantial restructuring of knowledge” and deeper, higher quality learning.¹⁴

The organization and structure of learning environments impacts knowledge construction behaviors in online collaborative groups.^12,15 Learner collaboration that results in deep learning, especially at the intersection of interprofessional education and the virtual learning environment, requires that educators provide some degree of scaffolding to guide the interaction between learners and support knowledge construction.¹⁶ Providing structured guidance to prompt communication between learners, also known as dialogue scaffolding, may support collaborative knowledge construction.^14,16,17 However, little is known about how to design online curricula to include dialogue scaffolding that promotes collaborative construction of knowledge among health professions students.

We report on the development, pilot implementation, and evaluation of an asynchronous, medication-focused, mobile learning module containing structured dialogue prompts aiming to support knowledge construction behaviors among medical and pharmacy students. Our evaluation focuses on answering a specific question: How effective are structured dialogue prompts (based on Gunawardena’s knowledge construction framework) at stimulating higher-phase knowledge construction behaviors and positively impacting learning outcomes?

METHODS

To address the logistical challenges that often hamper the implementation of interprofessional training opportunities, we created a learning module that was delivered via the PIVOT med (Practice Improvement using Virtual Online Training) mobile app (HoloDox, LLC, Palo Alto, CA) for asynchronous, online collaboration between medical and pharmacy students.^6-8 Through the mobile app, students can access clinical case information at their convenience, exchange messages with their interprofessional partner, and build knowledge together.

This pilot interventional study was conducted from January through July 2018 and consisted of three parts: developing the module (January-March), including the case and dialogue prompts; implementing the module (March); and conducting a limited efficacy evaluation of the module. This evaluation focused on two outcomes: impact of the dialogue prompts on knowledge construction behaviors and the quality of the collaborative care plan as measured by scores for accuracy and completeness; and student feedback on the prompts (March-July). The University of California, San Francisco Committee on Human Research approved this study.

The PIVOT study design, including the intervention, process, knowledge outcomes, and evaluation is presented in Figure 1. The PIVOT med mobile app is an educational tool developed to support health professions student engagement (Figure 2) through simulation of real-world patient scenarios. The first learning module delivered via PIVOT med was piloted at UCSF in 2016 to support second-year medical students’ development of diagnostic reasoning skills through analysis of a complex case of systemic lupus erythematosus.¹⁸ Based on its success and a desire to increase interprofessional learning experiences for undergraduate students, we revised the case and the learning module structure to support knowledge construction among pharmacy and medical students.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Pilot study framework for PIVOT, a mobile learning module that provides guidance to health professions students that is intended to encourage interprofessional collaboration. PIVOT stands for “Practice Improvement using Virtual Online Training.”

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

The PIVOT app interface home screen

The PIVOT med app allows students to work together asynchronously, communicating only through the app, to manage a complex and evolving clinical case. Students received clinical case data, including the patient’s medical history, physical examination findings, laboratory results, and multimedia content including a video interview with the patient and radiographic images. Student pairs communicated through instant messaging (referred to as “team chats”) (Figure 3), and the app captured user entered data (ie, answers to questions/assignments, including care plans) as well as text exchanges (ie, the team chat dialogue), and supported distribution of survey questions.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

The PIVOT app interface “Team Chat” instant messaging

To enhance collaborative learning, authenticity and clinical relevance for the students, two physicians and two pharmacists revised the case used in the original pilot to focus on increasingly complex medication management issues that would benefit from a collaborative approach between the referring physician and a pharmacist consultant.

The module was designed for medical and pharmacy student pairs to work together on a virtual, interactive case of a 44-year-old woman presenting with fatigue, joint pain, and low-grade fever. In the revised case, the patient’s clinical course evolved over two weeks, with new case details revealed in three separate phases: the patient is diagnosed with systemic lupus erythematosus (phase 1), starts treatment (phase 2), and develops treatment complications (phase 3). Students used the PIVOT med app to review clinical case data, communicate with their partner via team chat and work together to formulate three separate collaborative care plans, one for each phase. The learning objectives and structure of the module were designed to support students as they constructed knowledge in the following domains: clinical knowledge of systemic lupus erythematosus and development of optimal therapeutic strategies.

To support the process of student interactions and knowledge construction in our mobile-learning module and to explore the impact of dialogue scaffolding both on students’ knowledge construction behaviors and their performance on the collaborative care plans, we used Gunawardena’s knowledge construction framework to design “high guidance” and “low guidance” dialogue prompts.¹¹ Matching sets of structured dialogue prompts, one high guidance and one low guidance, were created for each phase of the clinical case with the aim of facilitating creation and subsequent application of new knowledge between medical and pharmacy students (Appendix 1). We expected student pairs who received high guidance prompts to engage in higher phases of knowledge construction behaviors than pairs receiving low guidance prompts. Each set of dialogue prompts and associated collaborative care plans focused on medications and therapeutic strategies, as this is where, in practice, we would expect the most prevalent interactions between pharmacists and physicians.

To improve construct validity, we piloted the dialogue prompts with two medical and two pharmacy students not participating in the study and then revised these to maximize clarity and minimize wordiness. As shown in Appendix 1, the high guidance and low guidance prompts contained overlapping language and instructed students to “collaborate” with their colleagues to address the clinical questions that were posed. The high guidance prompts were designed to encourage collaboration on clinical questions via higher levels of interaction and knowledge construction behaviors. Students under the high guidance condition were generally instructed to: share individual professional opinions based on specific questions posed for each phase of the case (ie, achieve phase I knowledge construction); explore differences of professional opinion (ie, achieve phase II knowledge construction); articulate the clinical rationale for proposed recommendations (ie, achieve phase III knowledge construction); discuss how past experiences informed their thinking (ie, achieve phase IV knowledge construction); and, formulate a mutually agreeable plan (ie, achieve phase V knowledge construction).

Implementation included student recruitment and orientation to the module, preparation for launch of the module, and technological support throughout the two-week pilot. The 16 third-year medical students were completing their clinical clerkship and the module was a component of the curriculum. They were free to opt out of the study, but all consented to participate. The pharmacy students were in their third year (n=8) or fourth year (n=8) of the curriculum. The students were randomly assigned to medicine-pharmacy pairs, and pairs were randomly assigned to either the low guidance or high guidance condition, with third- and fourth-year pharmacy students evenly distributed between low guidance and high guidance groups. Orientation sessions were conducted to introduce the learning module, learning objectives, and expectations regarding participation in the study. Students were not informed about the study question regarding high guidance and low guidance prompts and their impact on knowledge construction until after the study.

In preparation for launch of the module, we worked extensively with the software development team to create a detailed calendar of events that specified the date and time when the app would release information and/or send notifications (including high guidance and low guidance prompts) to student pairs. Faculty members and the software development team supported students during the pilot phase, mainly addressing technological issues (eg, trouble logging into the system, difficulty uploading collaborative care plans).

The evaluation of the PIVOT med module included analyses of text-based dialogue coded for knowledge construction phases, the accuracy and completeness of each pairs’ collaborative care plans, and quantitative and qualitative participant feedback.

For the pilot study, we collected data to evaluate the impact of the level of guidance provided (ie, either high guidance or low guidance dialogue prompts) on both the frequency and phase of knowledge construction behaviors manifested in team chat dialogue (ie, text-based dialogue) within student pairs. We applied directed content analysis to all team chat data throughout the two-week curriculum.¹⁹ Using knowledge construction phases as codes, two investigators (LCF and JM) evaluated each sentence in the team chat dialogue and independently applied knowledge construction codes to each sentence. We segmented team chat dialogue to correspond with discussion of each plan (ie, collaborative care plans 1, 2, and 3) and tabulated knowledge construction behaviors for each collaborative care plan. Both investigators reviewed independently coded data and resolved discrepancies through discussion. We calculated descriptive statistics to compare the effect of the level of guidance on the frequencies of specific knowledge construction behaviors observed during the team chat dialogue. We computed the 95% CI values for the difference in the mean frequencies of knowledge construction behaviors between the two groups (ie, high guidance vs low guidance). Cohen’s d effect size values were calculated as the difference in the two groups' means divided by the average of their standard deviations to demonstrate the magnitude of the difference in the frequencies of knowledge construction behaviors between the two groups (ie, high guidance vs low guidance).

The collaborative care plans served as a proxy for clinical knowledge attained, and student pairs were free to determine the structure of their submitted plans. The clinician-investigators developed rubrics to assess the accuracy and completeness of collaborative care plans 1, 2, and 3 (a rubric for each distinct phase of the case) using a three-point scale (1=below expectations, 2=meets expectations, and 3=exceeds expectations).²⁰ All of the collaborative care plans (n=48) were stripped of information regarding the associated guidance level, and then randomly assigned to two of three clinician-investigators on our research team for scoring. Each collaborative care plan was scored independently according to the rubrics. Discrepancies were discussed among the raters who then came to agreement on the students’ final score based on the rubric. If not, a third rater with a different clinical background was consulted. We calculated descriptive statistics to compare the quality of collaborative care plans (ie, accuracy and completeness) in high guidance and low guidance pairs. We computed the 95% CI values for the difference in the mean collaborative care plan scores for accuracy and completeness between the two groups (ie, high guidance vs low guidance). Cohen’s d effect size values were calculated (ie, the difference in the two groups' means divided by the average of their standard deviations) to demonstrate the magnitude of the difference in the collaborative care plans scores between the two groups (ie, high guidance vs low guidance).

The medical and pharmacy student participants were encouraged to attend one of two post-study debrief sessions. In these sessions, one of the faculty members (LCF) described the different types of prompts students received and asked for feedback on their experience with either low guidance or high guidance prompts. Additionally, all participants were encouraged to complete an online survey that assessed their perceptions of the intervention and solicited their ideas for improvement. Using an open coding approach, we analyzed student feedback for satisfaction with the experience and perceptions of the guidance prompts.²¹

RESULTS

All 16 interprofessional pairs completed the curriculum and submitted three collaborative care plans. Overall, medical and pharmacy student pairs that received high guidance prompts interacted with greater frequency (ie, higher mean frequencies of knowledge construction behaviors) at all knowledge construction phases than pairs that received low guidance prompts (Table 2).

Though the extent of interactivity was greater in all groups that received a higher level of guidance, comparing the frequencies of interactive behaviors underscores the similarity in behavior patterns in both high guidance and low guidance groups. Sharing and comparing (phase I) accounted for 44% and 47% of all interactions in the high guidance and low guidance pairs, respectively. Most of these behaviors involved students providing clinical recommendations to or soliciting input from their partner. Across pairs, exploring dissonance (phase II interactions) occurred infrequently (in 7% of interactions occurring between pairs who received a high level of guidance and in 5% between pairs who received a low level of guidance). Co-construction (phase III) accounted for 26% and 31% of the interactions between students in the student pairs receiving high guidance and low guidance, respectively. Phase III behaviors were usually associated with sequential additions by partners that resulted in plan augmentation. We rarely observed pairs in either the high guidance or the low guidance groups testing their co-constructed knowledge (phase IV interactions; 4% and 2%, respectively). Reaching agreement and application (phase V interaction) occurred in 18% of the interactions among pairs that received high guidance and 15% of the pairs that received low guidance and included instances where consensus was reached on co-constructed knowledge, or when pairs explicitly agreed upon patient assessments. Significance was not tested due to small sample size and expectation of insufficient power (Table 2). The effect sizes of the differences between the high guidance and low guidance pairs (where effect size cutoffs of d=0.2, 0.5 and 0.8 are considered to be small, medium, and large, respectively) were medium to large for knowledge construction phases I, IV, V (Table 2).²² See Table 1 for knowledge construction definitions and for specific examples of each knowledge construction phase taken from the team chat dialogue.

Guidance level (ie, high or low) did not impact any of the care plan scores in terms of their accuracy and completeness (Table 3). Across all groups, there was much heterogeneity in collaborative care plan scores (as reflected by the wide standard deviations as shown in Table 3). Given that the 95% CI values for collaborative care plan scores all contained zero, we did not have sufficient evidence to conclude that there was a difference in the collaborative care plan means on the basis of either accuracy or completeness. From our data, the effect sizes of the differences in collaborative care plan accuracy and completeness between high guidance and low guidance pairs were low, with the exception of collaborative care plan 3 accuracy scores where there was a medium effect size.²²

All participants completed the survey (n=32). Because of scheduling differences, all of the medical students (n=16) but only three pharmacy students attended debrief session 1, while only pharmacy students (n=11) attended debrief session 2.

Survey data indicated that 98% of students agreed or strongly agreed that the module was an effective way to improve their knowledge about systemic lupus erythematosus, practice clinical reasoning, and improve their ability to communicate with colleagues from medicine or pharmacy. In the debrief sessions, technological issues with the PIVOT med app, including intermittency of the notifications function and difficulty with the text-editing interface, were mentioned as challenges.

While student feedback from the survey and debriefs helped us to better understand their learning experience during the pilot, their responses regarding the impact of both the high guidance and low guidance prompts on their own learning was mixed, and students expressed a variety of views, both positive and negative. Reflecting positively on the experience, one medical student felt the low guidance prompts allowed for a more realistic exchange of ideas and information, noting, “…I think it really allowed for me to think about, ‘Okay what's actually important about this patient,’ and practice communicating in that way…I think the openness (ie, minimal structure) allowed for a lot of discussion.”

Responses to the high guidance prompts were mixed. One pharmacy student appreciated that the high guidance prompt directed her and her partner to discuss differences of opinion (ie, to support phase II knowledge construction). However, one medical student suggested that this was unlikely to be effective because, as students, “…we tend to be polite and nice…especially when it's not a real patient's wellbeing.” Two medical students and one pharmacy student stated that the prompts were initially helpful but eventually became “redundant.” Another medical student explained that, at “…the beginning I thought it was more thorough going through each step, but then it started to seem a bit repetitive…” One pharmacy student stated that she felt that the prompts helped structure her own thinking and responses, but logistical constraints made answering each question “too challenging.” Another pharmacy student said that she “didn't really use” the high guidance prompts at all.

Two pharmacy students from pairs that received low guidance stated that they would have come to a consensus with their medical colleagues independently of any prompting because they are trained “…in a holistic way… we are (the patient’s) healthcare providers, so we should be on a common ground when treating a patient….there's just a natural tendency to (ask one another), ‘What do we think? What is the best plan?’"

DISCUSSION

Informed by the knowledge construction framework, we designed and piloted a case-based, mobile learning module to enhance interaction between medical and pharmacy students and, ultimately, support higher-phase knowledge construction.¹¹ This module, delivered via the PIVOT med app, allowed medical and pharmacy students to learn about the clinical manifestations and treatment of systemic lupus erythematosus and to practice interprofessional collaboration, communication, and clinical reasoning. The results of the knowledge construction coding of the team chats suggest that the high-guidance prompts provided dialogue scaffolds that enhanced student interaction and supported higher-phase knowledge construction behaviors more than the low-guidance prompts did. However, the feedback we received from the students suggested that the influence of the prompts is less clear. While both the high-guidance and low-guidance prompts helped focus some students’ thinking, several students expressed that repeated exposure to the detailed, high-guidance prompts during the module was unnecessary.

The pattern of student pairs’ knowledge construction behaviors generally corresponds with the findings from studies conducted in online learning environments, but with one key exception. While the majority of interactions between our student pairs were at phase I, as observed in other investigations, we found that both our high-guidance and low-guidance prompts were associated with a relatively high frequency of phase V interactions, accounting for approximately 20% of all interactions.^11,23,24 In other studies, phase V interactions only accounted for 0.4%, 1.9%, and 3% of total interactions.^11,24-26 Our study included complex clinical problems with no defined solution as the primary driver of learner interactivity in the PIVOT med module.²⁶ For this type of ill-structured problem-solving, the high-guidance and low-guidance prompts may have effectively supported phase V interactions to a greater extent than found in previously published studies and encouraged students to negotiate ideas, co-construct knowledge, and come to agreement to create each collaborative care plan.^26,27 Alternately, the activity itself may have required more phase V behaviors than activities in other studies, or phase V interaction behavior may be enmeshed in health professional culture, where members are normed to seek agreement on the care plan in a timely manner by virtue of their training. If so, this raises a key question: would further interventions to support higher-phase knowledge construction translate into learning benefits?

We also considered whether “over scripting” the guidance prompts might dampen the impact of the high-guidance condition.^16,28 In other words, a prompt that is too detailed or too prescriptive, may actually limit the level of knowledge construction by constraining students’ creativity and open discussion. Based on the feedback from the debrief sessions, some students felt the high-guidance prompts were excessive. If the prompts had been less scripted, we might have seen greater differences in knowledge construction between students who received the high-guidance prompts vs. those who received the low-guidance prompts. From van Aalst’s work, we know that enhanced learning outcomes have been observed in the settings of collaborative knowledge construction where student interactions have been scaffolded or guided.¹⁴ However, when more guidance than is needed is provided to students, students may disengage in ways that have negative effects. As we heard in the debrief sessions, some students felt that the excessive detail and repetition were unnecessary. Another concern is that some students might become dependent on the high level of guidance. In either case, too much guidance can be detrimental to students’ motivation and construction of new knowledge.^29-31 Further investigation into the role of structured guidance is warranted.

Our findings showed no difference in students’ knowledge outcomes based on the quality of the collaborative care plans. While this result may indicate that our intervention (ie, structured dialogue prompts) did not produce substantial enough difference in knowledge construction behaviors to yield different outcomes, it may also reflect the unintended consequences of choosing to provide students with only minimal guidance about the content and structure of the collaborative care plan. We decided that an overly prescriptive collaborative care plan structure could constrain the students’ knowledge construction and limit the variability in responses, so we did not require a specific structure/organization for the collaborative care plans. This decision resulted in considerable heterogeneity in the structure and level of detail within the plans (eg, some pairs had richly detailed discussions about their care plans during their team chat, but submitted only a few brief bullet points for their final collaborative care plan). In retrospect, if we had required students to use a general structure for the collaborative care plans (ie, asking for student pairs to include at least their patient assessment and plan in each care plan), then we would have been able to more evenly apply the rubrics in assessing the accuracy and completeness of the care plans. This would have allowed an “apples to apples” comparison without eliminating performance variation. These findings raise questions similar to those discussed in relation to the optimal amount of scripting in dialogue prompts. That is, the level of scripting must be considered not only in the design of the dialogue prompts, but also in the design of instructions provided to students regarding their work products.

To scaffold higher-phase knowledge construction and promote deeper learning in the setting of collaborative learning environments, some form of student guidance is necessary.³² Educators can reduce the risk of over-scripting by periodically checking on the dialogue and actively promoting productive dialogue through guidance tailored to the needs of the students.^{16,26,27,32,33} Given that individualization of guidance is of paramount importance to foster deep, high-quality learning, we expect that scaffolded dialogue, combined with customized facilitation by an instructor, will effectively support achievement of higher-phase knowledge construction.^30,31

There are many potential uses for PIVOT med and similar mobile learning modules to support knowledge construction for other disease states, patient populations, and interprofessional contexts. Future work to make logistical and technical refinements to the platform and to calibrate dialogue prompts with desired phases of knowledge construction behavior and outcomes will be important.

Limitations of this study include technological glitches with the mobile app that negatively impacted the interactions of some student pairs. The limited availability of students from both the medical and pharmacy programs resulted in the study being underpowered and prevented us from performing a full statistical evaluation of the results. Also, our inclusion of more senior learners rather than first- and second-year students, may have blunted the impact of the high guidance prompts on knowledge construction behaviors. Additionally, given that the debrief sessions were essentially uni-professional (ie, mostly medical students in the first session and only pharmacy students in the second), we were unable to gather feedback and insights from the interprofessional student pairs themselves.

In this pilot, we focused on the effectiveness of structured dialogue prompts to support high-level knowledge construction behaviors and their impact on primary learning outcomes (in our case, the accuracy and completeness of the collaborative care plans). We believe that the differences observed between the high guidance and low guidance pairs show potentially promising trends that warrant further investigation with a larger group of students. Though we chose not to apply an objective assessment of students’ knowledge of systemic lupus erythematosus, pre- and post-intervention in the current investigation, future studies of similar interventions should consider including a more robust evaluation of learning effects.

CONCLUSION

This study represents the successful development and implementation of a case-based, mobile-learning module using the PIVOT med app and evaluation of structured dialogue prompts to support knowledge construction in student pairs. While technological issues with the app hampered interactions for some student pairs, the module supported interprofessional collaboration as students learned about the clinical manifestations and treatment of systemic lupus erythematosus and allowed us to explore knowledge construction behaviors of pharmacy and medical student pairs as they worked together to solve complex, medication-focused problems in the virtual training environment. Our pilot shows that health professions educators may use the knowledge construction framework to both design and evaluate educational interventions for asynchronous learning. It also highlights opportunities for further investigation into the ways that we design interprofessional interventions and implement tailored guidance to most effectively support high-level knowledge construction in our trainees as they learn to work together to deliver high quality, collaborative clinical care.