Skip to main content

Main menu

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers: Moving from Injustice to Equity
  • Reviewers
    • Reviewer Instructions
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Other Publications

User menu

Search

  • Advanced search
American Journal of Pharmaceutical Education
  • Other Publications
American Journal of Pharmaceutical Education

Advanced Search

  • Articles
    • Current
    • Early Release
    • Archive
    • Rufus A. Lyman Award
    • Theme Issues
    • Special Collections
  • Authors
    • Author Instructions
    • Submission Process
    • Submit a Manuscript
    • Call for Papers: Moving from Injustice to Equity
  • Reviewers
    • Reviewer Instructions
    • Reviewer Recognition
    • Frequently Asked Questions (FAQ)
  • About
    • About AJPE
    • Editorial Team
    • Editorial Board
    • History
  • More
    • Meet the Editors
    • Webinars
    • Contact AJPE
  • Follow AJPE on Twitter
  • LinkedIn
Research ArticleRESEARCH BRIEF

Design and Evaluation of Video Podcasts for Providing Online Feedback on Formative Pharmaceutical Calculations Assessments

Michelle Flood, John C. Hayden, Bernadette Bourke, Paul J. Gallagher and Sam Maher
American Journal of Pharmaceutical Education December 2017, 81 (10) 6400; DOI: https://doi.org/10.5688/ajpe6400
Michelle Flood
School of Pharmacy, Royal College of Surgeons in Ireland, Dublin, Ireland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John C. Hayden
School of Pharmacy, Royal College of Surgeons in Ireland, Dublin, Ireland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bernadette Bourke
School of Pharmacy, Royal College of Surgeons in Ireland, Dublin, Ireland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paul J. Gallagher
School of Pharmacy, Royal College of Surgeons in Ireland, Dublin, Ireland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sam Maher
School of Pharmacy, Royal College of Surgeons in Ireland, Dublin, Ireland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Objective. To evaluate worked example video podcasts as a method of providing feedback to pharmacy interns for an online and formative pharmaceutical calculations assessment.

Methods. A theory-informed approach based on multimedia learning theory was used to design video podcasts as feedback on a calculations examination. A mixed-methods evaluation completed by pharmacy interns enrolled in Ireland’s National Pharmacy Internship Programme was used to establish cognitive and affective attitudes toward video podcasts compared with conventional written solutions.

Results. The majority of students found video podcasts were clear, helpful for learning, easy to understand, and useful as a method of feedback. Majority reported that they felt positively about standard written solutions. The evaluation suggested distinct benefits for each kind of feedback, something that has not been previously reported. Thematic analysis of qualitative data indicated useful features of video podcasts, including clear explanation, step-by-step approach, and synchronization of audio and visual information.

Conclusion. Respondents reported positive cognitive and affective attitudes toward video podcasts as online feedback. Video podcasts are a helpful and novel way of providing feedback on pharmaceutical calculations. A similar opinion of traditional written solutions suggests that students may benefit from both forms of feedback. Further study is required to identify the particular benefits associated with both kinds.

Keywords
  • video podcasts
  • pharmaceutical calculations
  • feedback
  • formative assessment

INTRODUCTION

Proficiency in completing pharmaceutical calculations is a core competency for health professionals as errors can compromise patient safety.1,2 This brief evaluated the potential of video podcasts as a novel method of providing feedback on a formative assessment of pharmaceutical calculations to final year pharmacy students enrolled in the National Pharmacy Internship Programme (NPIP) in the Republic of Ireland – a workplace-based program with academic content primarily delivered and formatively assessed online. Delivery of educational material online requires reconsideration of traditional methods of assessment and feedback, and video podcasts are an option relevant to calculations.3 Video podcasts based on the principles of Mayer’s cognitive theory of multimedia learning4 were designed as a novel method to provide feedback on a formative calculation assessment. We evaluated this approach and compared it to standard written solutions.

Podcasts are media (audio) files that are distributed via the Internet for use on personal computers and portable media devices. The term originally referred to audio files played on the iPod (Apple Inc., Cupertino, CA) portable media device.5 Further development has led to the use of video podcasts (sometimes referred to as vodcasts), which combine audio and visual outputs. These have been used extensively in educational settings and have shown to have a positive impact on the student learning experience, and the application of video podcasts as a method of providing feedback has been considered an area of potential future benefit to learners.6-8 In several studies, “worked example” video podcasts are described as a beneficial way of presenting complex material.9,10 “Worked example” video podcasts are multimedia clips used to explain problems, articulate reasoning and assist students in learning how to solve specific problems. The benefit of worked examples has been reported in disciplines such as mathematics, science and engineering, although none have been reported specifically relating to pharmaceutical calculations.11,12 This study provides an overview of the steps taken in design and evaluation of the feedback.

METHODS

NPIP students (n=162) were provided with a formative 20-item multiple-choice question (MCQ) assessment delivered via the virtual learning environment (VLE) in 2012/2013. Two kinds of feedback were developed and made available to all students. MCQs were separated into two groups of 10 questions, one group for video podcast feedback and the other for written solutions. Each group had questions of comparable difficulty level as determined using the item analysis and a difficulty ranking score ranging between 1 (0% to 20% achieving the correct answer) and 5 (81% to100% achieving the correct answer).

For the podcast group, 10 individual video podcasts were developed using Jing software (TechSmith, Okemos, MI), which enables simultaneous screencast and audio recordings of up to 5 minutes. A Bamboo digital stylus pen and tablet (Wacom, Kazo, Saitama Prefecture, Japan) were used to digitize and record handwritten solutions to each calculation – facilitating optimal synchronization between audio and visual materials in line with multimedia learning principles.4 Each video podcast took 30-45 minutes to develop and ranged in length from 2 to 5 minutes with a mean of 3 minutes. The written solution feedback was prepared using PowerPoint (Microsoft, Redmond, WA), here each step in the calculation was typed and there was no audio explanation. Both kinds of feedback were made available at the same time.

Evaluation was performed for both methods using a repeated measures design. Here, all students completed the evaluation of both types of feedback rather than allocating the students to groups receiving one feedback type only. A mixed methods survey was developed to evaluate the video podcasts. The combination of Likert rating scale questions and open-ended questions was used to strengthen the evaluation, and facilitate triangulation and comparison with other studies.8 This is especially important given the difficulty in achieving desirable response rates for surveys.13 The Likert rating scale questions are shown in Table 1. The open-ended questions asked respondents to describe which features of the video podcast feedback they found useful and provide suggestions for improvement. The survey was prepared and reviewed for face validity by the authors and one NPIP student. As per institutional requirements, all questions were optional and no incentive was offered. Ethical approval was granted by Lancaster University and permission to conduct the evaluation was obtained from the program director of the NPIP. All registered students were invited to take part in the evaluation and given 10 days to complete the survey, with a reminder after 8 days. Survey responses were collated and tabulated, and qualitative content analysis was used to analyze the open-ended responses.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1.

Respondent Feedback Toward Video Podcast and Standard Written Solution Formats

RESULTS

Fifty-three students (33% of those invited) responded to part or all of the survey. Forty-four students gave details of their demographics; 73% (n=32) were female and 27% (n=12) were male. Respondents’ age was also surveyed; 66% (n=29) were 20-25 years old, 20% (n=9) were 25-30 years old, and 14% (n=6) were over 30 years old. In this evaluation, positive cognitive attitudes were reported toward video podcasts and written solutions. The survey questions and responses are detailed in Table 1. The majority of participants agreed or strongly agreed that video podcasts were easy to understand (91%; n=31), clearly explained (91%; n=31), and a useful form of feedback (88%; n=30), and participants had a comparable view of written answers (Table 1). The themes and frequencies arising from the qualitative content analysis, and illustrative quotes are presented in Table 2. Qualitative analysis of the open-ended responses triangulated these positive findings, with respondents indicating that they felt positively about particular features of video podcasts, including clear explanations, stepwise approach to explanation and the audio/visual features (Table 2). Respondents also suggested areas for improvement such as expanding their application and technical aspects.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 2.

Video Podcasts: Useful Features and Areas for Improvement Reported by Respondents

DISCUSSION

As would have been predicted based on similar studies in other fields, respondents reported that they liked the feedback video podcasts and found them useful.8 The benefits reported in this study were consistent with those reported previously, namely: a clear explanation and a step-by-step approach,9 and the synchronization of delivery of audio and visual information.11 Additionally, respondents requested more examples using video podcasts and identified technical aspects (eg, to improve sound quality) that may improve the audio and/or visual aspects of the podcasts, again a finding common to other research.8

Video podcasts take longer to produce than typed solutions, so we wanted to understand if there were distinct benefits that make video podcasts a worthwhile endeavor for faculty members. To this end, we simultaneously evaluated traditional feedback. The addition of this comparison group showed that a comparable number of respondents felt similarly about written solutions, which was a relatively unexpected finding. The positive review of both methods by the same cohort indicates that there may be particular advantages to both, and this is supported by free-text comments that clearly indicate that there are features particular to video podcasts that were considered useful, that are not afforded by written solutions. While there are several possible reasons, one possibility is the relatively high proficiency of final year students who can likely determine their errors from a written solution. It has been previously suggested that those with relatively low prior knowledge derive most benefit from material presented in this manner.4 We aim to explore this empirically through a future study using video podcasts for feedback on pharmaceutical calculations in an undergraduate student cohort.

Our findings provide further support for use of video podcasts as a method of feedback provision based on self-reported cognitive and affective attitudes, and we report qualitative data that will help guide future instructional design relating to online feedback in our institution. Although a low response rate, and the self-report nature of the study are limitations of this evaluation, the preliminary findings are strengthened by triangulation with the qualitative data. Aside from determining face validity for the survey, no reliability or other validity testing was performed, and we acknowledge this as a potential limitation of the study. We acknowledge that positive findings require further study to comprehensively understand the relative benefit of each feedback type and have taken care not to overstate our findings.14

CONCLUSION

Video podcasts are a beneficial form of online feedback on pharmaceutical calculations. The data suggest that feedback in video podcast form may be a useful addition to traditional feedback. Data collected may be of interest for those involved in instructional design for online learners or those teaching pharmaceutical calculations. Our illustration of the use of principles of learning sciences research on video podcasts, and cognitive load theory in the design of online learning material for health professions education is timely, and currently highlighted as a topic of importance in the wider health professions education literature.15

ACKNOWLEDGMENTS

The authors would like to thank Prof. Don Passey from Lancaster University (UK) for his advice on the design of the evaluation and review of the manuscript.

  • Received March 14, 2017.
  • Accepted June 19, 2017.
  • © 2017 American Association of Colleges of Pharmacy

REFERENCES

  1. 1.↵
    1. Oldridge GJ,
    2. Gray KM,
    3. McDermott LM,
    4. Kirkpatrick CM
    . Pilot study to determine the ability of health-care professionals to undertake drug dose calculations. Intern Med J. 2004;34(6):316-319.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Rolfe S,
    2. Harper NJ
    . Ability of hospital doctors to calculate drug doses. BMJ. 1995;310(6988):1173-1174.
    OpenUrlFREE Full Text
  3. 3.↵
    1. Gikandi JW,
    2. Morrow D,
    3. Davis NE
    . Online formative assessment in higher education: a review of the literature. Comput Educ. 2011;57(4):2333-2351.
    OpenUrlCrossRef
  4. 4.↵
    1. Fiorella L,
    2. Mayer RE
    . Effects of observing the instructor draw diagrams on learning from multimedia messages. J Educ Psychol. 2016;108(4):528-546.
    OpenUrl
  5. 5.↵
    1. Boulos MN,
    2. Maramba I,
    3. Wheeler S
    . Wikis, blogs and podcasts: a new generation of Web-based tools for virtual collaborative clinical practice and education. BMC Med Educ. 2006;6(1):41.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Fernandez V,
    2. Simo P,
    3. Sallan JM
    . Podcasting: a new technological tool to facilitate good practice in higher education. Comput Educ. 2009;53(2):385-392.
    OpenUrl
  7. 7.↵
    1. Green SM,
    2. Voegeli D,
    3. Harrison M,
    4. et al
    . Evaluating the use of streaming video to support student learning in a first-year life sciences course for student nurses. Nurse Educ Today. 2003;23(4):255-261.
    OpenUrlPubMed
  8. 8.↵
    1. Kay RH
    . Exploring the use of video podcasts in education: a comprehensive review of the literature. Comp Hum Behav. 2012;28(3):820-831.
    OpenUrlCrossRef
  9. 9.↵
    1. Kay R,
    2. Kletskin I
    . Evaluating the use of problem-based video podcasts to teach mathematics in higher education. Comput Educ. 2012;59(2):619-627.
    OpenUrlCrossRef
  10. 10.↵
    1. Crippen KJ,
    2. Earl BL
    . Considering the efficacy of web-based worked examples in introductory chemistry. J Comput Math Sci Teach. 2004;23(2):151-168.
    OpenUrl
  11. 11.↵
    1. Griffin DK,
    2. Mitchell D,
    3. Thompson SJ
    . Podcasting by synchronising PowerPoint and voice: what are the pedagogical benefits? Comput Educ. 2009;53(2):532-539.
    OpenUrl
  12. 12.↵
    1. Sweller J
    . Cognitive load during problem solving: effects on learning. Cog Sci. 1988;12(2):257-285.
    OpenUrl
  13. 13.↵
    1. Fincham JE
    . Response rates and responsiveness for surveys, standards, and the Journal. Am J Pharm Educ. 2008;72(2):Article 43.
    OpenUrl
  14. 14.↵
    1. Leppink J,
    2. Winston K,
    3. O’Sullivan P
    . Statistical points and pitfalls-series-introduction. Perspect Med Educ. 2016;5(1):42-44.
    OpenUrl
  15. 15.↵
    1. Gooding H,
    2. Mann K,
    3. Armstrong E
    . Twelve tips for applying the science of learning to health professions education. Med Teach. 2017;39(1):26-31.
    OpenUrl
PreviousNext
Back to top

In this issue

American Journal of Pharmaceutical Education
Vol. 81, Issue 10
1 Dec 2017
  • Table of Contents
  • Index by author
Print
Download PDF
Email Article

Thank you for your interest in spreading the word on American Journal of Pharmaceutical Education.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Design and Evaluation of Video Podcasts for Providing Online Feedback on Formative Pharmaceutical Calculations Assessments
(Your Name) has sent you a message from American Journal of Pharmaceutical Education
(Your Name) thought you would like to see the American Journal of Pharmaceutical Education web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
1 + 1 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Design and Evaluation of Video Podcasts for Providing Online Feedback on Formative Pharmaceutical Calculations Assessments
Michelle Flood, John C. Hayden, Bernadette Bourke, Paul J. Gallagher, Sam Maher
American Journal of Pharmaceutical Education Dec 2017, 81 (10) 6400; DOI: 10.5688/ajpe6400

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Design and Evaluation of Video Podcasts for Providing Online Feedback on Formative Pharmaceutical Calculations Assessments
Michelle Flood, John C. Hayden, Bernadette Bourke, Paul J. Gallagher, Sam Maher
American Journal of Pharmaceutical Education Dec 2017, 81 (10) 6400; DOI: 10.5688/ajpe6400
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • ACKNOWLEDGMENTS
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

Similar AJPE Articles

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Relationships Between Myers-Briggs Type Indicators and NAPLEX Performances
  • Spirituality and Religiosity of Pharmacy Students
  • Lessons Learned from Evaluating a Process for Systematic Curriculum Review
Show more RESEARCH BRIEF

Related Articles

  • No related articles found.
  • Google Scholar

Keywords

  • video podcasts
  • pharmaceutical calculations
  • feedback
  • formative assessment

Home

  • AACP
  • AJPE

Articles

  • Current Issue
  • Early Release
  • Archive

Instructions

  • Author Instructions
  • Submission Process
  • Submit a Manuscript
  • Reviewer Instructions

About

  • AJPE
  • Editorial Team
  • Editorial Board
  • History
  • Contact

© 2021 American Journal of Pharmaceutical Education

Powered by HighWire