Teaching & Learning with AI
Contributors
University of South Florida, Innovative Education
Alternative Title
The Multimodal Video Evaluator (MVE): Systematic Multimedia Design & Evaluation Framework
Contributor
University of Central Florida. Faculty Center for Teaching and Learning; University of Central Florida. Division of Digital Learning; Teaching and Learning with AI Conference (2026 : Kissimmee, Fla.)
Description
Online higher education relies heavily on instructional videos, yet many faculty-produced media assets inadvertently induce high extraneous cognitive load. Instructional designers frequently face an “expert bottleneck,” lacking the operational bandwidth required to conduct manual, frame-by-frame pedagogical audits across expanding institutional media catalogs. To bridge this research-practice gap, this paper presents an operational blueprint for the Multimodal Video Evaluator (MVE), an open-access, AI-driven workflow designed to automate multimedia quality assurance. Built on a native multimodal artificial intelligence architecture, the MVE concurrently processes visual, auditory, and textual streams to evaluate media assets against 127 research-derived design guidelines rooted in the Cognitive Theory of Multimedia Learning (CTML) (Mayer, 2014; Mayer & Fiorella, 2022), human cognition (Xu et al., 2012; Marois & Ivanoff, 2005), video engagement metrics (Guo et al., 2014; Kim et al., 2014; Hansch et al., 2015), and visual scaffolding principles.
Pilot implementations revealed an initial leniency gap in version 1, which scored assets 10% to 15% higher than human experts. Through systematic prompt calibrations, strict semantic restrictions, and the introduction of mutually exclusive grading categories, version 2 eliminated this evaluation bias— improving total human-expert accuracy alignment from 81% to a near-expert 92%. This presentation summarizes how to replicate and configure the MVE for large language models like Google Gemini. The MVE offers a highly scalable, human-in-the-loop framework that compresses a traditional two-hour manual review into a one-minute automated scan, successfully shifting the focus of instructional design teams from mechanical compliance to high-impact pedagogical coaching.
Publication Date
6-11-2026
Location
Kissimmee
Publisher
University of Central Florida Libraries
Keywords:
Multimodal Video Evaluator; MVE; Automated Quality Assurance; Cognitive Theory of Multimedia Learning (CTML); Video Design Guidelines; Instructional Design; AI-Ready Knowledgebase; Knowledgebase Optimization; System Prompt Engineering; Macro-Level Big Data Analytics; Human-in-the-Loop AI Systems Custom AI Bot; Knowledgebase; System Prompt; Multimedia Learning; Vidieo Design; Multimedia Design Evaluation
Subjects
Artificial intelligence--Educational applications; Multimedia learning--Design guidelines; Instructional systems--Design; Educational technology--Evaluation; Big data; Universities and colleges--Quality control
Language
eng
Session Type
Presentation (30 Minutes)
Format
application/pdf
Extent
1 file (PDF, 28 slides)
Creator (Linked Data)
https://orcid.org/0009-0001-4154-2151
Collection
Teaching and Learning with AI Conference Presentations
Series
Teaching and Learning with AI Conference 2026
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Audience
Faculty; Instructional designers; Researchers; Administrators
Recommended Citation
Sato, Koichi, "The Multimodal Video Evaluator (MVE): Codifying Decades of Empirical Research into a Systematic Multimedia Design & Evaluation Framework" (2026). Teaching & Learning with AI. 17.
https://stars.library.ucf.edu/teaching-and-learning-with-ai/17