Learning Perceptual Embeddings With Two Related Tasks For Joint Predictions Of Media Interestingness And Emotions

Keywords

Multi-task feature extraction; Multimedia emotional impact prediction; Multimedia interestingness prediction; Perceptual embedding

Abstract

Integrating media elements of various medium, multimedia is capable of expressing complex information in a neat and compact way. Early studies have linked different sensory presentation in multimedia with the perception of human-like concepts. Yet, the richness of information in multimedia makes understanding and predicting user perceptions in multimedia content a challenging task both to the machine and the human mind. This paper presents a novel multi-task feature extraction method for accurate prediction of user perceptions in multimedia content. Differentiating from the conventional feature extraction algorithms which focus on perfecting a single task, the proposed model recognizes the commonality between different perceptions (e.g., interestingness and emotional impact), and attempts to jointly optimize the performance of all the tasks through uncovered commonality features. Using both a media interestingness dataset and a media emotion dataset for user perception prediction tasks, the proposed model attempts to simultaneously characterize the individualities of each task and capture the commonalities shared by both tasks, and achieves better accuracy in predictions than other competing algorithms on real-world datasets of two related tasks: MediaEval 2017 Predicting Media Interestingness Task and MediaEval 2017 Emotional Impact of Movies Task.

Publication Date

6-5-2018

Publication Title

ICMR 2018 - Proceedings of the 2018 ACM International Conference on Multimedia Retrieval

Number of Pages

420-427

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1145/3206025.3206071

Socpus ID

85053869136 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85053869136

This document is currently not available here.

Share

COinS