Title

Identifying User-Specific Facial Affects From Spontaneous Expressions With Minimal Annotation

Keywords

Facial affect detection; user-dependent model; weakly supervised learning

Abstract

This paper presents Personalized Affect Detection with Minimal Annotation (PADMA), a user-dependent approach for identifying affective states from spontaneous facial expressions without the need for expert annotation. The conventional approach relies on the use of key frames in recorded affect sequences and requires an expert observer to identify and annotate the frames. It is susceptible to user variability and accommodating individual differences is difficult. The alternative is a user-dependent approach, but it would be prohibitively expensive to collect and annotate data for each user. PADMA uses a novel Association-based Multiple Instance Learning (AMIL) method, which learns a personal facial affect model through expression frequency analysis, and does not need expert input or frame-based annotation. PADMA involves a training/calibration phase in which the user watches short video segments and reports the affect that best describes his/her overall feeling throughout the segment. The most indicative facial gestures are identified and extracted from the facial response video, and the association between gesture and affect labels is determined by the distribution of the gesture over all reported affects. Hence both the geometric deformation and distribution of key facial gestures are specially adapted for each user. We show results that demonstrate the feasibility, effectiveness and extensibility of our approach.

Publication Date

10-1-2016

Publication Title

IEEE Transactions on Affective Computing

Volume

7

Issue

4

Number of Pages

360-373

Document Type

Article

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/TAFFC.2015.2495222

Socpus ID

85027456005 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85027456005

This document is currently not available here.

Share

COinS