Discovering Motion Primitives for Unsupervised Grouping and One-Shot Learning of Human Actions, Gestures, and Expressions

    Authors

    Y. Yang; I. Saleemi;M. Shah

    Comments

    Authors: contact us about adding a copy of your work at [email protected]

    Abbreviated Journal Title

    IEEE Trans. Pattern Anal. Mach. Intell.

    Keywords

    Human actions; one-shot learning; unsupervised clustering; gestures; facial expressions; action representation; action recognition; motion; primitives; motion patterns; histogram of motion primitives; motion; primitives strings; Hidden Markov model; RECOGNITION; Computer Science, Artificial Intelligence; Engineering, Electrical &; Electronic

    Abstract

    This paper proposes a novel representation of articulated human actions and gestures and facial expressions. The main goals of the proposed approach are: 1) to enable recognition using very few examples, i.e., one or k-shot learning, and 2) meaningful organization of unlabeled datasets by unsupervised clustering. Our proposed representation is obtained by automatically discovering high-level subactions or motion primitives, by hierarchical clustering of observed optical flow in four-dimensional, spatial, and motion flow space. The completely unsupervised proposed method, in contrast to state-of-the-art representations like bag of video words, provides a meaningful representation conducive to visual interpretation and textual labeling. Each primitive action depicts an atomic subaction, like directional motion of limb or torso, and is represented by a mixture of four-dimensional Gaussian distributions. For one-shot and k-shot learning, the sequence of primitive labels discovered in a test video are labeled using KL divergence, and can then be represented as a string and matched against similar strings of training videos. The same sequence can also be collapsed into a histogram of primitives or be used to learn a Hidden Markov model to represent classes. We have performed extensive experiments on recognition by one and k-shot learning as well as unsupervised action clustering on six human actions and gesture datasets, a composite dataset, and a database of facial expressions. These experiments confirm the validity and discriminative nature of the proposed representation.

    Journal Title

    Ieee Transactions on Pattern Analysis and Machine Intelligence

    Volume

    35

    Issue/Number

    7

    Publication Date

    1-1-2013

    Document Type

    Article

    Language

    English

    First Page

    1635

    Last Page

    1648

    WOS Identifier

    WOS:000319060600008

    ISSN

    0162-8828

    Share

    COinS
     

    Accessibility Statement

    This item was created or digitized prior to April 24, 2027, or is a reproduction of legacy media created before that date. It is preserved in its original, unmodified state specifically for research, reference, or historical recordkeeping. In accordance with the ADA Title II Final Rule, the University Libraries provides accessible versions of archival materials upon request. To request an accommodation for this item, please submit an accessibility request form.