Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning

    Authors

    S. Ali;M. Shah

    Comments

    Authors: contact us about adding a copy of your work at STARS@ucf.edu

    Abbreviated Journal Title

    IEEE Trans. Pattern Anal. Mach. Intell.

    Keywords

    Action recognition; motion; video analysis; principal component; analysis; kinematic features; Computer Science, Artificial Intelligence; Engineering, Electrical &; Electronic

    Abstract

    We propose a set of kinematic features that are derived from the optical flow for human action recognition in videos. The set of kinematic features includes divergence, vorticity, symmetric and antisymmetric flow fields, second and third principal invariants of flow gradient and rate of strain tensor, and third principal invariant of rate of rotation tensor. Each kinematic feature, when computed from the optical flow of a sequence of images, gives rise to a spatiotemporal pattern. It is then assumed that the representative dynamics of the optical flow are captured by these spatiotemporal patterns in the form of dominant kinematic trends or kinematic modes. These kinematic modes are computed by performing Principal Component Analysis (PCA) on the spatiotemporal volumes of the kinematic features. For classification, we propose the use of multiple instance learning (MIL) in which each action video is represented by a bag of kinematic modes. Each video is then embedded into a kinematic-mode-based feature space and the coordinates of the video in that space are used for classification using the nearest neighbor algorithm. The qualitative and quantitative results are reported on the benchmark data sets.

    Journal Title

    Ieee Transactions on Pattern Analysis and Machine Intelligence

    Volume

    32

    Issue/Number

    2

    Publication Date

    1-1-2010

    Document Type

    Article

    Language

    English

    First Page

    288

    Last Page

    303

    WOS Identifier

    WOS:000272741500008

    ISSN

    0162-8828

    Share

    COinS