Content based video matching using spatiotemporal volumes

    Authors

    A. Basharat; Y. Zhai;M. Shah

    Comments

    Authors: contact us about adding a copy of your work at STARS@ucf.edu

    Abbreviated Journal Title

    Comput. Vis. Image Underst.

    Keywords

    video retrieval; video matching; spatiotemporal volumes; motion; segmentation; IMAGE; REPRESENTATION; RETRIEVAL; FEATURES; TEXTURE; Computer Science, Artificial Intelligence; Engineering, Electrical &; Electronic

    Abstract

    This paper presents a novel framework for matching video sequences using the spatiotemporal segmentation of videos. Instead of using appearance features for region correspondence across frames, we use interest point trajectories to generate video volumes. Point trajectories, which are generated using the SIFT operator, are clustered to form motion segments by analyzing their motion and spatial properties. The temporal correspondence between the estimated motion segments is then established based on most common SIFT correspondences. A two pass correspondence algorithm is used to handle splitting and merging regions. Spatiotemporal volumes are extracted using the consistently tracked motion segments. Next, a set of features including color, texture, motion, and SIFT descriptors are extracted to represent a volume. We employ an Earth Mover's Distance (EMD) based approach for the comparison of volume features. Given two videos, a bipartite graph is constructed by modeling the volumes as vertices and their similarities as edge weights. Maximum matching of this graph produces volume correspondences between the videos, and these volume matching scores are used to compute the final video matching score. Experiments for video retrieval were performed on a variety of videos obtained from different sources including BBC Motion Gallery and promising results were achieved. We present qualitative and quantitative analysis of retrieval along with a comparison with two baseline methods. (C) 2007 Elsevier Inc. All rights reserved.

    Journal Title

    Computer Vision and Image Understanding

    Volume

    110

    Issue/Number

    3

    Publication Date

    1-1-2008

    Document Type

    Article

    Language

    English

    First Page

    360

    Last Page

    377

    WOS Identifier

    WOS:000256047300005

    ISSN

    1077-3142

    Share

    COinS