Title

Learning Semantic Visual Vocabularies Using Diffusion Distance

Abstract

In this paper, we propose a novel approach for learning generic visual vocabulary. We use diffusion maps to au-tomatically learn a semantic visual vocabulary from ab-undant quantized midlevel features. Each midlevel feature is represented by the vector of pointwise mutual informa-tion (PMI). In this midlevel feature space, we believe the features produced by similar sources must lie on a certain manifold. To capture the intrinsic geometric relations be-tween features, we measure their dissimilarity using diffu-sion distance. The underlying idea is to embed the midlevel features into a semantic lower-dimensional space. Our goal is to construct a compact yet discriminative semantic visual vocabulary., , Although the conventional approach using k-means is good for vocabulary construction, its performance is sen-sitive to the size of the visual vocabulary. In addition, the learnt visual words are not semantically meaningful since the clustering criterion is based on appearance similarity only. Our proposed approach can effectively overcome these problems by capturing the semantic and geometric relations of the feature space using diffusion maps. Unlike some of the supervised vocabulary construction ap-proaches, and the unsupervised methods such as pLSA and LDA, diffusion maps can capture the local intrinsic geo-metric relations between the midlevel feature points on the manifold. We have tested our approach on the KTH action dataset, our own YouTube action dataset and the fifteen scene dataset, and have obtained very promising results. ©2009 IEEE.

Publication Date

1-1-2009

Publication Title

2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009

Number of Pages

461-468

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/CVPRW.2009.5206845

Socpus ID

70450170628 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/70450170628

This document is currently not available here.

Share

COinS