Semantic Image Search From Multiple Query Images

Abstract

This paper presents a novel search paradigm that uses mul-tiple images as input to perform semantic search of images. While earlier focuses on using single or multiple query im-ages to retrieve images with views of the same instance, the proposed paradigm uses each query image to discover com-mon concepts that are implicitly shared by all of the query images and retrieves images considering the found concepts. Our implementation uses high level visual features extracted from a deep convolutional network to retrieve images simi-lar to each query input. These images have associated text previously generated by implicit crowdsourcing. A Bag of Words (BoW) textual representation of each query image is built from the associated text of the retrieved similar images. A learned vector space representation of English words ex-tracted from a corpus of 100 billion words allows computing the conceptual similarity of words. The words that repre-sent the input images are used to find new words that share conceptual similarity across all the input images. These new words are combined with the representations of the input im-ages to obtain a BoW textual representation of the search, which is used to perform image retrieval. The retrieved im-ages are re-ranked to enhance visual similarity with respect to any of the input images. Our experiments show that the concepts found are meaningful and that they retrieve cor-rectly 72.43% of the images from the top 25, along with user ratings performed in the cases of study.

Publication Date

10-13-2015

Publication Title

MM 2015 - Proceedings of the 2015 ACM Multimedia Conference

Number of Pages

887-890

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1145/2733373.2806356

Socpus ID

84962787535 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84962787535

This document is currently not available here.

Share

COinS