Concurrence-Aware Long Short-Term Sub-Memories For Person-Person Action Recognition

Abstract

Recently, Long Short-Term Memory (LSTM) has become a popular choice to model individual dynamics for single-person action recognition. However, existing RNN models only focus on capturing the temporal dynamics of the person-person interactions by naively combining the activity dynamics of individuals or modeling them as a whole. This neglects the inter-related dynamics of how person-person interactions change over time. To this end, we propose a novel Concurrent Long Short-Term Memories (Co-LSTM) to model the long-term inter-related dynamics between two interacting people on the bonding boxes covering people. Specifically, for each frame, two sub-memory units store individual motion information, while a concurrent LSTM unit selectively integrates and stores inter-related motion information between interacting people from these two sub-memory units via a new co-memory cell. In experiments, we show the superior performance of Co-LSTM compared with the state-of-the-arts methods.

Publication Date

8-22-2017

Publication Title

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

Volume

2017-July

Number of Pages

2176-2183

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/CVPRW.2017.270

Socpus ID

85030265600 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85030265600

This document is currently not available here.

Share

COinS