University of Wollongong
Browse

Spatially and Temporally Structured Global to Local Aggregation of Dynamic Depth Information for Action Recognition

Download (2.63 MB)
journal contribution
posted on 2024-11-15, 08:24 authored by Yonghong Hou, Shuang Wang, Pichao Wang, Zhimin Gao, Wanqing LiWanqing Li
This paper presents an effective yet simple video representation for RGB-D based action recognition. It proposes to represent a depth map sequence into three pairs of structured dynamic images at body, part and joint levels respectively through hierarchical bidirectional rank pooling. Different from previous works that applied one Convolutional Neural Network (ConvNet) for each part/joint separately, one pair of structured dynamic images is constructed from depth maps at each granularity level and serves as the input of a ConvNet. The structured dynamic image not only preserves the spatial-temporal information but also enhances the structure information across both body parts/joints and different temporal scales. In addition, it requires low computational cost and memory to construct. This new representation, referred to as Spatially and Temporally Structured Dynamic Depth Images (STSDDI), aggregates from global to fine-grained levels motion and structure information in a depth sequence, and enables us to fine-tune the existing ConvNet models trained on image data for classification of depth sequences, without a need for training the models afresh. The proposed representation is evaluated on six benchmark datasets, namely, MSRAction3D, G3D, MSRDailyActivity3D, SYSU 3D HOI, UTD-MHAD and M2I datasets and achieves the state-of-the-art results on all six datasets.

History

Citation

Hou, Y., Wang, S., Wang, P., Gao, Z. & Li, W. (2018). Spatially and Temporally Structured Global to Local Aggregation of Dynamic Depth Information for Action Recognition. IEEE Access, 6 2206-2219.

Journal title

IEEE Access

Volume

6

Pagination

2206-2219

Language

English

RIS ID

118209

Usage metrics

    Categories

    Keywords

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC