University of Wollongong
Browse

A hybrid network for large-scale action recognition from RGB and depth modalities

Download (4.5 MB)
journal contribution
posted on 2024-11-15, 22:33 authored by Huogen Wang, Zhanjie Song, Wanqing LiWanqing Li, Pichao Wang
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. The paper presents a novel hybrid network for large-scale action recognition from multiple modalities. The network is built upon the proposed weighted dynamic images. It effectively leverages the strengths of the emerging Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) based approaches to specifically address the challenges that occur in large-scale action recognition and are not fully dealt with by the state-of-the-art methods. Specifically, the proposed hybrid network consists of a CNN based component and an RNN based component. Features extracted by the two components are fused through canonical correlation analysis and then fed to a linear Support Vector Machine (SVM) for classification. The proposed network achieved state-of-the-art results on the ChaLearn LAP IsoGD, NTU RGB+D and Multi-modal & Multi-view & Interactive (M2 I) datasets and outperformed existing methods by a large margin (over 10 percentage points in some cases).

History

Citation

Wang, H., Song, Z., Li, W. & Wang, P. (2020). A hybrid network for large-scale action recognition from RGB and depth modalities. Sensors (Switzerland), 20 (11), 1-25.

Language

English

RIS ID

143788

Usage metrics

    Categories

    Keywords

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC