Self-attention guided deep features for action recognition

Authors

Renyi Xiao, Tianjin University
Yonghong Hou, Tianjin UniversityFollow
Zihui Guo, Tianjin University
Chuankun Li, Tianjin UniversityFollow
Pichao Wang, Alibaba Group (U. S.) Inc.Follow
Wanqing Li, University of WollongongFollow

RIS ID

138459

Publication Details

Xiao, R., Hou, Y., Guo, Z., Li, C., Wang, P. & Li, W. (2019). Self-attention guided deep features for action recognition. IEEE International Conference on Multimedia and Expo (ICME) 2019 (pp. 1060-1065). United States: IEEE.

Abstract

Skeleton based human action recognition is an important task in computer vision. However, it is very challenging due to the complex spatio-temporal variations of skeleton joints. In this work, we propose an end-to-end trainable network consisting of a Deep Convolutional Model (DCM) and a Self-Attention Model (SAM) for human action recognition from skeleton data. Specifically, skeleton sequences are encoded into color images and fed into DCM to extract deep features. In the SAM, handcrafted features representing the motion degree of joints are extracted and the attention weights are learned by a simple yet effective linear mapping. The effectiveness of proposed method has been verified on NTU RGB+D, SYSU-3D and UTD-MHAD datasets and achieved state-of-the-art results.