A Novel Multi-feature Skeleton Representation for 3D Action Recognition

Chen，Lian1; Lu，Ke1,2; Gao，Pengcheng1; Xue，Jian1; Wang，Jinbao3

10.1007/978-3-030-68821-9_33

2021

0302-9743

1611-3349

LECT NOTES ARTIF INT

12665 LNCS

365-379

Deep-learning-based methods have been used for 3D action recognition in recent years. Methods based on recurrent neural networks (RNNs) have the advantage of modeling long-term context, but they focus mainly on temporal information and ignore the spatial relationships in each skeleton frame. In addition, it is difficult to handle a very long skeleton sequence using an RNN. Compared with an RNN, a convolutional neural network (CNN) is better able to extract spatial information. To model the temporal information of skeleton sequences and incorporate the spatial relationship in each frame efficiently using a CNN, this paper proposes a multi-feature skeleton representation for encoding features from original skeleton sequences. The relative distances between joints in each skeleton frame are computed from the original skeleton sequence, and several relative angles between the skeleton structures are computed. This useful information from the original skeleton sequence is encoded as pixels in grayscale images. To preserve more spatial relationships between input skeleton joints in these images, the skeleton joints are divided into five groups: one for the trunk and one for each arm and each leg. Relationships between joints in the same group are more relevant than those between joints in different groups. By rearranging pixels in encoded images, the joints that are mutually related in the spatial structure are adjacent in the images. The skeleton representations, composed of several grayscale images, are input to CNNs for action recognition. Experimental results demonstrate the effectiveness of the proposed method on three public 3D skeleton-based action datasets.

3D action recognition Convolutional neural network Deep learning Skeleton representation

其他

英语

EI

20211610234861

Convolutional neural networks ; Pattern recognition ; Pixels ; Recurrent neural networks

Biomechanics, Bionics and Biomimetics:461.3

2-s2.0-85104313554

Scopus

1.University of Chinese Academy of Sciences,Beijing,China
2.Peng Cheng Laboratory,Shenzhen,Vanke Cloud City Phase I Building 8, Xili Street, Nanshan District,China
3.Southern University of Science and Technology,Shenzhen,China

Chen，Lian,Lu，Ke,Gao，Pengcheng,et al. A Novel Multi-feature Skeleton Representation for 3D Action Recognition[C],2021:365-379.