Pose-Aided Video-based Person Re-Identification via Recurrent Graph Convolutional Network

Pan，Honghu1; Liu，Qiao2; Chen，Yongyong1; He，Yunqi3; Zheng，Yuan4; Zheng，Feng5; He，Zhenyu1

2023

10.1109/TCSVT.2023.3276996

IEEE Transactions on Circuits and Systems for Video Technology   影响因子和分区

1051-8215

1558-2205

Existing methods for video-based person re-identification (ReID) mainly learn the appearance feature of a given pedestrian via a feature extractor and a feature aggregator. However, the appearance models would fail to learn a large inter-class variance when different pedestrians have similar appearances. Considering that different pedestrians have different walking postures and body proportions, we propose to learn the discriminative pose feature beyond the appearance feature for video retrieval. Specifically, we implement a two-branch architecture to separately learn the appearance feature and pose feature, and then concatenate them together for inference. To learn the pose feature, we first detect the pedestrian pose in each frame through an off-the-shelf pose detector, and construct a temporal graph using the pose sequence. We then exploit a recurrent graph convolutional network (RGCN) to learn the node embeddings of the temporal pose graph, which devises a global information propagation mechanism to simultaneously achieve the neighborhood aggregation of intra-frame nodes and message passing among inter-frame graphs. Finally, we propose a dual-attention method (DAM) consisting of node-attention and time-attention to obtain the temporal graph representation from the node embeddings, where the self-attention mechanism is employed to learn the importance of each node and each frame. We verify the proposed method on three video-based ReID datasets, i.e., Mars, DukeMTMC and iLIDS-VID, whose experimental results demonstrate that the learned pose feature can effectively improve the performance of existing appearance models.

Convolutional neural networks Dams Feature extraction Legged locomotion Optical filters Representation learning Visualization

EI

英语

其他

20232214159910

Bandpass filters ; Computer vision ; Embeddings ; Feature extraction ; Graph neural networks ; Graph theory ; Information dissemination ; Message passing

Electric Filters:703.2 ; Information Theory and Signal Processing:716.1 ; Computer Programming:723.1 ; Data Processing and Image Processing:723.2 ; Artificial Intelligence:723.4 ; Computer Applications:723.5 ; Vision:741.2 ; Information Dissemination:903.2 ; Combinatorial Mathematics, Includes Graph Theory, Set Theory:921.4

ENGINEERING

2-s2.0-85160259017

Scopus

1.School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Shenzhen, China
2.National Center for Applied Mathematics in Chongqing, Chongqing Normal University, Chongqing, China
3.College of Information and Computer Engineering, Northeast Forestry University, Harbin, China
4.School of Computer Science, Inner Mongolia University, Huhehot, China
5.Southern University of Science and Technology, Shenzhen, China

Pan，Honghu,Liu，Qiao,Chen，Yongyong,et al. Pose-Aided Video-based Person Re-Identification via Recurrent Graph Convolutional Network[J]. IEEE Transactions on Circuits and Systems for Video Technology,2023,PP(99):1-1.

Pan，Honghu.,Liu，Qiao.,Chen，Yongyong.,He，Yunqi.,Zheng，Yuan.,...&He，Zhenyu.(2023).Pose-Aided Video-based Person Re-Identification via Recurrent Graph Convolutional Network.IEEE Transactions on Circuits and Systems for Video Technology,PP(99),1-1.

Pan，Honghu,et al."Pose-Aided Video-based Person Re-Identification via Recurrent Graph Convolutional Network".IEEE Transactions on Circuits and Systems for Video Technology PP.99(2023):1-1.