DIR-BHRNet: A Lightweight Network for Real-Time Vision-Based Multiperson Pose Estimation on Smartphones

Lan, Gongjin1; Wu, Yu1; Hao, Qi1,2

2024-07-01

10.1109/TII.2024.3421511

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS   影响因子和分区

1551-3203

1941-0050

Human pose estimation (HPE), particularly multiperson pose estimation (MPPE), has been applied in many domains, such as human-machine systems. However, the current MPPE methods generally run on powerful GPU systems and take a lot of computational costs. Real-time MPPE on mobile devices with low-performance computing is a challenging task. In this article, we propose a lightweight neural network, DIR-BHRNet, for real-time MPPE on smartphones. In DIR-BHRNet, we design a novel lightweight convolutional module, dense inverted residual (DIR), to improve accuracy by adding a depthwise convolution and a shortcut connection into the well-known inverted residual, and a novel efficient neural network structure, balanced HRNet (BHRNet), to reduce computational costs by reconfiguring the proper number of convolutional blocks on each branch. We evaluate DIR-BHRNet on the well-known COCO and CrowdPose datasets. The results show that DIR-BHRNet outperforms the state-of-the-art methods in terms of accuracy with a real-time computational cost. Finally, we implement the DIR-BHRNet on the current mainstream Android smartphones, which perform more than 10 FPS. The free-used executable file (Android 10), source code, and a video description of this work are publicly available on the page(1) to facilitate the development of real-time MPPE on smartphones.

Deep learning human pose estimation (HPE) multiperson pose estimation (MPPE) real time smartphones

SCI

英语

第一 ; 通讯

National Natural Science Foundation of China[62261160654] ; Shenzhen Fundamental Research Program[JCYJ20220818103006012] ; Shenzhen Key Laboratory of Robotics and Computer Vision[ZDSYS20220330160557001]

Automation & Control Systems ; Computer Science ; Engineering

Automation & Control Systems ; Computer Science, Interdisciplinary Applications ; Engineering, Industrial

WOS:001283801500001

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

Web of Science

1.Southern Univ Sci & Technol, Dept Comp Sci & Engn, Shenzhen 518055, Peoples R China
2.Southern Univ Sci & Technol, Res Inst Trustworthy Autonomous Syst, Shenzhen 518055, Peoples R China

Lan, Gongjin,Wu, Yu,Hao, Qi. DIR-BHRNet: A Lightweight Network for Real-Time Vision-Based Multiperson Pose Estimation on Smartphones[J]. IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS,2024.

Lan, Gongjin,Wu, Yu,&Hao, Qi.(2024).DIR-BHRNet: A Lightweight Network for Real-Time Vision-Based Multiperson Pose Estimation on Smartphones.IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS.

Lan, Gongjin,et al."DIR-BHRNet: A Lightweight Network for Real-Time Vision-Based Multiperson Pose Estimation on Smartphones".IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS (2024).