An Occlusion-Aware Framework for Real-Time 3D Pose Tracking

Fu, Mingliang1,2; Leng, Yuquan3; Luo, Haitao1; Zhou, Weijia1

2018-08

10.3390/s18082734

SENSORS   影响因子和分区

1424-8220

Random forest-based methods for 3D temporal tracking over an image sequence have gained increasing prominence in recent years. They do not require object's texture and only use the raw depth images and previous pose as input, which makes them especially suitable for textureless objects. These methods learn a built-in occlusion handling from predetermined occlusion patterns, which are not always able to model the real case. Besides, the input of random forest is mixed with more and more outliers as the occlusion deepens. In this paper, we propose an occlusion-aware framework capable of real-time and robust 3D pose tracking from RGB-D images. To this end, the proposed framework is anchored in the random forest-based learning strategy, referred to as RFtracker. We aim to enhance its performance from two aspects: integrated local refinement of random forest on one side, and online rendering based occlusion handling on the other. In order to eliminate the inconsistency between learning and prediction of RFtracker, a local refinement step is embedded to guide random forest towards the optimal regression. Furthermore, we present an online rendering-based occlusion handling to improve the robustness against dynamic occlusion. Meanwhile, a lightweight convolutional neural network-based motion-compensated (CMC) module is designed to cope with fast motion and inevitable physical delay caused by imaging frequency and data transmission. Finally, experiments show that our proposed framework can cope better with heavily-occluded scenes than RFtracker and preserve the real-time performance.

pose tracking occlusion handling online rendering motion compensation

SCI ; EI

英语

其他

National Science Foundation of China[51505470]

Chemistry ; Electrochemistry ; Instruments & Instrumentation

Chemistry, Analytical ; Electrochemistry ; Instruments & Instrumentation

WOS:000445712400334

MDPI

20183505743776

Decision trees ; Motion compensation ; Neural networks ; Rendering (computer graphics)

Data Processing and Image Processing:723.2 ; Systems Science:961

CHEMISTRY

Web of Science

1.Chinese Acad Sci, Shenyang Inst Automat, State Key Lab Robot, Shenyang 110016, Liaoning, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
3.Southern Univ Sci & Technol, Dept Mech & Energy Engn, Shenzhen 518055, Peoples R China

Fu, Mingliang,Leng, Yuquan,Luo, Haitao,et al. An Occlusion-Aware Framework for Real-Time 3D Pose Tracking[J]. SENSORS,2018,18(8).

Fu, Mingliang,Leng, Yuquan,Luo, Haitao,&Zhou, Weijia.(2018).An Occlusion-Aware Framework for Real-Time 3D Pose Tracking.SENSORS,18(8).

Fu, Mingliang,et al."An Occlusion-Aware Framework for Real-Time 3D Pose Tracking".SENSORS 18.8(2018).