Robust Data Association Against Detection Deficiency for Semantic SLAM

Xubin Lin1; Jiahao Ruan1; Yirui Yang1; Li He2; Yisheng Guan1; Hong Zhang2

2023

10.1109/TASE.2022.3233662

IEEE Transactions on Automation Science and Engineering   影响因子和分区

1558-3783

1558-3783

Robust and accurate object association is essential for precise 3D object landmark inference in semantic Simultaneous Localization and Mapping (SLAM), and yet remains challenging due to the detection deficiency caused by high miss detection rate, false alarm, occlusion and limited field-of-view, etc. The 2D location of an object is a crucial complementary cue to the appearance feature, especially in the case of associating objects across frames under large viewpoint changes. However, motion model or trajectory pattern based methods struggle to infer object motion reliably with a moving camera. In this paper, by exploiting the local projective warping consistency, a local homography based 2D motion inference method is proposed to sequentially estimate the object location along with uncertainty. By integrating the deep appearance feature and semantic information, an object association method, named HOA, which is robust to detection deficiency is proposed. Experimental evaluations suggest that the proposed motion prediction method is capable of maintaining a low cumulative error over a long duration, which enhances the object association performance in both accuracy and robustness. Note to Practitioners-This work aims to consistently associate 2D detection boxes corresponding to the same 3D object across images. In tasks of landmark-based navigation, collision avoidance, grasping and manipulation, objects in the task space are commonly simplified into 3D enveloping surfaces (e.g. cuboid or ellipsoid) by using 2D object detection boxes from multiple image views, and accurate data association is a prerequisite for precise enveloping surface reconstruction. This problem remains challenging considering the imperfect object detections, the appearance similarity of objects and the unpredictable trajectory of the moving camera. This work proposes a long-term reliable 2D location prediction algorithm that is capable of handling the complex motion of the target. Along with the appearance feature extracted by a retrain-free deep learning based model, this work proposes an object association method that can simultaneously deal with multiple objects with unknown object categories under the moving camera scenario.

Data association multiple object tracking semantic SLAM local homography

SCI ; EI

英语

其他

National Natural ScienceFoundation of China[62173096] ; Leading Talentsof Guangdong Province Program["2016LJ06G498","2019QN01X761"] ; Guangdong Province Special Fund for ModernAgricultural Industry Common Key Technology Research and DevelopmentInnovation Team[2019KJ129] ; Programfor Guangdong Yangfan Innovative and Entrepreneurial Teams[2017YT05G026]

Automation & Control Systems

Automation & Control Systems

WOS:000915771100001

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

20230613546126

Deep learning ; Feature extraction ; Location ; Mapping ; Motion estimation ; Object detection ; Object recognition ; Robotics ; Semantics ; Target tracking ; Three dimensional displays ; Trajectories ; Uncertainty analysis

Surveying:405.3 ; Ergonomics and Human Factors Engineering:461.4 ; Computer Peripheral Equipment:722.2 ; Data Processing and Image Processing:723.2 ; Robotics:731.5 ; Photographic Equipment:742.2 ; Probability Theory:922.1

IEEE

1.Biomimetic and Intelligent Robotics Laboratory (BIRL), Guangdong University of Technology, Guangzhou, China
2.Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China

Xubin Lin,Jiahao Ruan,Yirui Yang,et al. Robust Data Association Against Detection Deficiency for Semantic SLAM[J]. IEEE Transactions on Automation Science and Engineering,2023,PP(99):1-13.

Xubin Lin,Jiahao Ruan,Yirui Yang,Li He,Yisheng Guan,&Hong Zhang.(2023).Robust Data Association Against Detection Deficiency for Semantic SLAM.IEEE Transactions on Automation Science and Engineering,PP(99),1-13.

Xubin Lin,et al."Robust Data Association Against Detection Deficiency for Semantic SLAM".IEEE Transactions on Automation Science and Engineering PP.99(2023):1-13.