Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

Huang, Kemiao1; Hao, Qi1,2,3

10.1109/IROS51168.2021.9636311

2021

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2153-0858

978-1-6654-1715-0

2021 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)

6983-6989

SEP 27-OCT 01, 2021

null,null,ELECTR NETWORK

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Multi-object tracking (MOT) with camera-LiDAR fusion demands accurate results of object detection, affinity computation and data association in real time. This paper presents an efficient multi-modal MOT framework with online joint detection and tracking schemes and robust data association for autonomous driving applications. The novelty of this work includes: (1) development of an end-to-end deep neural network for joint object detection and correlation using 2D and 3D measurements; (2) development of a robust affinity computation module to compute occlusion-aware appearance and motion affinities in 3D space; (3) development of a comprehensive data association module for joint optimization among detection confidences, affinities and start-end probabilities. The experiment results on the KITTI tracking benchmark demonstrate the superior performance of the proposed method in terms of both tracking accuracy and processing speed.

Training Solid modeling Three-dimensional displays Tracking Robot vision systems Object detection Benchmark testing

第一 ; 通讯

英语

CPCI-S ; EI

Shenzhen Fundamental Research Program[JCYJ20200109141622964] ; Intel ICRI-IACV Research Fund[52514373]

Automation & Control Systems ; Computer Science ; Engineering ; Robotics

Automation & Control Systems ; Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Robotics

WOS:000755125505086

20220711623943

Benchmarking ; Cameras ; Deep neural networks ; Object detection ; Object recognition ; Optical radar

Highway Transportation:432 ; Ergonomics and Human Factors Engineering:461.4 ; Radar Systems and Equipment:716.2 ; Data Processing and Image Processing:723.2 ; Robot Applications:731.6 ; Optical Devices and Systems:741.3 ; Photographic Equipment:742.2

Web of Science

1.Southern Univ Sci & Technol, Dept Comp Sci & Engn, Shenzhen 518055, Peoples R China
2.Southern Univ Sci & Technol, Sifakis Res Inst Trustworthy Autonomous Syst, Shenzhen 518055, Peoples R China
3.Pazhou Lab, Guangzhou 510330, Peoples R China

Huang, Kemiao,Hao, Qi. Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2021:6983-6989.