Collaborative 3D Object Detection for Autonomous Vehicles via Learnable Communications

Wang, J.1; Zeng, Y.2; Gong, Y.1

2023-05-01

10.1109/TITS.2023.3272027

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS   影响因子和分区

1524-9050

1558-0016

3D object detection from LiDAR point cloud is a challenging task in autonomous driving systems. Collaborative perception can incorporate information from spatially diverse sensors and provide significant benefits for accurate 3D object detection from point clouds. In this work, we consider that the autonomous vehicle uses local point cloud data and combines information from neighboring infrastructures through wireless links for cooperative 3D object detection. However, information sharing among vehicles and infrastructures in predefined communication schemes may result in communication congestion and/or bring limited performance improvement. To this end, we propose a novel collaborative 3D object detection framework using an encoder-decoder network architecture and an attention-based learnable communications scheme. It consists of three components: a feature encoder network that maps point clouds into feature maps; an attention-based communication module that propagates compact and fine-grained query feature maps from the vehicle to support infrastructures, and optimizes attention weights between query and key to refine support feature maps; a region proposal network that fuses local feature maps and weighted support feature maps for 3D object detection. We evaluate the performance of the proposed framework on CARLA-3D, a new dataset that we synthesized using CARLA for 3D cooperative object detection. Experimental results and bandwidth consumption analysis show that the proposed collaborative 3D object detection framework achieves a better detection performance and communication bandwidth trade-off than five baseline 3D object detection models under different detection difficulties.

Three-dimensional displays Object detection Point cloud compression Feature extraction Collaboration Bandwidth Autonomous vehicles Collaborative perception learnable communications 3D object detection autonomous driving

SCI ; EI

英语

第一 ; 通讯

National Natural Science Foundation of China["62071212","62106095"] ; Guangdong Basic and Applied Basic Research Foundation[2019B1515130003] ; Guangdong Provincial Department of Education[2020ZDZX3057]

Engineering ; Transportation

Engineering, Civil ; Engineering, Electrical & Electronic ; Transportation Science & Technology

WOS:000988487400001

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC

20232114131612

Autonomous vehicles ; Economic and social effects ; Feature extraction ; Network architecture ; Object detection ; Object recognition ; Three dimensional displays ; Traffic congestion ; Vehicle to vehicle communications

Highway Transportation:432 ; Information Theory and Signal Processing:716.1 ; Radio Systems and Equipment:716.3 ; Computer Peripheral Equipment:722.2 ; Data Processing and Image Processing:723.2 ; Robot Applications:731.6 ; Social Sciences:971

ENGINEERING

Web of Science

1.Southern Univ Sci & Technol, Dept Elect & Elect Engn, Shenzhen 518055, Peoples R China
2.Southern Univ Sci & Technol, Res Inst Trustworthy Autonomous Syst, Shenzhen 518055, Peoples R China

Wang, J.,Zeng, Y.,Gong, Y.. Collaborative 3D Object Detection for Autonomous Vehicles via Learnable Communications[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2023,PP(99):1-13.

Wang, J.,Zeng, Y.,&Gong, Y..(2023).Collaborative 3D Object Detection for Autonomous Vehicles via Learnable Communications.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,PP(99),1-13.

Wang, J.,et al."Collaborative 3D Object Detection for Autonomous Vehicles via Learnable Communications".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS PP.99(2023):1-13.