Limited Information Aggregation for Collaborative Driving in Multi-Agent Autonomous Vehicles

Liang，Qingyi1,2; Liu，Jia2; Jiang，Zhengmin2,3; Yin，Jianwen2; Xu，Kun2; Li，Huiyun2

2024-07-01

10.1109/LRA.2024.3410159

IEEE Robotics and Automation Letters   影响因子和分区

2377-3766

— Multi-agent reinforcement learning (MARL) methods have emerged as a promising solution for multi-agent collaborative driving in the intersection and roundabout scenarios. However, these methods need large amounts of training data obtained from the interaction with the driving simulator, and learning from limited interaction remains significantly underdeveloped. In this letter, we propose an efficient MARL method to address this challenge. Our method enables each vehicle to receive limited messages from surrounding vehicles, which are then used to augment the input representation of the local driving policy. By predicting the next-step state based on the current augmented local state and action, our approach enhances the decision-making capability of each vehicle. Specifically, we design a Self-supervised Message Attention Encoding (SMAE) module that utilizes an attention mechanism to aggregate the received messages and local observations, generating a compact representation. Then, this representation is used in a self-supervised module to predict the next-step state. By jointly training the encoder module and the prediction module, each vehicle effectively leverages the most relevant components of the aggregated representation to improve the learning efficiency of driving policy and alleviate issues related to partial observability in making driving decisions. To validate the effectiveness of our approach, we conduct experiments using an open-source autonomous driving simulator. The simulation results demonstrate that our proposed method outperforms the IPPO, MAPPO and CoPO algorithms in terms of success rate, route completion rate, crash rate, and other relevant metrics.

collaborative driving information aggregation Multi-agent autonomous vehicles

SCI ; EI

英语

第一

20242416235190

Autonomous agents ; Autonomous vehicles ; Data mining ; Fertilizers ; Forecasting ; Multi agent systems ; Observability ; Reinforcement learning ; Signal encoding

Highway Transportation:432 ; Information Theory and Signal Processing:716.1 ; Data Processing and Image Processing:723.2 ; Artificial Intelligence:723.4 ; Control Systems:731.1 ; Robot Applications:731.6 ; Chemical Products Generally:804 ; Agricultural Chemicals:821.2 ; Management:912.2

2-s2.0-85195408823

Scopus

1.the Southern University of Science and Technology,Shenzhen,518055,China
2.the CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems,Shenzhen Institutes of Advanced Technology,Chinese Academy of Sciences,Shenzhen,518055,China
3.the University of Chinese Academy of Sciences,Beijing,101408,China

Liang，Qingyi,Liu，Jia,Jiang，Zhengmin,et al. Limited Information Aggregation for Collaborative Driving in Multi-Agent Autonomous Vehicles[J]. IEEE Robotics and Automation Letters,2024,9(7):6624-6631.

Liang，Qingyi,Liu，Jia,Jiang，Zhengmin,Yin，Jianwen,Xu，Kun,&Li，Huiyun.(2024).Limited Information Aggregation for Collaborative Driving in Multi-Agent Autonomous Vehicles.IEEE Robotics and Automation Letters,9(7),6624-6631.

Liang，Qingyi,et al."Limited Information Aggregation for Collaborative Driving in Multi-Agent Autonomous Vehicles".IEEE Robotics and Automation Letters 9.7(2024):6624-6631.