Smart Mobility with Agent-based Foundation Models: Towards Interactive and Collaborative Intelligent Vehicles

Bingyi Xia1; Peijia Xie1; Jiankun Wang1,2

2024

10.1109/TIV.2024.3457759

IEEE Transactions on Intelligent Vehicles   影响因子和分区

2379-8858

This letter reports the insights gained during a Distributed/Decentralized Hybrid Workshop on Foundation/Infrastructure Intelligence (FII), where we discussed the evolving role of Foundation Models in the field of intelligent vehicles. These models, pre-trained on multimodal data, have emerged as pivotal in the landscape of intelligent vehicles by leveraging their capabilities for high-level reasoning. Ongoing research focuses on these models to further improve scene perception and decision-making, aiming to develop adaptive systems for robot navigation and autonomous driving. However, for smart mobility across the Cyber-Physical-Social space, foundation intelligence should learn human-level knowledge to perform sophisticated interactions and collaborations based on human feedback. Agent-based Foundation Models, as the new training paradigm, can generate cross-domain actions consistent with perception information, paving the way to realize interactive and collaborative agents. This letter discusses the challenges of enhancing and leveraging the scene understanding and spatial reasoning capabilities of the pre-trained foundation model for smart mobility. It also offers insights into the embodied employment of foundation and infrastructure intelligence in enhancing multimodal interactions between robots, environments, and humans.

第一

1.Shenzhen Key Laboratory of Robotics Perception and Intelligence, Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China
2.Jiaxing Research Institute, Southern University of Science and Technology, Jiaxing, China

Bingyi Xia,Peijia Xie,Jiankun Wang. Smart Mobility with Agent-based Foundation Models: Towards Interactive and Collaborative Intelligent Vehicles[J]. IEEE Transactions on Intelligent Vehicles,2024,PP(99).

Bingyi Xia,Peijia Xie,&Jiankun Wang.(2024).Smart Mobility with Agent-based Foundation Models: Towards Interactive and Collaborative Intelligent Vehicles.IEEE Transactions on Intelligent Vehicles,PP(99).

Bingyi Xia,et al."Smart Mobility with Agent-based Foundation Models: Towards Interactive and Collaborative Intelligent Vehicles".IEEE Transactions on Intelligent Vehicles PP.99(2024).