Pavilion: Bridging photo-realism and robotics

Jiang，Fan; Hao，Qi

10.1109/ICRA.2019.8794235

2019-05-01

1050-4729

2577-087X

978-1-5386-8176-3

Proceedings - IEEE International Conference on Robotics and Automation

2019-May

8285-8290

20-24 May 2019

Montreal, QC, Canada

345 E 47TH ST, NEW YORK, NY 10017 USA

Institute of Electrical and Electronics Engineers Inc.

Simulation environments play a centric role in the research of sensor fusion and robot control. This paper presents Pavilion, a novel open-source simulation system, for robot perception and kinematic control based on the Unreal Engine and the Robot Operating System (ROS). The novelty of this work includes threefold: (1) developing a shader-based method to generate optical flow ground-truth data with the Unreal Engine, (2) developing a toolset to remove binary incompatibility between ROS and the Unreal Engine to enable real-time interaction, and (3) developing a method to directly import Simulation Description Format (SDF) robot models into the Unreal Engine at runtime. Finally, a Gazebo-compatible real-time simulation system is developed to enable training and evaluation of a large number of sensor fusion, planning, decision and control algorithms. The system can be implemented on both Linux and macOS, with the latest version of ROS. Various experiments have been performed to validate the superior performance of the proposed simulation environment over other state-of-the-art simulators in terms of number of modalities, simulation accuracy, latency and degree of integration difficulty.

Engines Robot sensing systems Data models Real-time systems Pipelines Optical sensors

第一

英语

CPCI-S ; EI

National Natural Science Foundation of China[61773197]

Automation & Control Systems ; Robotics

Automation & Control Systems ; Robotics

WOS:000494942306005

20193507384343

2-s2.0-85071512623

Scopus

Department of Computer Science and EngineeringSouthern University of Science and Technology,Shenzhen,518055,China

Jiang，Fan,Hao，Qi. Pavilion: Bridging photo-realism and robotics[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:Institute of Electrical and Electronics Engineers Inc.,2019:8285-8290.