An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement

Haojie Shi1,2; Tingguang Li1; Qingxu Zhu1; Jiapeng Sheng1; Lei Han1; Max Q.-H. Meng3

10.1109/ICRA57147.2024.10611560

2024-05-17

979-8-3503-8458-1

2024 IEEE International Conference on Robotics and Automation (ICRA)

13-17 May 2024

Yokohama, Japan

Learning-based methods have improved locomotion skills of quadruped robots through deep reinforcement learning. However, the sim-to-real gap and low sample efficiency still limit the skill transfer. To address this issue, we propose an efficient model-based learning framework that combines a world model with a policy network. We train a differentiable world model to predict future states and use it to directly supervise a Variational Autoencoder (VAE)-based policy network to imitate real animal behaviors. This significantly reduces the need for real interaction data and allows for rapid policy updates. We also develop a high-level network to track diverse commands and trajectories. Our simulated results show a tenfold sample efficiency increase compared to reinforcement learning methods such as PPO. In real-world testing, our policy achieves proficient command-following performance with only a two-minute data collection period and generalizes well to new speeds and paths.

其他

EI

1.Tencent Robotics X, China
2.Chinese University of Hong Kong
3.Department of Electronic and Electrical Engineering, Shenzhen Key Laboratory of Robotics Perception and Intelligence, Southern University of Science and Technology, Shenzhen, China

Haojie Shi,Tingguang Li,Qingxu Zhu,et al. An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement[C],2024.