No Need for Interactions: Robust Model-Based Imitation Learning using Neural ODE

Lin，Hao Chih1,2; Li，Baopu1; Zhou，Xin1; Wang，Jiankun3; Meng，Max Q.H.3,4

10.1109/ICRA48506.2021.9561635

2021

IEEE International Conference on Robotics and Automation (ICRA)

1050-4729

2577-087X

978-1-7281-9078-5

Proceedings - IEEE International Conference on Robotics and Automation

2021-May

11088-11094

MAY 30-JUN 05, 2021

null,Xian,PEOPLES R CHINA

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Interactions with either environments or expert policies during training are needed for most of the current imitation learning (IL) algorithms. For IL problems with no interactions, a typical approach is Behavior Cloning (BC). However, BC-like methods tend to be affected by distribution shift. To mitigate this problem, we come up with a Robust Model-Based Imitation Learning (RMBIL) framework that casts imitation learning as an end-to-end differentiable nonlinear closed-loop tracking problem. RMBIL applies Neural ODE to learn a precise multi-step dynamics and a robust tracking controller via Nonlinear Dynamics Inversion (NDI) algorithm. Then, the learned NDI controller will be combined with a trajectory generator, a conditional VAE, to imitate an expert's behavior. Theoretical derivation shows that the controller network can approximate an NDI when minimizing the training loss of Neural ODE. Experiments on Mujoco tasks also demonstrate that RMBIL is competitive to the state-of-the-art generative adversarial method (GAIL) and achieves at least 30% performance gain over BC in uneven surfaces.

Training Heuristic algorithms Neural networks Performance gain Approximation algorithms Generators Control theory

其他

英语

CPCI-S ; EI

Automation & Control Systems ; Robotics

Automation & Control Systems ; Robotics

WOS:000771405403087

20220911737826

Cloning ; Computer vision ; Controllers ; Ordinary differential equations

Genetic Engineering:461.8.1 ; Computer Applications:723.5 ; Control Equipment:732.1 ; Vision:741.2 ; Calculus:921.2

2-s2.0-85125495864

Scopus

1.The Baidu Research,United States
2.ETH Zurich,Switzerland
3.The Department of Electronic and Electrical Engineering,The Southern University of Science and Technology,Shenzhen,China
4.The Department of Electronic Engineering,The Chinese University of Hong Kong,Hong Kong

Lin，Hao Chih,Li，Baopu,Zhou，Xin,et al. No Need for Interactions: Robust Model-Based Imitation Learning using Neural ODE[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2021:11088-11094.