Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning

Castillo, Guillermo A.; Weng, Bowen; Zhang, Wei1; Hereid, Ayonga2

10.1109/ICRA40945.2020.9197175

2020

IEEE International Conference on Robotics and Automation (ICRA)

1050-4729

2577-087X

978-1-7281-7396-2

2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)

8746-8752

MAY 31-JUN 15, 2020

null,null,ELECTR NETWORK

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

This paper presents a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from the hybrid nature of the walking dynamics and the well-established hybrid zero dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, short training time, and less dependence on prior knowledge. We demonstrate the effectiveness of the proposed method on Cassie, a challenging 3D bipedal robot. The proposed solution produces stable limit walking cycles that can track various walking speed in different directions. Surprisingly, without specifically trained with disturbances to achieve robustness, it also performs robustly against various adversarial forces applied to the torso towards both the forward and the backward directions.

Legged locomotion Three-dimensional displays Trajectory Robustness Torso Robot kinematics

通讯

英语

EI ; CPCI-S

National Science Foundation[CNS-1552838]

Automation & Control Systems ; Engineering ; Robotics

Automation & Control Systems ; Engineering, Electrical & Electronic ; Robotics

WOS:000712319505104

20204309375257

Robust control ; Biped locomotion ; Dynamics ; Robotics ; Feedback control

Biomechanics, Bionics and Biomimetics:461.3 ; Artificial Intelligence:723.4 ; Automatic Control Principles and Applications:731 ; Control Systems:731.1 ; Robotics:731.5

Web of Science

1.Southern Univ Sci & Technol, SUSTech Inst Robot, Shenzhen, Peoples R China
2.Ohio State Univ, Mech & Aerosp Engn, Columbus, OH 43210 USA

Castillo, Guillermo A.,Weng, Bowen,Zhang, Wei,et al. Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2020:8746-8752.