Reinforcement learning meets hybrid zero dynamics: A case study for RABBIT

Castillo，Guillermo A.1; Weng，Bowen1; Hereid，Ayonga2; Wang，Zheng3; Zhang，Wei1,4

10.1109/ICRA.2019.8793627

2019-05-01

1050-4729

2577-087X

978-1-5386-8176-3

Proceedings - IEEE International Conference on Robotics and Automation

2019-May

284-290

20-24 May 2019

Montreal, QC, Canada

345 E 47TH ST, NEW YORK, NY 10017 USA

Institute of Electrical and Electronics Engineers Inc.

The design of feedback controllers for bipedal robots is challenging due to the hybrid nature of its dynamics and the complexity imposed by high-dimensional bipedal models. In this paper, we present a novel approach for the design of feedback controllers using Reinforcement Learning (RL) and Hybrid Zero Dynamics (HZD). Existing RL approaches for bipedal walking are inefficient as they do not consider the underlying physics, often requires substantial training, and the resulting controller may not be applicable to real robots. HZD is a powerful tool for bipedal control with local stability guarantees of the walking limit cycles. In this paper, we propose a non traditional RL structure that embeds the HZD framework into the policy learning. More specifically, we propose to use RL to find a control policy that maps from the robot's reduced order states to a set of parameters that define the desired trajectories for the robot's joints through the virtual constraints. Then, these trajectories are tracked using an adaptive PD controller. The method results in a stable and robust control policy that is able to track variable speed within a continuous interval. Robustness of the policy is evaluated by applying external forces to the torso of the robot. The proposed RL framework is implemented and demonstrated in OpenAI Gym with the MuJoCo physics engine based on the well-known RABBIT robot model.

Legged locomotion Hip Rabbits Robot kinematics Training Computational modeling

通讯

英语

CPCI-S ; EI

National Science Foundation[CNS-1552838]

Automation & Control Systems ; Robotics

Automation & Control Systems ; Robotics

WOS:000494942300035

20193507383636

2-s2.0-85071450874

Scopus

1.Electrical and Computer EngineeringOhio State University,Columbus,United States
2.EECSUniversity of Michigan,Ann Arbor,United States
3.Department of Mechanical EngineeringUniversity of Hong Kong,Hong Kong,Hong Kong
4.SUSTech Institute of RoboticsSouthern University of Science and Technology,China

Castillo，Guillermo A.,Weng，Bowen,Hereid，Ayonga,et al. Reinforcement learning meets hybrid zero dynamics: A case study for RABBIT[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:Institute of Electrical and Electronics Engineers Inc.,2019:284-290.