Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robot

Castillo, Guillermo A.1; Weng, Bowen1; Zhang, Wei2; Hereid, Ayonga3

10.1109/IROS51168.2021.9636467

2021

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2153-0858

978-1-6654-1715-0

2021 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)

5136-5143

SEP 27-OCT 01, 2021

null,null,ELECTR NETWORK

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

In this paper, a hierarchical and robust framework for learning bipedal locomotion is presented and successfully implemented on the 3D biped robot Digit built by Agility Robotics. We propose a cascade-structure controller that combines the learning process with intuitive feedback regulations. This design allows the framework to realize robust and stable walking with a reduced-dimensional state and action spaces of the policy, significantly simplifying the design and increasing the sampling efficiency of the learning method. The inclusion of feedback regulation into the framework improves the robustness of the learned walking gait and ensures the success of the sim-to-real transfer of the proposed controller with minimal tuning. We specifically present a learning pipeline that considers hardware-feasible initial poses of the robot within the learning process to ensure the initial state of the learning is replicated as close as possible to the initial state of the robot in hardware experiments. Finally, we demonstrate the feasibility of our method by successfully transferring the learned policy in simulation to the Digit robot hardware, realizing sustained walking gaits under external force disturbances and challenging terrains not incurred during the training process. To the best of our knowledge, this is the first time a learning-based policy is transferred successfully to the Digit robot in hardware experiments.

Legged locomotion Training Three-dimensional displays Trajectory planning Robot sensing systems Hardware Regulation

其他

英语

CPCI-S ; EI

National Natural Science Foundation of China[62073159]

Automation & Control Systems ; Computer Science ; Engineering ; Robotics

Automation & Control Systems ; Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Robotics

WOS:000755125504013

20220711623748

Biped locomotion ; Machine design ; Process control ; Reinforcement learning ; Robotics

Biomechanics, Bionics and Biomimetics:461.3 ; Mechanical Design:601 ; Artificial Intelligence:723.4 ; Control Systems:731.1 ; Robotics:731.5

Web of Science

1.Ohio State Univ, Elect & Comp Engn, Columbus, OH 43210 USA
2.Southern Univ Sci & Technol SUSTech, SUSTech Inst Robot, Shenzhen, Peoples R China
3.Ohio State Univ, Mech & Aerosp Engn, Columbus, OH 43210 USA

Castillo, Guillermo A.,Weng, Bowen,Zhang, Wei,et al. Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robot[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2021:5136-5143.