CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion

Hongxi Wang1; Haoxiang Luo1; Wei Zhang1,2; Hua Chen2,3

2024

10.1109/LRA.2024.3457379

IEEE Robotics and Automation Letters   影响因子和分区

2377-3774

Thanks to recent explosive developments of data-driven learning methodologies, reinforcement learning (RL) emerges as a promising solution to address the legged locomotion problem in robotics. In this paper, we propose CTS, a novel Concurrent Teacher-Student reinforcement learning architecture for legged locomotion over uneven terrains. Different from conventional teacher-student architecture that trains the teacher policy via RL first and then transfers the knowledge to the student policy through supervised learning, our proposed architecture trains teacher and student policy networks concurrently under the reinforcement learning paradigm. To this end, we develop a new training scheme based on a modified proximal policy gradient (PPO) method that exploits data samples collected from the interactions between both the teacher and the student policies with the environment. The effectiveness of the proposed architecture and the new training scheme is demonstrated through substantial quantitative simulation comparisons with the state-of-the-art approaches and extensive indoor and outdoor experiments with quadrupedal and point-foot bipedal robot platforms, showcasing robust and agile locomotion capability. Quantitative simulation comparisons show that our approach reduces the average velocity tracking error by up to 20% compared to the two-stage teacher-student, demonstrating significant superiority in addressing blind locomotion tasks. Videos are available at https://clearlab-sustech.github.io/concurrentTS/.

第一

1.School of System Design and Intelligent Manufacturing (SDIM), Southern University of Science and Technology, Shenzhen, China
2.LimX Dynamics, Shenzhen, China
3.Zhejiang University-University of Illinois Urbana-Champaign Institute (ZJUI), Haining, China

Hongxi Wang,Haoxiang Luo,Wei Zhang,et al. CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion[J]. IEEE Robotics and Automation Letters,2024,PP(99).

Hongxi Wang,Haoxiang Luo,Wei Zhang,&Hua Chen.(2024).CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion.IEEE Robotics and Automation Letters,PP(99).

Hongxi Wang,et al."CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion".IEEE Robotics and Automation Letters PP.99(2024).