A Parallel Evolutionary Algorithm with Value Decomposition for Multi-agent Problems

Li，Gao; Duan，Qiqi; Shi，Yuhui

10.1007/978-3-030-53956-6_57

2020

0302-9743

1611-3349

LECT NOTES ARTIF INT

12145 LNCS

616-627

Many real-world problems involve cooperation and/or competition among multiple agents. These problems often can be formulated as multi-agent problems. Recently, Reinforcement Learning (RL) has made significant progress on single-agent problems. However, multi-agent problems still cannot be easily solved by traditional RL algorithms. First, the multi-agent environment is considered as a non-stationary system. Second, most multi-agent environments only provide a shared team reward as feedback. As a result, agents may not be able to learn proper cooperative or competitive behaviors by traditional RL. Our algorithm adopts Evolution Strategies (ES) for optimizing policy which is used to control agents and a value decomposition method for estimating proper fitness for each policy. Evolutionary Algorithm is considered as a promising alternative for signal-agent problems. Owing to its simplicity, scalability, and efficiency on zeroth-order optimization, EAs can even outperform RLs on some tasks. In order to solve multi-agent problems by EA, a value decomposition method is used to decompose the team reward. Our method is parallel on multiple cores, which can speed up our algorithm significantly. We test our algorithm on two benchmarking environments, and the experiment results show that our algorithm is better than traditional RL and other representative gradient-free methods.

Evolutionary algorithm Multi-agent problems Reinforcement learning Value decomposition

第一 ; 通讯

英语

EI

20203108999767

Behavioral research ; Optimization ; Evolutionary algorithms ; Learning algorithms ; Decomposition methods ; Multi agent systems

Ergonomics and Human Factors Engineering:461.4 ; Artificial Intelligence:723.4 ; Machine Learning:723.4.2 ; Optimization Techniques:921.5 ; Social Sciences:971

2-s2.0-85088746137

Scopus

Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,China

Li，Gao,Duan，Qiqi,Shi，Yuhui. A Parallel Evolutionary Algorithm with Value Decomposition for Multi-agent Problems[C],2020:616-627.