Surrogate-Assisted Evolutionary Q-Learning for Black-Box Dynamic Time-Linkage Optimization Problems

Zhang，Tuo1; Wang，Handing1; Yuan，Bo2; Jin，Yaochu3; Yao，Xin2

2022

10.1109/TEVC.2022.3179256

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION   影响因子和分区

1089-778X

1941-0026

Dynamic time-linkage optimization problems (DTPs) are special dynamic optimization problems (DOPs) with the time-linkage property. The environment of DTPs changes not only over time, but also depends on the previous applied solutions. DTPs are hardly solved by existing dynamic evolutionary algorithms because they ignore the time-linkage property. In fact, they can be viewed as multiple decision making problems and solved by reinforcement learning. However, only some discrete DTPs are solved by reinforcement learning based evolutionary optimization algorithms with the assumption of observable objective functions. In this work, we propose a dynamic evolutionary optimization algorithm using surrogate-assisted Q-learning for continuous black-box DTPs. To observe the states of black-box DTPs, the state extraction and prediction methods are applied after the search process at each time step. Based on the learnt information, a surrogate-assisted Q-learning is introduced to evaluate and select candidate solutions in the continuous decision space in a long-term consideration. We evaluate the components of our proposed algorithm on various benchmark problems to study their behaviors. Results of comparative experiments indicate that the proposed algorithm outperforms other compared algorithms and performs robustly on DTPs with up to 30 decision variables and different dynamic changes.

black-box problem Convergence dynamic time-linkage optimization problem Evolutionary dynamic optimization Heuristic algorithms Optimization Prediction algorithms Q-learning Sociology Statistics surrogate model.

EI

英语

其他

20222412225849

Decision making ; Evolutionary algorithms ; Learning algorithms ; Optimization ; Reinforcement learning

Computer Programming:723.1 ; Artificial Intelligence:723.4 ; Machine Learning:723.4.2 ; Management:912.2 ; Optimization Techniques:921.5

COMPUTER SCIENCE

2-s2.0-85131734849

Scopus

1.School of Artificial Intelligence and the Collaborative Innovation Center of Quantum Information of Shaanxi Province, Xidian University, Xi’an 710071, China
2.Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Southern University of Science and Technology, Shenzhen 518055, China
3.Chair of Nature Inspired Computing and Engineering, Faculty of Technology, Bielefeld University, D-33615 Bielefeld, Germany

Zhang，Tuo,Wang，Handing,Yuan，Bo,et al. Surrogate-Assisted Evolutionary Q-Learning for Black-Box Dynamic Time-Linkage Optimization Problems[J]. IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION,2022,PP(99):1-1.

Zhang，Tuo,Wang，Handing,Yuan，Bo,Jin，Yaochu,&Yao，Xin.(2022).Surrogate-Assisted Evolutionary Q-Learning for Black-Box Dynamic Time-Linkage Optimization Problems.IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION,PP(99),1-1.

Zhang，Tuo,et al."Surrogate-Assisted Evolutionary Q-Learning for Black-Box Dynamic Time-Linkage Optimization Problems".IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION PP.99(2022):1-1.