Safety-Aware Optimal Control of Nonlinear Systems Using Off-Policy Reinforcement Learning*

Mingduo Lin1; Bo Zhao1; Hongbing Xia2; Derong Liu3

10.1109/CSIS-IAC60628.2023.10363836

2023

979-8-3503-0901-0

2023 International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC)

75-79

20-22 Oct. 2023

Shenzhen, China

In this paper, we investigate the safety-aware optimal control (SAOC) problem, which attempts to minimize a predefined performance index function while ensuring the safety of nonlinear systems. First, the barrier function-based system transformation is utilized to design an optimal control policy which maintains the system states located in the safety region. To deal with the input constraints, a non-quadratic cost function is imposed to the control input. Then, the Hamilton-Jacobi-Bellman equation is established to provide the solution of the SAOC problem. Moreover, by utilizing the off-policy Bellman equation, a data-based off-policy reinforcement learning (OPRL) algorithm is developed to obtain the safety-aware optimal controller in a model-free manner. To implement this algorithm, a data collection process with the barrier transform is executed to generate the off-policy trajectory data, and an actor-critic neural network structure with the least-square updating law is employed in the off-policy learning phase. Finally, a simulation example is provided to verify the effectiveness of the developed control method.

Safety-aware control reinforcement learning adaptive dynamic programming off-policy

其他

EI

20240415419451

Adaptive control systems ; Cost functions ; Dynamic programming ; Learning algorithms ; Learning systems ; Nonlinear systems

Artificial Intelligence:723.4 ; Machine Learning:723.4.2 ; Control Systems:731.1 ; Optimization Techniques:921.5 ; Systems Science:961

IEEE

1.School of Systems Science, Beijing Normal University, Beijing, China
2.School of Artificial Intelligence, Anhui University, Hefei, China
3.School of System Design and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China

Mingduo Lin,Bo Zhao,Hongbing Xia,et al. Safety-Aware Optimal Control of Nonlinear Systems Using Off-Policy Reinforcement Learning*[C],2023:75-79.