Improved and Stable Data-Based Tracking Control via Approximate Iterative Q-Learning

Zhantao Liang1; Weiming Liu2; Mingming Ha3; Derong Liu2

10.1109/ICCSSE59359.2023.10245683

2023

2832-188X

979-8-3503-3906-2

2023 9th International Conference on Control Science and Systems Engineering (ICCSSE)

275-280

16-18 June 2023

Shenzhen, China

In this paper, to avoid computing the feedward control input and to completely eliminate the tracking error, an improved cost function with respect to the tracking error is proposed, under which the optimal tracking controller can stabilize the tracking error system. Then, the approximate value-iteration-based Q-learning is given to solve the neuro-optimal control policy. Also, the finite error bound of the approximate Q-function is established. In addition, for guaranteeing the admissibility of the obtained control policies, some stability conditions for this data-based iterative Q-learning are developed. When the stability conditions are satisfied, it is ensured that the iterative control policies derived from the critic network and the trained action network with approximation errors will stabilize the tracking error system. Finally, a simulation example is employed to implement the data-based Q-learning algorithm and verify the present theoretical results.

Adaptive dynamic programming discrete-time nonlinear systems value-iteration-based Q-learning data-based tracking control stability analysis

其他

EI

20234214906875

Adaptive control systems ; Discrete time control systems ; Dynamic programming ; Errors ; Iterative methods ; Learning algorithms ; Navigation ; Reinforcement learning ; System stability

Artificial Intelligence:723.4 ; Machine Learning:723.4.2 ; Control Systems:731.1 ; Optimization Techniques:921.5 ; Numerical Methods:921.6 ; Systems Science:961

IEEE

1.School of Automation Guangdong University of Technology, Guangzhou, China
2.School of System Design and Intelligent Manufacturing Southern University of Science and Technology, Shenzhen, China
3.School of Intelligence Science and Technology and Institute of Artificial Intelligence University of Science and Technology Beijing, Beijing, China

Zhantao Liang,Weiming Liu,Mingming Ha,et al. Improved and Stable Data-Based Tracking Control via Approximate Iterative Q-Learning[C],2023:275-280.