Complete stability analysis of iterative adaptive critic designs with discounted cost

Liang, Zhantao1; Ha, Mingming2; Liu, Derong3,4; Wang, Yonghua1

2024-06-01

10.1007/s11071-024-09840-0

NONLINEAR DYNAMICS   影响因子和分区

0924-090X

1573-269X

In this paper, the stability of nonlinear systems under infinite-horizon discounted optimal control via adaptive dynamic programming method is analyzed. First, considering the adoption of function approximators during value iteration (VI), the iterative value function and control policy are shown to be continuous. Then, based on a verifiable condition on the approximation errors caused by the critic network, it is proved that the approximate value functions are bounded and positive definite. Further in the stability analysis, a stability condition as the termination criterion of approximate VI (AVI) is developed, which guarantees that the control policy derived from the obtained critic network makes the controlled system KL\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal{K}\mathcal{L}$$\end{document}-stable. Also, an upper bound function of the approximation errors caused by the action network is derived for ensuring that the system controlled by the trained action network remains KL\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal{K}\mathcal{L}$$\end{document}-stable. The KL\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal{K}\mathcal{L}$$\end{document}-stability of the closed-loop system is established by using the approximate value function to act as the Lyapunov function and estimate the region of attraction. Finally, the present theoretical results are applied to the simulation studies of the spacecraft rendezvous.

Adaptive dynamic programming Discounted value iteration Discrete-time nonlinear system Robust KL-stability

SCI ; EI

英语

通讯

National Natural Science Foundation of China[62073085]

Engineering ; Mechanics

Engineering, Mechanical ; Mechanics

WOS:001258049400007

SPRINGER

ENGINEERING

Web of Science

1.Guangdong Univ Technol, Sch Automat, Guangzhou 510006, Peoples R China
2.Univ Sci & Technol Beijing, Sch Intelligence Sci & Technol, Beijing 100083, Peoples R China
3.Southern Univ Sci & Technol, Sch Syst Design & Intelligent Mfg, Shenzhen 518055, Peoples R China
4.Univ Illinois, Dept Elect & Comp Engn, Chicago, IL 60607 USA

Liang, Zhantao,Ha, Mingming,Liu, Derong,et al. Complete stability analysis of iterative adaptive critic designs with discounted cost[J]. NONLINEAR DYNAMICS,2024.

Liang, Zhantao,Ha, Mingming,Liu, Derong,&Wang, Yonghua.(2024).Complete stability analysis of iterative adaptive critic designs with discounted cost.NONLINEAR DYNAMICS.

Liang, Zhantao,et al."Complete stability analysis of iterative adaptive critic designs with discounted cost".NONLINEAR DYNAMICS (2024).