LINEAR QUADRATIC OPTIMAL CONTROL PROBLEM WITH UNCERTAIN COEFFICIENT

不确定系数下的线性二次最优控制问题

陈坤祥

11849397

硕士

概率论与数理统计

熊捷

2020-05-26

2020-07-27

哈尔滨工业大学

深圳

In this paper, from the perspective of calculus, functional analysis, stochastic calculus and stochastic control theory, the LQ problem with uncertain coefficients is discussed by means of model analysis, theoretical derivation and calculation and other methods in accordance with the principle of gradual progress from easy to difficult. In this paper, from the deterministic linear quadratic optimal control model to the stochastic linear quadratic optimal control model, and then to the portfolio model whose utility is quadratic function, we solve them one by one.Firstly, this paper studies the linear quadratic optimal control model with the given initial value of $x$, the state equation of $dx_{t}=(Ax_{t}+Bu_{t}) dt$ and the cost functional of $J(A,u)= \int_{0}^{T} \frac{1}{2} (Qx_t^2+Ru_t^2+2Sx_tu_t)dt $. In this paper, it is assumed that $A$ is the uncertainty coefficient. In this case, the theoretical method of functional analysis can be used to simplify the cost functional into a tractable bivariate quadratic functional form. Then, by using the linear and positive definite properties of the functional for $u$, we get the convexity of $u$, so we can get the conclusion that the saddle point exists when $J(a, u)$ is concave. Next, we considers this model where there is saddle point that can convert the minimax problem to the maxmini problem. So, the optimal control $\bar{u}$ is solved by the traditional maximum principle. So we obtain the value function $J(A,\bar{u}) $, then maximize it with respect to $A $.Then further, this paper studies the stochastic model with Brownian motion. In this case, the state equation is $dx_{t}=(ax_{t}+u_{t})dt +cu_tdW_t$, where $a$ is the uncertainty coefficient and the cost functional is $J(a,u)=\mathbb{E} \int_{0}^{T}\frac{1}{2}(x_t^2+ru_t^2)dt$. For this model, this article also considers some solvable situations step by step. We solve the state equation in the form of the control variable $u$ and substituted into the cost functional. Then the cost functional can be represented as a quadratic functional of $u$ with the parameter $a$ by using the theoretical method of backward stochastic differential equation and functional analysis. Then, similar to the previous chapter, we find the conditions for the existence of saddle point. And then we still consider the existence of the saddle point where the minimax problem can be converted into the maxmini problem. We use the classical stochastic maximum principle firstly to solve the optimal control $\bar{u}$, and get the value function $J(a,\bar{u})$, which depends on $a$ only. We then maximize it about $a$. Subsequently,a special example without considering whether there is saddle point or not is given, and the relatively simple stochastic model is analyzed, discussed and solved.Finally, we consider applying it to an actual portfolio problem to illustrate the practical significance of our result. To achieve this goal, we also only need to consider one-dimensional variables, so this article assumes that there is only one risk-free asset and one risk asset. After derivation and calculation, we build the model that the state equation is $dx_{t}=(ax_{t}+bu_t)dt +cu_tdW_t$, and the cost functional is $J(b,u)=\mathbb{E} (\mu x^2_T -x_T)$. In this model, $b$ is the uncertainty coefficient. Different from the previous chapter, the uncertain coefficient of this chapter is on the control variable of the drift term. In order to solve the model, this paper still expresses the state variable as a function with parameters $b$ and control variable $u$, and substitute it into the cost functional. Then use the optimized theoretical method to analyze and compare the influence of parameters on the optimal solution. Finally, try to solve the model where the cost functional has only terminal utility.

本文拟从分析数学、泛函分析、随机分析及随机控制理论等相关角度出发，用模型分析，理论推导与计算等方法，本着从易到难、循序渐进的原则，对不确定系数下的LQ问题进行探讨。本文从确定性的线性二次最优控制模型到随机线性二次最优控制模型，再运用到效用为二次函数的投资组合模型，并逐一进行了求解。首先，本文研究了给定初始值为$x$的状态方程为$dx_{t}=(Ax_{t}+Bu_{t}) dt$，成本泛函为$J(A,u)= \int_{0}^{T} \frac{1}{2} (Qx_t^2+Ru_t^2+2Sx_tu_t)dt $的确定性的线性二次最优控制模型。本文假设该模型中$A$为不确定系数。这时可以利用泛函分析的理论方法将成本泛函化简成易处理的二元二次泛函形式，然后利用该泛函对$u$所具有的线性与正定性等优良性质，得到关于$u$的凸性，所以再加上$J(A,u)$关于$a$是凹的条件就可以得到鞍点存在的结论。接下来，针对该模型，本文考虑可以将minimax问题转换为maxmini问题这种鞍点存在的情况，并利用经典的最大值原理求解出最优控制$\bar{u}$，得到只关于$A$的价值函数$J(A,\bar{u})$，再对其关于$A$进行最大化即可。然后进一步地，本文研究了加入布朗运动的随机模型，其状态方程为$dx_{t}=(ax_{t}+u_{t})dt +cu_tdW_t$，其中$a$为不确定系数，成本泛函为$J(a,u)=\mathbb{E} \int_{0}^{T}\frac{1}{2}(x_t^2+ru_t^2)dt$。对于该模型，本文也按部就班地先考虑某些可解的情况。先将状态方程以带控制变量$u$的形式求解出来并代入成本泛函，此时可以利用倒向随机微分方程和泛函分析的理论方法将成本泛函表示成含参数$a$的关于$u$的二次泛函$J(a,u)$。然后类似于上一章，寻找鞍点存在的条件。另一方面，还是考虑minimax问题可以转换为maxmini问题这种鞍点存在的情况，并先用经典的随机最大值原理先求解出最优控制$\bar{u}$，得到只关于$a$的价值函数$J(a,\bar{u})$，再关于$a$对其进行最大化。随后，本文做了特殊的一个例子，对一个较为简单的且不一定存在鞍点的随机模型进行了具体的分析，讨论与求解。最后，考虑应用到一个实际的投资组合问题中，以说明该问题的实际意义。为了达到此目的，我们同样只需要在一维变量下进行，所以本文假设只有一个无风险资产和一个风险资产。经过推导计算处理，得到的模型中，状态方程为$dx_{t}=(ax_{t}+bu_t)dt +c u_tdW_t$，成本泛函为$J(b,u)=\mathbb{E} (\mu x^2_T -x_T)$，在这个模型中，$b$是不确定系数。与前一章不同，本章不确定系数在漂移项的控制变量上。为了求解该模型，本文仍然先将状态变量用含参数$b$与控制变量$u$的函数表示出来并代入成本泛函，再利用最优化的理论方法进行分析与比较参数对最优解存在的影响。最后试着对成本泛函只有终端效用的模型进行求解。

linear quadratic optimal control uncertainty coefficient saddle point maximum principle minimax problem

线性二次最优控制 不确定系数 鞍点 最大值原理 极小化极大问题

英语

联合培养

南方科技大学

Chen KX. LINEAR QUADRATIC OPTIMAL CONTROL PROBLEM WITH UNCERTAIN COEFFICIENT[D]. 深圳. 哈尔滨工业大学,2020.