GENERATIVE MODEL DRIVEN BY WINSLOW FUNCTIONAL

基于Winslow泛函的生成模型

叶飞杨

11849183

硕士

计算数学

张振

2020

2020-07-27

哈尔滨工业大学

深圳

Probabilistic generative models, also known as generative models, are a class of models with extremely high practical application value in statistics and machine learning. It can be used to model different types of data, such as images, sounds, and text data. It can also be incorporated into reinforcement learning in several ways. so it has been widely used in data prediction, image processing, and text generation. However, how to designan effective generative model is also very challenging. The core of generative model is to estimate the target distribution parametrically. To simplify the discussion somewhat, we will focus on generative models that work via the principle of minimizing Kullback-Leibler divergence (KL). There are many kinds of generating models. In particular, there is a special generative model with explicit density function, which is based on defining continuous, nonlinear transformations between two different spaces, called flow model. In other words, this kind of model starts from a simple distribution, combines it with a transformation. This transformation warps space in complicated ways and can yield a complicated distribution. If this mapping is carefully designed, the density is tractable too. This kind of models, such as NICE and RealNvp, define a clear and manageable probability density distribution by designing a reversible encoder. However, it also has some disadvantages, such as the complex network structure, resulting in large amount of calculation and long training time. This paper also considers such a special generation model, by finding the mapping between the initial distribution and the target distribution to estimate the target probability distribution. We find that there are many similarities between finding this mapping and the moving mesh method. The moving mesh method, also called adaptive grid method, is an iterative grid redistribution method based on the variational method. It can change the grid distribution near the large change area of the solution of PDEs, and is especially effective in the process of solving PDEs with singular solutions. Such a grid movement is controlled by the Winslow functional. However, we can find that mapping the samples to the region with higher probability density is similar to the process of moving the grid point to the region with larger gradient of solution. Therefore, we can use Winslow functional to establish the relationship between these two problems, and then use this energy functional to build a new generative model. Our work provides a new direction for the research of generation model. We have successfully applied the Winslow functional used in the moving mesh algorithm to the generation model, and transformed the problem of solving the target mapping into the problem of solving the partial differential equation. We have successfully verified the effectiveness of the algorithm in some numerical experiments. In addition, we have carried out some numerical experiments on the related generative models, and give some preliminary analysis and exploration on the results.

概率生成模型，也叫作生成模型，是在机器学习和概率统计问题中的一类具有极高实际应用价值的模型。它的应用十分广泛，可以用来对不同种类的数据进行建模，比如图像，声音，文本数据，同时它能够通过多种方式融入强化学习，所以在数据预测，图片处理，文本生成等领域有广泛的作用。但是如何设计一种高效且有效的生成模型，也是非常具有挑战性的。生成模型的关键步骤就是对目标分布进行参数化估计。为了在一定程度上简化讨论，在本文中我们将主要关注通过极小化交叉熵 (KL) 的原理工作的生成模型。生成模型的种类非常多，特别的，有一类特别的具有显式密度函数的生成模型，是基于定义两个不同空间之间的连续非线性变换来构造的，称为流模型。换句话说，这类模型从一个简单的分布出发，将其与一个变换相结合，以一种复杂的方式扭曲空间，进而得到复杂的分布。如果对该映射进行控制，那么对应的概率分布也是可以控制的。这类模型比如 NICE 模型与 RealNVP 方法，通过设计一个可逆的编码器，直接定义了明确的且可处理的概率密度分布。但是这类模型也有其缺点，比如网络结构复杂，导致计算量大，训练时间长。本文同样考虑这样一种特殊的生成模型，通过构造初始分布与目标分布之间的映射，来对目标概率分布进行估计。我们发现，求解这样一种映射与求解偏微分方程中的自适应网格法有很多相似之处。自适应网格方法是一种基于变分法的迭代网格构造方法，该方法能够改变偏微分方程解的变化较大区域附近的网格分布，在求解具有奇异解的偏微分方程过程中特别有效。而这样一种网格移动，是由Winslow 能量泛函所控制的。如果我们将把样本映射到概率密度较高的区域，看成将网格点移动到解的梯度较大的区域的过程。我们就能通过 Winslow 泛函来建立这两种问题的联系，进而将这种能量泛函运用到生成模型的构建中来。本文的数值实验结果，说明了这样一种基于 Winslow 泛函的生成模型是有效的。并且不论是使用数值求解还是使用神经网络求解问题，该方法都取得了不错的效果，能够在较低次数的迭代下达到较低的交叉熵。

Generative model Winslow functional Neural network

生成模型 Winslow 泛函 神经网络

英语

联合培养

南方科技大学

Ye FY. GENERATIVE MODEL DRIVEN BY WINSLOW FUNCTIONAL[D]. 深圳. 哈尔滨工业大学,2020.