Multi-task additive models for robust estimation and automatic structure discovery

Wang，Yingjie1; Chen，Hong2; Zheng，Feng3; Xu，Chen4; Gong，Tieliang4,5; Chen，Yanhong6

2020

1049-5258

Advances in Neural Information Processing Systems

2020-December

Additive models have attracted much attention for high-dimensional regression estimation and variable selection. However, the existing models are usually limited to the single-task learning framework under the mean squared error (MSE) criterion, where the utilization of variable structure depends heavily on a priori knowledge among variables. For high-dimensional observations in real environment, e.g., Coronal Mass Ejections (CMEs) data, the learning performance of previous methods may be degraded seriously due to the complex non-Gaussian noise and the insufficiency of a prior knowledge on variable structure. To tackle this problem, we propose a new class of additive models, called Multi-task Additive Models (MAM), by integrating the mode-induced metric, the structure-based regularizer, and additive hypothesis spaces into a bilevel optimization framework. Our approach does not require any priori knowledge of variable structure and suits for high-dimensional data with complex noise, e.g., skewed noise, heavy-tailed noise, and outliers. A smooth iterative optimization algorithm with convergence guarantees is provided to implement MAM efficiently. Experiments on simulations and the CMEs analysis demonstrate the competitive performance of our approach for robust estimation and automatic structure discovery.

其他

英语

EI

20212610554791

Additives ; Clustering algorithms ; Gaussian noise (electronic) ; Iterative methods ; Mean square error

Chemical Agents and Basic Industrial Chemicals:803 ; Information Sources and Analysis:903.1 ; Numerical Methods:921.6 ; Mathematical Statistics:922.2

2-s2.0-85101345888

Scopus

1.College of Informatics,Huazhong Agricultural University,China
2.College of Science,Huazhong Agricultural University,China
3.Department of Computer Science and Engineering,Southern University of Science and Technology,China
4.Department of Mathematics and Statistics,University of Ottawa,Canada
5.School of Computer Science and Technology,Xi’an Jiaotong University,China
6.National Space Science Center,Chinese Academy of Sciences,China

Wang，Yingjie,Chen，Hong,Zheng，Feng,et al. Multi-task additive models for robust estimation and automatic structure discovery[C],2020.