Multi-task Additive Models for Robust Estimation and Automatic Structure Discovery

Additive models have attracted much attention for high-dimensional regressionestimation and variable selection. However, the existing models are usually lim-ited to the single-task learning framework under the mean squared error (MSE)criterion, where the utilization of variable structure depends heavily on a prioriknowledge among variables. For high-dimensional observations in real environ-ment,e.g., Coronal Mass Ejections (CMEs) data, the learning performance ofprevious methods may be degraded seriously due to the complex non-Gaussiannoise and the insufficiency of a prior knowledge on variable structure. To tacklethis problem, we propose a new class of additive models, called Multi-task Addi-tive Models (MAM), by integrating the mode-induced metric, the structure-basedregularizer, and additive hypothesis spaces into a bilevel optimization framework.Our approach does not require any priori knowledge of variable structure and suitsfor high-dimensional data with complex noise,e.g., skewed noise, heavy-tailednoise, and outliers. A smooth iterative optimization algorithm with convergenceguarantees is provided to implement MAM efficiently. Experiments on simulationsand the CMEs analysis demonstrate the competitive performance of our approachfor robust estimation and automatic structure discovery.