Multi-Task Learning via Generalized Tensor Trace Norm

Zhang，Yi1; Zhang，Yu1; Wang，Wei2

10.1145/3447548.3467329

2021-08-14

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

2254-2262

The trace norm is widely used in multi-task learning as it can discover low-rank structures among tasks in terms of model parameters. Nowadays, with the emerging of big complex datasets and the popularity of deep learning techniques, tensor trace norms have been used for deep multi-task models. However, existing tensor trace norms cannot discover all the low-rank structures and they require users to determine the importance of their components manually. To solve those two issues, in this paper, we propose a Generalized Tensor Trace Norm (GTTN). The GTTN is defined as a convex combination of matrix trace norms of all possible tensor flattenings and hence it can discover all the possible low-rank structures. Based on the induced objective function with the GTTN, we can learn combination coefficients in the GTTN with several strategies. Experiments on real-world datasets demonstrate the effectiveness of the proposed GTTN.

low-rank structure multi-task learning tensor trace norm

第一

英语

EI

20213810905656

Deep learning ; Learning systems ; Multi-task learning ; Tensors

Data Processing and Image Processing:723.2 ; Algebra:921.1

2-s2.0-85114956167

Scopus

1.Southern University of Science and Technology,Shenzhen,China
2.Nanjing University,Nanjing,China

Zhang，Yi,Zhang，Yu,Wang，Wei. Multi-Task Learning via Generalized Tensor Trace Norm[C],2021:2254-2262.