ESSR: Evolving Sparse Sharing Representation for Multi-task Learning

Zhang，Yayu1; Qian，Yuhua1; Ma，Guoshuai1; Liang，Xinyan1; Liu，Guoqing1; Zhang，Qingfu2; Tang，Ke3

2023

10.1109/TEVC.2023.3272663

IEEE Transactions on Evolutionary Computation   影响因子和分区

1089-778X

1941-0026

Multi-task learning uses knowledge transfer among tasks to improve the generalization performance of all tasks. For deep multi-task learning, knowledge transfer is often implemented via sharing all hidden features of tasks. A major shortcoming is that it can lead to negative knowledge transfer across tasks when task correlation is weak. To overcome it, this paper proposes an evolutionary method to learn sparse sharing representations adaptively. By embedding the neural network optimization into evolutionary multitasking, our proposed method finds an optimal combination of tasks and sharing features. It can identify negative correlation and redundant features and then remove them from the hidden feature set. Thus, an optimal sparse sharing subnetwork can be produced for each task. Experiment results show that the proposed method achieve better learning performance with a smaller inference model than other related methods.

Adaptation models Correlation evolutionary multitasking optimization knowledge transfer Knowledge transfer Multi-task learning Multitasking Optimization sharing representation Task analysis Training

EI ; SCI

英语

其他

20232114132647

Deep learning ; Knowledge management ; Multitasking

Ergonomics and Human Factors Engineering:461.4 ; Digital Computers and Systems:722.4 ; Computer Applications:723.5 ; Information Retrieval and Use:903.3

COMPUTER SCIENCE

2-s2.0-85159829874

Scopus

1.Institute of Big Data Science and Industry, Shanxi University, Taiyuan, China
2.Department of Computer Science, City University of Hong Kong, Hong Kong, Hong Kong
3.the Department of Computer Science and Engineering, Shenzhen Key Laboratory of Computational Intelligence, Southern University of Science and Technology, Shenzhen, China

Zhang，Yayu,Qian，Yuhua,Ma，Guoshuai,et al. ESSR: Evolving Sparse Sharing Representation for Multi-task Learning[J]. IEEE Transactions on Evolutionary Computation,2023,PP(99):1-1.

Zhang，Yayu.,Qian，Yuhua.,Ma，Guoshuai.,Liang，Xinyan.,Liu，Guoqing.,...&Tang，Ke.(2023).ESSR: Evolving Sparse Sharing Representation for Multi-task Learning.IEEE Transactions on Evolutionary Computation,PP(99),1-1.

Zhang，Yayu,et al."ESSR: Evolving Sparse Sharing Representation for Multi-task Learning".IEEE Transactions on Evolutionary Computation PP.99(2023):1-1.