Evolutionary Large‐Scale Multi‐Objective Optimization in Deep Learning

This chapter presents two state‐of‐the‐art Evolutionary algorithms (EAs) for two typical multi‐objective optimization problems for deep learning: a gradient‐guided multi‐objective EA for training deep neural networks (DNNs) (termed GEMONN) and an action command‐based surrogate‐assisted multi‐objective EA for searching neural architectures of DNNs (termed ACEncoding). Moreover, the network sparsity is also optimized together with the training loss, which can reduce the network complexity and alleviate overfitting. Experimental results on single‐layer NNs, deep‐layer NNs, recurrent NNs, and CNNs demonstrate the effectiveness of the proposed GEMONN. In short, this work not only introduces a novel approach for training deep NNs but also enhances the performance of EAs in solving large‐scale optimization problems. For the proposed ACEncoding, a novel encoding scheme and a performance evaluator are presented for neural architecture search.