ENAO: Evolutionary Neural Architecture Optimization in the Approximate Continuous Latent Space of a Deep Generative Model

Zheng Li1; Xuan Rao2; Shaojie Liu3; Bo Zhao2; Derong Liu3

10.1109/IJCNN60899.2024.10650934

2024-07-05

2161-4393

979-8-3503-5932-9

2024 International Joint Conference on Neural Networks (IJCNN)

30 June-5 July 2024

Yokohama, Japan

Neural architecture search (NAS) has emerged as a transformative approach for automating the design of neural networks, demonstrating exceptional performance across a variety of tasks. Numerous NAS methods aim to optimize neural architectures within discrete or continuous search spaces, but each method possesses its own inherent limitations. Additionally, the search efficiency is notably impeded by suboptimal encoding methods, presenting an ongoing challenge. In response to these obstacles, this paper introduces a novel approach, evolutionary neural architecture optimization (ENAO), which optimizes architectures in an approximate continuous search space. ENAO begins with training a deep generative model to embed discrete architectures into a condensed latent space, leveraging unsupervised representation learning. Subsequently, evolutionary algorithm is employed to refine neural architectures within this approximate continuous latent space. Empirical comparisons against several NAS benchmarks underscore the effectiveness of the ENAO method. Thanks to its foundation in deep unsupervised representation learning, ENAO demonstrates a distinguished ability to identify high-quality architectures with fewer evaluations and achieve state-of-the-art result in NAS-Bench-201 dataset. Overall, the ENAO method is a promising approach for optimizing neural network architectures in an approximate continuous search space with evolutionary algorithms and may be a useful tool for researchers and practitioners in the field of NAS.

第一

1.Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China
2.School of Systems Science, Beijing Normal University, Beijing, China
3.School of System Design and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China

Zheng Li,Xuan Rao,Shaojie Liu,et al. ENAO: Evolutionary Neural Architecture Optimization in the Approximate Continuous Latent Space of a Deep Generative Model[C],2024.