Neural Architecture Search Based on Evolutionary Algorithms with Fitness Approximation

Pan，Chao; Yao，Xin

10.1109/IJCNN52387.2021.9533986

2021-07-18

International Joint Conference on Neural Networks (IJCNN)

2161-4393

978-1-6654-4597-9

Proceedings of the International Joint Conference on Neural Networks

2021-July

1-8

JUL 18-22, 2021

null,null,ELECTR NETWORK

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Designing advanced neural architectures to tackle specific tasks involves weeks or even months of intensive investigation by experts with rich domain knowledge. In recent years, neural architecture search (NAS) has attracted the interest of many researchers due to its ability to automatically design efficient neural architectures. Among different search strategies, evolutionary algorithms have achieved significant successes as derivative-free optimization algorithms. However, the tremendous computational resource consumption of the evolutionary neural architecture search dramatically restricts its application. In this paper, we explore how fitness approximation-based evolutionary algorithms can be applied to neural architecture search and propose NAS-EA-FA to accelerate the search process. We further exploit data augmentation and diversity of neural architectures to enhance the algorithm, and present NAS-EA-FA V2. Experiments show that NAS-EA-FA V2 is at least five times faster than other state-of-the-art neural architecture search algorithms like regularized evolution and iterative neural predictor on NASBench-101, and it is also the most effective and stable algorithm on NASBench-201. All the code used in this paper is available at https://github.com/fzjcdt/NAS-EA-FA.

Diversity Evolutionary Algorithm Fitness Approximation Neural Architecture Search

第一

英语

CPCI-S ; EI

Guangdong Provincial Key Laboratory[2020B121201001]

Computer Science ; Engineering

Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Engineering, Electrical & Electronic

WOS:000722581705066

20214110995695

Approximation algorithms ; Health ; Iterative methods ; Optimization

Medicine and Pharmacology:461.6 ; Mathematics:921 ; Optimization Techniques:921.5 ; Numerical Methods:921.6

2-s2.0-85116459542

Scopus

Southern University of Science and Technology,Guangdong Key Laboratory of Brain-inspired Intelligent Computation,Research Institute of Trustworthy Autonomous Systems,Department of Computer Science and Engineering,Shenzhen,518055,China

Pan，Chao,Yao，Xin. Neural Architecture Search Based on Evolutionary Algorithms with Fitness Approximation[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2021:1-8.