A Feedback-inspired Super-network Shrinking Framework for Flexible Neural Architecture Search

Xuan Rao1; Bo Zhao2; Derong Liu3; Haowei Lin4

10.1109/YAC59482.2023.10401612

2023

2837-8598

979-8-3503-0364-3

2023 38th Youth Academic Annual Conference of Chinese Association of Automation (YAC)

1233-1238

27-29 Aug. 2023

Hefei, China

Neural architecture search (NAS) has been proved to be effective in automatically designing the architectures of deep neural networks (DNNs). Currently, most of cell-based NAS methods are subjected to the prior rules of human experts on neural architectures (e.g., the topology-sharing strategy), but it is not in line with the sustainable development of automated machine learning (Auto-ML). In this paper, a feedback-inspired super-network shrinking (FSS) method is developed under the framework of differentiable architecture search (DARTS) such that neural networks with flexible architectures are derived without strong reliance on human experts. In FSS, the information entropy is utilized to measure the sparsity of the super-network and an adaptive shrinking method is developed to balance the cross-entropy and the sparsity entropy. The experiments on CIFAR10/100 demonstrate that the proposed ESF derives a set of neural architectures with competitive performance-computation trade-offs in one single search procedure, which is superior to the state-of-the-art NAS methods.

Deep learning neural architecture search Auto-ML DARTS super-network shrinking

其他

IEEE

1.School of Automation, Guangdong University of Technology, Guangzhou, China
2.School of Systems Science, Beijing Normal University, Beijing, China
3.School of System Design and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China
4.School of Advanced Manufacturing, Guangdong University of Technology, Jieyang, China

Xuan Rao,Bo Zhao,Derong Liu,et al. A Feedback-inspired Super-network Shrinking Framework for Flexible Neural Architecture Search[C],2023:1233-1238.