LECO: Improving Early Exiting via Learned Exits and Comparison-based Exiting Mechanism

Zhang, Jingfan1; Tan, Ming2; Dai, Pengyu3,4; Zhu, Wei5

2023

61st Annual Meeting of the Association-for-Computational-Linguistics / Student Research Workshop (ACL-SRW)

PROCEEDINGS OF THE 61ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL-SRW 2023, VOL 4

JUL 10-12, 2023

null,Toronto,CANADA

209 N EIGHTH STREET, STROUDSBURG, PA 18360 USA

ASSOC COMPUTATIONAL LINGUISTICS-ACL

Recently, dynamic early exiting has attracted much attention since it can accelerate the inference speed of pre-trained models (PTMs). However, previous work on early exiting has neglected the intermediate exits' architectural designs. In this work, we propose a novel framework, Learned Exits and COmparison-based early exiting (LECO) to improve PTMs' early exiting performances. First, to fully uncover the potentials of multi-exit BERT, we design a novel search space for intermediate exits and employ the idea of differentiable neural architecture search (DNAS) to design proper exit architectures for different intermediate layers automatically. Second, we propose a simple-yet-effective comparison-based early exiting mechanism (COBEE), which can help PTMs achieve better performance and speedup tradeoffs. Extensive experiments show that our LECO achieves the SOTA performances for multi-exit BERT training and dynamic early exiting.

其他

英语

EI ; CPCI-S

Computer Science

Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods

WOS:001181053700029

Web of Science

1.University of Ottawa, Canada
2.Southern University of Science and Technology, China
3.Chongqing University of Post and Telecommunication, China
4.Brunel University, London, United Kingdom
5.East China Normal University, China

Zhang, Jingfan,Tan, Ming,Dai, Pengyu,et al. LECO: Improving Early Exiting via Learned Exits and Comparison-based Exiting Mechanism[C]. 209 N EIGHTH STREET, STROUDSBURG, PA 18360 USA:ASSOC COMPUTATIONAL LINGUISTICS-ACL,2023.