A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization

Liu, Risheng1,2,3; Liu, Xuan1,2; Yuan, Xiaoming4; Zeng, Shangzhi4; Zhang, Jin5,6

2021

International Conference on Machine Learning (ICML)

2640-3498

INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139

139

JUL 18-24, 2021

null,null,ELECTR NETWORK

1269 LAW ST, SAN DIEGO, CA, UNITED STATES

JMLR-JOURNAL MACHINE LEARNING RESEARCH

Bi-level optimization model is able to capture a wide range of complex learning tasks with practical interest. Due to the witnessed efficiency in solving bi-level programs, gradient-based methods have gained popularity in the machine learning community. In this work, we propose a new gradient-based solution scheme, namely, the Bilevel Value-Function-based Interior-point Method (BVFIM). Following the main idea of the log-barrier interior-point scheme, we penalize the regularized value function of the lower level problem into the upper level objective. By further solving a sequence of differentiable unconstrained approximation problems, we consequently derive a sequential programming scheme. The numerical advantage of our scheme relies on the fact that, when gradient methods are applied to solve the approximation problem, we successfully avoid computing any expensive Hessian-vector or Jacobian-vector product. We prove the convergence without requiring any convexity assumption on either the upper level or the lower level objective. Experiments demonstrate the efficiency of the proposed BVFIM on non-convex bi-level problems.

通讯

英语

CPCI-S

National Natural Science Foundation of China[61922019,61733002,11971220] ; LiaoNing Revitalization Talents Program[XLYC1807088] ; Shenzhen Science and Technology Program[RCYX20200714114700072] ; Guangdong Basic and Applied Basic Research Foundation[2019A1515011152]

Computer Science

Computer Science, Artificial Intelligence

WOS:000683104606083

Web of Science

1.Dalian Univ Technol, DUT RU Int Sch Informat Sci & Engn, Dalian, Peoples R China
2.Key Lab Ubiquitous Network & Serv Software Liaoni, Dalian, Peoples R China
3.Pazhou Lab, Guangzhou, Peoples R China
4.Univ Hong Kong, Dept Math, Hong Kong, Peoples R China
5.Southern Univ Sci & Technol, Dept Math, Shenzhen, Peoples R China
6.Natl Ctr Appl Math Shenzhen, Shenzhen, Peoples R China

Liu, Risheng,Liu, Xuan,Yuan, Xiaoming,et al. A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization[C]. 1269 LAW ST, SAN DIEGO, CA, UNITED STATES:JMLR-JOURNAL MACHINE LEARNING RESEARCH,2021.