南方科技大学知识苑(SUSTech KC): Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems

题名	Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems
作者	Gao, Lucy 1; Ye, Jane J.2; Yin, Haian3 ; Zeng, Shangzhi 2; Zhang, Jin3,4
通讯作者	Zhang, Jin
发表日期	2022
会议名称	38th International Conference on Machine Learning (ICML)
ISSN	2640-3498
会议录名称	INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162
会议日期	JUL 17-23, 2022
会议地点	null,Baltimore,MD
出版地	1269 LAW ST, SAN DIEGO, CA, UNITED STATES
出版者	JMLR-JOURNAL MACHINE LEARNING RESEARCH
摘要	Gradient-based optimization methods for hyper-parameter tuning guarantee theoretical convergence to stationary solutions when for fixed upperlevel variable values, the lower level of the bilevel program is strongly convex (LLSC) and smooth (LLS). This condition is not satisfied for bilevel programs arising from tuning hyperparameters in many machine learning algorithms. In this work, we develop a sequentially convergent Value Function based Difference-of-Convex Algorithm with inexactness (VF-iDCA). We show that this algorithm achieves stationary solutions without LLSC and LLS assumptions for bilevel programs from a broad class of hyperparameter tuning applications. Our extensive experiments confirm our theoretical findings and show that the proposed VF-iDCA yields superior performance when applied to tune hyperparameters.
学校署名	通讯
语种	英语
相关链接	[来源记录]
收录类别	CPCI-S
资助项目	National Natural Science Foundation of China[11971220] ; Shenzhen Science and Technology Program[RCYX20200714114700072] ; Guangdong Basic and Applied Basic Research Foundation[2022B1515020082]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000922378802009
来源库	Web of Science
引用统计	被引频次[WOS]：0
成果类型	会议论文
条目标识符	http://sustech.caswiz.com/handle/2SGJ60CL/502107
专题	理学院_数学系
作者单位	1.Univ Waterloo, Dept Stat & Actuarial Sci, Waterloo, ON, Canada 2.Univ Victoria, Dept Math & Stat, Victoria, BC, Canada 3.Southern Univ Sci & Technol, Dept Math, SUSTech Int Natl Ctr Math, Shenzhen, Guangdong, Peoples R China 4.Natl Ctr Appl Math Shenzhen, Shenzhen, Guangdong, Peoples R China
通讯作者单位	数学系
推荐引用方式 GB/T 7714	Gao, Lucy,Ye, Jane J.,Yin, Haian,et al. Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems[C]. 1269 LAW ST, SAN DIEGO, CA, UNITED STATES:JMLR-JOURNAL MACHINE LEARNING RESEARCH,2022.