Boosted kernel ridge regression: Optimal learning rates and early stopping

Lin，Shao Bo1; Lei，Yunwen2; Zhou，Ding Xuan3

2019-02-01

JOURNAL OF MACHINE LEARNING RESEARCH   影响因子和分区

1532-4435

1533-7928

In this paper, we introduce a learning algorithm, boosted kernel ridge regression (BKRR), that combines L2-Boosting with the kernel ridge regression (KRR). We analyze the learning performance of this algorithm in the framework of learning theory. We show that BKRR provides a new bias-variance trade-off via tuning the number of boosting iterations, which is different from KRR via adjusting the regularization parameter. A (semi-)exponential bias-variance trade-off is derived for BKRR, exhibiting a stable relationship between the generalization error and the number of iterations. Furthermore, an adaptive stopping rule is proposed, with which BKRR achieves the optimal learning rate without saturation.

Boosting Integral operator Kernel ridge regression Learning theory

EI

英语

其他

COMPUTER SCIENCE

2-s2.0-85072647947

Scopus

1.Department of MathematicsWenzhou University,Wenzhou,China
2.Department of Computer Science and EngineeringSouthern University of Science and Technology,Shenzhen,China
3.School of Data Science and Department of MathematicsCity University of Hong Kong,Kowloon,Hong Kong

Lin，Shao Bo,Lei，Yunwen,Zhou，Ding Xuan. Boosted kernel ridge regression: Optimal learning rates and early stopping[J]. JOURNAL OF MACHINE LEARNING RESEARCH,2019,20.

Lin，Shao Bo,Lei，Yunwen,&Zhou，Ding Xuan.(2019).Boosted kernel ridge regression: Optimal learning rates and early stopping.JOURNAL OF MACHINE LEARNING RESEARCH,20.

Lin，Shao Bo,et al."Boosted kernel ridge regression: Optimal learning rates and early stopping".JOURNAL OF MACHINE LEARNING RESEARCH 20(2019).