Federated Learning at the Edge: An Interplay of Mini-batch Size and Aggregation Frequency

Weijie Liu1; Xiaoxi Zhang1; Jingpu Duan2; Carlee Joe-Wong3; Zhi Zhou1; Xu Chen1

10.1109/INFOCOMWKSHPS57453.2023.10226109

2023

2159-4228

978-1-6654-9428-1

IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)

1-6

20-20 May 2023

Hoboken, NJ, USA

Federated Learning (FL) is a distributed learning paradigm that can coordinate heterogeneous edge devices to perform model training without sharing private raw data. Prior works on the convergence analysis of FL have focused on mini-batch size and aggregation frequency separately. However, increasing the batch size and the number of local updates can differently affect model performance and system overhead. This paper proposes a novel model in quantifying the interplay of FL mini-batch size and aggregation frequency to navigate the unique trade-offs among convergence, completion time, and resource cost. We obtain a new convergence bound for synchronous FL with respect to these decision variables under heterogeneous training datasets at different devices. Based on this bound, we derive closed-form solutions for co-optimized mini-batch size and aggregation frequency, uniformly among devices. We then design an efficient exact algorithm to optimize heterogeneous mini-batch configurations, further improving the model accuracy. An adaptive control algorithm is also proposed to dynamically adjust the batch sizes and the number of local updates per round. Extensive experiments demonstrate the superiority of our offline optimized solutions and online adaptive algorithm.

Training Performance evaluation Time-frequency analysis Costs Closed-form solutions Federated learning Heuristic algorithms

其他

EI

20233814771241

Adaptive algorithms

Social Sciences:971

IEEE

1.Sun Yat-sen University
2.Southern University of Science and Technology
3.Carnegie Mellon University

Weijie Liu,Xiaoxi Zhang,Jingpu Duan,et al. Federated Learning at the Edge: An Interplay of Mini-batch Size and Aggregation Frequency[C],2023:1-6.