RLink: Accelerate On-Device Deep Reinforcement Learning with Inference Knowledge at the Edge

Tianyu Zeng1; Xiaoxi Zhang1; Daipeng Feng1; Jingpu Duan2,3; Zhi Zhou1; Xu Chen1

10.1109/MSN60784.2023.00093

2023-12-16

2994-3515

979-8-3503-5827-8

2023 19th International Conference on Mobility, Sensing and Networking (MSN)

14-16 Dec. 2023

Nanjing, China

Deep reinforcement learning (DRL) has been a successful paradigm in machine learning that enables solving complex control problems at the human level. However, the sampling and training efficiency of state-of-the-art DRL frameworks can not satisfy the stringent latency and throughput requirements of today’s mobile environments. Existing distributed and offline reinforcement learning algorithms along with the libraries for training acceleration are inherently designed for DRL tasks performed in the cloud rather than on distributed mobile devices, on which the computing resources are highly constrained, heterogeneous, and possibly dynamically changing. With the rise of edge computing and intelligence services, this paper presents RLink, a novel distributed training library to accelerate on-device deep reinforcement learning with inference knowledge at the edge. We leverage knowledge distillation to realize lightweight interaction between our on-device training task and the remote models that can provide inference knowledge. In this way, RLink is designed to be event-driven and agnostic to heterogeneous deep reinforcement learning algorithms and libraries. To tackle the communication bottleneck, a novel asynchronous sampling algorithm is proposed to facilitate real-time training in RLink. Tuned for unstable-connected mobile devices, RLink is robust and efficient by using a semantic-aware communication pipeline for lossless data compression. Extensive experimental results show that, compared with state-of-the-art algorithms and libraries, RLink can accelerate deep reinforcement learning at the edge with up to decuple speedups in convergence and ideal computational performance.

其他

1.School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
2.Institute of Future Networks, Southern University of Science and Technology, Shenzhen, China
3.Department of Communications, Peng Cheng Laboratory, Shenzhen, China

Tianyu Zeng,Xiaoxi Zhang,Daipeng Feng,et al. RLink: Accelerate On-Device Deep Reinforcement Learning with Inference Knowledge at the Edge[C],2023.