NON-LOCAL SELF-ATTENTION STRUCTURE FOR FUNCTION APPROXIMATION IN DEEP REINFORCEMENT LEARNING

Wang, Zhixiang1; Xiao, Xi1; Hu, Guangwu2; Yao, Yao1; Zhang, Dianyan1; Peng, Zhendong1; Li, Qing3; Xia, Shutao1,3

10.1109/ICASSP.2019.8682832

2019

1520-6149

978-1-4799-8132-8

2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)

3042-3046

12-17 May 2019

Brighton, UK

345 E 47TH ST, NEW YORK, NY 10017 USA

IEEE

Reinforcement learning is a framework to make sequential decisions. The combination with deep neural networks further improves the ability of this framework. Convolutional nerual networks make it possible to make sequential decisions based on raw pixels information directly and make reinforcement learning achieve satisfying performances in series of tasks. However, convolutional neural networks still have own limitations in representing geometric patterns and long-term dependencies that occur consistently in state inputs. To tackle with the limitation, we propose the self-attention architecture to augment the original network. It provides a better balance between ability to model long-range dependencies and computational efficiency. Experiments on Atari games illustrate that self-attention structure is significantly effective for function approximation in deep reinforcement learning.

reinforcement learning deep learning self-attention Atari game

其他

英语

CPCI-S

RD Program of Shenzhen[JCYJ20160531174259309] ; RD Program of Shenzhen[JCYJ20170307153032483] ; RD Program of Shenzhen[JCYJ20160331184440545] ; RD Program of Shenzhen[JCYJ20170307153157440] ; RD Program of Shenzhen[JCYJ 20170817115335418]

Acoustics ; Engineering

Acoustics ; Engineering, Electrical & Electronic

WOS:000482554003053

Web of Science

1.Tsinghua Univ, Beijing, Peoples R China
2.Shenzhen Inst Informat Technol, Sch Comp Sci, Shenzhen, Peoples R China
3.Southern Univ Sci & Technol, Pengcheng Lab, Shenzhen, Peoples R China

Wang, Zhixiang,Xiao, Xi,Hu, Guangwu,et al. NON-LOCAL SELF-ATTENTION STRUCTURE FOR FUNCTION APPROXIMATION IN DEEP REINFORCEMENT LEARNING[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA:IEEE,2019:3042-3046.