A Model-Based Exploration Policy in Deep Q-Network

Li，Shuailong1; Zhang，Wei1; Leng，Yuquan2; Zhang，Xin1

10.1109/DSInS54396.2021.9670573

2021

978-1-6654-0631-4

2021 International Conference on Digital Society and Intelligent Systems, DSInS 2021

336-343

3-4 Dec. 2021

Chengdu, China

Reinforcement learning has successfully been used in many applications and achieved prodigious performance (such as video games), and DQN is a well-known algorithm in RL. However, there are some disadvantages in practical applications, and the exploration and exploitation dilemma is one of them. To solve this problem, common strategies about exploration like ϵ-greedy have risen. Unfortunately, there are sample inefficient and ineffective because of the uncertainty of later exploration. In this paper, we propose a model-based exploration method that learns the state transition model to explore. Using the training rules of machine learning, we can train the state transition model networks to improve exploration efficiency and sample efficiency. We compare our algorithm with ϵ-greedy on the Deep Q-Networks (DQN) algorithm and apply it to the Atari 2600 games. Our algorithm outperforms the decaying ϵ-greedy strategy when we evaluate our algorithm across 14 Atari games in the Arcade Learning Environment (ALE).

exploration and exploitation dilemma model-based exploration method reinforcement learning

通讯

英语

EI

20220911706199

Computer aided instruction ; Efficiency

Artificial Intelligence:723.4 ; Computer Applications:723.5 ; Education:901.2 ; Production Engineering:913.1

2-s2.0-85125104191

Scopus

1.University of Chinese Academy of Sciences,Institutes for Robotics and Intelligent Manufacturing,Chinese Academy of Sciences,State Key Laboratory of Robotics,Shenyang Institute of Automation,Chinese Academy of Sciences,Shenyang,China
2.Southern University of Science and Technology,Department of Mechanical and Energy Engineering,Shenzhen,China

Li，Shuailong,Zhang，Wei,Leng，Yuquan,et al. A Model-Based Exploration Policy in Deep Q-Network[C],2021:336-343.