Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

Liu，Chuwei1; Rao，Xuan2; Zhao，Bo2; Liu，Derong3,4; Wei，Qinglai5,6; Wang，Yonghua1

2024-08-01

10.1016/j.epsr.2024.110404

Electric Power Systems Research   影响因子和分区

0378-7796

This paper aims to address the bidding strategy optimization in the real-time multi-participant electricity market with short-term load dynamics. In order to avoid the sub-optimal solution and the dependence on the complete information in traditional mathematical programming methods, an electricity market bidding strategy optimization algorithm based on deep reinforcement learning (DRL) is developed. While conventional reinforcement learning algorithms (e.g., Q-learning and deep Q-learning) are only capable of handling simple problems in discrete state spaces, the proximal policy optimization (PPO) algorithm is implemented in the bidding strategy optimization since it can optimize the bidding strategy in the continuous action and state spaces. In order to substantiate the aforementioned perspective, this paper conducts a two-part experimental study. First, experiments which consider the fixed demand load of market participants show that the developed method can reach the Nash equilibrium just like the bi-level optimization, and higher profits can be achieved by adjusting hyperparameters. Then, complex experiments which consider the time-varying demand load verify the DRL-based electricity market bidding strategy performs better than bi-level optimization-based methods and increases the profits of generators.

Bi-level optimization Deep reinforcement learning Electricity market bidding strategy Nash equilibrium Proximal policy optimization algorithm

EI

英语

其他

20242016105272

Computation theory ; Deep learning ; Electric industry ; Electric loads ; Game theory ; Learning algorithms ; Profitability ; Reinforcement learning

Ergonomics and Human Factors Engineering:461.4 ; Electric Power Systems:706.1 ; Computer Theory, Includes Formal Logic, Automata Theory, Switching Theory, Programming Theory:721.1 ; Artificial Intelligence:723.4 ; Machine Learning:723.4.2 ; Industrial Economics:911.2 ; Probability Theory:922.1

ENGINEERING

2-s2.0-85193203828

Scopus

1.School of Automation,Guangdong University of Technology,Guangzhou,510006,China
2.School of Systems Science,Beijing Normal University,Beijing,100875,China
3.School of System Design and Intelligent Manufacturing,Southern University of Science and Technology,Shenzhen,518055,China
4.Department of Electrical and Computer Engineering,University of Illinois Chicago,Chicago,60607,United States
5.The State key Laboratory for Management and Control of Complex Systems,Institute of Automation,Chinese Academy of Sciences,Beijing,100190,China
6.School of Artificial Intelligence,University of Chinese Academy of Sciences,Beijing,100049,China

Liu，Chuwei,Rao，Xuan,Zhao，Bo,et al. Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load[J]. Electric Power Systems Research,2024,233.

Liu，Chuwei,Rao，Xuan,Zhao，Bo,Liu，Derong,Wei，Qinglai,&Wang，Yonghua.(2024).Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load.Electric Power Systems Research,233.

Liu，Chuwei,et al."Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load".Electric Power Systems Research 233(2024).