A Budget-aware Incentive Mechanism for Vehicle-to-Grid via Reinforcement Learning

Tianxiang Zhu1; Xiaoxi Zhang1; Jingpu Duan2; Zhi Zhou1; Xu Chen1

10.1109/IWQoS57198.2023.10188695

2023

1548-615X

979-8-3503-9974-5

2023 IEEE/ACM 31st International Symposium on Quality of Service (IWQoS)

2023-June

1-10

19-21 June 2023

Orlando, FL, USA

With the increasing penetration of renewable energy and electric vehicles (EVs), the behavior of EVs' charging and discharging has shown great impact on the Micro Grid power load, motivating the development of Vehicle-to-Grid (V2G) technologies. However, the V2G market is still in its infancy, due to insufficient understanding of EV users' willingness and concerns. While many studies consider direct EV control, it's more realistic to indirectly affect users' behavior through monetary incentives. For better implementation flexibility, we advocate to display at charging piles strategically chosen incentives that are combined with electricity prices. Technically, this is the first model-free learning algorithm that can optimize incentives under unknown EV user reactions, increase the load control effectiveness and users' quality-of-service (QoS) simultaneously under a long-term incentive budget, and provide theoretical performance guarantees. We first construct a bi-level optimization framework to model the time-dependencies across our solutions. We then integrate primal-dual theories and upper-confidence bounds into reinforcement learning to balance power control and incentive consumption. A dynamic programming based algorithm is also proposed to maximize the aggregate user QoS. Finally, we prove bounded sub-optimality of our learning algorithm through theoretical analysis and conduct trace-driven simulations to demonstrate the advantages of our bi-level framework.

Vehicle-to-grid Renewable energy sources Q-learning Heuristic algorithms Simulation Quality of service Behavioral sciences

其他

EI

20233314568247

Budget control ; Dynamic programming ; Electric loads ; Learning algorithms ; Power control ; Power quality ; Quality control ; Quality of service ; Vehicle-to-grid ; Vehicles

Electric Power Systems:706.1 ; Electric Power Distribution:706.1.2 ; Artificial Intelligence:723.4 ; Machine Learning:723.4.2 ; Specific Variables Control:731.3 ; Quality Assurance and Control:913.3 ; Optimization Techniques:921.5

IEEE

1.Sun Yat-sen University, Guangzhou, China
2.Southern University of Science and Technology, Shenzhen, China

Tianxiang Zhu,Xiaoxi Zhang,Jingpu Duan,et al. A Budget-aware Incentive Mechanism for Vehicle-to-Grid via Reinforcement Learning[C],2023:1-10.