基于改进SAC算法的城轨列车混合储能系统动态功率分配策略

doi:10.3969/j.issn.1008-0198.2024.04.002

湖南电力 ›› 2024, Vol. 44 ›› Issue (4): 11-19.doi: 10.3969/j.issn.1008-0198.2024.04.002

• 特约专栏：新能源发电与储能技术 • 上一篇下一篇

基于改进SAC算法的城轨列车混合储能系统动态功率分配策略

贺庆辰, 秦斌

湖南工业大学电气与信息工程学院,湖南株洲 412007

收稿日期:2024-03-08 修回日期:2024-07-02 出版日期:2024-08-25 发布日期:2024-09-09
作者简介:贺庆辰(2000),男,硕士研究生,主要研究方向为城轨混合储能控制。秦斌(1963),男,教授,主要从事复杂电气系统建模与优化控制、风力发电智能控制、永磁牵引电机再生制动能量利用、微电网与混合动力汽车能量管理、信号处理与人工智能应用等方面的研究。
基金资助:
湖南省自然科学基金项目(2022JJ50074)

Dynamic Power Allocation Strategy for Hybrid Energy Storage System of Urban Rail Trains Based on Improved SAC Algorithm

HE Qingchen, QIN Bin

School of Electrical and Information Engineering, Hunan University of Technology, Zhuzhou 412007, China

Received:2024-03-08 Revised:2024-07-02 Online:2024-08-25 Published:2024-09-09

摘要/Abstract

摘要： 为平抑城轨列车牵引网电压波动,在采用车载超级电容和地面混合储能系统的基础上,提出一种基于强化学习的柔性动作-评价算法(soft actor-critic, SAC)的功率动态分配策略,用于提高直流牵引网节能稳压特性、实现车载超级电容寿命保护。首先建立城轨列车动力学模型,并针对SAC算法在城轨动态功率分配中训练时间长、收敛慢等问题,提出PEC-SAC算法。该算法结合了优先级经验回放、强调最近经验和余弦退火算法,通过增加最近经验的采样概率和动态调整学习率,提高了训练效率和收敛速度。根据目标设置状态空间、动作空间及奖励函数,使列车在与仿真环境交互中学习到混合储能系统最优能量控制策略。通过MATLAB/Simulink与Python联合搭建仿真平台,仿真结果表明,与SAC算法相比该方法稳压提高了0.36%,能耗降低了4.52%。

关键词: 城轨列车, 再生制动能量, 混合储能系统, 功率动态分配, 深度强化学习

Abstract: To smooth out voltage fluctuations in the traction network of urban rail trains, a power dynamic allocation strategy based on a soft actor-critic (SAC) with reinforcement learning is proposed based on the use of on-board supercapacitor and ground hybrid energy storage system. It is used to improve the energy-saving voltage stabilization characteristics of DC traction network and realize the life protection of on-board supercapacitor. Firstly, an urban rail train dynamics model is established, and the PEC-SAC algorithm is proposed to address the problems of long training time and slow convergence of SAC algorithm in urban rail dynamic power allocation. The algorithm combines prioritized experience replay, emphasizing recent experience and cosine annealing, and improves the learning rate by increasing the sampling probability of the recent experience and dynamically adjusting the learning rate, which improves the training efficiency and convergence speed. Then the state space, action space, and reward function are set up to realize that the train learns the optimal energy control strategy for the hybrid energy storage system in interaction with the simulation environment. The simulation platform is built through the joint simulation of MATLAB/Simulink and PYTHON, and the results show that the method improves the voltage stabilization by 0.36% and reduces the energy consumption by 4.52% compared to the SAC algorithm.

Key words: urban rail train, regenerative braking energy, hybrid energy storage system, power dynamic allocation, deep reinforcement learning

中图分类号:

TM911

贺庆辰, 秦斌. 基于改进SAC算法的城轨列车混合储能系统动态功率分配策略[J]. 湖南电力, 2024, 44(4): 11-19.

HE Qingchen, QIN Bin. Dynamic Power Allocation Strategy for Hybrid Energy Storage System of Urban Rail Trains Based on Improved SAC Algorithm[J]. Hunan Electric Power, 2024, 44(4): 11-19.

参考文献

[1] 周浪雅,王亦乐,谢余晨,等.站城融合背景下高速铁路综合枢纽短时客流预测研究[J].铁道学报,2023,45(4):1-7.
[2] QI L F,PAN H Y,PAN Y J,et al.A review of vibration energy harvesting in rail transportation field[J]. iScience, 2022,25(3):103849.
[3] 侯秀芳,冯晨,燕汉民,等.2023年中国内地城市轨道交通运营线路概况[J].都市快轨交通,2024, 37(1):10-16.
[4] 李志强,胡海涛,陈俊宇,等.城轨交通混合型再生制动能量利用系统及其控制策略[J].电力自动化设备,2023,43(11):1-14.
[5] 沈小军,曹戈.城轨交通制动能量回收超级电容储能阵列配置方法对比分析[J].电工技术学报, 2020,35(23):4988-4997.
[6] 郑文奇.城市轨道交通车载混合储能系统的控制策略及容量优化配置研究[D].南昌:华东交通大学,2020.
[7] KHODAPARASTAN M, MOHAMED A A, BRANDAUER W. Recuperation of regenerative braking energy in electric rail transit systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(8): 2831-2847.
[8] 章宝歌,李萍,张振,等.应用于城轨列车混合储能系统的能量管理策略[J].储能科学与技术,2020, 9(1):204-210.
[9] 朱志强,王欣,秦斌.城市轨道交通供电-牵引-混合储能系统设计与仿真[J].湖南电力,2024, 44(1):24-31.
[10] WANG X,LUO Y B,QIN B, et al. Power dynamic allocation strategy for urban rail hybrid energy storage system based on iterative learning control [J]. Energy, 2022, 245: 123263.
[11] 杨浩丰,刘冲,李彬,等.基于列车运行工况的城轨地面式混合储能系统控制策略研究[J].电工技术学报,2021,36(S1):168-178.
[12] LANCTOT M, LOCKHART E, LESPIAU J B, et al. Openspiel: a framework for reinforcement learning in games[EB/OL].[ 2020-09-29].https://arxiv.org/abs/1908.09453.
[13] LEVINE S, FINN C, DARRELL T, et al. End-to-end training of deep visuomotor policies[J]. The Journal of Machine Learning Research, 2016, 17(1): 1334-1373.
[14] YANG Z, ZHU F, LIN F. Deep-reinforcement-learning-based energy management strategy for supercapacitor energy storage systems in urban rail transit [J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(2): 1150-1160.
[15] ABDELHEDI R,LAHYANI A,AMMARI A C,et al.Reinforcement learning-based power sharing between batteries and supercapacitors in electric vehicles[C] //2018 IEEE International Conference on Industrial Technology (ICIT).Lyon,France.IEEE,2018:2072-2077.
[16] FUJIMOTO S,VAN HOOF H,MEGER D.Addressing function approximation error in actor-critic methods [C] //35th International Conference on Machine Learning,Stockholm,Sweden.PMLR,2018,80:1587-1596 .
[17] WANG X,LUO Y,QIN B,et al.Power allocation strategy for urban rail hess based on deep reinforcement learning sequential decision optimization[J]. IEEE Transactions on Transportation Electrification, 2023, 9(2): 2693-2710.
[18] HAARNOJA T,ZHOU A,HARTIKAINEN K,et al.Soft actor-critic algorithms and applications[EB/OL].[2019-01-29].https://arxiv.org/pdf/1812.05905/1000.
[19] HAARNOJA T,ZHOU A,ABBEEL P,et al.Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor[C] //35th International Conference on Machine Learning,Stockholm,Sweden.PMLR,2018,80:1861-1870.

基于改进SAC算法的城轨列车混合储能系统动态功率分配策略

Dynamic Power Allocation Strategy for Hybrid Energy Storage System of Urban Rail Trains Based on Improved SAC Algorithm

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价

[1]	朱志强, 王欣, 秦斌. 城市轨道交通供电-牵引-混合储能系统设计与仿真[J]. 湖南电力, 2024, 44(1): 24-31.
[2]	吴晓刚, 季青锋, 张有鑫, 刘林萍, 陈楠, 叶杰阳. 基于随机模型预测控制的电氢混合储能微电网弹性调度[J]. 湖南电力, 2023, 43(6): 116-123.
[3]	王淑强, 刘世件, 黄超, 梁家豪. 火电多市场联合运行策略研究[J]. 湖南电力, 2022, 42(6): 76-82.