A Data-Driven Energy Management Strategy Based on Deep Reinforcement Learning for Microgrid Systems
Tóm tắt
Due to the interactions among schedulable equipment and the uncertainty of microgrid (MG) systems, it becomes increasingly difficult to establish accurate mathematical models for energy management. To improve the stability and economy of MGs, a data-driven energy management strategy must be proposed. In this paper, distributed generators (DGs) and an energy storage system (ESS) are taken as the control objects, and a data-driven energy management strategy based on prioritized experience replay soft actor-critic (PERSAC) is proposed for MGs. First, we construct an MG energy management model with the objective of minimizing the operation cost. Second, the energy management model is formulated as a Markov decision process (MDP), and the PERSAC algorithm is used to solve the MDP. Moreover, the sampling rule of the training process is optimized by using the prioritized empirical replay (PER) method. The analysis of numerical examples proves the effectiveness and practicability of the algorithm. By controlling DGs and the ESS, the operation cost of the proposed algorithm is the lowest compared with other algorithms.
Tài liệu tham khảo
Aslam S, Herodotou H, Mohsin SM. A survey on deep learning methods for power load and renewable energy forecasting in smart microgrids. Renew Sustain Energy Rev. 2021;144.
Zia MF, Elbouchikhi E, Benbouzid M. Microgrids energy management systems: A critical review on methods, solutions, and prospects. Appl Energy. 2018;222:1033–55.
Valencia F, Collado J, Sáez D. Robust energy management system for a microgrid based on a fuzzy prediction interval model. IEEE Trans Smart Grid. 2015;7(3):1486–94.
Meng T, Lin Z, Shamash YA. Distributed cooperative control of battery energy storage systems in dc microgrids. IEEE/CAA Journal of Automatica Sinica. 2021;8(3):606–16.
Cosic A, Stadler M, Mansoor M, Zellinger M. Mixed-integer linear programming based optimization strategies for renewable energy communities. Energy. 2021;237.
Vitale F, Rispoli N, Sorrentino M, Rosen M. On the use of dynamic programming for optimal energy management of gridconnected reversible solid oxide cell-based renewable microgrids. Energy. 2021;225.
Hossain MA, Pota HR, Squartini S, Zaman F. Energy scheduling of community microgrid with battery cost using particle swarm optimisation. Appl Energy. 2019;254.
Nosratabadi SM, Jahandide M, Guerrero JM. Robust scenario-based concept for stochastic energy management of an energy hub contains intelligent parking lot considering convexity principle of CHP nonlinear model with triple operational zones[J]. Sustain Cities Soc. 2021;68.
Khosravi M, Azarinfar H, Nejati AS. Microgrids energy management in automated distribution networks by considering consumers comfort index. Int J Electr Power Energy Syst. 2022;139:108013.
Velasquez MA, Gomez BJ, Quijano N, Cadena AI. Intra-hour microgrid economic dispatch based on model predictive control. IEEE Trans Smart Grid. 2020;11(3):1968–79.
Chuan S, Shan G, Yu L. A model predictive control approach in microgrid considering multi-uncertainty of electric vehicles. Renew Energy. 2021;163:1385–96.
Gan L, Zhang P, Lee J. Data-Driven Energy Management System With Gaussian Process Forecasting and MPC for Interconnected Microgrids. IEEE Trans Sustainable Energy. 2021;12(1):695–704.
Li W, Wen S, Shi K, Yang Y. Neural Architecture Search With a Lightweight Transformer for Text-to-Image Synthesis. IEEE Transactions on Network Science and Engineering. 2022;9(3):1567–76.
Lyu B, Wen S, Shi K, Huang T. Multiobjective Reinforcement Learning-Based Neural Architecture Search for Efficient Portrait Parsing. IEEE Transactions on Cybernetics. 2021;1–12.
Li S, Li W, Wen S, Shi K. Auto-FERNet: A facial expression recognition network with architecture search. IEEE Transactions on Network Science and Engineering. 2021;8(3):2213–22.
Hodge VJ, Hawkins R, Alexander R. Deep reinforcement learning for drone navigation using sensor data[J]. Neural Comput Appl. 2021;33(6):2015–33.
Chen Q, Zhao W, Li L, Wang C. ES-DQN: A Learning Method for Vehicle Intelligent Speed Control Strategy Under Uncertain Cut-In Scenario. IEEE Trans Veh Technol. 2022;71(3):2472–84.
Xu B, Zhou Q, Shi J, Li S. Hierarchical q-learning network for online simultaneous optimization of energy efficiency and battery life of the battery/ultracapacitor electric vehicle. Journal of Energy Storage. 2022;46.
Alabdullah MH, Abido MA. Microgrid energy management using deep Q-network reinforcement learning. Alex Eng J. 2022;61(11):9069–78.
Botvinick M, Wang JX, Dabney W, Miller KJ. Deep reinforcement learning and its neuroscientific implications. Neuron. 2020;107(4):603–16.
Mathew A, Jolly MJ, Mathew J. Improved residential energy management system using priority double deep q-learning. Sustain Cities Soc. 2021;69.
Du Y, Zandi H, Kotevska O, Kurte K, Munk J. Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning. Appl Energy. 2021;281.
Si C, Tao Y, Qiu J, Lai J. Deep reinforcement learning based home energy management system with devices operational dependencies. Int J Mach Learn Cybern. 2021;12(6):1687–703.
Guo C, Wang X, Zheng Y, Zhang F. Optimal energy management of multi-microgrids connected to distribution system based on deep reinforcement learning. Int J Electr Power Energy Syst. 2021;131:107048.
Li X, Ma R. Operation control strategy for energy storage station after considering battery life in commercial park. High Voltage Engineering. 2020;46(1):62–70.
Haarnoja T, Zhou A. Soft actor-critic: Offpolicy maximum entropy deep reinforcement learning with a stochastic actor. ArXiv:1801012902018.
Hasselt HV, Guez A, Silver D. Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI conference on artificial intelligence. 2016;30.
Schaul T, Quan J, Antonoglou I. Prioritized experience replay. 2015. arXiv preprint arXiv:1511.05952.
Jalilibal Z, Amiri A, Castagliola P, Khoo MB. Monitoring the coefficient of variation: A literature review. Comput Ind Eng. 2021;161:107600.