Dynamic customer demand management: A reinforcement learning model based on real-time pricing and incentives

Renewable Energy Focus - Tập 46 - Trang 39-56 - 2023
Eduardo J. Salazar1, Mauricio E. Samper1, H. Daniel Patiño2
1Institute of Electrical Energy (IEE), National University of San Juan (UNSJ), and National Scientific and Technical Research Council (CONICET), San Juan, Argentina
2Institute of Automatics, Faculty of Engineering, National University of San Juan (UNSJ), San Juan, Argentina

Tài liệu tham khảo

Stoll, 2014, Including dynamic co2 intensity with demand response, Energy Policy, 65, 490, 10.1016/j.enpol.2013.10.044 Mohsenian-Rad, 2010, Optimal residential load control with price prediction in real-time electricity pricing environments, IEEE Trans. Smart Grid, 1, 120, 10.1109/TSG.2010.2055903 Garg, 2011, Economic and environmental implications of demand-side management options, Energy Policy, 39, 3076, 10.1016/j.enpol.2011.02.009 Yu, 2016, Supply – demand balancing for power management in smart grid: A stackelberg game approach, Appl. Energy, 164, 702, 10.1016/j.apenergy.2015.12.039 Deng, 2015, Fast distributed demand response with spatially and temporally coupled constraints in smart grid, IEEE Trans. Industr. Inf., 11, 1597, 10.1109/TII.2015.2408455 Valdes, 2019, Industry, flexibility, and demand response: Applying german energy transition lessons in chile, Energy Research and Social, Science, 54, 12 Aalami, 2010, Demand response modeling considering interruptible/ curtailable loads and capacity market programs, Appl. Energy, 87, 243, 10.1016/j.apenergy.2009.05.041 Good, 2017, Review and classification of barriers and enablers of demand response in the smart grid, Renew. Sustain. Energy Rev., 72, 57, 10.1016/j.rser.2017.01.043 Ji, 2022, Bidding strategy of two-layer optimization model for electricity market considering renewable energy based on deep reinforcement learning, Electronics, 11, 10.3390/electronics11193107 Singh, 2019, Simplified algorithm for dynamic demand response in smart homes under smart grid environment, IEEE PES GTD Grand International Conference and Exposition Asia (GTD Asia), 2019, 259, 10.1109/GTDAsia.2019.8715935 Annala, 2018, Regulation as an enabler of demand response in electricity markets and power systems, J. Clean. Prod., 195, 1139, 10.1016/j.jclepro.2018.05.276 B. Liu, Scheduling strategies of smart community with load aggregator-based demand response, in: 2018 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2), 2018, pp. 1–4. doi:10.1109/EI2.2018.8582656. Aaltonen, 2021, A simulation environment for training a reinforcement learning agent trading a battery storage, Energies, 14, 10.3390/en14175587 Lu, 2021, Data-driven real-time price-based demand response for industrial facilities energy management, Appl. Energy, 283, 116291, 10.1016/j.apenergy.2020.116291 Lu, 2020, Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management, Appl. Energy, 276, 115473, 10.1016/j.apenergy.2020.115473 Lu, 2019, Incentive-based demand response for smart grid with reinforcement learning and deep neural network, Appl. Energy, 236, 937, 10.1016/j.apenergy.2018.12.061 Lu, 2018, A dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach, Appl. Energy, 220, 220, 10.1016/j.apenergy.2018.03.072 Vázquez-Canteli, 2019, Reinforcement learning for demand response: A review of algorithms and modeling techniques, Appl. Energy, 235, 1072, 10.1016/j.apenergy.2018.11.002 Nakabi, 2021, Deep reinforcement learning for energy management in a microgrid with flexible demand, Sustainable Energy, Grids and Networks, 25, 100413, 10.1016/j.segan.2020.100413 Wang, 2016, Renewable energy-aware demand response for distributed data centers in smart grid, 2016 IEEE Green Energy and Systems Conference, IGSEC, 2016 Nandkeolyar, 2018, Management of time-flexible demand to provide power system frequency response, Technol. Smart-City Energy Secur. Power (ICSESP), 2018, 1 Fleschutz, 2021, The effect of price-based demand response on carbon emissions in european electricity markets: The importance of adequate carbon prices, Appl. Energy, 295, 117040, 10.1016/j.apenergy.2021.117040 N. Hajibandeh, M. Ehsan, S. Soleymani, M. Shafie-khah, J.P.S. Catalão, Modeling price- and incentive-based demand response strategies in the renewable-based energy markets, in: 2017 IEEE International Conference on Environment and Electrical Engineering and 2017 IEEE Industrial and Commercial Power Systems Europe, 2017, pp. 1–5. doi:10.1109/EEEIC.2017.7977701. S. Caron, G. Kesidis, Incentive-based energy consumption scheduling algorithms for the smart grid, 2010 First IEEE International Conference on Smart Grid Communications (2010) 391–396. doi:10.1109/smartgrid.2010.5622073. Chai, 2014, Demand response management with multiple utility companies: A two-level game approach, IEEE Trans. Smart Grid, 5, 722, 10.1109/TSG.2013.2295024 Noppakant, 2022, Improving energy management through demand response programs for low-rise university buildings, Sustainability, 14, 10.3390/su142114233 Ye, 2021, Real-time autonomous residential demand response management based on twin delayed deep deterministic policy gradient learning, Energies, 14, 10.3390/en14030531 Charoen, 2022, A demand response implementation with building energy management system, Energies, 15, 10.3390/en15031220 Xu, 2023, Real-time multi-energy demand response for high-renewable buildings, Energy Build., 281, 112764, 10.1016/j.enbuild.2022.112764 Dewangan, 2022, Peak-to-average ratio incentive scheme to tackle the peak-rebound challenge in tou pricing, Electric Power Syst. Res., 210, 108048, 10.1016/j.epsr.2022.108048 Sediqi, 2022, Impact of time-of-use demand response program on optimal operation of afghanistan real power system, Energies, 15, 10.3390/en15010296 Wang, 2022, How social learning affects customer behavior under the implementation of tou in the electricity retailing market, Energy Econ., 106, 105836, 10.1016/j.eneco.2022.105836 Khalid, 2019, Enhanced time-of-use electricity price rate using game theory, Electronics, 8, 10.3390/electronics8010048 Xu, 2021, A hybrid demand response mechanism based on real-time incentive and real-time pricing, Energy, 231, 10.1016/j.energy.2021.120940 Zhang, 2021, Testbed implementation of reinforcement learning-based demand response energy management system, Appl. Energy, 297, 117131, 10.1016/j.apenergy.2021.117131 Kong, 2020, Online pricing of demand response based on long short-term memory and reinforcement learning, Appl. Energy, 271, 114945, 10.1016/j.apenergy.2020.114945 Azuatalam, 2020, Reinforcement learning for whole-building hvac control and demand response, Energy and AI, 2, 10.1016/j.egyai.2020.100020 Peirelinck, 2021, Transfer learning in demand response: A review of algorithms for data-efficient modelling and control, Energy and AI, 100126 Jang, 2023, Deep reinforcement learning with planning guardrails for building energy demand response, Energy and AI, 11, 100204, 10.1016/j.egyai.2022.100204 Zhong, 2021, Deep reinforcement learning framework for dynamic pricing demand response of regenerative electric heating, Appl. Energy, 288, 116623, 10.1016/j.apenergy.2021.116623 Yu, 2017, Incentive-based demand response considering hierarchical electricity market: A stackelberg game approach, Appl. Energy, 203, 267, 10.1016/j.apenergy.2017.06.010 Yang, 2013, A game-theoretic approach for optimal time-of-use electricity pricing, IEEE Trans. Power Syst., 28, 884, 10.1109/TPWRS.2012.2207134 Fan, 2022, A deep reinforcement learning-based method for predictive management of demand response in natural gas pipeline networks, J. Clean. Prod., 335, 130274, 10.1016/j.jclepro.2021.130274 Lee, 2019, Reinforcement learning-based energy management of smart home with rooftop solar photovoltaic system, energy storage system, and home appliances, Sensors, 19, 10.3390/s19183937 Gholizadeh, 2021, Distributed learning applications in power systems: A review of methods, gaps, and challenges, Energies, 14, 10.3390/en14123654 Damjanovic, 2022, Deep reinforcement learning-based approach for autonomous power flow control using only topology changes, Energies, 15, 10.3390/en15196920 Viziteu, 2022, Smart scheduling of electric vehicles based on reinforcement learning, Sensors, 22, 10.3390/s22103718 Kim, 2022, Deep reinforcement learning-based real-time joint optimal power split for battery–ultracapacitor–fuel cell hybrid electric vehicles, Electronics, 11, 10.3390/electronics11121850 Barth, 2022, Distributed reinforcement learning for the management of a smart grid interconnecting independent prosumers, Energies, 15, 10.3390/en15041440 Kazimierski, 2021, Desarrollo fotovoltaico en san juan: un acercamiento al entramado de estrategias públicas para la transición energética, Ciencia, Docencia y Tecnología, 32 Samper, 2021, Grid parity analysis of distributed pv generation considering tariff policies in argentina, Energy Policy, 157, 112519, 10.1016/j.enpol.2021.112519 Naghizadeh, 2020, Condensed silhouette: an optimized filtering process for cluster selection in k-means, Proc. Comput. Sci., 176, 205, 10.1016/j.procs.2020.08.022