Abstract
Inverter-based resources (IBRs) are becoming more prominent due to the increasing penetration of renewable energy sources that reduce power system inertia, compromising power system stability and grid support services. At present, optimal coordination among generation technologies remains a significant challenge for frequency control services. This paper presents a novel networked multi-agent deep reinforcement learning (N—MADRL) scheme for optimal dispatch and frequency control services. First, we develop a model-free environment consisting of a photovoltaic (PV) plant, a wind plant (WP), and an energy storage system (ESS) plant. The proposed framework uses a combination of multi-agent actor-critic (MAAC) and soft actor-critic (SAC) schemes for optimal dispatch of active power, mitigating frequency deviations, aiding reserve capacity management, and improving energy balancing. Second, frequency stability and optimal dispatch are formulated in the N—MADRL framework using the physical constraints under a dynamic simulation environment. Third, a decentralised coordinated control scheme is implemented in the HPP environment using communication-resilient scenarios to address system vulnerabilities. Finally, the practicality of the N—MADRL approach is demonstrated in a Grid2Op dynamic simulation environment for optimal dispatch, energy reserve management, and frequency control. Results demonstrated on the IEEE 14 bus network show that compared to PPO and DDPG, N—MADRL achieves 42.10% and 61.40% higher efficiency for optimal dispatch, along with improvements of 68.30% and 74.48% in mitigating frequency deviations, respectively. The proposed approach outperforms existing methods under partially, fully, and randomly connected scenarios by effectively handling uncertainties, system intermittency, and communication resiliency.
RAS ID
82095
Document Type
Journal Article
Date of Publication
5-1-2025
Volume
18
Issue
10
Funding Information
Edith Cowan University
School
School of Engineering
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Publisher
MDPI
Identifier
Muhammad Ikram: https://orcid.org/0000-0001-6559-9067
Daryoush Habibi: https://orcid.org/0000-0002-7662-6830
Asma Aziz: https://orcid.org/0000-0003-3538-0536
Recommended Citation
Ikram, M., Habibi, D., & Aziz, A. (2025). Networked multi-agent deep reinforcement learning framework for the provision of ancillary services in hybrid power plants. DOI: https://doi.org/10.3390/en18102666
Comments
Ikram, M., Habibi, D., & Aziz, A. (2025). Networked multi-agent deep reinforcement learning framework for the provision of ancillary services in hybrid power plants. Energies, 18(10), 2666. https://doi.org/10.3390/en18102666