Deep Reinforcement Learning Based Active Network Management and Emergency Load-Shedding Control for Power Systems

Abstract

This paper presents two novel deep reinforcement learning (DRL) approaches aimed at solving complex power system control problems in a data-driven sense to maintain the stability of power systems. Specifically, we propose, respectively, SACPER (Soft Actor-Critic (SAC) with Prioritized Experience Replay (PER)) and Constrained Variational Policy Optimization (CVPO) DRL algorithms to address the sequential decision-making problem of active network management (ANM) in distributed power systems and optimizing emergency load shedding (ELS) control problems. First, we propose SACPER for the ANM problem, which prioritizes the training of samples with large errors and poor policy performance. Evaluation of SACPER in terms of stability improvement and convergence speed shows that the ANM problem is optimized and energy loss and operational constraint violations are minimized. Next, we introduce CVPO for the ELS control problem, which is formulated as the Safe Reinforcement Learning (SRL) framework to address safety constraint prioritization issues in power systems. We consider additional voltage variables in the network as strong constraints for SRL to achieve fast voltage recovery and minimize unnecessary energy loss, while ensuring good training performance and efficiency. To demonstrate the performances of SACPER, we apply it to ANM6-Easy environment. The CVPO algorithm is applied to IEEE 39-Bus and IEEE 300-Bus systems. The simulation results of SACPER and CVPO are validated through extensive comparisons with other state-of-the-art DRL approaches.