Power System Topology Control via Option-Critic Deep Reinforcement Learning

Abstract

In recent years, the integration of renewable energy sources into power systems has increased their complexity, making automated control and management more challenging. To address this issue, we propose OC-LSTM, a deep reinforcement learning (DRL) algorithm which integrates option-critic DRL with the long short-term memory (LSTM) neural network to efficiently manage power systems. The OC-LSTM algorithm extracts temporal features from the power system using the LSTM network and leverages the option-critic (OC) framework in DRL to learn policies for adjusting the system's topology, ensuring secure and efficient power transmission. Experimental results demonstrate that the OC-LSTM algorithm outperforms standard DRL algorithms during training, and ablation studies further confirm the effectiveness of LSTM in extracting power system features. Additionally, the OC-LSTM algorithm allows stable operation of the IEEE 5-Bus, IEEE 14-Bus and L2RPN WCCI 2020 power systems for 60 consecutive hours without the need for human intervention.