Safe deep reinforcement learning-assisted two-stage energy management for active power distribution networks with hydrogen fueling stations

Abstract

In a power-hydrogen coupled integrated energy system (PHCIES), hydrogen fueling stations (HFSs) with solar photovoltaic (PV) systems are crucial devices to support hydrogen demand and maintain a stable and near-zero pollutant emission-based PHCIES operation by producing a pollutant-free hydrogen. However, optimal operation management of HFSs is challenging because the electrolyzers, compressor, hydrogen storage system (HSS), and fuel cell in HFSs are interconnected and change dynamically under various operational environments. To resolve this issue, this paper presents a safe deep reinforcement learning (DRL)-assisted two-stage framework that ensures a reliable and economical PHCIES operation. In the first stage, day-ahead operational schedules of stand-alone PV systems, PV-enabled HFSs, on-load tap changers, and capacitor banks with hourly resolution are coordinated to minimize the system operation cost, electricity arbitrage cost, PV curtailment cost, and real power loss by solving a Volt-VAR control (VVC) optimization problem. In the second stage, a safe DRL algorithm with a 15-min resolution is employed to minimize the real power loss and maximize the HFS profit by rescheduling the reactive power of stand-alone PV systems and real/reactive power and hydrogen of HFSs. The proposed self-tuning adaptive safety module integrated in the DRL method ensures no violations of HSS’ state-of-hydrogen (SOH) and voltage magnitude in the PHCIES during the training process. Numerical examples conducted on a PHCIES (IEEE 33-bus system coupled with two PV-enabled HFSs) demonstrate the effectiveness of the proposed framework in terms of training convergence, SOH/voltage violation, real power loss, and profit of HFSs. © 2024 Elsevier Ltd