Three-Stage Deep Reinforcement Learning for Privacy-and Safety-Aware Smart Electric Vehicle Charging Station Scheduling and Volt/VAR Control

Abstract

This paper proposes a three-stage privacy-and safety-aware deep reinforcement learning framework for coordinating smart electric vehicle charging stations (EVCSs) integrated with a photovoltaic system/energy storage system (ESS) and Volt/VAR control in a power distribution system. The proposed framework aims to maximize the EVCS profit and minimize the network real power loss while ensuring zero ESS state of charge (SOC) and voltage violation as well as preserving the privacy of the EVCS net load schedule data. In Stage 1 with 30-min resolution, each charging station operator (CSO) agent of the EVCS performs day-ahead profitable real power charging/discharging of the ESS without violating its SOC constraint via a safety layer during training. In Stage 2, using the -differential privacy method, the CSO agents encrypt the EVCS net load schedule data delivered from Stage 1. In Stage 3 with 5-min resolution, the distribution system operator agent conducts real-time reactive power charging/discharging of the ESSs to minimize the real power loss while removing voltage violations completely via iterative safe exploration of the agent with iteration penalties during training. The proposed framework was assessed on the IEEE 33-bus system for its privacy preserving and safety performances. IEEE