Deep Reinforcement Learning-Based Real-Time Joint Optimal Power Split for Battery-Ultracapacitor-Fuel Cell Hybrid Electric Vehicles

Abstract

Hybrid energy storage systems for hybrid electric vehicles (HEVs) consisting of multiple complementary energy sources are becoming increasingly popular as they reduce the risk of running out of electricity and increase the overall lifetime of the battery. However, designing an efficient power split optimization algorithm for HEVs is a challenging task due to their complex structure. Thus, in this paper, we propose a model that jointly learns the optimal power split for a battery/ultracapacitor/fuel cell HEV. Concerning the mechanical system of the HEV, two propulsion machines with complementary operation characteristics are employed to achieve higher efficiency. Additionally, to train and evaluate the model, standard driving cycles and real driving cycles are employed as input to the mechanical system. Then, given the inputs, a temporal attention long short-term memory model predicts the next time step velocity, and through that velocity, the predicted load power and its corresponding optimal power split is computed by a soft actor-critic deep reinforcement learning model whose training phase is aided by shaped reward functions. In contrast to global optimization techniques, the local velocity and load power prediction without future knowledge of the driving cycle is a step toward real-time optimal energy management. The experimental results show that the proposed method is robust to different initial states of charge values, better allocates the power to the energy sources and thus better manages the state of charge of the battery and the ultracapacitor. Additionally, the use of two motors significantly increases the efficiency of the system, and the prediction step is shown to be a reliable way to plan the HESS power split in advance.