An Improved Soft Actor-Critic-Based Energy Management Strategy of Fuel Cell Hybrid Vehicles with a Nonlinear Fuel Cell Degradation Model

Abstract

With the rapid development of artificial intelligence, deep reinforcement learning (DRL)-based energy management strategies (EMSs) have become an important research direction for hybrid electric vehicles recently, which still face some problems such as fragile convergence characteristics, slower convergence speed, and unsatisfactory optimization effects. In this research, a novel DRL algorithm, i.e. an improved soft actor-critic (ISAC) algorithm is applied to the EMS of a fuel cell hybrid vehicle (FCHV), in which the priority experience replay (PER) and emphasizing recent experience (ERE) methods are adopted to improve the convergence performance of the algorithm and to enhance the FCHV fuel economy. In addition, the fuel cell durability is also considered in the proposed EMS based on a nonlinear fuel cell degradation model while considering the fuel economy. Results indicate that the FCHV fuel consumption of the proposed EMS is decreased by 7.87%, 2.79%, and 2.44% compared to that of the deep deterministic policy gradient (DDPG)-based, the twin delayed deep deterministic policy gradient (TD3)-based, and the SAC-based EMSs respectively while the fuel consumption gap to the dynamic programming-based EMS is narrowed to 2.37% by the proposed EMS. Moreover, the proposed EMS presents the best training performance considering both the convergence speed and stability, and the convergence speed of the proposed EMS is increased by an average of 47.89% compared to that of the other DRL-based EMSs. Furthermore, the fuel cell durability is improved by more than 95% using the proposed EMS compared to that of the EMS without considering the fuel cell degradation.