A Controllable Agent by Subgoals in Path Planning Using Goal-Conditioned Reinforcement Learning

Abstract

The aim of path planning is to search for a path from the starting point to the goal. Numerous studies, however, have dealt with a single predefined goal. That is, an agent who has completed learning cannot reach other goals that have not been visited in the training. In the present study, we propose a novel reinforcement learning (RL) framework for an agent reachable to any subgoal as well as the final goal in path planning. To do this, we utilize goal-conditioned RL and propose bidirectional memory editing to obtain various bidirectional trajectories of the agent. Bidirectional memory editing can generate various behavior and subgoals of the agent from the limited trajectory. Then, the generated subgoals and behaviors of the agent are trained on the policy network so that the agent can reach any subgoals from any starting point. In addition, we present reward shaping for the short path of the agent to reach the goal. In the experimental result, the agent was able to reach the various goals that had never been visited by the agent during the training. We confirmed that the agent could perform difficult missions, such as a round trip, and the agent used the shorter route with reward shaping.