GOME-NGU: visual navigation under sparse reward via Goal-Oriented Memory Encoder with Never Give Up

Abstract

In this paper, we propose the Goal-Oriented Memory Encoder (GOME) with Never Give Up (NGU) algorithm to enhance visual navigation in sparse-reward environments. Our approach addresses the critical need for efficient exploration and exploitation when dealing with high-dimensional visual data and limited spatial information. In our proposed GOME-NGU, we first utilize NGU for sufficient exploration, and then apply GOME to effectively leverage the information acquired during the exploration phase for exploitation. GOME stores information about goals and their associated rewards obtained during the exploration phase as pairs. During the exploitation phase, the stored goals are prioritized based on the current location of the agent and arranged in order of proximity to enable the agent to act optimally. To demonstrate the efficiency of the proposed GOME-NGU, two experimental categories were considered. Specifically, (i) to validate the sufficiency of exploration using NGU, we measured the states of the environment and recorded the number of times the agent visited each state, and (ii) to confirm that GOME optimizes the paths to the goals nearest the location of the agent during the exploitation phase, we measured the frequency of the agent reaching nearby goals at least twice. For both (i) and (ii), training was conducted using NVIDIA Isaac Gym, and validation was performed by migrating to Isaac Sim. In addition, for (ii), we validated the efficiency of the proposed GOME-NGU using a Husky robot in a real-world setting. Based on the experimental results, the proposed GOME-NGU demonstrated enhanced performance in both exploration and exploitation in sparse-reward environments.