Train a real-world local path planner in one hour via partially decoupled reinforcement learning and vectorized diversity

Abstract

Deep Reinforcement Learning (DRL) has exhibited efficacy in resolving the Local Path Planning (LPP) problem. However, its practical application remains significantly constrained due to its limited training efficiency and generalization capability. To address these challenges, we propose a solution termed Color, which includes an Actor-Sharer-Learner (ASL) training framework designed to improve efficiency, and a fast yet diverse simulator named Sparrow aimed at elevating both efficiency and generalization. Specifically, the ASL employs a Vectorized Data Collection (VDC) mode to enhance data collection, decouples the model optimization from data collection to expedite data consumption, and partially connects the two procedures with a Time Feedback Mechanism (TFM) to evade data underuse or overuse. Meanwhile, the Sparrow simulator utilizes a 2-Dimensional (2D) grid-based world, simplified kinematics, matrix operation, and conversion-free data flow to achieve a lightweight design. The lightness facilitates vectorized diversity, allowing for rapid and diversified simulation across numerous copies of the vectorized environments, thereby significantly enhancing both efficiency and generalization capacity. Comprehensive experiments demonstrate that with merely one hour of simulation training, Color achieves impressive arrival rates of 84% and 90% on 32 simulated and 42 real-world LPP scenarios, respectively. The code and video of this paper are accessible on our website.