Vehicle Control with Prediction Model Based Monte-Carlo Tree Search

Abstract

In this paper, we propose a model-based Monte-Carlo Tree Search (model-based MCTS) algorithm for the vehicle planning and control problem. While driving vehicles, we need to predict the future states of other vehicles to avoid collisions. However, because the movements of the vehicles are determined by the intentions of the human drivers, we need to use a prediction model which captures the intention of the human behavior. In our model-based MCTS algorithm, we introduce a neural-network-based prediction model which predicts the behaviors of the human drivers. Unlike conventional MCTS algorithms, our method estimates the rewards and Q-values based on intention-considering future states, not from the pre-defined deterministic models or self-play methods. For the evaluation, we use environments where the other vehicles follow the trajectories of pre-collected driver datasets. Our method shows novel results in the collision avoidance and success rate of the driving, compared to other reinforcement learning and imitation learning algorithms.