Deep Reinforcement Learning Multi-UAV Trajectory Control for Target Tracking

Abstract

In this article, we propose a novel deep reinforcement learning (DRL) approach for controlling multiple unmanned aerial vehicles (UAVs) with the ultimate purpose of tracking multiple first responders (FRs) in challenging 3-D environments in the presence of obstacles and occlusions. We assume that the UAVs receive noisy distance measurements from the FRs which are of two types, i.e., Line of Sight (LoS) and non-LoS (NLoS) measurements and which are used by the UAV agents in order to estimate the state (i.e., position) of the FRs. Subsequently, the proposed DRL-based controller selects the optimal joint control actions according to the Cramer-Rao lower bound (CRLB) of the joint measurement likelihood function to achieve high tracking performance. Specifically, the optimal UAV control actions are quantified by the proposed reward function, which considers both the CRLB of the entire system and each UAV's individual contribution to the system, called global reward and difference reward, respectively. Since the UAVs take actions that reduce the CRLB of the entire system, tracking accuracy is improved by ensuring the reception of high quality LoS measurements with high probability. Our simulation results show that the proposed DRL-based UAV controller provides a highly accurate target tracking solution with a very low runtime cost.