Distributed Online Handover Decisions for Energy Efficiency in Dense HetNets

Abstract

In this paper, we consider the problem of handover decision making in the context of a dense heterogeneous network with a macro base station and multiple small base stations. We propose a distributed deep Q-learning based algorithm that minimizes the overall energy consumption by taking into account both the energy consumption from transmission and hand over overheads. The proposed algorithm is performed in a distributed and interactive manner in which a centralized training agent manages the replay buffer for training its deep Q-network, by gathering state, action, and reward information reported from distributed handover agents. We perform several numerical evaluations and demonstrate that the proposed algorithm provides 10% to 30% energy savings over other contemporary handover mechanisms depending on handover overhead costs.