Deep Reinforcement Learning for Scalable Dynamic Bandwidth Allocation in RAN Slicing With Highly Mobile Users

Abstract

Radio Access Network (RAN) slicing is a key technology in 5G communication systems. It dynamically allocates network resources such as bandwidth and time slots to each RAN slice, meeting the quality of service (QoS) requirements of each slice on a common underlying 5G infrastructure. This RAN slicing problem normally has a large number of resource combinations with a practical number of RAN slices. However, most Q-learning based Deep reinforcement learning (DRL) algorithms cannot successfully converge with the size of the action space. To address this issue, we introduce the architecture of Action Factorization (AF) with a soft-max layer, which aids exploration by decomposing a large action space into multiple independent sub-action spaces. In addition, RAN slicing problem is facing a performance issue for highly mobile users. To improve the performance of this problem, we use current channel information and future channel information predicted by long short-term memory (LSTM). We then propose a DRL architecture combined with AF and LSTM for bandwidth allocation in RAN slicing. Furthermore, we point out that the QoS requirements used as a performance metric in existing studies are inconsistent with the QoE achievement from the user's point of view. Therefore, we introduce new metrics, data rate indicators (DRI), to compensate the discrepancy. Through extensive simulations, we confirm that our proposed solution efficiently allocates bandwidth to each slice for a reasonable number of slices by maximizing the sum of rewards from QoE achievement for each user under high mobility.