Deep Q-Learning Based Online Beam Tracking for Hybrid Beamforming Systems

Abstract

This paper investigates an online beam tracking problem in multi-user downlink hybrid beamforming systems. A base station (BS) equipped with a hybrid beamforming antenna system tracks multiple users with mobility by controlling its analog beamforming precoder. To ensure stable wireless services for mobile users, a codebook-based online deep Q-learning (DQL) is proposed, operating through feedback of signal-to-interference-plus-noise ratio (SINR). Specifically, to maximize the achievable sum rate, the DQL agent deployed at the BS intelligently adjusts the analog beamforming precoder based on a beamforming codebook with uniformly quantized angles in the first phase, guided by the SINR feedback from the users. Subsequently, after the analog precoder is fixed, the digital precoder is designed using zero-forcing (ZF) to simultaneously support multiple users. Numerical results demonstrate that the proposed online beam tracking can efficiently adjust its analog precoder according to the users’ mobility, leading to a significant improvement in the sum rate performance. © 2023 IEEE.