Dynamic Multichannel Access via Multi-agent Reinforcement Learning: Throughput and Fairness Guarantees

Abstract

A multichannel random access system is considered in which each user accesses a single channel among multiple orthogonal channels to communicate with an access point (AP). Users arrive to the system at random and be activated for a certain period of time slots and then disappear from the system. Under such dynamic network environment, we propose a distributed multichannel access protocol based on multi-agent reinforcement learning (RL) to improve both throughput and fairness between users. Unlike the previous approaches adjusting channel access probabilities at each time slot, the proposed RL algorithm deterministically selects a set of channel access policies for several consecutive time slots. To effectively reduce the complexity of the proposed RL algorithm, we adopt a branching dueling Q-network architecture and propose a training methodology for producing proper Q-values under time-varying user sets. Numerical results demonstrate that the proposed scheme significantly improve both throughput and fairness.