Learning to schedule network resources throughput and delay optimally using q+-learning
- Authors
- Bae, Jeongmin; Lee, Joohyun; Chong, Song
- Issue Date
- Apr-2021
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Keywords
- Network resource management; throughput and delay optimality; reinforcement learning; upper confidence bound
- Citation
- IEEE/ACM Transactions on Networking, v.29, no.2, pp 750 - 763
- Pages
- 14
- Indexed
- SCIE
SCOPUS
- Journal Title
- IEEE/ACM Transactions on Networking
- Volume
- 29
- Number
- 2
- Start Page
- 750
- End Page
- 763
- URI
- https://scholarworks.bwise.kr/erica/handle/2021.sw.erica/106214
- DOI
- 10.1109/tnet.2021.3051663
- ISSN
- 1063-6692
1558-2566
- Abstract
- As network architecture becomes complex and the user requirement gets diverse, the role of efficient network resource management becomes more important. However, existing throughput-optimal scheduling algorithms such as the max-weight algorithm suffer from poor delay performance. In this paper, we present reinforcement learning-based network scheduling algorithms for a single-hop downlink scenario which achieve throughput-optimality and converge to minimal delay. To this end, we first formulate the network optimization problem as a Markov decision process (MDP) problem. Then, we introduce a new state-action value function called Q+-function and develop a reinforcement learning algorithm called Q+-learning with UCB (Upper Confidence Bound) exploration which guarantees small performance loss during a learning process. We also derive an upper bound of the sample complexity in our algorithm, which is more efficient than the best known bound from Q-learning with UCB exploration by a factor of γ2 where γ is the discount factor of the MDP problem. Finally, via simulation, we verify that our algorithm shows a delay reduction of up to 40.8% compared to the max-weight algorithm over various scenarios. We also show that the Q+-learning with UCB exploration converges to an ϵ-epsilon-optimal policy 10 times faster than Q-learning with UCB.
- Files in This Item
-
Go to Link
- Appears in
Collections - COLLEGE OF ENGINEERING SCIENCES > SCHOOL OF ELECTRICAL ENGINEERING > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.