SCARL: Attentive reinforcement learning-based scheduling in a multi-resource heterogeneous clusteropen access
- Authors
- Cheong, M.[Cheong, M.]; Lee, H.[Lee, H.]; Yeom, I.[Yeom, I.]; Woo, H.[Woo, H.]
- Issue Date
- 2019
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Keywords
- attention; attentive embedding; attentive reinforcement learning; Cluster resource management
- Citation
- IEEE Access, v.7, pp.153432 - 153444
- Indexed
- SCIE
SCOPUS
- Journal Title
- IEEE Access
- Volume
- 7
- Start Page
- 153432
- End Page
- 153444
- URI
- https://scholarworks.bwise.kr/skku/handle/2021.sw.skku/13814
- DOI
- 10.1109/ACCESS.2019.2948150
- ISSN
- 2169-3536
- Abstract
- Advanced reinforcement learning (RL) technologies have recently increased the opportunity for automating several tasks in cluster management at scale by exploiting repetitive logs of cluster operation and building a learning model for resource allocation and job scheduling. Yet, this trend of adopting RL in the domain of cluster management has not fully addressed the diversity and heterogeneity of jobs and machines in modern cluster environments. In this paper, we present an RL-based scheduler for a multi-resource cluster, namely SCARL (SCheduler with Attentive Reinforcement Learning), concentrating on intricate cluster operating conditions with different resource requirements and capabilities. Specifically, we employ attentive embedding and factored-action scheduling that together efficiently incorporate time-varying interdependency of jobs and machines in RL processing; they enable an end-to-end scalable policy for scheduling diverse jobs on heterogeneous machines. To the best of our knowledge, we are the first to employ attention mechanism in RL-based cluster resource management. Through experiments, we demonstrate that our approach is competitive with existing heuristic methods under various cluster simulation configurations, e.g., an average 9.2 % enhancement in slowdown over the shortest job first algorithm. Additionally, the approach yields stable performance with our test cluster for running synthetic workloads based on real traces. © 2013 IEEE.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - Information and Communication Engineering > School of Electronic and Electrical Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/skku/handle/2021.sw.skku/13814)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.