Design and implementation of dynamic I/O control scheme for large scale distributed file systems
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Kim, Sunggon | - |
dc.contributor.author | Sim, Alex | - |
dc.contributor.author | Wu, Kesheng | - |
dc.contributor.author | Byna, Suren | - |
dc.contributor.author | Son, Yong Seok | - |
dc.date.accessioned | 2023-02-21T02:40:54Z | - |
dc.date.available | 2023-02-21T02:40:54Z | - |
dc.date.issued | 2022-12 | - |
dc.identifier.issn | 1386-7857 | - |
dc.identifier.issn | 1573-7543 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/60819 | - |
dc.description.abstract | In this work, we have analyzed the input/output (I/O) activities of Cori, which is a high-performance computing system at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory. Our analysis results indicate that most users do not adjust storage configurations but rather use the default settings. In addition, owing to the interference from many applications running simultaneously, the performance varies based on the system status. To configure file systems autonomously in complex environments, we developed DCA-IO, a dynamic distributed file system configuration adjustment algorithm that utilizes the system log information to adjust storage configurations automatically. Our scheme aims to improve the application performance and avoid interference from other applications without user intervention. Moreover, DCA-IO uses the existing system logs and does not require code modifications, an additional library, or user intervention. To demonstrate the effectiveness of DCA-IO, we performed experiments using I/O kernels of real applications in both an isolated small-sized Lustre environment and Cori. Our experimental results shows that our scheme can improve the performance of HPC applications by up to 263% with the default Lustre configuration. | - |
dc.format.extent | 16 | - |
dc.language | 영어 | - |
dc.language.iso | ENG | - |
dc.publisher | SPRINGER | - |
dc.title | Design and implementation of dynamic I/O control scheme for large scale distributed file systems | - |
dc.type | Article | - |
dc.identifier.doi | 10.1007/s10586-022-03640-0 | - |
dc.identifier.bibliographicCitation | CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, v.25, no.6, pp 4423 - 4438 | - |
dc.description.isOpenAccess | Y | - |
dc.identifier.wosid | 000833449400001 | - |
dc.identifier.scopusid | 2-s2.0-85135260170 | - |
dc.citation.endPage | 4438 | - |
dc.citation.number | 6 | - |
dc.citation.startPage | 4423 | - |
dc.citation.title | CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS | - |
dc.citation.volume | 25 | - |
dc.type.docType | Article | - |
dc.publisher.location | 미국 | - |
dc.subject.keywordAuthor | High-performance computing | - |
dc.subject.keywordAuthor | Distributed dynamic resource management | - |
dc.subject.keywordAuthor | Autonomous control | - |
dc.subject.keywordAuthor | Parallel and distributed file system | - |
dc.subject.keywordAuthor | Cloud system | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Theory & Methods | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.