Soft Memory Box: A Virtual Shared Memory Framework for Fast Deep Neural Network Training in Distributed High Performance Computing
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Ahn, Shinyoung | - |
dc.contributor.author | Kim, Joongheon | - |
dc.contributor.author | Lim, Eunji | - |
dc.contributor.author | Kang, Sungwon | - |
dc.date.available | 2019-01-22T14:20:13Z | - |
dc.date.issued | 2018-05-08 | - |
dc.identifier.issn | 2169-3536 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/1495 | - |
dc.description.abstract | Deep learning is one of the major promising machine learning methodologies. Deep learning is widely used in various application domains, e.g., image recognition, voice recognition, and natural language processing. In order to improve learning accuracy, deep neural networks have evolved by: 1) increasing the number of layers and 2) increasing the number of parameters in massive models. This implies that distributed deep learning platforms need to evolve to: 1) deal with huge/complex deep neural networks and 2) process with high-performance computing resources for massive training data. This paper proposes a new virtual shared memory framework, called Soft Memory Box (SMB), which enables sharing the memory of remote node among distributed processes in the nodes so as to improve communication performance via parameter sharing. According to data-intensive performance evaluation results, the communication time of deep learning using the proposed SMB is 2.1 times faster than that using the massage passing interface (MPI). In addition, the communication time of the SMB-based asynchronous parameter update becomes 2-7 times faster than that using the MPI depending on deep learning models and the number of deep learning workers. | - |
dc.format.extent | 12 | - |
dc.publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC | - |
dc.title | Soft Memory Box: A Virtual Shared Memory Framework for Fast Deep Neural Network Training in Distributed High Performance Computing | - |
dc.type | Article | - |
dc.identifier.doi | 10.1109/ACCESS.2018.2834146 | - |
dc.identifier.bibliographicCitation | IEEE ACCESS, v.6, pp 26493 - 26504 | - |
dc.description.isOpenAccess | N | - |
dc.identifier.wosid | 000434945000001 | - |
dc.identifier.scopusid | 2-s2.0-85046765445 | - |
dc.citation.endPage | 26504 | - |
dc.citation.startPage | 26493 | - |
dc.citation.title | IEEE ACCESS | - |
dc.citation.volume | 6 | - |
dc.type.docType | Article | - |
dc.publisher.location | 미국 | - |
dc.subject.keywordAuthor | High performance computing | - |
dc.subject.keywordAuthor | distributed computing | - |
dc.subject.keywordAuthor | soft memory box | - |
dc.subject.keywordAuthor | shared memory | - |
dc.subject.keywordAuthor | deep neural network | - |
dc.subject.keywordAuthor | distributed deep learning | - |
dc.subject.keywordPlus | RECOGNITION | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Telecommunications | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Information Systems | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Telecommunications | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
84, Heukseok-ro, Dongjak-gu, Seoul, Republic of Korea (06974)02-820-6194
COPYRIGHT 2019 Chung-Ang University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.