Communication-Efficient Randomized Algorithm for Multi-Kernel Online Federated Learning
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Hong, Songnam | - |
dc.contributor.author | Chae, Jeongmin | - |
dc.date.accessioned | 2022-12-20T05:01:38Z | - |
dc.date.available | 2022-12-20T05:01:38Z | - |
dc.date.created | 2022-01-05 | - |
dc.date.issued | 2022-12 | - |
dc.identifier.issn | 0162-8828 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/172807 | - |
dc.description.abstract | Online federated learning (OFL) is a promising framework to learn a sequence of global functions from distributed sequential data at local devices. In this framework, we first introduce a single kernel-based OFL (termed S-KOFL) by incorporating random-feature (RF) approximation, online gradient descent (OGD), and federated averaging (FedAvg). As manifested in the centralized counterpart, an extension to multi-kernel method is necessary. Harnessing the extension principle in the centralized method, we construct a vanilla multi-kernel algorithm (termed vM-KOFL) and prove its asymptotic optimality. However, it is not practical as the communication overhead grows linearly with the size of a kernel dictionary. Moreover, this problem cannot be addressed via the existing communication-efficient techniques (e.g., quantization and sparsification) in the conventional federated learning. Our major contribution is to propose a novel randomized algorithm (named eM-KOFL), which exhibits similar performance to vM-KOFL while maintaining low communication cost. We theoretically prove that eM-KOFL achieves an optimal sublinear regret bound. Mimicking the key concept of eM-KOFL in an efficient way, we propose a more practical pM-KOFL having the same communication overhead as S-KOFL. Via numerical tests with real datasets, we demonstrate that pM-KOFL yields the almost same performance as vM-KOFL (or eM-KOFL) on various online learning tasks. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | IEEE COMPUTER SOC | - |
dc.title | Communication-Efficient Randomized Algorithm for Multi-Kernel Online Federated Learning | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Hong, Songnam | - |
dc.identifier.doi | 10.1109/TPAMI.2021.3129809 | - |
dc.identifier.scopusid | 2-s2.0-85120081394 | - |
dc.identifier.wosid | 000880661400094 | - |
dc.identifier.bibliographicCitation | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, v.44, no.12, pp.9872 - 9886 | - |
dc.relation.isPartOf | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE | - |
dc.citation.title | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE | - |
dc.citation.volume | 44 | - |
dc.citation.number | 12 | - |
dc.citation.startPage | 9872 | - |
dc.citation.endPage | 9886 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Artificial Intelligence | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.subject.keywordPlus | E-learning | - |
dc.subject.keywordPlus | Collaborative Work | - |
dc.subject.keywordPlus | Downlink | - |
dc.subject.keywordPlus | Federated learning | - |
dc.subject.keywordPlus | Kernel | - |
dc.subject.keywordPlus | Kernel-based learning | - |
dc.subject.keywordPlus | Online learning | - |
dc.subject.keywordPlus | Predictive models | - |
dc.subject.keywordPlus | Reproducing kernel hilbert space | - |
dc.subject.keywordPlus | Reproducing Kernel Hilbert spaces | - |
dc.subject.keywordPlus | Uplink | - |
dc.subject.keywordPlus | Optimization | - |
dc.subject.keywordAuthor | Collaborative work | - |
dc.subject.keywordAuthor | Data models | - |
dc.subject.keywordAuthor | Downlink | - |
dc.subject.keywordAuthor | Federated learning | - |
dc.subject.keywordAuthor | Kernel | - |
dc.subject.keywordAuthor | kernel-based learning | - |
dc.subject.keywordAuthor | online learning | - |
dc.subject.keywordAuthor | Predictive models | - |
dc.subject.keywordAuthor | reproducing kernel Hilbert space (RKHS) | - |
dc.subject.keywordAuthor | Servers | - |
dc.subject.keywordAuthor | Uplink | - |
dc.identifier.url | https://ieeexplore.ieee.org/document/9625795 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.