Fully-Decentralized Multi-Kernel Online Learning over Networks

Abstract

Fully decentralized online learning with multiple kernels (named FDOMKL) is studied, where each node in a network learns a sequence of global functions in an online fashion without the control of a central server. Every node finds the best global function only using information from its one-hop neighboring nodes via online alternating direction method of multipliers (ADMM) and the network-wise Hedge algorithm. The learning framework for an individual node is based on kernel learning and the proposed algorithm successfully harness multi-kernel method to find the best common function over the entire network. To the best of our knowledge, this is the first work that proposes a fully-decentralized online learning algorithm based on multiple kernels. The proposed FDOMKL preserves privacy by maintaining the local data at the edge nodes and exchanging model parameters only. We prove that FDOMKL achieves a sublinear regret bound compared with the best kernel function in hindsight under certain assumptions. In addition, numerical tests on real time-series datasets demonstrate the superiority of the proposed algorithm in terms of learning accuracy and network consistency compared to state-of-the-art single kernel methods.