Quantized Distributed Online Kernel Learning

Abstract

In this paper we propose a communication-efficient kernel-based learning method by means of random-feature approximation and quantization. The proposed algorithm is named quantized distributed online kernel learning (QDOKL). We theoretically prove that QDOKL over N time slots can achieve an optimal sublinear regret \mathrm{O}(\sqrt{N}), provided that a quantization level scales with \sqrt{N}. Our analysis implies that every node in the network can learn a common function having a diminishing gap from the best function in hindsight. We verify our theoretical result via numerical tests with real datasets on online regression tasks. Also, it is demonstrated that QDOKL can achieve the almost same accuracy as the unquantized counterpart while having a lower communication overhead.