Soft-Sign Stochastic Gradient Descent Algorithm for Wireless Federated Learning
- Authors
- Lee, Seunghoon; Park, Chanho; Hong, Songnam; Eldar, Yonina C.; Lee, Namyoon
- Issue Date
- Nov-2021
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Citation
- IEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC, v.2021, no.September, pp.241 - 245
- Indexed
- SCOPUS
- Journal Title
- IEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC
- Volume
- 2021
- Number
- September
- Start Page
- 241
- End Page
- 245
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/140382
- DOI
- 10.1109/SPAWC51858.2021.9593212
- ISSN
- 0000-0000
- Abstract
- Federated learning over wireless networks requires aggregating locally computed gradients at a server where the mobile devices send statistically distinct gradient information over heterogenous communication links. This paper proposes a Bayesian approach for wireless federated learning referred to as soft-sign stochastic gradient descent (soft-signSGD). The idea of soft-signSGD is to aggregate the one-bit quantized local gradients at the server by jointly exploiting i) the prior distributions of the local gradients, ii) the gradient quantizer function, and iii) channel distributions. This aggregation method is optimal in the sense of minimizing the mean-squared error (MSE) under a simplified Gaussian prior assumption on the local gradient. From simulations, we demonstrate that soft-signSGD considerably outperforms the conventional sign stochastic gradient descent algorithm when training and testing neural networks using the MNIST dataset and the CIFAR-10 dataset over heterogeneous wireless networks.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 융합전자공학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/140382)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.