Communication-Efficient Federated Learning Based on Compressed Sensing

Chengxi Li, Gang Li, Pramod K. Varshney

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, we investigate the problem of federated learning (FL) in a communication-constrained environment of Internet of Things (IoT), where multiple IoT clients train a global model collectively by communicating model updates with a central server instead of sending raw datasets. To ease the communication burden in IoT systems, several approaches have been proposed for the FL tasks including sparsification methods and data quantization strategies. To overcome the shortcomings of the existing methods, we propose two new FL algorithms based on compressed sensing (CS) referred to as the CS-FL algorithm and the 1-bit CS-FL algorithm, both of which compress the upstream and downstream data while communicating between the clients and the central server. The proposed algorithms improve upon the existing algorithms by letting the clients send analog and 1-bit data respectively to the server after compression with a random measurement matrix. Based on that, in CS-FL and 1-bit CS-FL, the clients update the model locally utilizing the result of sparse reconstruction obtained by iterative hard thresholding (IHT) and binary iterative hard thresholding (BIHT), respectively. Experiments conducted on the MNIST and the Fashion-MNIST datasets reveal the superiority of the proposed algorithm over the baseline algorithms, SignSGD with a majority vote, FL-STC and FedAvg.

Original languageEnglish (US)
JournalIEEE Internet of Things Journal
DOIs
StateAccepted/In press - 2021

Keywords

  • 1-bit quantization.
  • Compressed sensing
  • Data models
  • federated learning
  • Internet of Things
  • Internet of Things (IoT)
  • Quantization (signal)
  • Servers
  • Signal processing algorithms
  • Tools
  • Training data

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Communication-Efficient Federated Learning Based on Compressed Sensing'. Together they form a unique fingerprint.

Cite this