Federated Learning with Soft Clustering

Chengxi Li, Gang Li, Pramod K. Varshney

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, we consider the problem of federated learning (FL) with training data that are not independent and identically distributed (non-IID) across the clients. To cope with data heterogeneity, Iterative Federated Clustering Algorithm (IFCA) has been proposed. IFCA partitions the clients into a number of clusters and lets the clients in the same cluster optimize a shared model. However, in IFCA, the clusters are non-overlapping, which leads to an inefficient utilization of the local information since the knowledge of a client is used by only one cluster during each round. To capture the complex nature of real-world data, soft clustering methods with overlapping clusters have been proposed that attain superior performance over the hard ones. Motivated by this, we propose a new algorithm named Federated Learning with Soft Clustering (FLSC) by combining the strengths of soft clustering and IFCA, where the clients are partitioned into overlapping clusters and the information of each participating client is used by multiple clusters simultaneously during each round. The experimental results show that FLSC achieves better learning performance on the classification tasks on the MNIST and Fashion-MNIST datasets, compared with the state-of-the-art baseline methods, i.e., the global model method and IFCA.

Original languageEnglish (US)
JournalIEEE Internet of Things Journal
DOIs
StateAccepted/In press - 2021

Keywords

  • Clustering algorithms
  • Computational modeling
  • Data heterogeneity
  • Data models
  • federated learning
  • Servers
  • soft clustering.
  • Task analysis
  • Training
  • Training data

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Federated Learning with Soft Clustering'. Together they form a unique fingerprint.

Cite this