Byzantine Resilient Non-Convex SCSG with Distributed Batch Gradient Computations

Saikiran Bulusu, Prashant Khanduri, Swatantra Kafle, Pranay Sharma, Pramod K. Varshney

Research output: Contribution to journalArticlepeer-review

Abstract

Distributed learning is an important paradigm in the current machine learning algorithms with large datasets. In this paper, distributed stochastic optimization problem of minimizing a nonconvex function in an adversarial setting is considered. A robust variant of the stochastic variance-reduced algorithm is proposed. In the distributed setup, we assume that a fraction of worker nodes (WNs) can be Byzantines. We assume that the batch gradients are computed at the WNs and the stochastic gradients are computed at the central node (CN). We provide the convergence rate of the proposed algorithm which employs the design of a novel filtering rule that is independent of the problem dimension. Furthermore, we capture the effect of Byzantines present in the network on the convergence performance of the algorithm. We evaluate the performance of the proposed algorithm and present the simulation results using real world datasets, in addition to providing the theoretical guarantees.

Original languageEnglish (US)
Pages (from-to)754-766
Number of pages13
JournalIEEE Transactions on Signal and Information Processing over Networks
Volume7
DOIs
StatePublished - 2021

Keywords

  • Nonconvex optimization
  • Stochastic gradient descent
  • distributed optimization
  • variance reduction

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Byzantine Resilient Non-Convex SCSG with Distributed Batch Gradient Computations'. Together they form a unique fingerprint.

Cite this