Evolutionary training of deep neural networks on heterogeneous computing environments

Subodh Kalia, Chilukuri K. Mohan, Ramakrishna Nemani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep neural networks are typically trained using gradient-based optimizers such as error backpropagation. This study proposes a framework based on Evolutionary Algorithms (EAs) to train deep neural networks without gradients. The network parameters, which may vary up to millions, are considered optimization variables. We demonstrate the training of an encoder-decoder segmentation network (U-Net) and Long Short-Term Memory (LSTM) model using (μ + λ)-ES, Genetic Algorithm, and Particle Swarm Optimization. The framework can train models with forward propagation on machines with different hardware in a cluster computing environment. We compare prediction results from the two models trained using our framework and backpropagation. We show that the neural networks can be trained in less time on CPUs as compared to the training on specialized compute-intensive GPUs.

Original languageEnglish (US)
Title of host publicationGECCO 2022 Companion - Proceedings of the 2022 Genetic and Evolutionary Computation Conference
PublisherAssociation for Computing Machinery, Inc
Pages2318-2321
Number of pages4
ISBN (Electronic)9781450392686
DOIs
StatePublished - Jul 9 2022
Externally publishedYes
Event2022 Genetic and Evolutionary Computation Conference, GECCO 2022 - Virtual, Online, United States
Duration: Jul 9 2022Jul 13 2022

Publication series

NameGECCO 2022 Companion - Proceedings of the 2022 Genetic and Evolutionary Computation Conference

Conference

Conference2022 Genetic and Evolutionary Computation Conference, GECCO 2022
Country/TerritoryUnited States
CityVirtual, Online
Period7/9/227/13/22

Keywords

  • evolutionary algorithms
  • heuristics
  • neural networks
  • parallelization

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Computational Mathematics
  • Theoretical Computer Science

Fingerprint

Dive into the research topics of 'Evolutionary training of deep neural networks on heterogeneous computing environments'. Together they form a unique fingerprint.

Cite this