Experience-Driven Congestion Control: When Multi-Path TCP Meets Deep Reinforcement Learning

Zhiyuan Xu, Jian Tang, Chengxiang Yin, Yanzhi Wang, Guoliang Xue

Research output: Contribution to journalArticle

20 Scopus citations

Abstract

In this paper, we aim to study networking problems from a whole new perspective by leveraging emerging deep learning, to develop an experience-driven approach, which enables a network or a protocol to learn the best way to control itself from its own experience (e.g., runtime statistics data), just as a human learns a skill. We present design, implementation and evaluation of a deep reinforcement learning (DRL)-based control framework, DRL-CC (DRL for Congestion Control), which realizes our experience-driven design philosophy on multi-path TCP (MPTCP) congestion control. DRL-CC utilizes a single (instead of multiple independent) agent to dynamically and jointly perform congestion control for all active MPTCP flows on an end host with the objective of maximizing the overall utility. The novelty of our design is to utilize a flexible recurrent neural network, LSTM, under a DRL framework for learning a representation for all active flows and dealing with their dynamics. Moreover, we, for the first time, integrate the above LSTM-based representation network into an actor-critic framework for continuous (congestion) control, which leverages the emerging deterministic policy gradient to train critic, actor, and LSTM networks in an end-to-end manner. We implemented DRL-CC based on the MPTCP implementation in the Linux kernel. The experimental results show that 1) DRL-CC consistently and significantly outperforms a few well-known MPTCP congestion control algorithms in terms of goodput without sacrificing fairness, 2) it is flexible and robust to highly-dynamic network environments with time-varying flows, and 3) it is friendly to regular TCP.

Original languageEnglish (US)
Article number8664598
Pages (from-to)1325-1336
Number of pages12
JournalIEEE Journal on Selected Areas in Communications
Volume37
Issue number6
DOIs
StatePublished - Jun 1 2019

Keywords

  • AI
  • congestion control
  • deep learning
  • experience-driven control
  • multi-path TCP
  • TCP

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Experience-Driven Congestion Control: When Multi-Path TCP Meets Deep Reinforcement Learning'. Together they form a unique fingerprint.

  • Cite this