PnP-DRL: A Plug-and-Play Deep Reinforcement Learning Approach for Experience-Driven Networking

Zhiyuan Xu, Kun Wu, Weiyi Zhang, Jian Tang, Yanzhi Wang, Guoliang Xue

Research output: Contribution to journalArticlepeer-review

Abstract

While Deep Reinforcement Learning has emerged as a de facto approach to many complex experience-driven networking problems, it remains challenging to deploy DRL into real systems. Due to the random exploration or half-trained deep neural networks during the online training process, the DRL agent may make unexpected decisions, which may lead to system performance degradation or even system crash. In this paper, we propose PnP-DRL, an offline-trained, plug and play DRL solution, to leverage the batch reinforcement learning approach to learn the best control policy from pre-collected transition samples without interacting with the system. After being trained without interaction with systems, our Plug and Play DRL agent will start working seamlessly, without additional exploration or possible disruption of the running systems. We implement and evaluate our PnP-DRL solution on a prevalent experience-driven networking problem, Dynamic Adaptive Streaming over HTTP (DASH). Extensive experimental results manifest that 1) The existing batch reinforcement learning method has its limits; 2) Our approach PnP-DRL significantly outperforms classical adaptive bitrate algorithms in average user Quality of Experience (QoE); 3) PnP-DRL, unlike the state-of-the-art online DRL methods, can be off and running without learning gaps, while achieving comparable performances.

Original languageEnglish (US)
Article number9454317
Pages (from-to)2476-2486
Number of pages11
JournalIEEE Journal on Selected Areas in Communications
Volume39
Issue number8
DOIs
StatePublished - Aug 2021

Keywords

  • batch reinforcement learning
  • deep reinforcement learning
  • Experience-driven networking

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'PnP-DRL: A Plug-and-Play Deep Reinforcement Learning Approach for Experience-Driven Networking'. Together they form a unique fingerprint.

Cite this