Actor-critic deep reinforcement learning for dynamic multichannel access

Chen Zhong, Ziyang Lu, Mustafa C Gursoy, Senem Velipasalar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider the dynamic multichannel access problem, which can be formulated as a partially observable Markov decision process (POMDP). We first propose a model-free actor-critic deep reinforcement learning based framework to explore the sensing policy. To evaluate the performance of the proposed sensing policy and the framework's tolerance against uncertainty, we test the framework in scenarios with different channel switching patterns and consider different switching probabilities. Then, we consider a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons with the Deep-Q network (DQN) based framework proposed in [1], in terms of both average reward and the time efficiency.

Original languageEnglish (US)
Title of host publication2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages599-603
Number of pages5
ISBN (Electronic)9781728112954
DOIs
StatePublished - Feb 20 2019
Event2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Anaheim, United States
Duration: Nov 26 2018Nov 29 2018

Publication series

Name2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings

Conference

Conference2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018
CountryUnited States
CityAnaheim
Period11/26/1811/29/18

Fingerprint

Reinforcement learning
Uncertainty

Keywords

  • Actor-critic
  • Channel selection
  • Deep reinforcement learning
  • POMDP

ASJC Scopus subject areas

  • Information Systems
  • Signal Processing

Cite this

Zhong, C., Lu, Z., Gursoy, M. C., & Velipasalar, S. (2019). Actor-critic deep reinforcement learning for dynamic multichannel access. In 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings (pp. 599-603). [8646405] (2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/GlobalSIP.2018.8646405

Actor-critic deep reinforcement learning for dynamic multichannel access. / Zhong, Chen; Lu, Ziyang; Gursoy, Mustafa C; Velipasalar, Senem.

2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. p. 599-603 8646405 (2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhong, C, Lu, Z, Gursoy, MC & Velipasalar, S 2019, Actor-critic deep reinforcement learning for dynamic multichannel access. in 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings., 8646405, 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 599-603, 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018, Anaheim, United States, 11/26/18. https://doi.org/10.1109/GlobalSIP.2018.8646405
Zhong C, Lu Z, Gursoy MC, Velipasalar S. Actor-critic deep reinforcement learning for dynamic multichannel access. In 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. p. 599-603. 8646405. (2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings). https://doi.org/10.1109/GlobalSIP.2018.8646405
Zhong, Chen ; Lu, Ziyang ; Gursoy, Mustafa C ; Velipasalar, Senem. / Actor-critic deep reinforcement learning for dynamic multichannel access. 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 599-603 (2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings).
@inproceedings{d000b2a5d8b9467e8120b94fe7a123a7,
title = "Actor-critic deep reinforcement learning for dynamic multichannel access",
abstract = "We consider the dynamic multichannel access problem, which can be formulated as a partially observable Markov decision process (POMDP). We first propose a model-free actor-critic deep reinforcement learning based framework to explore the sensing policy. To evaluate the performance of the proposed sensing policy and the framework's tolerance against uncertainty, we test the framework in scenarios with different channel switching patterns and consider different switching probabilities. Then, we consider a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons with the Deep-Q network (DQN) based framework proposed in [1], in terms of both average reward and the time efficiency.",
keywords = "Actor-critic, Channel selection, Deep reinforcement learning, POMDP",
author = "Chen Zhong and Ziyang Lu and Gursoy, {Mustafa C} and Senem Velipasalar",
year = "2019",
month = "2",
day = "20",
doi = "10.1109/GlobalSIP.2018.8646405",
language = "English (US)",
series = "2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "599--603",
booktitle = "2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings",

}

TY - GEN

T1 - Actor-critic deep reinforcement learning for dynamic multichannel access

AU - Zhong, Chen

AU - Lu, Ziyang

AU - Gursoy, Mustafa C

AU - Velipasalar, Senem

PY - 2019/2/20

Y1 - 2019/2/20

N2 - We consider the dynamic multichannel access problem, which can be formulated as a partially observable Markov decision process (POMDP). We first propose a model-free actor-critic deep reinforcement learning based framework to explore the sensing policy. To evaluate the performance of the proposed sensing policy and the framework's tolerance against uncertainty, we test the framework in scenarios with different channel switching patterns and consider different switching probabilities. Then, we consider a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons with the Deep-Q network (DQN) based framework proposed in [1], in terms of both average reward and the time efficiency.

AB - We consider the dynamic multichannel access problem, which can be formulated as a partially observable Markov decision process (POMDP). We first propose a model-free actor-critic deep reinforcement learning based framework to explore the sensing policy. To evaluate the performance of the proposed sensing policy and the framework's tolerance against uncertainty, we test the framework in scenarios with different channel switching patterns and consider different switching probabilities. Then, we consider a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons with the Deep-Q network (DQN) based framework proposed in [1], in terms of both average reward and the time efficiency.

KW - Actor-critic

KW - Channel selection

KW - Deep reinforcement learning

KW - POMDP

UR - http://www.scopus.com/inward/record.url?scp=85063079872&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063079872&partnerID=8YFLogxK

U2 - 10.1109/GlobalSIP.2018.8646405

DO - 10.1109/GlobalSIP.2018.8646405

M3 - Conference contribution

T3 - 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings

SP - 599

EP - 603

BT - 2018 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2018 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -