TY - JOUR
T1 - Deep Reinforcement Learning-Based Edge Caching in Wireless Networks
AU - Zhong, Chen
AU - Gursoy, M. Cenk
AU - Velipasalar, Senem
N1 - Funding Information:
ACKNOWLEDGMENT The information, data, or work presented herein was funded in part by National Science Foundation (NSF) under Grant 1739748, Grant 1816732 and by the Advanced Research Projects Agency-Energy (ARPA-E), U.S. Department of Energy, under Award Number DE-AR0000940. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.
Funding Information:
Manuscript received June 2, 2019; revised October 25, 2019; accepted December 30, 2019. Date of publication January 21, 2020; date of current version March 6, 2020. The information, data, or work presented herein was funded in part by National Science Foundation (NSF) under Grant 1739748, Grant 1816732 and by the Advanced Research Projects Agency-Energy (ARPA-E), U.S. Department of Energy, under Award Number DE-AR0000940. The associate editor coordinating the review of this article and approving it for publication was L. Zhang. (Corresponding author: M. Cenk Gursoy.) The authors are with the Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY 13244 USA (e-mail: czhong03@syr.edu; mcgursoy@syr.edu; svelipas@syr.edu). Digital Object Identifier 10.1109/TCCN.2020.2968326
Publisher Copyright:
© 2015 IEEE.
PY - 2020/3
Y1 - 2020/3
N2 - With the purpose to offload data traffic in wireless networks, content caching techniques have recently been studied intensively. Using these techniques and caching a portion of the popular files at the local content servers, the users can be served with less delay. Most of the content replacement policies are based on the content popularity, that depends on the users' preferences. In practice, such information varies over time. Therefore, an approach to determine the file popularity patterns must be incorporated into caching policies. In this context, we study content caching at the wireless network edge using a deep reinforcement learning framework with Wolpertinger architecture. In particular, we propose deep actor-critic reinforcement learning based policies for both centralized and decentralized content caching. For centralized edge caching, we aim at maximizing the cache hit rate. In decentralized edge caching, we consider both the cache hit rate and transmission delay as performance metrics. The proposed frameworks are assumed to neither have any prior information on the file popularities nor know the potential variations in such information. Via simulation results, the superiority of the proposed frameworks is verified by comparing them with other policies, including least frequently used (LFU), least recently used (LRU), and first-in-first-out (FIFO) policies.
AB - With the purpose to offload data traffic in wireless networks, content caching techniques have recently been studied intensively. Using these techniques and caching a portion of the popular files at the local content servers, the users can be served with less delay. Most of the content replacement policies are based on the content popularity, that depends on the users' preferences. In practice, such information varies over time. Therefore, an approach to determine the file popularity patterns must be incorporated into caching policies. In this context, we study content caching at the wireless network edge using a deep reinforcement learning framework with Wolpertinger architecture. In particular, we propose deep actor-critic reinforcement learning based policies for both centralized and decentralized content caching. For centralized edge caching, we aim at maximizing the cache hit rate. In decentralized edge caching, we consider both the cache hit rate and transmission delay as performance metrics. The proposed frameworks are assumed to neither have any prior information on the file popularities nor know the potential variations in such information. Via simulation results, the superiority of the proposed frameworks is verified by comparing them with other policies, including least frequently used (LFU), least recently used (LRU), and first-in-first-out (FIFO) policies.
KW - Actor-critic algorithms
KW - deep reinforcement learning
KW - edge caching
UR - http://www.scopus.com/inward/record.url?scp=85078406788&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85078406788&partnerID=8YFLogxK
U2 - 10.1109/TCCN.2020.2968326
DO - 10.1109/TCCN.2020.2968326
M3 - Article
AN - SCOPUS:85078406788
SN - 2332-7731
VL - 6
SP - 48
EP - 61
JO - IEEE Transactions on Cognitive Communications and Networking
JF - IEEE Transactions on Cognitive Communications and Networking
IS - 1
M1 - 8964499
ER -