Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We address the problem of monitoring a set of binary stochastic processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states (normal or anomalous). Based on the received observations, the decision-maker first determines whether to declare that the number of anomalies has exceeded the threshold or to continue taking observations. When the decision is to continue, it then decides whether to collect observations at the next time instant or defer it to a later time. If it chooses to collect observations, it further determines the subset of processes to be probed. To devise this three-step sequential decision-making process, we use a Bayesian formulation wherein we learn the posterior probability on the states of the processes. Using the posterior probability, we construct a Markov decision process and solve it using deep actor-critic reinforcement learning. Via numerical experiments, we demonstrate the superior performance of our algorithm compared to the traditional model-based algorithms.

Original languageEnglish (US)
Title of host publication2021 IEEE Global Communications Conference, GLOBECOM 2021 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728181042
DOIs
StatePublished - 2021
Event2021 IEEE Global Communications Conference, GLOBECOM 2021 - Madrid, Spain
Duration: Dec 7 2021Dec 11 2021

Publication series

Name2021 IEEE Global Communications Conference, GLOBECOM 2021 - Proceedings

Conference

Conference2021 IEEE Global Communications Conference, GLOBECOM 2021
Country/TerritorySpain
CityMadrid
Period12/7/2112/11/21

Keywords

  • Active hypothesis testing
  • actor-critic algorithm
  • anomaly detection
  • change-point detection
  • deep reinforcement learning
  • dynamic decision-making
  • sequential sensing

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Health Informatics

Fingerprint

Dive into the research topics of 'Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing'. Together they form a unique fingerprint.

Cite this