Anomaly Detection under Controlled Sensing Using Actor-Critic Reinforcement Learning

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider the problem of detecting anomalies among a given set of processes using their noisy binary sensor measurements. The noiseless sensor measurement corresponding to a normal process is 0, and the measurement is 1 if the process is anomalous. The decision-making algorithm is assumed to have no knowledge of the number of anomalous processes. The algorithm is allowed to choose a subset of the sensors at each time instant until the confidence level on the decision exceeds the desired value. Our objective is to design a sequential sensor selection policy that dynamically determines which processes to observe at each time and when to terminate the detection algorithm. The selection policy is designed such that the anomalous processes are detected with the desired confidence level while incurring minimum cost which comprises the delay in detection and the cost of sensing. We cast this problem as a sequential hypothesis testing problem within the framework of Markov decision processes, and solve it using the actor-critic deep reinforcement learning algorithm. This deep neural network-based algorithm offers a low complexity solution with good detection accuracy. We also study the effect of statistical dependence between the processes on the algorithm performance. Through numerical experiments, we show that our algorithm is able to adapt to any unknown statistical dependence pattern of the processes.

Original languageEnglish (US)
Title of host publication2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728154787
DOIs
StatePublished - May 2020
Event21st IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2020 - Atlanta, United States
Duration: May 26 2020May 29 2020

Publication series

NameIEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC
Volume2020-May

Conference

Conference21st IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2020
CountryUnited States
CityAtlanta
Period5/26/205/29/20

Keywords

  • Active hypothesis testing
  • optimal sequential selection
  • quickest state estimation
  • reinforcement learning

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Information Systems

Fingerprint Dive into the research topics of 'Anomaly Detection under Controlled Sensing Using Actor-Critic Reinforcement Learning'. Together they form a unique fingerprint.

Cite this