A real-time audio monitoring framework with limited data for constrained devices

Asif Salekin, Shabnam Ghaffarzadegan, Zhe Feng, John Stankovic

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

An effective and non-invasive audio monitoring system needs to be capable of simultaneous real-time detection of multiple audio events in many different environments, and locally executable on resource constrained devices, such as, smart microphones. A major challenge in this research domain is having limited available annotated data. This paper presents a novel framework to generate robust detection models of environmental and human audio events with limited available data. The framework presents the generation of a large synthetic dataset using limited data for any audio event, a novel computationally efficient feature modeling technique, named Audio2Vec, that is robust against environmental variations, and identifies and exploits the syntactic relation between audio states represented by the features and the targeted audio events. The presented framework achieves 10.3% higher F-1 scores compared to the best baseline approaches. To demonstrate the effectiveness of the framework we implemented a real-time audio monitoring system that simultaneously detects 10 audio events on a Raspberry Pi 3B and evaluate it in real home and in-car settings, that achieve F-1 scores of 0.96 and 0.956, respectively.

Original languageEnglish (US)
Title of host publicationProceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages98-105
Number of pages8
ISBN (Electronic)9781728105703
DOIs
StatePublished - May 2019
Externally publishedYes
Event15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019 - Santorini Island, Greece
Duration: May 29 2019May 31 2019

Publication series

NameProceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019

Conference

Conference15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019
CountryGreece
CitySantorini Island
Period5/29/195/31/19

    Fingerprint

Keywords

  • Audio Event Detection
  • Embedding
  • Feature Modeling
  • Limited Data
  • Raspberry Pi
  • Real-Time
  • Resource Constrained Devices

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Health Informatics
  • Instrumentation
  • Transportation
  • Communication

Cite this

Salekin, A., Ghaffarzadegan, S., Feng, Z., & Stankovic, J. (2019). A real-time audio monitoring framework with limited data for constrained devices. In Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019 (pp. 98-105). [8804744] (Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DCOSS.2019.00036