A real-time audio monitoring framework with limited data for constrained devices

Asif Salekin, Shabnam Ghaffarzadegan, Zhe Feng, John Stankovic

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

An effective and non-invasive audio monitoring system needs to be capable of simultaneous real-time detection of multiple audio events in many different environments, and locally executable on resource constrained devices, such as, smart microphones. A major challenge in this research domain is having limited available annotated data. This paper presents a novel framework to generate robust detection models of environmental and human audio events with limited available data. The framework presents the generation of a large synthetic dataset using limited data for any audio event, a novel computationally efficient feature modeling technique, named Audio2Vec, that is robust against environmental variations, and identifies and exploits the syntactic relation between audio states represented by the features and the targeted audio events. The presented framework achieves 10.3% higher F-1 scores compared to the best baseline approaches. To demonstrate the effectiveness of the framework we implemented a real-time audio monitoring system that simultaneously detects 10 audio events on a Raspberry Pi 3B and evaluate it in real home and in-car settings, that achieve F-1 scores of 0.96 and 0.956, respectively.

Original languageEnglish (US)
Title of host publicationProceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages98-105
Number of pages8
ISBN (Electronic)9781728105703
DOIs
StatePublished - May 2019
Externally publishedYes
Event15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019 - Santorini Island, Greece
Duration: May 29 2019May 31 2019

Publication series

NameProceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019

Conference

Conference15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019
CountryGreece
CitySantorini Island
Period5/29/195/31/19

Fingerprint

monitoring
Equipment and Supplies
event
Monitoring
Syntactics
Microphones
Railroad cars
Research
microphones
resources
time
Datasets
Rubus

Keywords

  • Audio Event Detection
  • Embedding
  • Feature Modeling
  • Limited Data
  • Raspberry Pi
  • Real-Time
  • Resource Constrained Devices

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Health Informatics
  • Instrumentation
  • Transportation
  • Communication

Cite this

Salekin, A., Ghaffarzadegan, S., Feng, Z., & Stankovic, J. (2019). A real-time audio monitoring framework with limited data for constrained devices. In Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019 (pp. 98-105). [8804744] (Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DCOSS.2019.00036

A real-time audio monitoring framework with limited data for constrained devices. / Salekin, Asif; Ghaffarzadegan, Shabnam; Feng, Zhe; Stankovic, John.

Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 98-105 8804744 (Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Salekin, A, Ghaffarzadegan, S, Feng, Z & Stankovic, J 2019, A real-time audio monitoring framework with limited data for constrained devices. in Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019., 8804744, Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019, Institute of Electrical and Electronics Engineers Inc., pp. 98-105, 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019, Santorini Island, Greece, 5/29/19. https://doi.org/10.1109/DCOSS.2019.00036
Salekin A, Ghaffarzadegan S, Feng Z, Stankovic J. A real-time audio monitoring framework with limited data for constrained devices. In Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 98-105. 8804744. (Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019). https://doi.org/10.1109/DCOSS.2019.00036
Salekin, Asif ; Ghaffarzadegan, Shabnam ; Feng, Zhe ; Stankovic, John. / A real-time audio monitoring framework with limited data for constrained devices. Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 98-105 (Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019).
@inproceedings{ab1521ebe6514438bb052a59c6911400,
title = "A real-time audio monitoring framework with limited data for constrained devices",
abstract = "An effective and non-invasive audio monitoring system needs to be capable of simultaneous real-time detection of multiple audio events in many different environments, and locally executable on resource constrained devices, such as, smart microphones. A major challenge in this research domain is having limited available annotated data. This paper presents a novel framework to generate robust detection models of environmental and human audio events with limited available data. The framework presents the generation of a large synthetic dataset using limited data for any audio event, a novel computationally efficient feature modeling technique, named Audio2Vec, that is robust against environmental variations, and identifies and exploits the syntactic relation between audio states represented by the features and the targeted audio events. The presented framework achieves 10.3{\%} higher F-1 scores compared to the best baseline approaches. To demonstrate the effectiveness of the framework we implemented a real-time audio monitoring system that simultaneously detects 10 audio events on a Raspberry Pi 3B and evaluate it in real home and in-car settings, that achieve F-1 scores of 0.96 and 0.956, respectively.",
keywords = "Audio Event Detection, Embedding, Feature Modeling, Limited Data, Raspberry Pi, Real-Time, Resource Constrained Devices",
author = "Asif Salekin and Shabnam Ghaffarzadegan and Zhe Feng and John Stankovic",
year = "2019",
month = "5",
doi = "10.1109/DCOSS.2019.00036",
language = "English (US)",
series = "Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "98--105",
booktitle = "Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019",

}

TY - GEN

T1 - A real-time audio monitoring framework with limited data for constrained devices

AU - Salekin, Asif

AU - Ghaffarzadegan, Shabnam

AU - Feng, Zhe

AU - Stankovic, John

PY - 2019/5

Y1 - 2019/5

N2 - An effective and non-invasive audio monitoring system needs to be capable of simultaneous real-time detection of multiple audio events in many different environments, and locally executable on resource constrained devices, such as, smart microphones. A major challenge in this research domain is having limited available annotated data. This paper presents a novel framework to generate robust detection models of environmental and human audio events with limited available data. The framework presents the generation of a large synthetic dataset using limited data for any audio event, a novel computationally efficient feature modeling technique, named Audio2Vec, that is robust against environmental variations, and identifies and exploits the syntactic relation between audio states represented by the features and the targeted audio events. The presented framework achieves 10.3% higher F-1 scores compared to the best baseline approaches. To demonstrate the effectiveness of the framework we implemented a real-time audio monitoring system that simultaneously detects 10 audio events on a Raspberry Pi 3B and evaluate it in real home and in-car settings, that achieve F-1 scores of 0.96 and 0.956, respectively.

AB - An effective and non-invasive audio monitoring system needs to be capable of simultaneous real-time detection of multiple audio events in many different environments, and locally executable on resource constrained devices, such as, smart microphones. A major challenge in this research domain is having limited available annotated data. This paper presents a novel framework to generate robust detection models of environmental and human audio events with limited available data. The framework presents the generation of a large synthetic dataset using limited data for any audio event, a novel computationally efficient feature modeling technique, named Audio2Vec, that is robust against environmental variations, and identifies and exploits the syntactic relation between audio states represented by the features and the targeted audio events. The presented framework achieves 10.3% higher F-1 scores compared to the best baseline approaches. To demonstrate the effectiveness of the framework we implemented a real-time audio monitoring system that simultaneously detects 10 audio events on a Raspberry Pi 3B and evaluate it in real home and in-car settings, that achieve F-1 scores of 0.96 and 0.956, respectively.

KW - Audio Event Detection

KW - Embedding

KW - Feature Modeling

KW - Limited Data

KW - Raspberry Pi

KW - Real-Time

KW - Resource Constrained Devices

UR - http://www.scopus.com/inward/record.url?scp=85071935180&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071935180&partnerID=8YFLogxK

U2 - 10.1109/DCOSS.2019.00036

DO - 10.1109/DCOSS.2019.00036

M3 - Conference contribution

AN - SCOPUS:85071935180

T3 - Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019

SP - 98

EP - 105

BT - Proceedings - 15th Annual International Conference on Distributed Computing in Sensor Systems, DCOSS 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -