Emotion Recognition Robust to Indoor Environmental Distortions and Non-targeted Emotions Using Out-of-distribution Detection

G. A.O. Ye, Asif Salekin, Kristina Gordon, Karen Rose, Hongning Wang, John Stankovic

Research output: Contribution to journalArticlepeer-review

Abstract

The rapid development of machine learning on acoustic signal processing has resulted in many solutions for detecting emotions from speech. Early works were developed for clean and acted speech and for a fixed set of emotions. Importantly, the datasets and solutions assumed that a person only exhibited one of these emotions. More recent work has continually been adding realism to emotion detection by considering issues such as reverberation, de-amplification, and background noise, but often considering one dataset at a time, and also assuming all emotions are accounted for in the model. We significantly improve realistic considerations for emotion detection by (i) more comprehensively assessing different situations by combining the five common publicly available datasets as one and enhancing the new dataset with data augmentation that considers reverberation and de-amplification, (ii) incorporating 11 typical home noises into the acoustics, and (iii) considering that in real situations a person may be exhibiting many emotions that are not currently of interest and they should not have to fit into a pre-fixed category nor be improperly labeled. Our novel solution combines CNN with out-of-data distribution detection. Our solution increases the situations where emotions can be effectively detected and outperforms a state-of-the-art baseline.

Original languageEnglish (US)
Article number15
JournalACM Transactions on Computing for Healthcare
Volume3
Issue number2
DOIs
StatePublished - Apr 2022

Keywords

  • convolutional neural networks
  • distance
  • emotion detection
  • noise
  • out-of-distribution detection
  • reverberation
  • Synthetic datasets

ASJC Scopus subject areas

  • Computer Science Applications
  • Software
  • Information Systems
  • Medicine (miscellaneous)
  • Health Informatics
  • Health Information Management
  • Biomedical Engineering

Fingerprint

Dive into the research topics of 'Emotion Recognition Robust to Indoor Environmental Distortions and Non-targeted Emotions Using Out-of-distribution Detection'. Together they form a unique fingerprint.

Cite this