Classification of affect using deep learning on brain blood flow data

Research output: Contribution to journalArticle

Abstract

We present a convolutional neural network- and long short-term memory-based method to classify the valence level of a computer user based on functional near infrared spectroscopy data. Convolutional neural networks are well suited for capturing the spatial characteristics of functional near infrared spectroscopy data. And long short-term memories are demonstrated to be good at learning temporal patterns of unknown length in time series data. We explore these methods in a combined layered architecture in order to improve classification accuracy. We conducted an experiment with 20 participants, wherein they were subjected to emotion inducing stimuli while their brain activity was measured using functional near infrared spectroscopy. Self-report surveys were administered after each stimulus to gauge participants' self-assessment of their valence. The resulting classification using these survey labels as ground truth provided a three-class classification accuracy 77.89% in across subject cross-validation. This method also shows promise for generalization to other classification tasks using functional near infrared spectroscopy data.

Original languageEnglish (US)
JournalJournal of Near Infrared Spectroscopy
DOIs
StatePublished - Jan 1 2019

Fingerprint

Near infrared spectroscopy
Brain
Blood
Neural networks
Gages
Labels
Time series
Deep learning
Experiments
Long short-term memory

Keywords

  • convolutional neural network
  • Emotion classification
  • long short-term memory

ASJC Scopus subject areas

  • Spectroscopy

Cite this

@article{6c67d98791ee45f6becad5a24a8b13b6,
title = "Classification of affect using deep learning on brain blood flow data",
abstract = "We present a convolutional neural network- and long short-term memory-based method to classify the valence level of a computer user based on functional near infrared spectroscopy data. Convolutional neural networks are well suited for capturing the spatial characteristics of functional near infrared spectroscopy data. And long short-term memories are demonstrated to be good at learning temporal patterns of unknown length in time series data. We explore these methods in a combined layered architecture in order to improve classification accuracy. We conducted an experiment with 20 participants, wherein they were subjected to emotion inducing stimuli while their brain activity was measured using functional near infrared spectroscopy. Self-report surveys were administered after each stimulus to gauge participants' self-assessment of their valence. The resulting classification using these survey labels as ground truth provided a three-class classification accuracy 77.89{\%} in across subject cross-validation. This method also shows promise for generalization to other classification tasks using functional near infrared spectroscopy data.",
keywords = "convolutional neural network, Emotion classification, long short-term memory",
author = "Danushka Bandara and Hirshfield, {Leanne M} and Senem Velipasalar",
year = "2019",
month = "1",
day = "1",
doi = "10.1177/0967033519837986",
language = "English (US)",
journal = "Journal of Near Infrared Spectroscopy",
issn = "0967-0335",
publisher = "I M Publications",

}

TY - JOUR

T1 - Classification of affect using deep learning on brain blood flow data

AU - Bandara, Danushka

AU - Hirshfield, Leanne M

AU - Velipasalar, Senem

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We present a convolutional neural network- and long short-term memory-based method to classify the valence level of a computer user based on functional near infrared spectroscopy data. Convolutional neural networks are well suited for capturing the spatial characteristics of functional near infrared spectroscopy data. And long short-term memories are demonstrated to be good at learning temporal patterns of unknown length in time series data. We explore these methods in a combined layered architecture in order to improve classification accuracy. We conducted an experiment with 20 participants, wherein they were subjected to emotion inducing stimuli while their brain activity was measured using functional near infrared spectroscopy. Self-report surveys were administered after each stimulus to gauge participants' self-assessment of their valence. The resulting classification using these survey labels as ground truth provided a three-class classification accuracy 77.89% in across subject cross-validation. This method also shows promise for generalization to other classification tasks using functional near infrared spectroscopy data.

AB - We present a convolutional neural network- and long short-term memory-based method to classify the valence level of a computer user based on functional near infrared spectroscopy data. Convolutional neural networks are well suited for capturing the spatial characteristics of functional near infrared spectroscopy data. And long short-term memories are demonstrated to be good at learning temporal patterns of unknown length in time series data. We explore these methods in a combined layered architecture in order to improve classification accuracy. We conducted an experiment with 20 participants, wherein they were subjected to emotion inducing stimuli while their brain activity was measured using functional near infrared spectroscopy. Self-report surveys were administered after each stimulus to gauge participants' self-assessment of their valence. The resulting classification using these survey labels as ground truth provided a three-class classification accuracy 77.89% in across subject cross-validation. This method also shows promise for generalization to other classification tasks using functional near infrared spectroscopy data.

KW - convolutional neural network

KW - Emotion classification

KW - long short-term memory

UR - http://www.scopus.com/inward/record.url?scp=85064278842&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064278842&partnerID=8YFLogxK

U2 - 10.1177/0967033519837986

DO - 10.1177/0967033519837986

M3 - Article

JO - Journal of Near Infrared Spectroscopy

JF - Journal of Near Infrared Spectroscopy

SN - 0967-0335

ER -