A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data

Danushka Bandara, Stephen Song, Leanne M Hirshfield, Senem Velipasalar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

We describe a method of achieving emotion classification using ECG and EDA data. There have been many studies conducted on usage of heart rate and EDA data to quantify the arousal level of a user [1–3]. Researchers have identified a connection between a person’s ECG data and the positivity or negativity of their emotional state [4]. The goal of this work is to extend this idea to human computer interaction domain. We will explore whether the valence/arousal level of a subject’s response to computer based stimuli is predictable using ECG and EDA, and whether or not that information can complement recordings of participants’ cognitive data to form a more accurate depiction of emotional state. The experiment consists of presenting three types of stimuli, both interactive and noninteractive, to 9 subjects and recording their physiological response via ECG and EDA data as well as fNIRS device. The stimuli were selected from validated methods of inducing emotion including DEAP dataset [5], Multi Attribute Task Battery [6] and Tetris video game [7]. The participants’ responses were captured using Self-Assessment Manikin [8] surveys which were used as the ground truth labels. The resulting data was analyzed using Machine Learning. The results provide new avenues of research in combining physiological data to classify emotion.

Original languageEnglish (US)
Title of host publicationFoundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings
PublisherSpringer Verlag
Pages287-298
Number of pages12
Volume9743
ISBN (Print)9783319399546
DOIs
StatePublished - 2016
Event10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016 - Toronto, Canada
Duration: Jul 17 2016Jul 22 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9743
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016
CountryCanada
CityToronto
Period7/17/167/22/16

Fingerprint

Electrocardiography
Complement
Human computer interaction
Learning systems
Labels
Self-assessment
Video Games
Heart Rate
Emotion
Electrocardiogram
Positivity
Battery
Machine Learning
Person
Quantify
Classify
Attribute
Experiments
Interaction
Experiment

Keywords

  • Arousal
  • Electrocardiography
  • Electrodermal activity
  • fNIRS
  • Human computer interaction
  • Valence

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Bandara, D., Song, S., Hirshfield, L. M., & Velipasalar, S. (2016). A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data. In Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings (Vol. 9743, pp. 287-298). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9743). Springer Verlag. https://doi.org/10.1007/978-3-319-39955-3_27

A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data. / Bandara, Danushka; Song, Stephen; Hirshfield, Leanne M; Velipasalar, Senem.

Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. Vol. 9743 Springer Verlag, 2016. p. 287-298 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9743).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Bandara, D, Song, S, Hirshfield, LM & Velipasalar, S 2016, A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data. in Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. vol. 9743, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9743, Springer Verlag, pp. 287-298, 10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016, Toronto, Canada, 7/17/16. https://doi.org/10.1007/978-3-319-39955-3_27
Bandara D, Song S, Hirshfield LM, Velipasalar S. A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data. In Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. Vol. 9743. Springer Verlag. 2016. p. 287-298. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-39955-3_27
Bandara, Danushka ; Song, Stephen ; Hirshfield, Leanne M ; Velipasalar, Senem. / A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data. Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. Vol. 9743 Springer Verlag, 2016. pp. 287-298 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{ecfc3a7f24a947ccb7e8e4d6b8490f95,
title = "A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data",
abstract = "We describe a method of achieving emotion classification using ECG and EDA data. There have been many studies conducted on usage of heart rate and EDA data to quantify the arousal level of a user [1–3]. Researchers have identified a connection between a person’s ECG data and the positivity or negativity of their emotional state [4]. The goal of this work is to extend this idea to human computer interaction domain. We will explore whether the valence/arousal level of a subject’s response to computer based stimuli is predictable using ECG and EDA, and whether or not that information can complement recordings of participants’ cognitive data to form a more accurate depiction of emotional state. The experiment consists of presenting three types of stimuli, both interactive and noninteractive, to 9 subjects and recording their physiological response via ECG and EDA data as well as fNIRS device. The stimuli were selected from validated methods of inducing emotion including DEAP dataset [5], Multi Attribute Task Battery [6] and Tetris video game [7]. The participants’ responses were captured using Self-Assessment Manikin [8] surveys which were used as the ground truth labels. The resulting data was analyzed using Machine Learning. The results provide new avenues of research in combining physiological data to classify emotion.",
keywords = "Arousal, Electrocardiography, Electrodermal activity, fNIRS, Human computer interaction, Valence",
author = "Danushka Bandara and Stephen Song and Hirshfield, {Leanne M} and Senem Velipasalar",
year = "2016",
doi = "10.1007/978-3-319-39955-3_27",
language = "English (US)",
isbn = "9783319399546",
volume = "9743",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "287--298",
booktitle = "Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings",
address = "Germany",

}

TY - GEN

T1 - A more complete picture of emotion using electrocardiogram and electrodermal activity to complement cognitive data

AU - Bandara, Danushka

AU - Song, Stephen

AU - Hirshfield, Leanne M

AU - Velipasalar, Senem

PY - 2016

Y1 - 2016

N2 - We describe a method of achieving emotion classification using ECG and EDA data. There have been many studies conducted on usage of heart rate and EDA data to quantify the arousal level of a user [1–3]. Researchers have identified a connection between a person’s ECG data and the positivity or negativity of their emotional state [4]. The goal of this work is to extend this idea to human computer interaction domain. We will explore whether the valence/arousal level of a subject’s response to computer based stimuli is predictable using ECG and EDA, and whether or not that information can complement recordings of participants’ cognitive data to form a more accurate depiction of emotional state. The experiment consists of presenting three types of stimuli, both interactive and noninteractive, to 9 subjects and recording their physiological response via ECG and EDA data as well as fNIRS device. The stimuli were selected from validated methods of inducing emotion including DEAP dataset [5], Multi Attribute Task Battery [6] and Tetris video game [7]. The participants’ responses were captured using Self-Assessment Manikin [8] surveys which were used as the ground truth labels. The resulting data was analyzed using Machine Learning. The results provide new avenues of research in combining physiological data to classify emotion.

AB - We describe a method of achieving emotion classification using ECG and EDA data. There have been many studies conducted on usage of heart rate and EDA data to quantify the arousal level of a user [1–3]. Researchers have identified a connection between a person’s ECG data and the positivity or negativity of their emotional state [4]. The goal of this work is to extend this idea to human computer interaction domain. We will explore whether the valence/arousal level of a subject’s response to computer based stimuli is predictable using ECG and EDA, and whether or not that information can complement recordings of participants’ cognitive data to form a more accurate depiction of emotional state. The experiment consists of presenting three types of stimuli, both interactive and noninteractive, to 9 subjects and recording their physiological response via ECG and EDA data as well as fNIRS device. The stimuli were selected from validated methods of inducing emotion including DEAP dataset [5], Multi Attribute Task Battery [6] and Tetris video game [7]. The participants’ responses were captured using Self-Assessment Manikin [8] surveys which were used as the ground truth labels. The resulting data was analyzed using Machine Learning. The results provide new avenues of research in combining physiological data to classify emotion.

KW - Arousal

KW - Electrocardiography

KW - Electrodermal activity

KW - fNIRS

KW - Human computer interaction

KW - Valence

UR - http://www.scopus.com/inward/record.url?scp=84978891271&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84978891271&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-39955-3_27

DO - 10.1007/978-3-319-39955-3_27

M3 - Conference contribution

AN - SCOPUS:84978891271

SN - 9783319399546

VL - 9743

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 287

EP - 298

BT - Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings

PB - Springer Verlag

ER -