Classification of affect using deep learning on brain blood flow data

We present a convolutional neural network- and long short-term memory-based method to classify the valence level of a computer user based on functional near infrared spectroscopy data. Convolutional neural networks are well suited for capturing the spatial characteristics of functional near infrared spectroscopy data. And long short-term memories are demonstrated to be good at learning temporal patterns of unknown length in time series data. We explore these methods in a combined layered architecture in order to improve classification accuracy. We conducted an experiment with 20 participants, wherein they were subjected to emotion inducing stimuli while their brain activity was measured using functional near infrared spectroscopy. Self-report surveys were administered after each stimulus to gauge participants' self-assessment of their valence. The resulting classification using these survey labels as ground truth provided a three-class classification accuracy 77.89% in across subject cross-validation. This method also shows promise for generalization to other classification tasks using functional near infrared spectroscopy data.