Truthiness: Challenges associated with employing machine learning on neurophysiological sensor data

The use of neurophysiological sensors in HCI research is increasing in use and sophistication, largely because such sensors offer the potential benefit of providing “ground truth” in studies, and also because they are expected to underpin future adaptive systems. Sensors have shown significant promise in the efforts to develop measurements to help determine users’ mental and emotional states in real-time, allowing the system to use that information to adjust user experience. Most of the sensors used generate a substantial amount of data, a high dimensionality and volume of data that requires analysis using powerful machine learning algorithms. However, in the process of developing machine learning algorithms to make sense of the data and subject’s mental or emotional state under experimental conditions, researchers often rely on existing and imperfect measures to provide the “ground truth” needed to train the algorithms. In this paper, we highlight the different ways in which researchers try to establish ground truth and the strengths and limitations of those approaches. The paper concludes with several suggestions and specific areas that require more discussion.