Psychophysiological Arousal in Young Children Who Stutter: An Interpretable AI Approach

Harshit Sharma, Yi Xiao, Victoria Tumanova, Asif Salekin

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


The presented first-of-its-kind study effectively identifies and visualizes the second-by-second pattern differences in the physiological arousal of preschool-age children who do stutter (CWS) and who do not stutter (CWNS) while speaking perceptually fluently in two challenging conditions: speaking in stressful situations and narration. The first condition may affect children's speech due to high arousal; the latter introduces linguistic, cognitive, and communicative demands on speakers. We collected physiological parameters data from 70 children in the two target conditions. First, we adopt a novel modality-wise multiple-instance-learning (MI-MIL) approach to classify CWS vs. CWNS in different conditions effectively. The evaluation of this classifier addresses four critical research questions that align with state-of-the-art speech science studies' interests. Later, we leverage SHAP classifier interpretations to visualize the salient, fine-grain, and temporal physiological parameters unique to CWS at the population/group-level and personalized-level. While group-level identification of distinct patterns would enhance our understanding of stuttering etiology and development, the personalized-level identification would enable remote, continuous, and real-time assessment of stuttering children's physiological arousal, which may lead to personalized, just-in-time interventions, resulting in an improvement in speech fluency. The presented MI-MIL approach is novel, generalizable to different domains, and real-time executable. Finally, comprehensive evaluations are done on multiple datasets, presented framework, and several baselines that identified notable insights on CWSs' physiological arousal during speech production.

Original languageEnglish (US)
Article number137
JournalProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Issue number3
StatePublished - Sep 7 2022


  • Affective Computing
  • Arousal Detection
  • Children Who Stutter
  • Deep Learning
  • Explainable AI
  • Machine Learning
  • Multi-modal Fusion
  • Multiple Instance Learning
  • Sensors
  • Stuttering

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Psychophysiological Arousal in Young Children Who Stutter: An Interpretable AI Approach'. Together they form a unique fingerprint.

Cite this