Abstract
The presented first-of-its-kind study effectively identifies and visualizes the second-by-second pattern differences in the physiological arousal of preschool-age children who do stutter (CWS) and who do not stutter (CWNS) while speaking perceptually fluently in two challenging conditions: speaking in stressful situations and narration. The first condition may affect children's speech due to high arousal; the latter introduces linguistic, cognitive, and communicative demands on speakers. We collected physiological parameters data from 70 children in the two target conditions. First, we adopt a novel modality-wise multiple-instance-learning (MI-MIL) approach to classify CWS vs. CWNS in different conditions effectively. The evaluation of this classifier addresses four critical research questions that align with state-of-the-art speech science studies' interests. Later, we leverage SHAP classifier interpretations to visualize the salient, fine-grain, and temporal physiological parameters unique to CWS at the population/group-level and personalized-level. While group-level identification of distinct patterns would enhance our understanding of stuttering etiology and development, the personalized-level identification would enable remote, continuous, and real-time assessment of stuttering children's physiological arousal, which may lead to personalized, just-in-time interventions, resulting in an improvement in speech fluency. The presented MI-MIL approach is novel, generalizable to different domains, and real-time executable. Finally, comprehensive evaluations are done on multiple datasets, presented framework, and several baselines that identified notable insights on CWSs' physiological arousal during speech production.
Original language | English (US) |
---|---|
Article number | 137 |
Journal | Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies |
Volume | 6 |
Issue number | 3 |
DOIs | |
State | Published - Sep 7 2022 |
Keywords
- Affective Computing
- Arousal Detection
- Children Who Stutter
- Deep Learning
- Explainable AI
- Machine Learning
- Multi-modal Fusion
- Multiple Instance Learning
- Sensors
- Stuttering
ASJC Scopus subject areas
- Human-Computer Interaction
- Hardware and Architecture
- Computer Networks and Communications