Demo abstract: KinVocal: Detecting agitated Vocal events

Many elderly who are suffering from dementia are also suffering from agitation. While most assisted living facilities and home health care situations rely upon caregivers to monitor and record agitation of their patients, the accuracy is limited because the caregiver must be present during the agitation and must record the event properly. Accurate 24-7 data would help physicians with improved diagnoses and care. To solve this problem we developed KinVocal, a system that continuously monitors and detects agitated vocal events and can be used for the elderly population suffering from dementia. KinVocal, using a novel combination of acoustic signal processing and multiple text mining techniques, automatically detects and records the 8 major vocal agitations for dementia patients as defined by the medical community. This includes: constant unwarranted request for attention or help, making verbal sexual advances, crying, screaming, laughing, cursing, speaking in repetitive sentences, and negativism. The novelty of KinVocal includes the comprehensiveness of addressing all 8 vocal events, using the text of the vocalizations only when accurate, combining text and acoustic features when necessary, and employing text mining and feature identification. A comprehensive performance evaluation includes using data from Youtube and movies, controlled experiments, and real in-home deployments. The results show high accuracy for all 8 vocal events.