HirePreter: A Framework for Providing Fine-grained Interpretation for Automated Job Interview Analysis

Wasifur Rahman, Sazan Mahbub, Asif Salekin, Md Kamrul Hasan, Ehsan Hoque

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

There has been a rise in automated technologies to screen potential job applicants through affective signals captured from video-based interviews. These tools can make the interview process scalable and objective, but they often provide little to no information of how the machine learning model is making crucial decisions that impacts the livelihood of thousands of people. We built an ensemble model - by combining Multiple-Instance-Learning and Language-Modeling based models - that can predict whether an interviewee should be hired or not. Using both model-specific and model-agnostic interpretation techniques, we can decipher the most informative time-segments and features driving the model's decision making. Our analysis also shows that our models are significantly impacted by the beginning and ending portions of the video. Our model achieves 75.3% accuracy in predicting whether an interviewee should be hired on the ETS Job Interview dataset. Our approach can be extended to interpret other video-based affective computing tasks like analyzing sentiment, measuring credibility, or coaching individuals to collaborate more effectively in a team.

Original languageEnglish (US)
Title of host publication2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665400213
DOIs
StatePublished - 2021
Event9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2021 - Nara, Japan
Duration: Sep 28 2021Oct 1 2021

Publication series

Name2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2021

Conference

Conference9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2021
Country/TerritoryJapan
CityNara
Period9/28/2110/1/21

Keywords

  • Fairness in AI
  • Interpretability
  • Job Interview Analysis

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Cognitive Neuroscience

Fingerprint

Dive into the research topics of 'HirePreter: A Framework for Providing Fine-grained Interpretation for Automated Job Interview Analysis'. Together they form a unique fingerprint.

Cite this