Optimizing Features in Active Machine Learning for Complex Qualitative Content Analysis

Jasy Liew Suet Yan, Nancy McCracken, Shichun Zhou, Kevin Crowston

Research output: Chapter in Book/Entry/PoemConference contribution

23 Scopus citations

Abstract

We propose a semi-automatic approach for content analysis that leverages machine learning (ML) being initially trained on a small set of hand-coded data to perform a first pass in coding, and then have human annotators correct machine annotations in order to produce more examples to retrain the existing model incrementally for better performance. In this “active learning” approach, it is equally important to optimize the creation of the initial ML model given less training data so that the model is able to capture most if not all positive examples, and filter out as many negative examples as possible for human annotators to correct. This paper reports our attempt to optimize the initial ML model through feature exploration in a complex content analysis project that uses a multidimensional coding scheme, and contains codes with sparse positive examples. While different codes respond optimally to different combinations of features, we show that it is possible to create an optimal initial ML model using only a single combination of features for codes with at least 100 positive examples in the gold standard corpus.

Original languageEnglish (US)
Title of host publicationWorkshop on Language Technologies and Computational Social Science, Science 2014 at the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings
EditorsCristian Danescu-Niculescu-Mizil, Jacob Eisenstein, Kathleen McKeown, Noah A. Smith
PublisherAssociation for Computational Linguistics (ACL)
Pages44-48
Number of pages5
ISBN (Electronic)9781941643105
StatePublished - 2014
Event2014 ACL Workshop on Language Technologies and Computational Social Science, Science 2014 at the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Baltimore, United States
Duration: Jun 26 2014 → …

Publication series

NameProceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)0736-587X

Conference

Conference2014 ACL Workshop on Language Technologies and Computational Social Science, Science 2014 at the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014
Country/TerritoryUnited States
CityBaltimore
Period6/26/14 → …

ASJC Scopus subject areas

  • Computer Science Applications
  • Linguistics and Language
  • Language and Linguistics

Fingerprint

Dive into the research topics of 'Optimizing Features in Active Machine Learning for Complex Qualitative Content Analysis'. Together they form a unique fingerprint.

Cite this