Knowledge Tracing to Model Learning in Online Citizen Science Projects

Kevin G Crowston, Carsten Oesterlund, Tae Kyoung Lee, Corey Jackson, Mahboobeh Harandi, Sarah Allen, Sara Bahaadini, Scotty Coughlin, Aggelos Katsaggelos, Shane Larson, Neda Rohani, Joshua Smith, Laura Trouille, Michael Zevin

Research output: Contribution to journalArticle

Abstract

We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

Original languageEnglish (US)
JournalIEEE Transactions on Learning Technologies
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

model learning
citizen
science
Systems science
Image classification
retirement
learning
Learning systems
promotion
simulation
ability

Keywords

  • Citizen science
  • machine learning
  • training

ASJC Scopus subject areas

  • Education
  • Engineering(all)
  • Computer Science Applications

Cite this

Knowledge Tracing to Model Learning in Online Citizen Science Projects. / Crowston, Kevin G; Oesterlund, Carsten; Lee, Tae Kyoung; Jackson, Corey; Harandi, Mahboobeh; Allen, Sarah; Bahaadini, Sara; Coughlin, Scotty; Katsaggelos, Aggelos; Larson, Shane; Rohani, Neda; Smith, Joshua; Trouille, Laura; Zevin, Michael.

In: IEEE Transactions on Learning Technologies, 01.01.2019.

Research output: Contribution to journalArticle

Crowston, KG, Oesterlund, C, Lee, TK, Jackson, C, Harandi, M, Allen, S, Bahaadini, S, Coughlin, S, Katsaggelos, A, Larson, S, Rohani, N, Smith, J, Trouille, L & Zevin, M 2019, 'Knowledge Tracing to Model Learning in Online Citizen Science Projects', IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2019.2936480
Crowston, Kevin G ; Oesterlund, Carsten ; Lee, Tae Kyoung ; Jackson, Corey ; Harandi, Mahboobeh ; Allen, Sarah ; Bahaadini, Sara ; Coughlin, Scotty ; Katsaggelos, Aggelos ; Larson, Shane ; Rohani, Neda ; Smith, Joshua ; Trouille, Laura ; Zevin, Michael. / Knowledge Tracing to Model Learning in Online Citizen Science Projects. In: IEEE Transactions on Learning Technologies. 2019.
@article{2622a4f8dd294761ba2894403c174038,
title = "Knowledge Tracing to Model Learning in Online Citizen Science Projects",
abstract = "We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.",
keywords = "Citizen science, machine learning, training",
author = "Crowston, {Kevin G} and Carsten Oesterlund and Lee, {Tae Kyoung} and Corey Jackson and Mahboobeh Harandi and Sarah Allen and Sara Bahaadini and Scotty Coughlin and Aggelos Katsaggelos and Shane Larson and Neda Rohani and Joshua Smith and Laura Trouille and Michael Zevin",
year = "2019",
month = "1",
day = "1",
doi = "10.1109/TLT.2019.2936480",
language = "English (US)",
journal = "IEEE Transactions on Learning Technologies",
issn = "1939-1382",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Knowledge Tracing to Model Learning in Online Citizen Science Projects

AU - Crowston, Kevin G

AU - Oesterlund, Carsten

AU - Lee, Tae Kyoung

AU - Jackson, Corey

AU - Harandi, Mahboobeh

AU - Allen, Sarah

AU - Bahaadini, Sara

AU - Coughlin, Scotty

AU - Katsaggelos, Aggelos

AU - Larson, Shane

AU - Rohani, Neda

AU - Smith, Joshua

AU - Trouille, Laura

AU - Zevin, Michael

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

AB - We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

KW - Citizen science

KW - machine learning

KW - training

UR - http://www.scopus.com/inward/record.url?scp=85071543202&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071543202&partnerID=8YFLogxK

U2 - 10.1109/TLT.2019.2936480

DO - 10.1109/TLT.2019.2936480

M3 - Article

AN - SCOPUS:85071543202

JO - IEEE Transactions on Learning Technologies

JF - IEEE Transactions on Learning Technologies

SN - 1939-1382

ER -