Structure learning in human sequential decision-making

Daniel Acuña, Paul Schrater

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

We use graphical models and structure learning to explore how people learn policies in sequential decision making tasks. Studies of sequential decision-making in humans frequently find suboptimal performance relative to an ideal actor that knows the graph model that generates reward in the environment. We argue that the learning problem humans face also involves learning the graph structure for reward generation in the environment. We formulate the structure learning problem using mixtures of reward models, and solve the optimal action selection problem using Bayesian Reinforcement Learning. We show that structure learning in one and two armed bandit problems produces many of the qualitative behaviors deemed suboptimal in previous studies. Our argument is supported by the results of experiments that demonstrate humans rapidly learn and exploit new reward structure.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
Pages1-8
Number of pages8
StatePublished - Dec 1 2009
Externally publishedYes
Event22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 - Vancouver, BC, Canada
Duration: Dec 8 2008Dec 11 2008

Publication series

NameAdvances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

Other

Other22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
CountryCanada
CityVancouver, BC
Period12/8/0812/11/08

ASJC Scopus subject areas

  • Information Systems

Fingerprint Dive into the research topics of 'Structure learning in human sequential decision-making'. Together they form a unique fingerprint.

  • Cite this

    Acuña, D., & Schrater, P. (2009). Structure learning in human sequential decision-making. In Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference (pp. 1-8). (Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference).