TY - GEN
T1 - Structure learning in human sequential decision-making
AU - Acuña, Daniel
AU - Schrater, Paul
PY - 2009/12/1
Y1 - 2009/12/1
N2 - We use graphical models and structure learning to explore how people learn policies in sequential decision making tasks. Studies of sequential decision-making in humans frequently find suboptimal performance relative to an ideal actor that knows the graph model that generates reward in the environment. We argue that the learning problem humans face also involves learning the graph structure for reward generation in the environment. We formulate the structure learning problem using mixtures of reward models, and solve the optimal action selection problem using Bayesian Reinforcement Learning. We show that structure learning in one and two armed bandit problems produces many of the qualitative behaviors deemed suboptimal in previous studies. Our argument is supported by the results of experiments that demonstrate humans rapidly learn and exploit new reward structure.
AB - We use graphical models and structure learning to explore how people learn policies in sequential decision making tasks. Studies of sequential decision-making in humans frequently find suboptimal performance relative to an ideal actor that knows the graph model that generates reward in the environment. We argue that the learning problem humans face also involves learning the graph structure for reward generation in the environment. We formulate the structure learning problem using mixtures of reward models, and solve the optimal action selection problem using Bayesian Reinforcement Learning. We show that structure learning in one and two armed bandit problems produces many of the qualitative behaviors deemed suboptimal in previous studies. Our argument is supported by the results of experiments that demonstrate humans rapidly learn and exploit new reward structure.
UR - http://www.scopus.com/inward/record.url?scp=77952543081&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77952543081&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:77952543081
SN - 9781605609492
T3 - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
SP - 1
EP - 8
BT - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
T2 - 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
Y2 - 8 December 2008 through 11 December 2008
ER -