CNI Speaker Series: Nathaniel Daw

Nathaniel Daw
Princeton Neuroscience Institute

Evaluation in sequential decision tasks: habits and beyond

Because of the computational complexity of exactly computing utilities in expectation over a series of future states, it is believed that human and animal brains use a range of shortcuts to simplify or approximate evaluation in sequential tasks such as mazes or chess. In particular, pre-computing (via model-free reinforcement learning) and storing action preferences has been taken as a formal model of habits -- both healthy and in disorders such as drug abuse. I review evidence that humans and animals trade off deliberative and habitual strategies to action evaluation, and then turn to in-progress research investigating additional possibilities outside this dichotomy. These include partial pre-computation by storing decision variable precursors, and nonparametric estimation based on memories of individual episodes. Options like these enrich the notions of habit and deliberation, and also further complicate attempts to build a resource-rational account of self-control via weighing the costs and benefits of different ways of computing decision variables.


A pizza lunch will be served.