ELSC Media

Bert Kappen: Integrating control, inference and learning. Is it what the brain does?

Information, Control, and Learning: The Ingredients of Intelligent Behavior.

September 2016, Jerusalem


Intelligent systems, whether natural or artificial, must act in a world that is highly unpredictable. To plan actions with uncertainty is a stochastic optimal control problem. However, there are two fundamental problems: the optimal control solution is intractable to compute and intractable to represent due the non-trivial state dependence of the optimal control. This has prevented application of stochastic optimal control theory to robotics or as a model for the brain so far. The path integral control theory describes a class of control problems whose solution can be computed as an inference computation through Monte Carlo sampling. The sampling can be made more efficient by adaptive importance sampling. This defines a recursive learning problem, where a better importance sampler is learned from self-generated data. I formalize the intuitive notion that the efficiency of the importance sampling is related to the proximity of the sampling control to the optimal control. Secondly, I show how parametrized feed-back control functions can be estimated using the cross entropy method . I finally discuss how these ideas can be used as an abstract model for sensorimotor control.

See video