Prof. Naftali Tishby

Past-future information bottleneck in dynamical systems.

Creutzig, F, Globerson A, Tishby N.  2009.  

Abstract:

Biological systems need to process information in real time and must trade off accuracy of presentation and coding costs. Here we operationalize this trade-off and develop an information-theoretic framework that selectively extracts information of the input past that is predictive about the output future, obtaining a generalized eigenvalue problem. Thereby, we unravel the input history in terms of structural phase transitions corresponding to additional dimensions of a state space. We elucidate the relation to canonical correlation analysis and give a numerical example. Altogether, this work relates information-theoretic optimization to the joint problem of system identification and model reduction.

Journal:

Physical review. E, Statistical, nonlinear, and soft matter physics

Volume:

79

Issue:

4 Pt 1

Pagination:

041925

Date Published:

2009 Apr

Custom 1:

http://www.ncbi.nlm.nih.gov/pubmed/19518274?dopt=Abstract