Efficient representation as a design principle for neural coding and computation

Efficient representation as a design principle for neural coding and computation William Bialek,a Rob R. de Ruyter van Steveninckb and Naftali Tishbyc aJoseph Henry Laboratories of Physics, Lewis–Sigler Institute for Integrative Genomics, and Princeton Center for Theoretical Physics, Princeton University, Princeton, NJ 08544 USA bDepartment of Physics and Program in Neuroscience, Indiana University, Bloomington, IN 47405, USA cSchool of Computer Science and Engineering, and Interdisciplinary Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel (Dated: October 25, 2018) Does the brain construct an efficient representation of the sensory world? We review progress on this question, focusing on a series of experiments in the last decade which use fly vision as a model system in which theory and experiment can confront each other. Although the idea of efficient representation has been productive, clearly it is incomplete since it doesn’t tell us which bits of sensory information are most valuable to the organism. We suggest that an organism which maximizes the (biologically meaningful) adaptive value of its actions given fixed resources should have internal representations of the outside world that are optimal in a very specific information theoretic sense: they maximize the information about the future of sensory inputs at a fixed value of the information about their past. This principle contains as special cases computations which the brain seems to carry out, and it should be possible to test this optimization directly. We return to the fly visual system and report the results of preliminary experiments that are in encouraging agreement with theory. I. INTRODUCTION Since Shannon’s original work [1] there has been the hope that information theory would provide not only a guide to the design of engineered communication sys- tems but also a framework for understanding information processing in biological systems. One of the most con- crete implementations of this idea is the proposal that computations in the brain serve to construct an efficient (perhaps even maximally efficient) representation of in- coming sensory data [2, 3, 4]. Since efficient coding schemes are matched, at least implicitly, to the distri- bution of input signals, this means that what the brain computes—perhaps down to the properties of individ- ual neurons—should be predictable from the statistical structure of the sensory world. This is a very attractive picture, and points toward general theoretical principles rather than just a set of small models for different small pieces of the brain. More precisely, this picture suggests a research program that could lead to an experimentally testable theory. Our research efforts, over several years, have been in- fluenced by these ideas of efficient representation. On the one hand, we have found evidence for this sort of optimization in the responses of single neurons in the fly visual system, especially once we developed tools for ex- ploring the responses to more naturalistic sensory inputs. On the other hand, we have been concerned that simple implementations of information theoretic optimization principles must be wrong, because they implicitly at- tach equal value to all possible bits of information about the world. In response to these concerns, we have been trying to develop alternative approaches, still grounded in information theory but not completely agnostic about the value of information. Guided by our earlier results, we also want to phrase these theoretical ideas in a way that suggests new experiments. What we have outlined here is an ambitious program, and certainly we have not reached anything like com- pletion. The invitation to speak at the International Symposium on Information Theory in 2006 seemed like a good occasion for a progress report, so that is what we present here. It is much easier to convey the sense of ‘work in progress’ when speaking than when writing, and we hope that the necessary formalities of text do not obscure the fact that we are still groping for the correct formulation of our ideas. We also hope that, incomplete as it is, others will find the current state of our under- standing useful and perhaps even provocative. II. SOME RESULTS FROM THE FLY VISUAL SYSTEM The idea of efficient representation in the brain has motivated a considerable amount of work over several decades. We begin by reviewing some of what has been done along these lines, focusing on one experimental test- ing ground, the motion sensitive neurons in the fly visual system. Many animals, in particular those that fly, rely on vi- sual motion estimation to navigate through the world. The sensory–motor system responsible for this task, loosely referred to as the optomotor control loop, has been the subject of intense investigation in the fly, both in behavioral [5] and in electrophysiological studies. In particular, Bishop and Keehn [6] described wide field motion sensitive cells in the fly’s lobula plate

…(Full text truncated)…

This content is AI-processed based on ArXiv data.