When sufficiency is insufficient: the functional information bottleneck for identifying probabilistic neural representations

The neural basis of probabilistic computations remains elusive, even amidst growing evidence that humans and other animals track their uncertainty. Recent work has proposed that probabilistic representations arise naturally in task-optimized neural networks trained without explicitly probabilistic inductive biases. However, prior work has lacked clear criteria for distinguishing probabilistic representations, those that perform transformations characteristic of probabilistic computation, from heuristic neural codes that merely reformat inputs. We propose a novel information bottleneck framework, the functional information bottleneck (fIB), that crucially evaluates a neural representation based not only on its statistical sufficiency but also on its minimality, allowing us to disambiguate heuristic from probabilistic coding. To demonstrate the power of this framework, we study a variety of task-optimized neural networks that had been suggested to develop probabilistic representations in earlier work: networks trained to perform static inference tasks (such as cue combination and coordinate transformation) or dynamic state estimation tasks (Kalman filtering). In contrast to earlier claims, our minimality requirement reveals that probabilistic representations fail to emerge in these networks: they do not develop minimal codes of Bayesian posteriors in their hidden layer activities, and instead rely on heuristic input recoding. Therefore, it remains an open question under which conditions truly probabilistic representations emerge in neural networks. More generally, our work provides a stringent framework for identifying probabilistic neural codes. Thus, it lays the foundation for systematically examining whether, how, and which posteriors are represented in neural circuits during complex decision-making tasks.

💡 Research Summary

The paper tackles a fundamental problem in neuroscience and machine learning: how to determine whether a neural network’s internal activity truly encodes Bayesian posterior distributions, rather than merely reformatting its inputs. To address this, the authors introduce the functional Information Bottleneck (fIB), a framework that evaluates neural representations on two complementary criteria: (1) Statistical sufficiency – the representation must contain enough information to decode the target posterior p(z|X); and (2) Minimality – the representation must discard unnecessary information about the raw input X, such that X cannot be reconstructed from it. Only when both conditions are met does the representation qualify as a specific probabilistic representation.

Traditional Information Bottleneck (IB) methods rely on estimating mutual information I(X;R) and I(R;Z), which is notoriously difficult in high‑dimensional deep networks. fIB circumvents this by training probe decoders on the hidden activations of a fully trained “performer” network. A probe is first trained to predict the posterior; successful decoding demonstrates sufficiency. A second probe is trained to reconstruct the original input; poor reconstruction indicates minimality. If the second probe succeeds, the hidden activity is essentially a copy of the input, leading to the authors’ “readout fallacy”: sufficiency alone is insufficient to claim a probabilistic code.

The authors apply fIB to three families of task‑optimized networks that previous work suggested develop Bayesian representations:

1. Static cue combination – two Poisson neural populations (each 50 neurons) encode the same latent stimulus z, modulated by trial‑by‑trial gain variables ν (nuisance). The network must infer z from the two cues.
2. Static coordinate transformation – two independent populations encode latent variables z₁ and z₂; the network must output the sum z = z₁ + z₂, requiring marginalization over the two latent sources.
3. Dynamic Kalman filtering – a recurrent network estimates a 1‑D latent state zₜ from a sequence of noisy observations xₜ, with process noise Q and measurement noise ν.

All networks are trained without any explicit probabilistic inductive bias; they are simply optimized to minimize task loss. After training, the authors probe the hidden layers. The results are strikingly consistent across tasks:

• The posterior can be decoded from the hidden activity (sufficiency holds).
• The original input X can also be decoded with high accuracy (minimality fails).

Thus, the hidden representations are not compressed Bayesian codes but rather “copycat” transformations that retain essentially all input information. Even in the recurrent Kalman‑filtering case, where the network must integrate information over time, the hidden state does not exhibit the expected compression of past observations; instead, it preserves the full observation history.

The paper distinguishes strong versus weak probabilistic representations. A strong representation achieves true representational compression: downstream circuits can access only the posterior, with all nuisance information eliminated. A weak representation allows a linear readout to extract the posterior while still retaining extraneous input features in orthogonal subspaces. The networks examined satisfy neither condition; they retain full input information.

Key methodological contributions include:

• Probe‑based information measurement as a tractable alternative to mutual‑information estimation.
• Explicit minimality test, highlighting the readout fallacy that has plagued prior work.
• Clarification that the presence of a sufficient statistic (the raw input) does not imply a probabilistic code.

The authors also discuss limitations: the generative models are deliberately simple (Gaussian tuning, Poisson spiking) to keep the true posterior analytically tractable; the approach does not train networks with an IB objective, so α (the trade‑off parameter) is not explored; and the probe methodology, while practical, provides only an approximation of information content.

In conclusion, the study provides a stringent, experimentally feasible framework for identifying genuine probabilistic neural codes. It demonstrates that task‑optimized networks, even when performing Bayesian‑optimal behavior, do not automatically develop minimal Bayesian representations. This challenges the prevailing view that probabilistic computation emerges “for free” in well‑trained deep networks and underscores the need for explicit constraints or architectural pressures if one wishes to induce true Bayesian coding in artificial or biological systems.