Maximally Informative Observables and Categorical Perception

We formulate the problem of perception in the framework of information theory, and prove that categorical perception is equivalent to the existence of an observable that has the maximum possible information on the target of perception. We call such an observable maximally informative. Regardless whether categorical perception is real, maximally informative observables can form the basis of a theory of perception. We conclude with the implications of such a theory for the problem of speech perception.

💡 Research Summary

The paper reframes perception as an information‑theoretic problem by treating the perceptual target (X) and the sensory observation (Y) as random variables. Mutual information I(X;Y) quantifies how much uncertainty about X is reduced by observing Y. The authors define a “maximally informative observable” (MIO) as an observation that achieves the theoretical upper bound I(X;Y)=H(X), which occurs precisely when the conditional entropy H(X|Y) equals zero. In this situation Y determines X uniquely, even though Y may vary continuously in the physical world. The authors prove that categorical perception—where a continuous stimulus dimension is perceived as discrete categories—is mathematically equivalent to the existence of an MIO.

The paper then explores the practical implications of MIOs. Real sensory data often consist of multiple dimensions (e.g., frequency, amplitude, temporal pattern). Each dimension may provide only partial information about X, forming “sub‑maximally informative observables.” By combining several sub‑MIOs, the brain can approximate the full information content of X, a process that aligns with known multisensory integration mechanisms.

Applying the framework to speech perception, the authors argue that certain acoustic cues (such as formant trajectories) act as MIOs for phonemic categories. When a particular acoustic pattern maps one‑to‑one onto a phoneme, listeners experience a sharp categorical boundary despite continuous acoustic variation. Variability introduced by accent, prosody, or speaking rate can degrade the MIO, requiring auxiliary cues (context, lexical knowledge) or adaptive learning to restore near‑maximal information. The paper models this adaptation using Bayesian updating and information‑maximization learning rules, suggesting concrete experimental designs (psychophysical tasks, neuroimaging) to test for the presence of MIOs.

In conclusion, the authors propose that the concept of a maximally informative observable provides a unifying, quantitative foundation for categorical perception across sensory modalities. It bridges classic psychophysical observations with modern computational neuroscience, offering testable predictions for speech perception, language acquisition, and the design of artificial speech‑recognition systems. By focusing on the information content of sensory signals rather than their raw physical properties, the framework promises a more principled understanding of how the brain extracts discrete perceptual categories from continuous environmental input.