The brain interprets visual information through learned regularities, a computation formalized as probabilistic inference under a prior. The visual cortex establishes priors for this inference, some delivered through established top-down connections that inform low-level cortices about statistics represented at higher levels in the cortical hierarchy. While evidence shows that adaptation leads to priors reflecting the structure of natural images, it remains unclear whether similar priors can be flexibly acquired when learning a specific task. To investigate this, we built a generative model of V1 optimized for a simple discrimination task and analyzed it together with large-scale recordings from mice performing an analogous task. In line with recent approaches, we assumed that neuronal activity in V1 corresponds to latent posteriors in the generative model, enabling investigation of task-related priors in neuronal responses. To obtain a flexible test bed, we extended the VAE formalism so that a task can be acquired efficiently by reusing previously learned representations. Task-specific priors learned by this Task-Amortized VAE were used to investigate biases in mice and model when presenting stimuli that violated trained task statistics. Mismatch between learned task statistics and incoming sensory evidence produced signatures of uncertainty in stimulus category in the TAVAE posterior, reflecting properties of bimodal response profiles in V1 recordings. The task-optimized generative model accounted for key characteristics of V1 population activity, including within-day updates to population responses. Our results confirm that flexible task-specific contextual priors can be learned on demand by the visual system and deployed as early as the entry level of visual cortex.
Deep learning models, including discriminative and generative models, have been shown to successfully model neuronal responses in the visual system of the brain (Khaligh-Razavi & Kriegeskorte, 2014;Yamins & DiCarlo, 2016;Lotter et al., 2020;Csikor et al., 2025;Zhuang et al., 2021). These models were assessed through the efficiency of predicting neuronal responses to natural images or natural videos. However, visual cortical responses are not only determined by the stimulus itself but also by non-stimulus attributes, such as the task the visual system is faced with (De Lange et al., 2018;Lange & Haefner, 2017;2022). Notably, recent studies have demonstrated strong and systematic task-specific biases as early as the earliest stage of the visual cortex, the V1 (Corbo et al., Published as a conference paper at ICLR 2026ICLR 2022;;2025). Understanding these systematic biases requires that we understand the computational principles behind the changes occuring at the early stages of processing when learning a novel task.
A coherent normative framework for biological perception is probabilistic inference: learning is assumed to deliver a generative model of the environment, in which latent generative factors are inferred when a stimulus is observed (Yuille & Kersten, 2006;Fiser et al., 2010). In this context, neuronal activity in the visual cortex is interpreted as a representation of the posterior either by establishing a point estimate (Olshausen & Field, 1996;Schwartz & Simoncelli, 2001), approximating it through sampling (Lee & Mumford, 2003;Orbán et al., 2016), or through variational approximation (Ma et al., 2006), and biases can be formalized through learned priors. The entry stage to the visual cortex, the V1, learns elementary features of the environment (Hubel & Wiesel, 1959), suggesting a latent representation that has a close to linear relationship with the stimulus, i.e. suggesting a linear generative model underlying inferences (Olshausen & Field, 1996). The hierarchy of visual cortical regions permits a prior over the features represented in V1, as hierarchically higher layers that represent more complex statistics can establish contextual priors for lower layers through top-down connections (Lee & Mumford, 2003). Indeed, animal experiments established that higher level cortical activity delivers rich, structured influence on V1 (Lee & Nguyen, 2001;Chen et al., 2014;Kok et al., 2016;Ziemba et al., 2019) and these influences were shown to be aligned with the contextual priors acquired by a hierarchical generative model trained end-to-end on natural images (Csikor et al., 2025).
While these contextual priors reflect the regularities of the natural environment, these do not encompass more specific regularities that can arise when learning a task. In fact, it remains an open question if task-related contextual priors can be established in V1. In this paper we test the hypothesis that inference in a generative model that acquires task-specific priors underlies systematic biases in the V1 of task-trained mice.
To build a model of V1 that is primarily shaped by natural image statistics but is adapted to a task, we adopt a Variational Autoencoder approach (Kingma & Welling, 2013;Rezende et al., 2014) as these can perform flexible inference and thus provide an opportunity to investigate task-induced biases in the posterior. In this model, the latent variables of a Variational Autoencoder are identified with neurons of V1. Correspondence between the experiment and the model is established through matching the response properties of biological and model neurons. Latent activations are assumed to correspond to membrane potentials of neurons and firing rates are obtained by calculating the magnitude of responses (absolute value). We train the V1 model of VAE on natural images. To obtain representations matching that of V1, this VAE is constrained to have a linear generative model (Geadah et al., 2024), and use an extension that both produces more consistent performance in inference and better matching of contrast-dependent V1 responses (Catoni et al., 2024). Starting from the VAE model of V1, we extend the VAE formalism to adopt specific tasks, by adapting the variational posterior. In a standard approach, to obtain a new amortized variational posterior, the VAE needs to be retrained. This is neither efficient in terms of data requirements, nor biologically plausible, as it could steer away from a crucial representation obtained during the animal’s development. Instead, we are seeking a way to build upon a learned representation when adapting to a new context. We propose a principled extension of the VAE formalism that is capable of flexibly learning contextual priors. The Task-Amortized VAE (TAVAE) reuses the likelihood of the task-general VAE and obtains task-specific posteriors without retraining the original amortized posterior. We train the TAVAE on a discrimination task matching a task mice were trained
This content is AI-processed based on open access ArXiv data.