뇌 회로 기반 정책 게이팅 통합 모델 GateMod

📝 Abstract

The ability to compose acquired skills to plan and execute behaviors is a hallmark of natural intelligence. Yet, despite remarkable cross-disciplinary efforts, a principled account of how task structure shapes gating and how such computations could be delivered in neural circuits, remains elusive. Here we introduce GateMod, an interpretable theoretically grounded computational model linking the emergence of gating to the underlying decision-making task, and to a neural circuit architecture. We first develop GateFrame, a normative framework casting policy gating into the minimization of the free energy. This framework, relating gating rules to task, applies broadly across neuroscience, cognitive and computational sciences. We then derive GateFlow, a continuous-time energy based dynamics that provably converges to GateFrame optimal solution. Convergence, exponential and global, follows from a contractivity property that also yields robustness and other desirable properties. Finally, we derive a neural circuit from GateFlow, GateNet. This is a soft-competitive recurrent circuit whose components perform local and contextual computations consistent with known dendritic and neural processing motifs. We evaluate GateMod across two different settings: collective behaviors in multiagent systems and human decision-making in multi-armed bandits. In all settings, GateMod provides interpretable mechanistic explanations of gating and quantitatively matches or outperforms established models. GateMod offers a unifying framework for neural policy gating, linking task objectives, dynamical computation, and circuit-level mechanisms. It provides a framework to understand gating in natural agents beyond current explanations and to equip machines with this ability.

💡 Analysis

The ability to compose acquired skills to plan and execute behaviors is a hallmark of natural intelligence. Yet, despite remarkable cross-disciplinary efforts, a principled account of how task structure shapes gating and how such computations could be delivered in neural circuits, remains elusive. Here we introduce GateMod, an interpretable theoretically grounded computational model linking the emergence of gating to the underlying decision-making task, and to a neural circuit architecture. We first develop GateFrame, a normative framework casting policy gating into the minimization of the free energy. This framework, relating gating rules to task, applies broadly across neuroscience, cognitive and computational sciences. We then derive GateFlow, a continuous-time energy based dynamics that provably converges to GateFrame optimal solution. Convergence, exponential and global, follows from a contractivity property that also yields robustness and other desirable properties. Finally, we derive a neural circuit from GateFlow, GateNet. This is a soft-competitive recurrent circuit whose components perform local and contextual computations consistent with known dendritic and neural processing motifs. We evaluate GateMod across two different settings: collective behaviors in multiagent systems and human decision-making in multi-armed bandits. In all settings, GateMod provides interpretable mechanistic explanations of gating and quantitatively matches or outperforms established models. GateMod offers a unifying framework for neural policy gating, linking task objectives, dynamical computation, and circuit-level mechanisms. It provides a framework to understand gating in natural agents beyond current explanations and to equip machines with this ability.

📄 Content

Humans and other animals can dynamically combine previously acquired skills to plan and execute complex behaviors [98]. This ability -widely regarded as a hallmark of natural intelligence -is crucial to survival and flexible problem solving [52,64]. Yet, understanding how the brain implements this capability [33,83,98] and, consequently, how it could inspire computational models for autonomous decision-making agents [77,64,103] is a central challenge with broad implications across neuroscience, engineering and artificial intelligence (AI).

In neuroscience, a growing body of experimental evidence suggests that the prefrontal cortex (PFC) may play a key role in synthesizing complex decision policies for a given task by combining behavioral schemas. This process may be implemented via a gating mechanism that regulates information flow across brain circuits [81,45]. Theoretical work further suggests that compositional mechanisms can be modeled via architectures that combine Recurrent Neural Networks (RNNs) and mixture-of-experts (MoE) frameworks [104,98,48]. Here, a gating network -often implementing a softmax rule -modulates the use of the appropriate schemas/skills (i.e., the experts) based on the environment inputs and the underlying task. While impressive, these approaches yield insights that are often specific to the data, the task, and the network architecture considered.

To both advance a more general understanding of knowledge composition mechanisms and engineer such principles in autonomous agents, researchers have increasingly turned to robotics as a testbed for designing, validating, and benchmarking gating-based computational models, often using motor control tasks as reference problems [77]. Given a task, behavioral schemas are associated to primitives (reusable policies) that are combined into a single policy, with weights assigned by a gating mechanism. This approach has inspired the design of layered architectures such as MOSAIC and Hammer [107,27]. In these architectures, the output of fast, specialized, controllers is linearly combined by a slower, more flexible, mechanism. Weights are determined by a gating rule -again, a softmax. This gating rule is also central to sensorimotor control schemes based on the MoE framework [43], such as the one in [95].

Beyond robotics, the MoE framework [60,108,46] has become a cornerstone of modern AI systems, including Large Language Models (LLMs) and in-context decision-making methods [16,67]. Compared with the neuroscience literature, these advances focus primarily on computational principles, typically implementing them in artificial networks. Two main architectures to combine expert outputs are [16,60]:

(i) dense, fully activated MoE, where all experts contribute to the final output, typically via softmax gating [43]; (ii) sparse, selective MoE, where only a subset of experts is chosen, typically determined via an argmax-based selection [90,30] or Gumbel-softmax [44,68]. In brief, the gating rule, a core determinant of the performance, is selected by the network designer rather than emerging from the properties of the task.

Despite remarkable cross-disciplinary efforts, most explanations remain tied to specific network architectures, tasks, and datasets. As a result, a principled account of how the task shapes gating computations and, in turn, how such computations drive the organization of the neural circuits that implement them, remains elusive. What appears to be missing is a theoretically grounded and interpretable computational model that applies broadly across neuroscience, cognitive science, and machine learning. Such a model should provide a unifying account of gating and integrate naturally with existing conceptual frameworks.

At present, it remains unclear what general and broadly applicable objective a given gating rule is optimizing, nor how its functional requirements are instantiated mechanistically in neural circuits.

To address this gap, we develop GateMod, a computational model grounded into the minimization of the free energy. GateMod yields a quantitative characterization of gating as an energy model. It casts gating within a variational, normative, formulation that explicitly relates task to gating mechanism to the energy landscape. It further specifies a neural circuit capable of implementing these computations, making the role of each circuit element interpretable in view of the task. In doing so, GateMod yields several implications, including highlighting the central role of in-context computation and suggests that dendritic processing may be a key biological substrate supporting gating mechanisms in natural circuits.

GateMod consists of three key components. The first is GateFrame, a normative framework for primitives gating formalized via an optimization problem. In GateFrame, the policy is computed by combining a set of primitives, e.g., schemas, skills for natural agents, or reusable sensorimotor controllers for artific

This content is AI-processed based on ArXiv data.