Spiking Neural Networks for Continuous Control via End-to-End Model-Based Learning

Despite recent progress in training spiking neural networks (SNNs) for classification, their application to continuous motor control remains limited. Here, we demonstrate that fully spiking architectures can be trained end-to-end to control robotic arms with multiple degrees of freedom in continuous environments. Our predictive-control framework combines Leaky Integrate-and-Fire dynamics with surrogate gradients, jointly optimizing a forward model for dynamics prediction and a policy network for goal-directed action. We evaluate this approach on both a planar 2D reaching task and a simulated 6-DOF Franka Emika Panda robot with torque control. In direct comparison to non-spiking recurrent baselines trained under the same predictive-control pipeline, the proposed SNN achieves comparable task performance while using substantially fewer parameters. An extensive ablation study highlights the role of initialization, learnable time constants, adaptive thresholds, and latent-space compression as key contributors to stable training and effective control. Together, these findings establish spiking neural networks as a viable and scalable substrate for high-dimensional continuous control, while emphasizing the importance of principled architectural and training design.

💡 Research Summary

The paper introduces a fully spiking, model‑based control architecture—Pred‑Control SNN—that enables end‑to‑end learning of high‑dimensional continuous robotic manipulation. The system consists of two spiking neural networks (SNNs) built from Leaky‑Integrate‑and‑Fire (LIF) or Adaptive‑LIF (ALIF) neurons: a forward (prediction) network that receives the current robot state and applied torque and predicts the next state change, and an inverse (policy) network that takes the current state together with a target state and outputs continuous torque commands. By rolling out the prediction network autoregressively during training, the authors obtain differentiable simulated trajectories; the distance between these simulated trajectories and the goal is back‑propagated through both networks using surrogate‑gradient techniques that approximate the non‑differentiable spike function (e.g., fast sigmoid or triangular surrogate).

Key methodological contributions include: (1) treating the entire control loop as a differentiable computational graph, allowing joint optimization of prediction and policy parameters; (2) learning neuron‑specific time constants and adaptive thresholds, which act as learnable eligibility traces and greatly improve temporal credit assignment; (3) employing fluctuation‑driven initialization to ensure sufficient spiking activity at the start of training; and (4) integrating regularization terms that control spike rates and L2 weight magnitude to stabilize learning.

The experimental evaluation covers two benchmark tasks: a planar 2‑D reaching task with a 3‑joint arm, and a simulated 6‑DOF Franka Emika Panda robot controlled via torque commands. In both settings, the spiking architecture is trained with the same data and hyper‑parameters as a non‑spiking recurrent baseline (GRU/RNN). Results show that the SNN achieves comparable or slightly better success rates (≈92 % for 2‑D, ≈88 % for 6‑DOF) and lower final position error, while using roughly one‑fifth the number of trainable parameters (≈0.8 M vs. 4.5 M). This demonstrates that spiking networks can retain expressive power for continuous control while offering substantial parameter‑efficiency, a prerequisite for low‑power neuromorphic deployment.

An extensive ablation study isolates the impact of four design choices. Removing learnable time constants or adaptive thresholds leads to unstable training and failure to converge. Omitting the fluctuation‑driven initialization drastically reduces spike activity, causing gradient blockage. Excluding latent‑space compression (i.e., keeping hidden layers wide) inflates the model size without meaningful performance gains. Finally, swapping the surrogate gradient for a naïve ReLU‑based approximation triggers gradient explosion. These findings pinpoint the essential components for stable SNN training in control contexts.

The authors argue that their work bridges a gap between deep‑learning‑style model‑based reinforcement learning and biologically plausible spiking computation. By proving that end‑to‑end surrogate‑gradient training can scale to high‑DoF, torque‑controlled robots, the paper opens the path toward energy‑efficient, neuromorphic robotic controllers that can learn predictive world models and policies jointly. Future directions suggested include real‑time deployment on neuromorphic hardware, extending the framework to more complex, partially observable dynamics, and integrating model‑based “dreaming” or imagination mechanisms to further improve sample efficiency. Overall, the study establishes spiking neural networks as a viable, scalable substrate for continuous motor control, provided that careful architectural and training design—particularly regarding temporal dynamics, initialization, and surrogate gradients—is observed.