Spiking Neural Network Architecture Search: A Survey

This survey paper presents a comprehensive examination of Spiking Neural Network (SNN) architecture search (SNNaS) from a unique hardware/software co-design perspective. SNNs, inspired by biological neurons, have emerged as a promising approach to neuromorphic computing. They offer significant advantages in terms of power efficiency and real-time resource-constrained processing, making them ideal for edge computing and IoT applications. However, designing optimal SNN architectures poses significant challenges, due to their inherent complexity (e.g., with respect to training) and the interplay between hardware constraints and SNN models. We begin by providing an overview of SNNs, emphasizing their operational principles and key distinctions from traditional artificial neural networks (ANNs). We then provide a brief overview of the state of the art in NAS for ANNs, highlighting the challenges of directly applying these approaches to SNNs. We then survey the state of the art in SNN-specific NAS approaches. Finally, we conclude with insights into future research directions for SNN research, emphasizing the potential of hardware/software co-design in unlocking the full capabilities of SNNs. This survey aims to serve as a valuable resource for researchers and practitioners in the field, offering a holistic view of SNNaS and underscoring the importance of a co-design approach to harness the true potential of neuromorphic computing.

💡 Research Summary

This survey provides a comprehensive overview of Spiking Neural Network architecture search (SNNaS) with a strong emphasis on hardware‑software co‑design. It begins by outlining the fundamental differences between spiking neural networks and conventional artificial neural networks, highlighting the event‑driven, temporally sparse nature of spikes, the variety of neuron models (Leaky‑Integrate‑and‑Fire, Hodgkin‑Huxley, Izhikevich), and the multiple encoding schemes (rate‑based, time‑to‑first‑spike, phase, burst, spatio‑temporal). These choices directly affect both algorithmic performance and hardware implementation cost, making the definition of the search space far more complex than in ANN‑NAS.

The paper then reviews the four principal training paradigms for SNNs—surrogate‑gradient back‑propagation, ANN‑to‑SNN conversion, spike‑timing‑dependent plasticity (STDP), and evolutionary optimization (EO). Each method solves the non‑differentiability problem in a different way, but also introduces distinct constraints on architecture design, energy consumption, and latency, especially when the target platform is a neuromorphic chip with limited weight resolution and synaptic connectivity.

SNNaS is framed as a three‑stage pipeline: (1) search‑space definition, (2) search strategy, and (3) performance evaluation. In the search‑space stage, recent works move beyond naïvely replacing ReLU with LIF neurons and instead employ cell‑based directed acyclic graphs, hierarchical motifs, and biologically inspired motifs that capture temporal dynamics. The survey catalogs the main search strategies—evolutionary algorithms, reinforcement learning, gradient‑based methods, and Bayesian optimization—pointing out that multi‑objective optimization (accuracy, energy, latency, memory footprint, spike sparsity) is intrinsic to SNN design.

Evaluation metrics are broadened beyond classification accuracy to include latency, energy per inference, and hardware resource utilization. Zero‑shot proxies, surrogate performance models, and one‑shot supernet techniques are described as “training‑free” or “training‑light” methods that can cut search cost by orders of magnitude, making SNNaS feasible for resource‑constrained research groups.

A central thesis of the survey is that hardware‑software co‑design is not optional; the temporal dynamics of spikes must be modeled together with hardware constraints from the outset. The authors present empirical evidence that co‑exploration of network topology and hardware configuration yields superior energy‑delay products compared with sequential optimization pipelines.

Finally, the paper outlines promising future directions: (i) more efficient surrogate‑gradient formulations that respect hardware quantization, (ii) hybrid Bayesian‑evolutionary frameworks that can handle high‑dimensional, multi‑objective spaces, (iii) closed‑loop hardware‑in‑the‑loop evaluation platforms to reduce the simulation‑to‑hardware gap, (iv) search spaces explicitly built around spatio‑temporal encoding and attention mechanisms, and (v) standardized toolchains and benchmarks that reflect event‑driven workloads rather than static image datasets. In sum, the survey positions SNNaS as an emerging but rapidly maturing field, arguing that integrating hardware‑aware search spaces, multi‑objective optimization, and training‑free evaluation will unlock the full potential of spiking networks for ultra‑low‑power AI at the edge.