Scalability and Optimization Strategies for GPU Enhanced Neural Networks (GeNN)

Simulation of spiking neural networks has been traditionally done on high-performance supercomputers or large-scale clusters. Utilizing the parallel nature of neural network computation algorithms, GeNN (GPU Enhanced Neural Network) provides a simulation environment that performs on General Purpose NVIDIA GPUs with a code generation based approach. GeNN allows the users to design and simulate neural networks by specifying the populations of neurons at different stages, their synapse connection densities and the model of individual neurons. In this report we describe work on how to scale synaptic weights based on the configuration of the user-defined network to ensure sufficient spiking and subsequent effective learning. We also discuss optimization strategies particular to GPU computing: sparse representation of synapse connections and occupancy based block-size determination.

💡 Research Summary

The paper presents a comprehensive study of scalability and performance‑optimisation techniques for GeNN (GPU Enhanced Neural Network), a simulation framework that enables large‑scale spiking neural network (SNN) experiments on commodity NVIDIA GPUs. Traditional SNN simulators rely on supercomputers or large clusters because the sheer number of neurons and synapses demands massive parallelism and memory bandwidth. GeNN addresses this gap by employing a code‑generation pipeline: users describe neuron populations, synaptic connection densities, and neuron models (e.g., LIF, Izhikevich) in a high‑level language, and GeNN automatically emits highly‑optimised CUDA kernels and host‑side C++ code. This approach removes the need for low‑level GPU programming while guaranteeing that the generated code respects the hardware’s architectural constraints.

Two core contributions are introduced. First, the authors develop a synaptic‑weight scaling method that compensates for the dilution of input current that occurs as network size grows. By analysing the total number of synapses (N_syn) and the user‑specified connection probability (p_conn), they derive a scaling factor that adjusts each weight before simulation starts:
\