Resting Neurons, Active Insights: Improving Input Sparsification for Large Language Models

Large Language Models (LLMs) achieve state-of-the-art performance across a wide range of applications, but their massive scale poses significant challenges for both efficiency and interpretability. Structural pruning, which reduces model size by removing redundant computational units such as neurons, has been widely explored as a solution, and this study devotes to input sparsification, an increasingly popular technique that improves efficiency by selectively activating only a subset of entry values for each input. However, existing approaches focus primarily on computational savings, often overlooking the representational consequences of sparsification and leaving a noticeable performance gap compared to full models. In this work, we first reinterpret input sparsification as a form of dynamic structural pruning. Motivated by the spontaneous baseline firing rates observed in biological neurons, we introduce a small set of trainable spontaneous neurons that act as compensatory units to stabilize activations in sparsified LLMs. Experiments demonstrate that these auxiliary neurons substantially reduce the sparsification-induced performance gap while generalizing effectively across tasks.

💡 Research Summary

The paper tackles the efficiency‑accuracy trade‑off of large language models (LLMs) by reinterpreting input sparsification as a form of dynamic structural pruning and by introducing a biologically inspired corrective mechanism called “spontaneous neurons.”
Input sparsification works by zero‑masking low‑magnitude activation entries for each input, which can be mathematically expressed as Y = W·S(X) where S(·) selects a subset of input features. This is equivalent to pruning neurons on a per‑sample basis, i.e., dynamic neuron pruning. While this reduces the number of weight‑channel loads and yields wall‑clock speed‑ups, it also destabilizes the model’s internal representations because different inputs activate largely disjoint neuron subsets, leading to a noticeable performance gap relative to the dense model.
Inspired by spontaneous firing in biological neural circuits—baseline activity that persists without external stimuli—the authors propose adding a small set of trainable, input‑independent activation vectors α to each transformer block. The forward pass becomes Y = W·S(X) + W·α. Because α does not depend on X, the term W·α can be folded into a bias vector b after training, incurring zero extra computation at inference time.
Training proceeds in two stages. First, the base sparsification function S(·) is kept unchanged (e.g., the thresholds used in TEAL or CA‑TS). Second, a calibration dataset is used to minimize the KL divergence between the output distribution of the full dense model f(X) and that of the sparsified model augmented with α, i.e., L = KL( f(X) ‖ f(S(X); α) ). This distillation forces the spontaneous neurons to capture the average residual error introduced by sparsification, effectively learning a data‑driven bias that compensates for missing activations.
Theoretical analysis shows that if the residual error e(X)=WX−W·S(X) has a non‑zero mean, adding a constant bias b equal to E