Arxiv 2512.16855

Current LLM compression strategies predominantly utilize quantization and pruning. Quantization methods reduce model precision, but uniform approaches like 4-bit or 8-bit quantization often degrade performance on tasks involving long-range dependencies or nuanced contextual coherence [6]. Mixed-precision quantization, which allocates variable bit-widths across layers, improves efficiency but introduces an immense combinatorial search space, growing as B 2L for an L-layer model with B bit-width options [7]. Similarly, pruning removes redundant attention heads or feed-forward neurons [8], yet aggressive pruning can undermine a model’s sequential coherence and context handling [9]. Combining quantization with pruning exacerbates these challenges, exponentially enlarging the search space. Moreover, existing approaches typically rely on expensive retraining or knowledge distillation [10], which may be impractical in resourcelimited scenarios [11]. Recent automated strategies based on rein-forcement learning [12] or Bayesian optimization [13] still primarily depend on heuristic approaches without formal assurances for preserving essential linguistic properties like coherence, factual accuracy, or contextual consistency [14], [15], [16]. Additionally, these methods often overlook layer-specific sensitivities to compression, despite evidence suggesting different layers serve distinct linguistic purposes [17]. Thus, an automated, formally grounded compression approach that systematically addresses these issues is urgently needed.

To overcome these limitations, we introduce a novel framework, Temporal Logic-Guided Compression of Large Language Models (TOGGLE). TOGGLE utilizes Signal Temporal Logic (STL) [18]-a formal specification language-to define and preserve critical linguistic properties during compression. Leveraging STL, TOGGLE employs robustness-guided Bayesian optimization to systematically explore the joint quantization-pruning space (layer-wise bit-widths and pruning ratios), ensuring the resulting compressed models meet formally specified behavioral requirements. Additionally, TOGGLE supports runtime adaptability by dynamically controlling the trade-off between inference quality and energy efficiency through configurable operating modes. Our key contributions are as follows:

Large Language Models (LLMs) are mainly built upon the Transformer architecture, which is known for revolutionizing Natural Language Processing (NLP) tasks such as text generation, summarization, and reasoning. The core innovation of the transformer lies in its attention mechanism, particularly multi-head attention, which prioritizes relevant input elements by computing attention scores across the sequence. For each input token, the model derives three parameters: Query (Q), Key (K), and Value (V ), calculated through linear transformations applied to the input embeddings. Attention scores assess token importance, and masks can exclude specific tokens, such as padding. Each head processes distinct subsets of Q, K, and V , enabling parallel analysis of diverse token relationships. The outputs of individual heads are concatenated and linearly combined to produce the final attention output, enhancing the model’s ability to capture complex dependencies.

To formalize the compression problem, we define the following variables and notations (summarized in Table I). Let Mbase represent the uncompressed base LLM and L = {l1, l2, . . . , ln} be the set of layers in Mbase, where n is the total number of layers. The model parameters (weights and biases) are denoted collectively as W . Let x = (x1, x2, . . . , xt) denote the sequence of input tokens processed by the LLM model up to generation step t, where each xi is a token from the vocabulary V . We define Ccomponents as the set of distinct parameter groups within each layer that can be targeted for compression. For instance,

…(Full text truncated)…