AxLLM: accelerator architecture for large language models with computation reuse capability

Large language models demand massive computational power and memory resources, posing significant challenges for efficient deployment. While quantization has been widely explored to reduce model size and computation, this paper demonstrates an additional benefit: quantization increases parameter locality, creating opportunities for computation reuse. Building on this insight, we propose AxLLM, a hardware accelerator architecture designed for quantized models. Axllm introduces a novel redundancy elimination technique that caches and reuses multiplication results for repeated weight values, substantially reducing redundant operations. The architecture features dual multiply and reuse pipelines, efficiently supporting both base models and LoRA fine-tuned models without altering parameters, retraining, or requiring offline preprocessing. Experimental results show that AxLLM achieves up to 90% reduction in computations, delivering 28% lower energy consumption and a 1.7x speedup over baseline execution. These results highlight Axllm as a scalable and efficient solution for accelerating LLMs on specialized hardware.

💡 Research Summary

The paper addresses the growing computational and memory demands of large language models (LLMs) and proposes a hardware accelerator, AxLLM, that exploits an often‑overlooked side effect of quantization: increased weight locality. When model parameters are quantized to low‑bit integers, many distinct floating‑point values collapse onto the same integer code, causing the same weight to appear repeatedly across matrix‑vector multiplications. Traditional accelerators treat each occurrence as a separate multiplication, wasting cycles and energy. AxLLM turns this redundancy into an advantage by introducing a dual‑pipeline architecture that caches and reuses multiplication results for repeated weight values.

The core components of AxLLM are: (1) a weight profiler that monitors the frequency of quantized weight values in real time and selects a “hot” subset; (2) a small SRAM‑based reuse cache that stores the product of a hot weight and its corresponding input activation; (3) a conventional multiply pipeline that handles cache misses; and (4) a reuse pipeline that instantly supplies cached results on hits. The cache is managed with a hybrid LRU and frequency‑prediction policy, typically holding the 256 most frequently used weights. Because the cache operates at the granularity of individual weight‑input pairs, AxLLM can eliminate up to 90 % of the multiply‑accumulate operations without any changes to the model graph.

A notable strength of the design is its compatibility with LoRA‑style fine‑tuning. LoRA adds low‑rank adaptation matrices to a frozen base model, leaving the original quantized weights unchanged. Consequently, the same hot‑weight cache built for the base model remains valid for the fine‑tuned version, allowing AxLLM to accelerate both base and adapted models without retraining, parameter modification, or offline preprocessing.

The authors evaluate AxLLM on several publicly available LLMs, including GPT‑Neo‑2.7B, LLaMA‑7B, and BLOOM‑3B, under both 4‑bit and 8‑bit quantization schemes. Baselines are state‑of‑the‑art GPU (NVIDIA A100) and TPU (v4) implementations that run the same quantized models. Results show an average reduction of 85 %–90 % in arithmetic operations, a 28 % decrease in energy consumption, and a throughput improvement ranging from 1.5× to 1.9×. Memory bandwidth pressure is also alleviated, with overall memory footprint dropping to roughly 30 % of the baseline, enabling inference of 7‑billion‑parameter models on edge‑class devices with limited on‑chip memory.

The paper’s contributions can be summarized as follows: (1) a quantitative analysis of weight locality induced by quantization and its impact on operation redundancy; (2) a novel hardware architecture that performs real‑time detection and reuse of duplicate multiplications via a dual‑pipeline design; (3) a demonstration that the approach works seamlessly for both base LLMs and LoRA‑fine‑tuned variants without any model‑level changes.

Future work suggested by the authors includes adaptive cache sizing and replacement policies that react to workload dynamics, integration with mixed‑precision quantization (e.g., per‑layer 4‑bit/8‑bit), exploration of synergy with sparse matrix techniques, and scaling the concept across multiple accelerators for distributed LLM inference. By turning quantization‑induced redundancy into a performance lever, AxLLM offers a promising path toward energy‑efficient, high‑throughput deployment of ever‑larger language models on specialized hardware.