Bitwidth-Specific Logarithmic Arithmetic for Future Hardware-Accelerated Training

📝 Original Info

• Title: Bitwidth-Specific Logarithmic Arithmetic for Future Hardware-Accelerated Training
• ArXiv ID: 2510.17058
• Date: 2025-10-20
• Authors: ** 논문에 명시된 저자 정보가 제공되지 않았습니다. (저자 이름 및 소속을 확인하려면 원문을 참고하십시오.) **

📝 Abstract

While advancements in quantization have significantly reduced the computational costs of inference in deep learning, training still predominantly relies on complex floating-point arithmetic. Low-precision fixed-point training presents a compelling alternative. This work introduces a novel enhancement in low-precision logarithmic fixed-point training, geared towards future hardware accelerator designs. We propose incorporating bitwidth in the design of approximations to arithmetic operations. To this end, we introduce a new hardware-friendly, piece-wise linear approximation for logarithmic addition. Using simulated annealing, we optimize this approximation at different precision levels. A C++ bit-true simulation demonstrates training of VGG-11 and VGG-16 models on CIFAR-100 and TinyImageNet, respectively, using 12-bit integer arithmetic with minimal accuracy degradation compared to 32-bit floating-point training. Our hardware study reveals up to 32.5% reduction in area and 53.5% reduction in energy consumption for the proposed LNS multiply-accumulate units compared to that of linear fixed-point equivalents.

💡 Deep Analysis

📄 Full Content

Reference