Hardware-Friendly Input Expansion for Accelerating Function Approximation

One-dimensional function approximation is a fundamental problem in scientific computing and engineering applications. While neural networks possess powerful universal approximation capabilities, their optimization process is often hindered by flat loss landscapes induced by parameter-space symmetries, leading to slow convergence and poor generalization, particularly for high-frequency components. Inspired by the principle of \emph{symmetry breaking} in physics, this paper proposes a hardware-friendly approach for function approximation through \emph{input-space expansion}. The core idea involves augmenting the original one-dimensional input (e.g., $x$) with constant values (e.g., $π$) to form a higher-dimensional vector (e.g., $[π, π, x, π, π]$), effectively breaking parameter symmetries without increasing the network’s parameter count. We evaluate the method on ten representative one-dimensional functions, including smooth, discontinuous, high-frequency, and non-differentiable functions. Experimental results demonstrate that input-space expansion significantly accelerates training convergence (reducing LBFGS iterations by 12% on average) and enhances approximation accuracy (reducing final MSE by 66.3% for the optimal 5D expansion). Ablation studies further reveal the effects of different expansion dimensions and constant selections, with $π$ consistently outperforming other constants. Our work proposes a low-cost, efficient, and hardware-friendly technique for algorithm design.

💡 Research Summary

**
The paper addresses a fundamental challenge in scientific computing: approximating one‑dimensional functions with neural networks. Although feed‑forward networks are universal approximators, their training often suffers from flat regions in the loss landscape caused by parameter‑space symmetries (e.g., hidden‑layer neuron permutations). These symmetries generate many equivalent minima, slow convergence, and poor generalization, especially for functions with high‑frequency components or discontinuities.

Inspired by the physics concept of symmetry breaking, the authors propose a hardware‑friendly “input‑space expansion” technique. The original scalar input (x) is padded with a constant value (c) (typically (\pi)) to create a higher‑dimensional vector, e.g., (