Analysis of Fourier Neural Operators via Effective Field Theory

Fourier Neural Operators (FNOs) have emerged as leading surrogates for solver operators for various functional problems, yet their stability, generalization and frequency behavior lack a principled explanation. We present a systematic effective field theory analysis of FNOs in an infinite-dimensional function space, deriving closed recursion relations for the layer kernel and four-point vertex and then examining three practically important settings-analytic activations, scale-invariant cases and architectures with residual connections. The theory shows that nonlinear activations inevitably couple frequency inputs to high frequency modes that are otherwise discarded by spectral truncation, and experiments confirm this frequency transfer. For wide networks, we derive explicit criticality conditions on the weight initialization ensemble that ensure small input perturbations maintain a uniform scale across depth, and we confirm experimentally that the theoretically predicted ratio of kernel perturbations matches the measurements. Taken together, our results quantify how nonlinearity enables neural operators to capture non-trivial features, supply criteria for hyperparameter selection via criticality analysis, and explain why scale-invariant activations and residual connections enhance feature learning in FNOs. Finally, we translate the criticality theory into a practical criterion-matched initialization (calibration) procedure; on a standard PDEBench Burgers benchmark, the calibrated FNO exhibits markedly more stable optimization, faster convergence, and improved test error relative to a vanilla FNO.

💡 Research Summary

This paper presents a systematic effective field theory (EFT) analysis of Fourier Neural Operators (FNOs) operating in infinite‑dimensional function spaces. By treating random weight initialization and stochastic gradient descent as sources of noise, the authors model the network as a statistical field and focus on connected correlators: the two‑point kernel K^{(l)} and the four‑point vertex V^{(l)}. In the infinite‑width limit, pre‑activations become Gaussian processes and the vertex scales as O(1/n), which yields closed layer‑wise recursions for the kernel.

A key insight is that pointwise nonlinearities become convolutional expansions in Fourier space: σ(g)̂(f)=∑_{m≥1}σ_m (ĝ∗…∗ĝ)(f). Consequently, nonlinear layers inevitably generate modes beyond the spectral truncation band, transferring energy from low‑frequency inputs to high‑frequency components that would otherwise be discarded. For analytic activations (e.g., tanh, polynomial) the m‑th term is suppressed by the network width as O(n^{-(m‑1)}), so high‑frequency leakage diminishes for wide networks. In contrast, scale‑invariant activations such as ReLU have σ_m independent of m, leading to strong frequency coupling, broad spectral support, and slower decay of correlations across depth.

The authors examine three practically important regimes:

1. Analytic activations – they derive explicit convolution‑type recursion formulas for the kernel and for parallel/perpendicular susceptibilities χ_{∥}, χ_{⊥}, which control signal propagation.

2. Scale‑invariant activations – kernels are described via a position‑space correlation ρ^{(l)}; the resulting spectral support is wide and the system exhibits a different universality class with slower decay of correlations.

3. Residual FNOs – introducing a residual gain γ yields the update Z^{(l+1)} = R^{(l+1)}σ(Z^{(l)}) + γ Z^{(l)}. EFT shows that γ controls the retention of post‑truncation energy; stability requires γ ≤ γ_c = χ_{∥}^{‑1} – 1.

For wide networks the paper provides explicit criticality conditions on the weight‑variance σ_w^2 and bias‑variance σ_b^2 that enforce χ_{∥}=χ_{⊥}=1. Under these conditions, the kernel’s scale remains uniform across layers, preventing signal explosion or vanishing. Empirical measurements of kernel perturbations across random initializations match the theoretical predictions within one standard deviation.

Leveraging the criticality analysis, the authors propose a practical “criterion‑matched initialization” (calibration) algorithm. The procedure automatically tunes σ_w and σ_b so that the susceptibilities equal one at initialization. When applied to the PDEBench Burgers benchmark, the calibrated FNO exhibits markedly more stable optimization, faster loss decay, and roughly 15 % lower test error compared with a vanilla, uncalibrated FNO.

Overall, the work quantifies how nonlinearity induces frequency transfer in FNOs, establishes rigorous design criteria for stable deep operator learning, and translates the theory into a concrete initialization scheme that improves real‑world surrogate modeling performance.