Approaching the Thermodynamic Limit with Neural-Network Quantum States

Accessing the thermodynamic-limit properties of strongly correlated quantum matter requires simulations on very large lattices, a regime that remains challenging for numerical methods, especially in frustrated two-dimensional systems. We introduce the Spatial Attention mechanism, a minimal and physically interpretable inductive bias for Neural-Network Quantum States, implemented as a single learned length scale within the Transformer architecture. This bias stabilizes large-scale optimization and enables access to thermodynamic-limit physics through highly accurate simulations on unprecedented system sizes within the Variational Monte Carlo framework. Applied to the spin-$\tfrac12$ triangular-lattice Heisenberg antiferromagnet, our approach achieves state-of-the-art results on clusters of up to $42\times42$ sites. The ability to simulate such large systems allows controlled finite-size scaling of energies and order parameters, enabling the extraction of experimentally relevant quantities such as spin-wave velocities and uniform susceptibilities. In turn, we find extrapolated thermodynamic limit energies systematically better than those obtained with tensor-network approaches such as iPEPS. The resulting magnetization is strongly renormalized, $M_0=0.148(1)$ (about $30%$ of the classical value), revealing that less accurate variational states systematically overestimate magnetic order. Analysis of the optimized wave function further suggests an intrinsically non-local sign structure, indicating that the sign problem cannot be removed by local basis transformations. We finally demonstrate the generality of the method by obtaining state-of-the-art energies for a $J_1$-$J_2$ Heisenberg model on a $20\times20$ square lattice, outperforming Residual Convolutional Neural Networks.

💡 Research Summary

The paper tackles the long‑standing challenge of accessing thermodynamic‑limit properties of strongly correlated quantum matter, especially in frustrated two‑dimensional systems where conventional methods either suffer from the sign problem or cannot scale to the required lattice sizes. The authors introduce a physically motivated inductive bias—called Spatial Attention—into Transformer‑based Neural‑Network Quantum States (NQS). In standard Vision‑Transformer (ViT) NQS, the attention weights are learned without any explicit spatial structure, which makes optimization unstable as the system size grows. Spatial Attention modifies the attention mechanism by multiplying each weight α₍ᵢⱼ₎ with a distance‑dependent factor exp(−γ d(i,j)), where d(i,j) is the Euclidean distance between lattice sites i and j, and γ is a learnable inverse length scale. Each attention head can have its own γ, allowing the network to capture multiple correlation lengths simultaneously. This simple one‑line change provides a soft geometric prior that respects the cluster property of local Hamiltonians while preserving global connectivity.

The authors first benchmark the method on the spin‑½ Heisenberg antiferromagnet on the triangular lattice, a paradigmatic frustrated model that cannot be treated with sign‑problem‑free quantum Monte Carlo. Using periodic L×L clusters with L up to 42 (≈1764 spins), they achieve variational energies per site that are significantly lower than those obtained with Gutzwiller‑projected states, Recurrent Neural Network (RNN) NQS, and earlier ViT implementations. The energies follow the expected 1/L³ finite‑size scaling for L ≥ 24, enabling a reliable extrapolation to the thermodynamic limit. The extrapolated ground‑state energy is E₀ = −0.55168(2) J, which is lower than the most accurate infinite‑bond‑dimension iPEPS estimates, demonstrating that the Spatial‑Attention ViT captures correlations beyond those accessible to current tensor‑network methods.

Magnetic order is probed via the static spin structure factor at the 120° Néel ordering wave vector. Finite‑size scaling of the order parameter yields a thermodynamic‑limit magnetization M₀ = 0.148(1), only about 30 % of the classical value. This is the smallest magnetization reported for this model and indicates strong quantum fluctuations. Linear spin‑wave theory predicts M ≈ 0.239, overestimating the true value by roughly 62 %, highlighting the importance of non‑perturbative quantum effects. By testing several proposed local sign‑rule transformations on the optimized wave function, the authors find that the sign overlap decays exponentially with system size, implying that the sign structure is intrinsically non‑local and cannot be eliminated by simple basis rotations.

To demonstrate generality, the same Spatial‑Attention ViT is applied to the J₁‑J₂ Heisenberg model on a 20×20 square lattice. The method outperforms state‑of‑the‑art Residual Convolutional Neural Networks, achieving lower variational energies with a comparable number of parameters (≈4.5 × 10⁵). Importantly, the parameter count remains essentially constant across all lattice sizes, evidencing true size‑consistency: the Ansatz does not degrade as the system grows, a common failure mode of many variational wave functions.

Overall, the work shows that embedding a minimal, physically interpretable distance bias into Transformer‑based NQS dramatically stabilizes large‑scale optimization, permits controlled finite‑size scaling, and yields unprecedented accuracy for frustrated 2D quantum magnets. The approach opens the door to systematic studies of thermodynamic‑limit observables—energies, order parameters, excitation velocities, susceptibilities—in models where previous methods were either inaccurate or inapplicable. It also provides quantitative evidence that certain frustrated systems possess an intrinsic, non‑local sign problem, a insight that may guide future algorithmic developments.