Addressing general measurements in quantum Monte Carlo

Quantum Monte Carlo is one of the most promising approaches for dealing with large-scale quantum many-body systems. It has played an extremely important role in understanding strongly correlated physics. However, two fundamental problems, namely the sign problem and general measurement issues, have seriously hampered its scope of application. We propose a universal scheme to tackle the problems of general measurement. The target observables are expressed as the ratio of two types of partition functions $\langle \mathrm{O} \rangle=\bar{Z}/Z$, where $\bar{Z}=\mathrm{tr} (\mathrm{Oe^{-βH}})$ and $Z=\mathrm{tr} (\mathrm{e^{-βH}})$. These two partition functions can be estimated separately within the reweight-annealing frame, and then be connected by an easily solvable reference point. We have successfully applied this scheme to XXZ model and transverse field Ising model, from 1D to 2D systems, from two-body to multi-body correlations and even non-local disorder operators, and from equal-time to imaginary-time correlations. The reweighting path is not limited to physical parameters, but also works for space and time. Essentially, this scheme solves the long-standing problem of calculating the overlap between different distribution functions in mathematical statistics, which can be widely used in statistical problems, such as quantum many-body computation, big data and machine learning.

💡 Research Summary

Quantum Monte Carlo (QMC) methods excel at simulating large quantum many‑body systems, but they have long suffered from two fundamental obstacles: the notorious sign problem and the difficulty of measuring off‑diagonal (general) observables. While the sign problem remains an open challenge, this paper tackles the second issue by introducing a universal “bipartite reweight‑annealing” (BRA) scheme that transforms any observable into a ratio of two partition functions, (\langle O\rangle = \bar Z / Z), where (\bar Z = \mathrm{tr}(O e^{-\beta H})) and (Z = \mathrm{tr}(e^{-\beta H})).

In standard QMC, diagonal observables share the same configuration space as the partition function, allowing a direct estimator (\langle O\rangle = \sum_i O_i W_i / \sum_i W_i). Off‑diagonal operators, however, generate a completely different set of configurations, so the two partition functions have no overlap and their ratio cannot be sampled directly. The authors resolve this by applying the reweight‑annealing idea—originally used to connect the same‑type partition functions at different physical parameters—to the two different partition functions separately.

The core of the method is to construct two independent annealing paths: one for (\bar Z) (red) and one for (Z) (blue). Along each path the system parameter (which can be a coupling constant, system size, distance, or even an auxiliary Hamiltonian interpolation) is changed in small increments (\delta J) such that adjacent probability distributions overlap strongly. For adjacent points the ratio of partition functions is obtained exactly by importance sampling, \