Expectation-maximization for low-SNR multi-reference alignment

We study the multi-reference alignment (MRA) problem of recovering a signal from noisy observations acted on by unknown random circular shifts. While the information-theoretic limits of MRA are well characterized in many settings, the algorithmic behavior at low signal-to-noise ratio (SNR), the regime of practical interest, remains poorly understood. In this paper, we analyze the expectation-maximization (EM) algorithm, a widely used method for MRA, and characterize its convergence dynamics and initialization dependence in the low-SNR limit. On the convergence side, we prove a two-phase phenomenon near the ground truth as $\mathrm{SNR}\to 0$: an initial contraction with error decaying as $\exp(-, \mathrm{SNR} \cdot t)$ followed by a much slower phase scaling as $\exp(- ,\mathrm{SNR}^2 \cdot t)$, where $t$ is the iteration number. This yields an iteration-complexity lower bound $T \gtrsim \mathrm{SNR}^{-2}$ to reach a small fixed target accuracy, revealing a severe computational bottleneck at low SNR. We also identify a finite-sample instability, which we term \emph{Ghost of Newton}, in which EM initially approaches the ground truth but later diverges, degrading reconstruction quality. On the bias side, we analyze EM in the noise-only setting ($\mathrm{SNR}=0$), a regime referred to as Einstein from Noise, to highlight its pronounced sensitivity to initialization. We prove that the EM map preserves the Fourier phases of the initialization across all iterations, while the corresponding Fourier magnitudes contract toward zero at a slow rate of $(1+T)^{-1/2}$. Consequently, although the amplitudes vanish in the limit of $T \to \infty$ iterations, the reconstructed structure continues to reflect the geometry encoded by the template’s Fourier phases. Together, these results expose fundamental computational and initialization-driven limitations of EM for MRA in the low-SNR regime.

💡 Research Summary

**
This paper provides a comprehensive theoretical investigation of the Expectation‑Maximization (EM) algorithm applied to the multi‑reference alignment (MRA) problem in the low‑signal‑to‑noise‑ratio (SNR) regime, a setting that is highly relevant for many practical imaging applications such as cryo‑electron microscopy. The authors first formalize the MRA model: each observation is a randomly circularly shifted copy of an unknown signal x★ corrupted by additive Gaussian noise, with SNR defined as ‖x★‖²/(dσ²). EM iteratively alternates between an E‑step that computes posterior probabilities (responsibilities) over all possible shifts for each observation, and an M‑step that updates the signal estimate as a weighted average of the observations using those responsibilities. While EM is guaranteed to increase the likelihood, its behavior when the noise dominates the signal has remained poorly understood.

The core contribution lies in a two‑phase convergence analysis of the population EM operator (the limit as the number of samples n → ∞). By expanding the Jacobian of the EM map in powers of SNR, the authors show that the leading linear term scales with SNR and the next‑order term scales with SNR². Consequently, when the current estimate is close to the true signal, the error contracts initially at a rate proportional to exp(−c·SNR·t). After a transient period, the quadratic term dominates and the contraction slows to exp(−c′·SNR²·t). This “two‑phase” behavior leads directly to an iteration‑complexity lower bound: to achieve any fixed accuracy ε, EM must run for at least T ≳ SNR⁻² iterations. In practical terms, when SNR is on the order of 10⁻³, thousands of EM iterations are required, which explains the severe computational bottleneck observed in low‑SNR experiments.

Beyond the asymptotic population analysis, the paper tackles the finite‑sample setting. Here the responsibilities are estimated from a finite number of observations, introducing stochastic fluctuations. The authors identify a novel instability they call the “Ghost of Newton”: for modest sample sizes, EM initially follows the population trajectory and reduces error, but stochastic deviations eventually cause the iterates to diverge from the true signal. By carefully bounding higher‑order moments of the empirical responsibilities, they prove that a sample complexity of n ≳ C·SNR⁻³ is necessary to avoid this phenomenon. This matches known information‑theoretic limits but reveals that EM requires essentially the same scaling to be algorithmically stable.

The paper also explores the extreme case of SNR = 0, i.e., observations consist solely of noise. In this “Einstein from Noise” regime, the EM map preserves the Fourier phases of the initial template for all iterations while the Fourier magnitudes decay as (1+T)⁻¹/². Hence, even after infinitely many iterations the amplitudes vanish, but the reconstructed object still reflects the geometric structure encoded in the initial phases. In high dimensions, finite‑sample effects cause a slow drift of these phases, which the authors quantify and demonstrate experimentally. This analysis highlights a subtle bias: EM can produce seemingly structured reconstructions even when no signal is present, solely due to the choice of initialization.

To mitigate the identified issues, the authors propose a mini‑batch EM scheme that reduces variance in the responsibility estimates and empirically alleviates both the Ghost of Newton and phase‑drift problems. They also discuss connections to alternative methods such as method‑of‑moments estimators, which achieve the information‑theoretic sample complexity but avoid the iterative bias inherent to EM.

In summary, the paper establishes that in the low‑SNR regime EM suffers from two fundamental limitations: (1) a computational barrier requiring O(SNR⁻²) iterations to converge, and (2) a statistical barrier requiring O(SNR⁻³) samples to prevent divergence. Moreover, EM’s dependence on the initial template’s Fourier phases persists even in pure noise, leading to potential confirmation bias. These findings provide a rigorous foundation for understanding why EM often underperforms in high‑noise imaging tasks and motivate the development of alternative algorithms or improved initialization and variance‑reduction strategies for practical low‑SNR applications.