Learning a distance measure from the information-estimation geometry of data

We introduce the Information-Estimation Metric (IEM), a novel form of distance function derived from an underlying continuous probability density over a domain of signals. The IEM is rooted in a fundamental relationship between information theory and estimation theory, which links the log-probability of a signal with the errors of an optimal denoiser, applied to noisy observations of the signal. In particular, the IEM between a pair of signals is obtained by comparing their denoising error vectors over a range of noise amplitudes. Geometrically, this amounts to comparing the score vector fields of the blurred density around the signals over a range of blur levels. We prove that the IEM is a valid global distance metric and derive a closed-form expression for its local second-order approximation, which yields a Riemannian metric. For Gaussian-distributed signals, the IEM coincides with the Mahalanobis distance. But for more complex distributions, it adapts, both locally and globally, to the geometry of the distribution. In practice, the IEM can be computed using a learned denoiser (analogous to generative diffusion models) and solving a one-dimensional integral. To demonstrate the value of our framework, we learn an IEM on the ImageNet database. Experiments show that this IEM is competitive with or outperforms state-of-the-art supervised image quality metrics in predicting human perceptual judgments.

💡 Research Summary

The paper introduces the Information‑Estimation Metric (IEM), a novel distance function derived from the geometry of an underlying continuous probability density over a signal domain. The authors start from a fundamental information‑estimation relationship: the pointwise I‑MMSE formula, which expresses the log‑density of a blurred version of the data (obtained by passing the signal through a Gaussian channel yγ = γx + wγ) in terms of the minimum‑mean‑square‑error (MMSE) denoising error across all signal‑to‑noise ratios (SNR) γ. By invoking the Tweedie‑Miyasawa identity, the denoising error is shown to be directly proportional to the score field ∇log p_yγ(yγ) of the blurred density.

Combining these two identities yields a representation of the log‑probability ratio between two points x₁ and x₂ as an Itô integral of the difference of their score fields over γ. Taking the expected quadratic variation of this stochastic process leads to the definition of IEM:

IEM(x₁, x₂, Γ) = ∫₀^Γ E_{wγ}