Learning the score under shape constraints

Score estimation has recently emerged as a key modern statistical challenge, due to its pivotal role in generative modelling via diffusion models. Moreover, it is an essential ingredient in a new approach to linear regression via convex $M$-estimation, where the corresponding error densities are projected onto the log-concave class. Motivated by these applications, we study the minimax risk of score estimation with respect to squared $L^2(P_0)$-loss, where $P_0$ denotes an underlying log-concave distribution on $\mathbb{R}$. Such distributions have decreasing score functions, but on its own, this shape constraint is insufficient to guarantee a finite minimax risk. We therefore define subclasses of log-concave densities that capture two fundamental aspects of the estimation problem. First, we establish the crucial impact of tail behaviour on score estimation by determining the minimax rate over a class of log-concave densities whose score function exhibits controlled growth relative to the quantile levels. Second, we explore the interplay between smoothness and log-concavity by considering the class of log-concave densities with a scale restriction and a $(β,L)$-Hölder assumption on the log-density for some $β\in [1,2]$. We show that the minimax risk over this latter class is of order $L^{2/(2β+1)}n^{-β/(2β+1)}$ up to poly-logarithmic factors, where $n$ denotes the sample size. When $β< 2$, this rate is faster than could be obtained under either the shape constraint or the smoothness assumption alone. Our upper bounds are attained by a locally adaptive, multiscale estimator constructed from a uniform confidence band for the score function. This study highlights intriguing differences between the score estimation and density estimation problems over this shape-constrained class.

💡 Research Summary

This paper investigates the fundamental statistical problem of estimating the score function ψ₀ = f₀′/f₀ of a univariate density f₀ under the squared L²(P₀) loss L(ĥψ,ψ₀) = ∫(ĥψ − ψ₀)² f₀, where P₀ is the distribution induced by f₀. The focus is on log‑concave densities, which guarantee that the score is monotone decreasing, but monotonicity alone does not ensure a finite minimax risk. To obtain meaningful rates the authors introduce two complementary subclasses of log‑concave densities.

Tail‑growth restriction.
For parameters γ∈(0,1] and L>0 they require
|ψ₀(x)| ≤ L · min{F₀(x), 1 − F₀(x)}^{−(1−γ)/2} for all x,
where F₀ is the cdf of f₀. This condition bounds the speed at which the score can diverge near the distribution’s quantiles. Under this class the minimax risk satisfies
Mₙ ≈ L² · n^{−(γ∧1/3)} (up to poly‑logarithmic factors).
A phase transition occurs at γ = 1/3: for larger γ the bulk of the distribution drives the error, while for smaller γ the tails dominate. The authors also refine the bound by incorporating an upper bound r on the Fisher information I(f₀) = ∫ψ₀² f₀, showing that the risk scales linearly with r.

Smoothness restriction.
They further assume that the log‑density log f₀ belongs to a Hölder class (β,L) with β∈