Sharp Concentration Inequalities: Phase Transition and Mixing of Orlicz Tails with Variance

In this work, we investigate how to develop sharp concentration inequalities for sub-Weibull random variables, including sub-Gaussian and sub-exponential distributions. Although the random variables may not be sub-Guassian, the tail probability around the origin behaves as if they were sub-Gaussian, and the tail probability decays align with the Orlicz $Ψ_α$-tail elsewhere. Specifically, for independent and identically distributed (i.i.d.) ${X_i}{i=1}^n$ with finite Orlicz norm $|X|{Ψ_α}$, our theory unveils that there is an interesting phase transition at $α= 2$ in that $\PPł(ł|\sum_{i=1}^n X_i \r| \geq t\r)$ with $t > 0$ is upper bounded by $2\expł(-C\maxł{\frac{t^2}{n|X|{Ψ_α}^2},\frac{t^α}{ n^{α-1} |X|{Ψ_α}^α}\r}\r)$ for $α\geq 2$, and by $2\expł(-C\minł{\frac{t^2}{n|X|{Ψ_α}^2},\frac{t^α}{ n^{α-1} |X|{Ψ_α}^α}\r}\r)$ for $1\leq α\leq 2$ with some positive constant $C$. In many scenarios, it is often necessary to distinguish the standard deviation from the Orlicz norm when the latter can exceed the former greatly. To accommodate this, we build a new theoretical analysis framework, and our sharp, flexible concentration inequalities involve the variance and a mixing of Orlicz $Ψ_α$-tails through the min and max functions. Our theory yields new, improved concentration inequalities even for the cases of sub-Gaussian and sub-exponential distributions with $α= 2$ and $1$, respectively. We further demonstrate our theory on martingales, random vectors, random matrices, and covariance matrix estimation. These sharp concentration inequalities can empower more precise non-asymptotic analyses across different statistical and machine learning applications.

💡 Research Summary

This paper presents a significant advancement in the theory of concentration inequalities for sums of sub-Weibull random variables, which encompass sub-Gaussian (α=2) and sub-exponential (α=1) distributions as special cases. The central aim is to derive sharp, non-asymptotic tail bounds for the sum S_n = Σ_{i=1}^n X_i of independent random variables under the minimal assumption of a finite Orlicz ψ_α-norm, ∥X∥_{Ψα} < ∞.

The authors’ first major discovery is a phase transition in the tail behavior of S_n at α=2. For i.i.d. mean-zero variables, they prove that the optimal concentration inequality takes two distinct forms:

• For α ≥ 2: P(|S_n| ≥ t) ≤ 2 exp( -C max{ t²/(n∥X∥²_{Ψα}), t^α/(n^{α-1}∥X∥^α_{Ψα}) } ).
• For 1 ≤ α ≤ 2: P(|S_n| ≥ t) ≤ 2 exp( -C min{ t²/(n∥X∥²_{Ψα}), t^α/(n^{α-1}∥X∥^α_{Ψα}) } ). The key innovation for α ≥ 2 is the use of the max operator, contrasting with the min operator found in prior literature (e.g., Boucheron et al., Kuchibhotla & Chakrabortty). This max-type bound is strictly sharper for large deviations (large t), correctly capturing the genuine Ψα tail (decaying as exp(-Ct^α)), while still preserving the sub-Gaussian behavior (exp(-Ct²)) for small deviations near the origin. This result closes an important theoretical gap, delivering the first sharp, density-free concentration inequality with a true Ψα tail for α ≥ 2 under the sole condition of a finite Orlicz norm.

The second major contribution addresses the common scenario where the standard deviation σ_X is much smaller than the Orlicz norm ∥X∥_{Ψα} (e.g., Bernoulli variables with small success probability). The authors develop a novel and flexible moment framework that explicitly decouples the variance from the heavier Orlicz tail. They introduce a condition of the form E