Sparse Estimators and the Oracle Property, or the Return of Hodges Estimator

We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity whenever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly in finite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.

💡 Research Summary

The paper revisits the “oracle property” introduced by Fan and Li (2001, 2002, 2004) and demonstrates that the attractive features commonly associated with this property are in fact a double‑edged sword when they arise from sparsity. The oracle property, as defined by Fan and Li, consists of two components: (i) variable‑selection consistency, meaning that coefficients that are truly zero are estimated as exactly zero with probability tending to one, and (ii) asymptotic efficiency for the non‑zero coefficients, i.e., the estimator of the selected sub‑model has the same first‑order asymptotic distribution as the ordinary maximum‑likelihood estimator (MLE). The authors argue that both components are natural consequences of a stronger “sparsity property” – the requirement that an estimator never shrinks a truly non‑zero coefficient to zero with any positive probability.

From a decision‑theoretic perspective, the authors prove a general theorem: for any loss function (L(\theta,\hat\theta)) and any estimator (\hat\theta) satisfying the sparsity property, the maximal risk (\sup_{\theta}E_{\theta}