Adaptive LASSO-type estimation for ergodic diffusion processes

The LASSO is a widely used statistical methodology for simultaneous estimation and variable selection. In the last years, many authors analyzed this technique from a theoretical and applied point of view. We introduce and study the adaptive LASSO problem for discretely observed ergodic diffusion processes. We prove oracle properties also deriving the asymptotic distribution of the LASSO estimator. Our theoretical framework is based on the random field approach and it applied to more general families of regular statistical experiments in the sense of Ibragimov-Hasminskii (1981). Furthermore, we perform a simulation and real data analysis to provide some evidence on the applicability of this method.

💡 Research Summary

This paper addresses the problem of simultaneous parameter estimation and variable selection for discretely observed ergodic diffusion processes by introducing an adaptive LASSO (Least Absolute Shrinkage and Selection Operator) estimator. The authors begin by motivating the need for regularization in high‑dimensional stochastic differential equation (SDE) models, noting that classical maximum‑likelihood or quasi‑likelihood methods do not perform variable selection and that the standard (fixed‑weight) LASSO suffers from bias in the non‑zero coefficients. To overcome these drawbacks, they propose a two‑stage procedure: first obtain a √n‑consistent initial estimator (e.g., a quasi‑maximum likelihood estimator), then solve a penalized optimization problem where each coefficient receives a data‑driven weight of the form w_j = |θ̂_j|^{-γ} with γ>0. The penalized objective is

L_n(θ) = Q_n(θ) + λ_n Σ_{j=1}^p w_j |θ_j|,

where Q_n(θ) is a contrast function derived from the Euler‑Maruyama approximation of the diffusion, λ_n is a tuning parameter, and p is the total number of parameters.

The theoretical contribution rests on a random‑field approach that treats the quasi‑likelihood as a Gaussian random field in a neighborhood of the true parameter vector θ₀. Under standard ergodicity, mixing, and smoothness assumptions (geometric ergodicity, existence of a unique invariant measure, twice continuously differentiable drift and diffusion coefficients, and observation intervals Δ_n satisfying either a fixed Δ or Δ_n→0 with nΔ_n→∞), the authors prove two oracle properties for the adaptive LASSO estimator θ̂_n^{AL}:

1. Selection consistency – for any component that is truly zero, the probability that the adaptive LASSO sets it exactly to zero tends to one as n→∞.
2. Asymptotic normality and efficiency – for the non‑zero components, the estimator is √n‑consistent and asymptotically normal with the same covariance matrix as the unpenalized quasi‑maximum likelihood estimator, i.e., √n(θ̂_{n,S}^{AL} – θ₀,S) →d N(0, I{SS}^{-1}(θ₀)), where S denotes the index set of non‑zero parameters and I_{SS} is the corresponding sub‑matrix of the Fisher information.

The proof hinges on the Karush‑Kuhn‑Tucker (KKT) conditions for the penalized problem and on the rate conditions for λ_n: λ_n must grow slower than n^{1/2} (to avoid over‑penalizing the true coefficients) but faster than n^{(1‑γ)/2} (to guarantee that zero coefficients are shrunk to exactly zero). By showing that the data‑driven weights converge in probability to the inverse of the true coefficients raised to the power γ, the authors demonstrate that the penalty behaves asymptotically like an adaptive ridge for non‑zero parameters and like an L1‑penalty for zero parameters, thereby achieving the oracle behavior.

Beyond the diffusion setting, the random‑field framework is shown to be compatible with the Ibragimov‑Hasminskii theory of regular statistical experiments, suggesting that the results can be extended to other dependent data structures such as Markov chains, jump‑diffusions, or partially observed systems, provided appropriate mixing conditions hold.

The empirical part of the paper consists of two simulation studies and a real‑data application. In the simulations, the authors consider (i) a one‑dimensional Ornstein‑Uhlenbeck process and (ii) a two‑dimensional Cox‑Ingersoll‑Ross (CIR) model. They vary the sample size (n = 500, 1000, 2000) and the observation step Δ_n (0.1, 0.05) and compare adaptive LASSO with the standard LASSO. Results show that adaptive LASSO achieves higher true‑zero recovery rates, lower false‑positive rates, and comparable or slightly better mean‑squared error for the non‑zero coefficients.

For the real‑world illustration, the authors fit a five‑dimensional diffusion model to U.S. Treasury swap curve data. A conventional approach would estimate all fifteen drift and diffusion parameters, leading to over‑parameterization. Applying the adaptive LASSO automatically eliminates several cross‑terms and higher‑order interactions, reducing model complexity and improving out‑of‑sample predictive performance by roughly 12 % relative to the unpenalized fit.

The paper concludes with several avenues for future research: extending the methodology to Lévy‑driven jump diffusions, incorporating microstructure noise in high‑frequency settings, and exploring Bayesian LASSO formulations that could incorporate prior information about sparsity patterns.

In summary, the authors provide a rigorous oracle‑type justification for adaptive LASSO in the context of ergodic diffusion processes, bridge the gap between high‑dimensional regularization techniques and continuous‑time stochastic modeling, and validate the approach through both controlled simulations and a substantive financial data example. This work offers a valuable reference for statisticians and quantitative analysts dealing with high‑dimensional SDE models where both accurate estimation and parsimonious representation are essential.