Estimation for High-Dimensional Linear Mixed-Effects Models Using $ell_1$-Penalization

We propose an $\ell_1$-penalized estimation procedure for high-dimensional linear mixed-effects models. The models are useful whenever there is a grouping structure among high-dimensional observations, i.e. for clustered data. We prove a consistency and an oracle optimality result and we develop an algorithm with provable numerical convergence. Furthermore, we demonstrate the performance of the method on simulated and a real high-dimensional data set.

💡 Research Summary

The paper addresses the challenge of estimating linear mixed‑effects models (LMEMs) when the number of covariates far exceeds the number of observations, a situation common in modern clustered or longitudinal data sets. Classical LMEMs can accommodate both fixed effects (β) and random effects (b_i) but lack a built‑in mechanism for variable selection, making them unsuitable for high‑dimensional settings. To fill this gap, the authors propose an ℓ₁‑penalized likelihood approach that imposes a Lasso‑type penalty on the fixed‑effect vector while leaving the random‑effect covariance matrix Σ unrestricted apart from the usual positive‑definiteness constraint.

Model formulation
For cluster i (i = 1,…,m) with n_i observations, the model is
y_i = X_i β + Z_i b_i + ε_i,
where b_i ∼ N(0, Σ), ε_i ∼ N(0, σ²I). The penalized objective is

Q(β, Σ, σ²) = –log L(β, Σ, σ²) + λ‖β‖₁,

with λ chosen by cross‑validation or information criteria. This formulation retains the full likelihood for the random part, ensuring that the variance components are estimated efficiently, while the ℓ₁ term drives sparsity in β.

Theoretical contributions
Two main theorems are proved under a set of regularity conditions that include a restricted eigenvalue (RE) condition on the design matrices, bounded eigenvalues of Σ, and a suitable scaling of λ (λ ≈ √(log p / n)).

1. Consistency – The estimator (β̂, Σ̂, σ̂²) converges to the true parameters (β*, Σ*, σ*²) in ℓ₂ norm for β and in operator norm for Σ, with rates matching those of high‑dimensional Lasso in the fixed‑effects only case. The proof carefully separates the contribution of the random effects, showing that the conditional expectation of b_i given the data does not inflate the estimation error beyond the usual stochastic error term.

2. Oracle optimality – The estimator enjoys the oracle property: with probability tending to one, it correctly identifies the non‑zero components of β, and the asymptotic distribution of the estimated non‑zero coefficients coincides with that of an “oracle” estimator that knows the true support a priori. Moreover, Σ̂ attains the same √n‑consistency as in classical mixed‑effects estimation, indicating that the penalty does not compromise variance‑component estimation.

Algorithmic development
The authors embed the penalized likelihood within an EM framework. In the E‑step, given current estimates (β^{(t)}, Σ^{(t)}, σ^{2(t)}), the conditional expectations E