Sparse estimation via nonconcave penalized likelihood in a factor analysis model

We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates cannot be obtained when the number of variables is much larger than the number of observations. Furthermore, even if the maximum likelihood estimates are available, the rotation technique does not often produce a sufficiently sparse solution. In order to handle these problems, this paper introduces a penalized likelihood procedure that imposes a nonconvex penalty on the factor loadings. We show that the penalized likelihood procedure can be viewed as a generalization of the traditional two-step approach, and the proposed methodology can produce sparser solutions than the rotation technique. A new algorithm via the EM algorithm along with coordinate descent is introduced to compute the entire solution path, which permits the application to a wide variety of convex and nonconvex penalties. Monte Carlo simulations are conducted to investigate the performance of our modeling strategy. A real data example is also given to illustrate our procedure.

💡 Research Summary

The paper addresses the longstanding difficulty of obtaining sparse factor loadings in high‑dimensional factor analysis, where the number of observed variables far exceeds the number of observations. The conventional two‑step routine—maximum‑likelihood estimation (MLE) of the loading matrix Λ followed by an orthogonal or oblique rotation (e.g., Varimax, Promax)—fails in two respects. First, MLE becomes numerically unstable or infeasible when p≫n because the sample covariance matrix is singular and its inverse cannot be computed. Second, the rotation step, being a post‑hoc orthogonal transformation, often does not produce sufficiently sparse loadings; it merely re‑expresses the same information without explicit regularization, and the resulting solutions can retain many small but non‑zero entries that hinder interpretability.

To overcome these limitations, the authors propose a penalized‑likelihood framework that directly imposes a non‑convex penalty on the elements of Λ. The objective function is

  Q(Λ,Ψ) = –ℓ(Λ,Ψ) + ∑{i,j} P_λ(|λ{ij}|),

where ℓ(Λ,Ψ) is the usual log‑likelihood of the factor model and P_λ(·) denotes a non‑convex penalty such as SCAD (Smoothly Clipped Absolute Deviation), MCP (Minimax Concave Penalty), or an L_q norm with 0<q<1. These penalties retain the sparsity‑inducing property of the L1 (lasso) penalty while reducing bias for large coefficients, thereby yielding more accurate estimates of truly non‑zero loadings.

Estimation proceeds via an Expectation–Maximization (EM) algorithm combined with coordinate descent (CD) in the M‑step. In the E‑step, given current estimates of Λ and Ψ, the conditional expectations E