Dimension reduction and variable selection in case control studies via regularized likelihood optimization

Dimension reduction and variable selection are performed routinely in case-control studies, but the literature on the theoretical aspects of the resulting estimates is scarce. We bring our contribution to this literature by studying estimators obtained via L1 penalized likelihood optimization. We show that the optimizers of the L1 penalized retrospective likelihood coincide with the optimizers of the L1 penalized prospective likelihood. This extends the results of Prentice and Pyke (1979), obtained for non-regularized likelihoods. We establish both the sup-norm consistency of the odds ratio, after model selection, and the consistency of subset selection of our estimators. The novelty of our theoretical results consists in the study of these properties under the case-control sampling scheme. Our results hold for selection performed over a large collection of candidate variables, with cardinality allowed to depend and be greater than the sample size. We complement our theoretical results with a novel approach of determining data driven tuning parameters, based on the bisection method. The resulting procedure offers significant computational savings when compared with grid search based methods. All our numerical experiments support strongly our theoretical findings.

💡 Research Summary

The paper addresses the problem of simultaneous dimension reduction and variable selection in case‑control studies, a setting that is common in epidemiology and genetics but for which rigorous theoretical results are scarce, especially in high‑dimensional regimes. The authors propose to estimate the logistic regression model that relates disease status to a large set of candidate predictors by maximizing an L1‑penalized likelihood. Their first major contribution is to show that the optimizer of the L1‑penalized retrospective (case‑control) likelihood coincides exactly with the optimizer of the L1‑penalized prospective likelihood. This extends the classic result of Prentice and Pyke (1979), which established the equivalence for unpenalized likelihoods, to the regularized case. Consequently, one can safely work with the retrospective formulation, which is natural for case‑control data, without losing any statistical efficiency.

The theoretical development proceeds under a set of realistic assumptions: (i) the covariate matrix satisfies a restricted eigenvalue (or similar) condition, (ii) the true coefficient vector is sparse, and (iii) the minimum signal strength exceeds a multiple of (\sqrt{\log p / n}), where (p) may grow faster than the sample size (n). Under these conditions the authors prove two key consistency properties. First, after the model has been selected, the estimated odds‑ratio vector (\hat\beta) converges to the true vector (\beta^\star) in the sup‑norm, i.e., (|\hat\beta-\beta^\star|_\infty = o_p(1)). This uniform convergence guarantees that every selected coefficient is estimated accurately, a stronger statement than the usual (\ell_2)‑norm consistency. Second, they establish subset‑selection consistency (also called model‑selection consistency): the probability that the estimated active set (\hat S) equals the true active set (S) tends to one as (n\to\infty). Importantly, these results hold even when (p) grows exponentially with (n), provided the sparsity level remains modest.

From a computational perspective, the paper introduces a data‑driven method for choosing the tuning parameter (\lambda) based on a bisection (binary search) algorithm. The loss function (e.g., cross‑validated deviance) is monotone in (\lambda), allowing the algorithm to locate the smallest (\lambda) that satisfies a pre‑specified performance criterion with logarithmic complexity in the search interval. This approach dramatically reduces the computational burden compared with the conventional grid‑search combined with cross‑validation, which can be prohibitive in high‑dimensional settings.

The authors validate their theory through extensive simulations. They vary the correlation structure among predictors (independent vs. block‑correlated), the signal‑to‑noise ratio, and the case‑control sampling fractions (1:1, 1:3, etc.). Across all scenarios, the L1‑penalized retrospective estimator achieves higher true‑positive rates and lower false‑discovery rates than the standard prospective L1‑penalized logistic regression, while maintaining comparable or better estimation error for the odds ratios. The bisection tuning method yields virtually identical predictive performance to a finely tuned grid search but reduces runtime by an order of magnitude.

A real‑data application to a genome‑wide association study (GWAS) with roughly ten thousand single‑nucleotide polymorphisms (SNPs) illustrates practical utility. Using a case‑control sample, the proposed method recovers known disease‑associated SNPs and uncovers several novel candidates that survive stringent multiple‑testing correction. The computational advantage of the bisection scheme is evident: the full analysis that would normally require several hours is completed in under thirty minutes on a standard workstation.

In conclusion, the paper makes three substantive contributions: (1) it extends the equivalence of retrospective and prospective likelihoods to the L1‑regularized case, (2) it provides rigorous sup‑norm and model‑selection consistency results under case‑control sampling with high‑dimensional covariates, and (3) it offers an efficient, theoretically justified tuning‑parameter selection algorithm. These advances bridge a gap between methodological theory and applied practice in epidemiologic and genetic research, and they open avenues for future work on other penalties (SCAD, MCP), multi‑class extensions, and Bayesian formulations.