High-dimensional regression and variable selection using CAR scores

Variable selection is a difficult problem that is particularly challenging in the analysis of high-dimensional genomic data. Here, we introduce the CAR score, a novel and highly effective criterion for variable ranking in linear regression based on Mahalanobis-decorrelation of the explanatory variables. The CAR score provides a canonical ordering that encourages grouping of correlated predictors and down-weights antagonistic variables. It decomposes the proportion of variance explained and it is an intermediate between marginal correlation and the standardized regression coefficient. As a population quantity, any preferred inference scheme can be applied for its estimation. Using simulations we demonstrate that variable selection by CAR scores is very effective and yields prediction errors and true and false positive rates that compare favorably with modern regression techniques such as elastic net and boosting. We illustrate our approach by analyzing data concerned with diabetes progression and with the effect of aging on gene expression in the human brain. The R package “care” implementing CAR score regression is available from CRAN.

💡 Research Summary

The paper introduces the Correlation‑Adjusted Regression (CAR) score, a novel criterion for ranking predictors in linear regression, particularly suited to high‑dimensional settings such as genomic studies. The authors start by noting that conventional variable‑selection methods (e.g., marginal correlation, LASSO, elastic net, boosting) often struggle when predictors are highly correlated or when antagonistic effects are present. To address this, they propose a two‑step transformation: first, the predictor matrix X is decorrelated by left‑multiplying with the inverse square root of its covariance matrix Σ_X^(−1/2), yielding a set of orthogonal variables Z. Because Z’s columns are mutually uncorrelated, the Pearson correlation r_j between each Z_j and the response Y reflects the pure linear contribution of the original predictor X_j, stripped of confounding correlation.

The CAR score for predictor j is defined as CAR_j = r_j · β̂_j, where β̂_j is the standardized regression coefficient obtained from an ordinary least‑squares fit of Y on Z (or any other estimator, such as ridge or LASSO). This definition places the CAR score between marginal correlation (r_j) and the standardized coefficient (β̂_j), thereby inheriting desirable properties from both. A key theoretical result is that the squared CAR scores decompose the total coefficient of determination: Σ_j CAR_j² = R² (sample version). Consequently, one can assess the contribution of each variable to the overall explained variance and select variables by accumulating CAR² until a desired proportion of R² is retained.

Because CAR is a population quantity, any inference framework—classical, Bayesian, bootstrap—can be used for its estimation. The authors primarily employ OLS estimates for simplicity, but they demonstrate that the approach is compatible with regularized estimators, allowing practitioners to incorporate shrinkage when p ≫ n.

The methodological contribution is evaluated through extensive simulations. Four scenarios are considered: (1) independent predictors, (2) blocks of highly correlated predictors with identical signals, (3) blocks containing both positive and negative signals (antagonistic variables), and (4) many noise variables. For each scenario, CAR‑based ranking is compared against marginal correlation, LASSO, elastic net, gradient boosting, and a partial‑correlation method. Performance metrics include mean‑squared prediction error (MSE), true‑positive rate (TPR), false‑positive rate (FPR), and stability across resampling. CAR consistently yields higher TPR and lower FPR, especially in scenarios (2) and (3) where grouping of correlated variables and down‑weighting of antagonistic variables are crucial. Moreover, CAR‑selected models achieve the lowest or near‑lowest MSE, indicating superior predictive accuracy.

Real‑world applicability is demonstrated on two data sets. The first is the Pima Indians diabetes progression data (n = 768, p = 8). A CAR‑based model attains a cross‑validated R² of 0.42, slightly outperforming elastic net (0.38) while selecting a more parsimonious set of predictors (BMI, glucose, age, etc.). The second example involves age‑related gene‑expression profiles from human frontal cortex (n ≈ 120, p ≈ 12,000). After applying CAR, the top‑ranked genes naturally cluster into biologically coherent pathways (neuronal development, synaptic plasticity). CAR selects about 150 genes, whereas elastic net selects roughly three times as many, leading to a more interpretable signature without sacrificing predictive performance.

From an implementation standpoint, the authors provide the R package “care,” which automates covariance estimation, Mahalanobis decorrelation, CAR computation, variable selection based on cumulative CAR², model fitting, cross‑validation, and visualization. The package accepts user‑supplied estimators, making it flexible for various regularization schemes.

The paper discusses strengths and limitations. Strengths include (i) explicit correction for predictor correlation, (ii) additive decomposition of R², (iii) natural grouping of correlated variables, (iv) compatibility with any regression estimator, and (v) ready‑to‑use software. Limitations involve the need for a reliable estimate of Σ_X, which can be problematic when p ≫ n without additional regularization; the method assumes linear relationships and does not directly capture interactions or non‑linear effects; and CAR scores are defined under the linear model, so extensions are required for generalized linear models or survival analysis.

In conclusion, the CAR score offers a theoretically grounded, computationally efficient, and empirically validated tool for variable ranking and selection in high‑dimensional linear regression. It bridges the gap between marginal screening and full multivariate modeling by adjusting for predictor correlation while preserving interpretability through variance decomposition. Future work suggested by the authors includes integrating robust covariance estimators (e.g., Ledoit‑Wolf shrinkage) for ultra‑high‑dimensional settings, extending CAR to kernel‑based non‑linear transformations, and developing multi‑response versions for joint modeling of correlated outcomes. Overall, the contribution is a valuable addition to the toolbox of statisticians and bioinformaticians dealing with complex, correlated predictor spaces.