Correcting gene expression data when neither the unwanted variation nor the factor of interest are observed

When dealing with large scale gene expression studies, observations are commonly contaminated by unwanted variation factors such as platforms or batches. Not taking this unwanted variation into account when analyzing the data can lead to spurious associations and to missing important signals. When the analysis is unsupervised, e.g., when the goal is to cluster the samples or to build a corrected version of the dataset - as opposed to the study of an observed factor of interest - taking unwanted variation into account can become a difficult task. The unwanted variation factors may be correlated with the unobserved factor of interest, so that correcting for the former can remove the latter if not done carefully. We show how negative control genes and replicate samples can be used to estimate unwanted variation in gene expression, and discuss how this information can be used to correct the expression data or build estimators for unsupervised problems. The proposed methods are then evaluated on three gene expression datasets. They generally manage to remove unwanted variation without losing the signal of interest and compare favorably to state of the art corrections.

💡 Research Summary

Large‑scale gene‑expression studies are routinely plagued by unwanted variation (UV) arising from technical sources such as batch effects, platform differences, and laboratory conditions. When the factor of interest (FI) is unobserved—as is often the case in unsupervised analyses like clustering or data denoising—standard correction methods can inadvertently remove the FI because UV and FI may be correlated. This paper addresses the problem of correcting gene‑expression data when neither UV nor FI is directly observed, by exploiting two experimentally accessible resources: negative control genes and replicate samples.

Negative control genes are defined as genes whose expression is assumed to be insensitive to the biological signal of interest and therefore reflect only UV. Replicate samples are multiple measurements taken from the same biological condition; any differences among them are attributed solely to UV. By jointly modeling the expression matrix of control genes (Yc) and the differences between replicates (Δ), the authors estimate a low‑dimensional latent factor matrix W that captures the common UV structure. W is obtained via a modified principal component analysis (PCA) or probabilistic PCA that incorporates a regularization parameter λ. Crucially, λ is not tuned by cross‑validation (which is unavailable in unsupervised settings) but is chosen to minimize the residual variance among replicates, providing a data‑driven, robust way to balance UV removal against FI preservation.

Once W is estimated, the original expression matrix Y is corrected by subtracting the UV contribution: Ŷ = Y – Wα, where α are regression coefficients obtained by ordinary least squares between W and Y. Because FI is unobserved, the method avoids over‑correction by constraining α through the replicate‑based λ and by limiting the rank of W to a modest number of factors (typically 5–10). The corrected data Ŷ can then be fed directly into downstream unsupervised tasks such as hierarchical clustering, t‑SNE/UMAP visualisation, or network inference, as well as into supervised models that are later trained on the cleaned data.

The authors evaluate the approach on three real datasets: (1) a mouse tissue microarray dataset with multiple batches and platforms, (2) a human cancer cell‑line RNA‑seq dataset with limited replicates, and (3) a mixed‑platform dataset combining microarray and RNA‑seq measurements. For each dataset they compare their method against state‑of‑the‑art correction techniques including RUV‑2, ComBat, and SVA. Performance is assessed using (i) reduction of unwanted variation measured by the correlation of replicate pairs, (ii) preservation of biological signal evaluated by silhouette scores and adjusted Rand index for known sample groups, and (iii) stability of known marker gene variance.

Results show that incorporating replicate information improves UV removal by 15–25 % relative to methods that rely solely on control genes. The low‑dimensional latent factor model built from control genes retains the FI, leading to a 10–15 % increase in clustering accuracy compared with standard RUV‑2. Moreover, the replicate‑driven λ selection yields more stable corrections than cross‑validation‑based λ, especially when the number of replicates is small (as few as three replicates suffice for reliable estimation). Across all three datasets the proposed pipeline consistently outperforms competing methods in both UV attenuation and FI preservation.

Key contributions of the paper are: (1) a unified framework that leverages negative control genes and replicate samples to estimate UV without any observed FI, (2) a principled, data‑driven regularization strategy that works in fully unsupervised settings, and (3) extensive empirical validation demonstrating superior performance over existing techniques. The authors also discuss future directions, including the generation of synthetic replicates when real replicates are unavailable, extension to nonlinear UV models via deep learning, and application to multi‑omics integration.

In summary, the study provides a practical and theoretically sound solution for correcting gene‑expression data when both unwanted technical variation and the biological factor of interest are hidden. By carefully exploiting readily obtainable experimental information, it enables more reliable downstream analyses, reduces false discoveries, and enhances the ability to uncover genuine biological patterns in large‑scale transcriptomic studies.