Inference of Sparse Networks with Unobserved Variables. Application to Gene Regulatory Networks

Networks are a unifying framework for modeling complex systems and network inference problems are frequently encountered in many fields. Here, I develop and apply a generative approach to network inference (RCweb) for the case when the network is sparse and the latent (not observed) variables affect the observed ones. From all possible factor analysis (FA) decompositions explaining the variance in the data, RCweb selects the FA decomposition that is consistent with a sparse underlying network. The sparsity constraint is imposed by a novel method that significantly outperforms (in terms of accuracy, robustness to noise, complexity scaling, and computational efficiency) Bayesian methods and MLE methods using l1 norm relaxation such as K-SVD and l1–based sparse principle component analysis (PCA). Results from simulated models demonstrate that RCweb recovers exactly the model structures for sparsity as low (as non-sparse) as 50% and with ratio of unobserved to observed variables as high as 2. RCweb is robust to noise, with gradual decrease in the parameter ranges as the noise level increases.

💡 Research Summary

The paper introduces RC‑web, a novel algorithm for inferring sparse bipartite networks when many of the influencing variables are unobserved. The authors start by framing the problem as a factor‑analysis‑like decomposition of an observed data matrix G (M samples of N observed variables) into a product of two matrices R (M × P latent variables) and C (P × N weighted adjacency matrix) plus residual noise Υ: G = RC + Υ. The key assumption is that C is sparse, i.e., each observed variable is regulated by only a few latent factors.

Standard factor analysis suffers from rotational ambiguity: any orthogonal matrix Q yields an equally valid decomposition R Q⁻¹·Q·C, making the underlying network structure unidentifiable without extra constraints. Existing sparse‑FA approaches either impose an ℓ₁ penalty (e.g., K‑SVD, sparse PCA) or use Bayesian priors, but they either approximate the ℓ₀ sparsity poorly or are computationally intensive.

RC‑web tackles the ℓ₀ sparsity directly by inserting an invertible matrix B into the singular‑value decomposition (SVD) of G. Writing G = U S Vᵀ and setting R̂ = U S B⁻¹, Ĉ = (B Vᵀ)ᵀ, the problem reduces to finding B such that Ĉ is as sparse as possible, i.e., minimizing ‖V B‖₀ under the constraint det(B) > 1 to avoid degenerate solutions. This reformulation converts the original combinatorial problem into a series of convex‑like sub‑problems that can be solved efficiently.

The algorithm proceeds column‑wise for B. For each column, it starts with the smallest right singular vector of V (denoted vₛ) as an initial guess. It then iteratively removes rows of V (corresponding to observed variables) that contribute most to the current smallest singular direction. The removal is guided by the left singular vector of the current reduced matrix V_{ω₀}, selecting the index with the largest absolute entry. After each removal, the inverse of V_{ω₀}ᵀ V_{ω₀} is updated by a rank‑one update, avoiding full matrix inversion. The process stops when the smallest singular value of the reduced matrix approaches zero or when the singular vector converges, at which point the removed rows define the zero pattern of the current column of Ĉ.

Key theoretical properties:

• Convergence is guaranteed within at most (N − P + 1) removals because after that many rows the reduced matrix cannot have full rank P.
• All heavy linear algebra operations are performed in the latent space of dimension P, which is typically much smaller than N or M, yielding substantial memory and CPU savings.
• The method is robust to moderate noise because the sparsity constraint dominates the optimization, and the singular‑vector perturbation theory ensures that small noise only slightly perturbs the direction of the smallest singular vector.

Empirical validation uses simulated bipartite networks of two topologies: Erdős‑Rényi (Poisson degree) and scale‑free (power‑law degree). For each topology, latent variables are drawn from a uniform distribution, and the observed data matrix G is generated via G = RC plus additive Gaussian noise of varying variance. The authors compare RC‑web against four state‑of‑the‑art baselines: (A) PSMF (Bayesian matrix factorization), (B) BFRM (Bayesian factor regression), (C) emPCA (MLE‑based sparse PCA), and (D) K‑SVD.

Performance metrics include ℓ₀ recovery accuracy (fraction of correctly identified non‑zero entries), precision, recall, and reconstruction error. Across a wide range of parameters (M, N, P, sparsity levels from 50 % to near‑dense, and noise levels up to 15 % of signal variance), RC‑web consistently outperforms the baselines. Notably, even when the true network is only 50 % sparse (i.e., half of the possible edges are present), RC‑web recovers the exact adjacency matrix, whereas the other methods’ accuracy drops sharply. Noise robustness is demonstrated by a gradual decline in performance rather than a catastrophic failure. Computationally, RC‑web runs 2–3× faster than K‑SVD and 5–10× faster than the Bayesian methods on the same hardware.

The authors discuss two inherent ambiguities: (1) scaling and permutation indeterminacy—any diagonal scaling matrix D can be absorbed into R and C without changing the product, meaning that without external information the latent factors cannot be uniquely assigned to physical entities; (2) the need for perturbation or prior knowledge to map latent variables to biological factors, which can be supplied by knock‑down experiments or known pathway information.

Limitations are acknowledged: the method assumes linear relationships and may struggle with extremely dense networks or pathological singular‑value spectra where the initial smallest singular vector is a poor approximation. Nevertheless, the authors provide theoretical justification (based on perturbation bounds) and empirical evidence that the algorithm still converges in practice.

In conclusion, RC‑web offers a principled, efficient, and highly accurate solution to the problem of inferring sparse regulatory networks in the presence of hidden variables. By directly targeting ℓ₀ sparsity through a clever SVD‑based reparameterization, it overcomes the rotational ambiguity of traditional factor analysis and outperforms existing Bayesian and ℓ₁‑based approaches. The framework is readily extensible to nonlinear interactions and promises practical utility for large‑scale gene‑regulatory network reconstruction, especially when combined with experimental perturbations to resolve the remaining scaling/permutation ambiguities.