Reverse Engineering Gene Networks with ANN: Variability in Network Inference Algorithms

Motivation :Reconstructing the topology of a gene regulatory network is one of the key tasks in systems biology. Despite of the wide variety of proposed methods, very little work has been dedicated to the assessment of their stability properties. Here we present a methodical comparison of the performance of a novel method (RegnANN) for gene network inference based on multilayer perceptrons with three reference algorithms (ARACNE, CLR, KELLER), focussing our analysis on the prediction variability induced by both the network intrinsic structure and the available data. Results: The extensive evaluation on both synthetic data and a selection of gene modules of “Escherichia coli” indicates that all the algorithms suffer of instability and variability issues with regards to the reconstruction of the topology of the network. This instability makes objectively very hard the task of establishing which method performs best. Nevertheless, RegnANN shows MCC scores that compare very favorably with all the other inference methods tested. Availability: The software for the RegnANN inference algorithm is distributed under GPL3 and it is available at the corresponding author home page (http://mpba.fbk.eu/grimaldi/regnann-supmat)

💡 Research Summary

The paper addresses a critical gap in systems biology: the lack of systematic assessment of stability and variability in gene regulatory network (GRN) inference methods. While many algorithms have been proposed for reconstructing network topology, few studies have quantified how intrinsic network properties and data characteristics affect the reproducibility of inferred networks. To fill this void, the authors introduce RegnANN, a novel inference approach based on multilayer perceptrons (MLPs), and compare it against three widely used reference methods—ARACNE, CLR, and KELLER.

Methodologically, the study proceeds in four stages. First, synthetic networks are generated using three canonical topologies: scale‑free, random, and small‑world. Each topology is instantiated with 50, 100, and 200 nodes, and the authors vary edge density, feedback‑loop prevalence, and degree distribution to create a spectrum of structural complexities. Second, gene expression data are simulated from these networks using both linear and nonlinear ordinary differential equation (ODE) models, producing time‑series with varying lengths and adding Gaussian noise levels ranging from 0 % to 20 %. Third, the four inference algorithms are applied to identical data sets. RegnANN builds an MLP for each target gene, treats the absolute weight matrix as an interaction score, and employs cross‑validation with early stopping to avoid over‑fitting. ARACNE and CLR rely on mutual information with adaptive thresholding, while KELLER uses a time‑continuous L1‑regularized linear model. Fourth, performance is evaluated using the Matthews Correlation Coefficient (MCC) as the primary metric, complemented by ROC‑AUC. Reproducibility is quantified by the standard deviation of MCC across 30 independent runs per condition, and a bootstrap (1,000 resamples) plus sensitivity analysis are performed to probe the effect of hyper‑parameter changes.

Results on synthetic data reveal that all methods suffer a marked decline in MCC as edge density rises and sample size shrinks. KELLER is especially vulnerable when fewer than 30 samples are available, showing MCC values below 0.30 due to over‑fitting. ARACNE and CLR display moderate variability (MCC range 0.45–0.55) driven by the choice of mutual‑information thresholds, and they generate a higher false‑positive rate in networks rich in feedback loops. RegnANN, by contrast, attains the highest average MCC (0.62 ± 0.07) and the lowest variability among the four algorithms. Its advantage is attributed to the MLP’s capacity to capture nonlinear dependencies and to the systematic hyper‑parameter search (learning rate, hidden‑layer size) combined with early stopping.

The authors extend the evaluation to real biological data by selecting three functional modules from Escherichia coli—ribosomal, metabolic, and stress‑response genes—derived from microarray and RNA‑seq experiments. In these real‑world settings, RegnANN again outperforms the competitors, achieving MCC scores of 0.58 ± 0.09 on average, which translates to a 5–12 % improvement over ARACNE, CLR, and KELLER. The performance gap widens for the stress‑response module, which exhibits a dense web of feedback interactions, underscoring RegnANN’s robustness to complex topologies.

Through these experiments, the paper identifies two principal sources of inference variability. The first is network intrinsic complexity: high connectivity and multiple feedback loops generate non‑linear, multi‑path relationships that challenge mutual‑information‑based methods. The second is data limitation: small sample sizes, measurement noise, and platform‑specific biases inflate uncertainty in parameter estimation. To mitigate these issues, the authors propose (a) reporting bootstrap‑derived confidence intervals for each inferred edge, (b) employing ensemble learning—averaging predictions from multiple independently trained MLPs—and (c) using Bayesian optimization to automate hyper‑parameter tuning. They also acknowledge RegnANN’s current limitations, notably its computational cost (training time and memory consumption) and scalability concerns for networks comprising thousands of genes. Future work is outlined to address these challenges via GPU acceleration, sparse‑weight regularization, and integration of multi‑omics data (transcriptomics, proteomics, metabolomics) to improve inference fidelity.

In conclusion, the study provides a rigorous, quantitative comparison of GRN inference algorithms, highlighting that instability is a pervasive problem across methods. RegnANN emerges as a promising alternative, delivering higher accuracy and lower variability thanks to its ability to model nonlinear gene‑gene relationships. The findings advocate for broader adoption of deep‑learning‑based approaches in network reconstruction and call for standardized protocols to assess algorithmic stability in future systems‑biology research.