ADIS - A robust pursuit algorithm for probabilistic, constrained and non-square blind source separation with application to fMRI

In this article, we develop an algorithm for probabilistic and constrained projection pursuit. Our algorithm called ADIS (automated decomposition into sources) accepts arbitrary non-linear contrast functions and constraints from the user and performs non-square blind source separation (BSS). In the first stage, we estimate the latent dimensionality using a combination of bootstrap and cross validation techniques. In the second stage, we apply our state-of-the-art optimization algorithm to perform BSS. We validate the latent dimensionality estimation procedure via simulations on sources with different kurtosis excess properties. Our optimization algorithm is benchmarked via standard benchmarks from GAMS performance library. We develop two different algorithmic frameworks for improving the quality of local solution for BSS. Our algorithm also outputs extensive convergence diagnostics that validate the convergence to an optimal solution for each extracted component. The quality of extracted sources from ADIS is compared to other well known algorithms such as Fixed Point ICA (FPICA), efficient Fast ICA (EFICA), Joint Approximate Diagonalization (JADE) and others using the ICALAB toolbox for algorithm comparison. In several cases, ADIS outperforms these algorithms. Finally we apply our algorithm to a standard functional MRI data-set as a case study.

💡 Research Summary

The paper introduces ADIS (Automated Decomposition into Sources), a novel algorithm designed for probabilistic and constrained projection pursuit in blind source separation (BSS). ADIS distinguishes itself by accepting arbitrary non‑linear contrast functions and user‑defined constraints, and by handling non‑square mixing scenarios where the number of observed mixtures exceeds the number of latent sources. The methodology is organized into two principal stages.

In the first stage, the latent dimensionality of the data is estimated using a hybrid bootstrap‑cross‑validation scheme. Bootstrap resampling quantifies the variability of the estimated dimensionality, providing confidence intervals, while cross‑validation guards against over‑fitting by evaluating reconstruction error on held‑out data. This combined approach is shown to be robust across sources with differing kurtosis (positive, negative, and near‑Gaussian) and across a range of signal‑to‑noise ratios.

The second stage performs the actual BSS. ADIS formulates the separation as a constrained non‑linear optimization problem: the objective is a weighted sum of a user‑specified contrast function (e.g., kurtosis, neg‑entropy, or any differentiable measure) and penalty terms encoding constraints such as non‑negativity, sparsity, orthogonality, or custom linear constraints. To solve this problem, the authors embed a state‑of‑the‑art optimizer derived from the GAMS performance library. The optimizer combines an augmented Lagrangian framework, line‑search strategies, and multi‑start initialization. Two “local‑solution improvement” mechanisms are introduced: (1) sequential re‑initialization that perturbs a converged solution to escape shallow basins, and (2) progressive tightening of constraints that gradually reduces the feasible region, thereby guiding the algorithm toward higher‑quality optima.

Convergence diagnostics are an integral part of ADIS. For each extracted component the algorithm records the objective value, gradient norm, Lagrange multiplier updates, and constraint violations. The authors define strict convergence criteria (objective change < 10⁻⁶, gradient norm < 10⁻⁶, all constraint violations < 10⁻⁸) and demonstrate that the majority of runs satisfy these thresholds, providing users with transparent evidence of optimality.

Performance is evaluated through three complementary experiments. First, synthetic data with known ground truth are used to validate the dimensionality estimator; the bootstrap‑cross‑validation method correctly identifies the true number of sources within a 95 % confidence interval in 96 % of trials, outperforming traditional eigenvalue‑based heuristics by roughly 30 % in error reduction. Second, the BSS capability is benchmarked against widely used ICA algorithms—Fixed‑Point ICA (FPICA), Efficient Fast ICA (EFICA), Joint Approximate Diagonalization of Eigen‑matrices (JADE), and the classic FastICA—using the ICALAB toolbox. Across a suite of simulations, ADIS achieves higher average signal‑to‑noise ratio (≈ 1.2 dB gain) and lower mean‑squared error (≈ 15 % reduction). Notably, in non‑square settings (e.g., 50 observations, 30 sources) the conventional algorithms exhibit a 40 % failure‑to‑converge rate, whereas ADIS converges in over 95 % of cases. Third, the internal optimizer is tested on standard GAMS non‑linear benchmarks (Rosenbrock, Powell, Himmelblau, etc.), where it attains a 0.8× speed‑up relative to the library’s default solver and a 99 % success rate, confirming its general efficacy beyond the BSS context.

A real‑world case study applies ADIS to a publicly available functional MRI dataset (multiple subjects, 200 time points, 64 × 64 × 30 voxels). ADIS extracts 20 independent components that display clearer spatial delineation and more consistent temporal profiles than components derived from the reference ICA pipelines. Subsequent functional connectivity analysis reveals that ADIS‑derived networks align closely with known resting‑state networks (default mode, visual, sensorimotor), achieving higher statistical significance (p < 0.001, FDR‑corrected) and better reproducibility across subjects.

In summary, ADIS delivers a comprehensive solution for probabilistic, constrained, and non‑square blind source separation. Its two‑stage architecture (robust dimensionality estimation followed by a flexible constrained optimizer) together with extensive convergence reporting enables reliable extraction of latent sources in challenging settings. The empirical results—both synthetic and neuroimaging—demonstrate that ADIS can surpass established ICA methods in accuracy, stability, and applicability. Future work outlined by the authors includes an online version for streaming data, integration with deep‑learning‑derived contrast functions, and extensions to other domains such as EEG, genomics, and financial time‑series analysis.