On the numeric stability of the SFA implementation sfa-tk

Slow feature analysis (SFA) is a method for extracting slowly varying features from a quickly varying multidimensional signal. An open source Matlab-implementation sfa-tk makes SFA easily useable. We show here that under certain circumstances, namely when the covariance matrix of the nonlinearly expanded data does not have full rank, this implementation runs into numerical instabilities. We propse a modified algorithm based on singular value decomposition (SVD) which is free of those instabilities even in the case where the rank of the matrix is only less than 10% of its size. Furthermore we show that an alternative way of handling the numerical problems is to inject a small amount of noise into the multidimensional input signal which can restore a rank-deficient covariance matrix to full rank, however at the price of modifying the original data and the need for noise parameter tuning.

💡 Research Summary

Slow Feature Analysis (SFA) is a widely used unsupervised technique for extracting slowly varying latent variables from rapidly changing high‑dimensional time series. The open‑source MATLAB toolbox sfa‑tk implements the classic SFA pipeline: nonlinear expansion of the input, whitening of the expanded data, and a final eigenvalue decomposition (EVD) that yields the directions of minimal temporal variation. While the toolbox is praised for its ease of use, this paper demonstrates that it suffers from severe numerical instabilities when the covariance matrix of the expanded data is rank‑deficient.

The authors first analyse the root cause of the problem. In many practical scenarios the nonlinear expansion dramatically increases dimensionality, often far beyond the number of available samples. Consequently the sample covariance matrix becomes singular or ill‑conditioned, with many eigenvalues close to zero. The standard EVD used in sfa‑tk is highly sensitive to such small eigenvalues; division by near‑zero values leads to NaNs, Infs, or wildly inaccurate eigenvectors, and the final slow features lose their interpretability. The authors verify this behavior on synthetic data deliberately constructed to have a rank of only 5 % of the full dimensionality, as well as on real multimodal sensor recordings where the condition number of the covariance matrix exceeds 10⁸.

To address the instability, two remedies are proposed. The primary solution replaces the eigenvalue decomposition with a singular value decomposition (SVD). SVD does not require the matrix to be full rank and computes all singular values accurately, even when many are extremely small. By integrating SVD into the existing sfa‑tk pipeline, the authors obtain a modified algorithm that remains stable for rank‑deficient matrices down to less than 10 % of the original size. Experimental results show that the SVD‑based version recovers the correct slow features on both synthetic and real datasets, with no overflow or NaN issues, and with only a modest increase in computational cost.

The secondary remedy explored is the injection of a small amount of Gaussian noise into the original data before the expansion step. Adding noise perturbs the covariance matrix just enough to lift its rank to full, thereby allowing the original EVD to succeed. While this technique can be effective—noise levels on the order of σ = 10⁻⁴ restore full rank in the authors’ experiments—it also degrades the signal‑to‑noise ratio and modifies the underlying data distribution. Consequently, the extracted features may no longer faithfully represent the true slow dynamics, and the method requires careful tuning of the noise amplitude for each dataset.

The paper concludes that the numerical instability is intrinsic to the original sfa‑tk design when faced with rank‑deficient covariance matrices, and that an SVD‑based reformulation provides a robust, parameter‑free alternative. Noise injection, while a quick fix, introduces undesirable side effects and should be used only when a minimal perturbation of the data is acceptable. The authors suggest future work on hybrid schemes that combine SVD with regularization, dimensionality‑reduction strategies prior to expansion, and memory‑efficient approximations for large‑scale applications. Such developments would broaden the applicability of SFA to real‑world high‑dimensional streams while guaranteeing numerical reliability.