Auto-associative models, nonlinear Principal component analysis, manifolds and projection pursuit

In this paper, auto-associative models are proposed as candidates to the generalization of Principal Component Analysis. We show that these models are dedicated to the approximation of the dataset by a manifold. Here, the word “manifold” refers to the topology properties of the structure. The approximating manifold is built by a projection pursuit algorithm. At each step of the algorithm, the dimension of the manifold is incremented. Some theoretical properties are provided. In particular, we can show that, at each step of the algorithm, the mean residuals norm is not increased. Moreover, it is also established that the algorithm converges in a finite number of steps. Some particular auto-associative models are exhibited and compared to the classical PCA and some neural networks models. Implementation aspects are discussed. We show that, in numerous cases, no optimization procedure is required. Some illustrations on simulated and real data are presented.

💡 Research Summary

The paper introduces auto‑associative models as a generalization of Principal Component Analysis (PCA) and proposes a projection‑pursuit algorithm to construct low‑dimensional manifolds that approximate high‑dimensional data. An auto‑associative function Fₙ: ℝᵖ → ℝᵖ is defined by d orthonormal direction vectors a₁,…,a_d and d continuously differentiable regression functions s₁,…,s_d satisfying the bi‑directional condition P_{a_j}∘s_k = δ_{jk} Id, where P_{a_j} denotes orthogonal projection onto a_j. This condition guarantees that the zero‑set of Fₙ is a d‑dimensional manifold (Theorem 1). Consequently, any random vector X can be decomposed into a manifold component and a residual ε = Fₙ(X); the residual variance σ²(ε) measures the variance of X outside the manifold.

The core algorithm (Algorithm 1) iteratively performs four steps: (A) Axis selection – maximize a chosen index I (e.g., projected variance) over unit vectors orthogonal to previously selected axes; (P) Projection – compute the principal variable Y_k = P_{a_k}(R_{k‑1}); (R) Regression – estimate the conditional expectation s_k(t) = E