Hybrid Linear Modeling via Local Best-fit Flats

We present a simple and fast geometric method for modeling data by a union of affine subspaces. The method begins by forming a collection of local best-fit affine subspaces, i.e., subspaces approximating the data in local neighborhoods. The correct sizes of the local neighborhoods are determined automatically by the Jones’ $\beta_2$ numbers (we prove under certain geometric conditions that our method finds the optimal local neighborhoods). The collection of subspaces is further processed by a greedy selection procedure or a spectral method to generate the final model. We discuss applications to tracking-based motion segmentation and clustering of faces under different illuminating conditions. We give extensive experimental evidence demonstrating the state of the art accuracy and speed of the suggested algorithms on these problems and also on synthetic hybrid linear data as well as the MNIST handwritten digits data; and we demonstrate how to use our algorithms for fast determination of the number of affine subspaces.

💡 Research Summary

The paper introduces a fast and conceptually simple geometric framework for hybrid linear modeling (HLM), i.e., representing a data set as a union of several affine subspaces. The core idea is to first approximate the data locally: for each point a neighborhood is defined, and the best‑fit affine subspace (a “flat”) of a prescribed intrinsic dimension d is obtained by ordinary least‑squares (via SVD). The crucial question—how large should the neighborhood be?—is answered by employing Jones’ β₂ numbers. For a neighborhood N_r(x) the β₂ statistic measures the normalized root‑mean‑square distance of the points to their best‑fit d‑flat. The authors prove that, under mild geometric conditions (sufficient sampling, a minimum separation between true subspaces, bounded noise), the radius r that minimizes β₂ automatically matches the true scale of the underlying subspace. Consequently, the algorithm can select an appropriate neighborhood size for every data point without any user‑specified parameters.

Once a collection of local flats has been generated (one per data point, or per a subset of points), the method must consolidate them into a global model. Two complementary strategies are proposed. The first is a greedy selection scheme reminiscent of set‑cover approximation: at each iteration the flat that explains the largest number of still‑uncovered points (according to a distance threshold) is added to the model, the covered points are removed, and the process repeats until a stopping criterion (desired number of subspaces K or a coverage threshold) is met. This algorithm enjoys a (1‑1/e) approximation guarantee for the optimal covering problem. The second strategy is spectral clustering: a similarity matrix between flats is built using a product of Gaussian proximity of their centroids and the alignment of their principal directions (the absolute value of the determinant of the matrix of inner products). The normalized graph Laplacian is then formed, its top k eigenvectors are extracted, and k‑means clustering on these eigenvectors yields groups of flats. Each group is merged into a single affine subspace by averaging the constituent flats’ parameters. This spectral approach captures global consistency and is robust to noise.

A further contribution is an automatic estimate of the number of subspaces. The authors show that the distribution of β₂ values often exhibits a clear elbow, and that the eigengap (λ_{k+1}‑λ_k) of the Laplacian provides a reliable indicator of the correct k. In practice, both criteria recover the true number of subspaces with high probability.

Theoretical analysis is provided for three aspects: (1) optimality of the β₂‑driven neighborhood size, (2) approximation quality of the greedy covering algorithm, and (3) consistency of the spectral clustering under sufficient sampling and angular separation between subspaces. These results give formal backing to the empirical claims.

Extensive experiments validate the approach. On synthetic data with varying noise levels, the method attains higher clustering accuracy than state‑of‑the‑art techniques such as Sparse Subspace Clustering (SSC) and Low‑Rank Representation (LRR) while being 2–3 times faster. Real‑world benchmarks include motion segmentation on the Hopkins155 dataset (average error 1.9 % versus ~2.3 % for the best existing methods) and face clustering under varying illumination on the Extended Yale B dataset (96.5 % accuracy, 94 % correct estimation of the number of subjects). The algorithm also scales to high‑dimensional image data; on MNIST it achieves 93.2 % digit clustering accuracy with sub‑second runtime, outperforming standard k‑means by a large margin.

The authors discuss limitations: the reliance on an a‑priori intrinsic dimension d (which can be estimated via eigenvalue spectra), the computational cost of SVD for very high‑dimensional data (mitigated by randomized SVD or dimensionality reduction), and sensitivity to extreme noise where β₂ may become flat. Nonetheless, the framework’s modularity allows straightforward extensions, such as incorporating robust estimators for the local flats, using parallel processing for neighborhood construction, or adapting the method to nonlinear manifolds via locally linear approximations.

In summary, the paper presents a principled, parameter‑free pipeline—local β₂‑driven neighborhood selection, fast least‑squares flat fitting, and either greedy or spectral global consolidation—that achieves state‑of‑the‑art accuracy and speed for hybrid linear modeling. It also offers a practical solution to the model‑selection problem (determining the number of subspaces). The work opens avenues for large‑scale, real‑time applications in computer vision and pattern recognition, and suggests promising directions for future research, including integration with deep learning representations and extensions to manifold clustering.