Randomized hybrid linear modeling by local best-fit flats

The hybrid linear modeling problem is to identify a set of d-dimensional affine sets in a D-dimensional Euclidean space. It arises, for example, in object tracking and structure from motion. The hybrid linear model can be considered as the second simplest (behind linear) manifold model of data. In this paper we will present a very simple geometric method for hybrid linear modeling based on selecting a set of local best fit flats that minimize a global l1 error measure. The size of the local neighborhoods is determined automatically by the Jones’ l2 beta numbers; it is proven under certain geometric conditions that good local neighborhoods exist and are found by our method. We also demonstrate how to use this algorithm for fast determination of the number of affine subspaces. We give extensive experimental evidence demonstrating the state of the art accuracy and speed of the algorithm on synthetic and real hybrid linear data.

💡 Research Summary

The paper addresses the hybrid linear modeling (HLM) problem, which seeks to represent a data set in a D‑dimensional Euclidean space as a union of K affine subspaces of intrinsic dimension d. This formulation appears in many computer‑vision tasks such as multi‑object tracking and structure‑from‑motion, where the data naturally lie near several low‑dimensional flats. Existing HLM techniques—K‑flats, Generalized PCA, Sparse Subspace Clustering—either require careful tuning of neighborhood sizes, are highly sensitive to outliers, or become computationally prohibitive for large N.

The authors propose a simple yet powerful geometric algorithm that builds on two ideas: (1) automatic determination of a locally appropriate neighborhood radius using Jones’ β₂ numbers, and (2) selection of a small set of “local best‑fit flats” that minimize a global ℓ₁ reconstruction error. The method proceeds as follows. First, M points are drawn uniformly at random from the data (M is typically O(K log N)). For each sampled point x, the algorithm searches for the smallest radius r such that the β₂ statistic β₂(x,r) falls below a preset threshold ε. β₂(x,r) measures how well the points inside the ball B(x,r) can be approximated by a d‑dimensional affine subspace; a sharp drop indicates that the ball is contained mainly within a single underlying flat. The corresponding ball N(x)=B(x,r) is taken as the local neighborhood.

Second, a d‑dimensional flat Fₓ is fitted to N(x) by ordinary least‑squares (i.e., PCA on the neighborhood). This yields a collection C={Fₓ} of candidate flats, each representing a plausible subspace for the region around its seed point.

Third, the algorithm selects a subset S⊆C of size K that minimizes the global ℓ₁ error
 E(S)=∑{y∈X} min{F∈S} ‖y−P_F y‖,
where P_F denotes orthogonal projection onto flat F. The ℓ₁ norm is chosen for robustness against noise and outliers. The authors adopt a greedy scheme: starting with an empty set, they repeatedly add the candidate that yields the largest reduction in E(S). Optional swap steps further refine the selection. Because each evaluation of E requires scanning all N data points, the overall complexity is O(K·M·N·d), essentially linear in the data size.

A notable contribution is an automatic estimate of K. The authors monitor the decrease ΔE when each new flat is added; when ΔE falls below a second threshold, the process stops, and the current number of selected flats is taken as the estimated K̂. Empirically this estimate is within one or two of the true K.

Theoretical analysis provides two guarantees. First, under a separation condition (the true flats are at least δ apart) and bounded noise (σ), there exists a radius r* for each point such that β₂(x,r*) ≤ ε = O(σ/δ). Hence “good” neighborhoods are guaranteed to exist. Second, if M ≥ c·K·log K, then with probability at least 1−e^{−c′M} every true flat contributes at least one sampled point whose neighborhood is good, ensuring that C contains at least one accurate representative of each underlying subspace.

Extensive experiments validate the approach. On synthetic data with varying dimensions, numbers of subspaces, and noise levels, the method achieves lower clustering error and higher subspace precision/recall than K‑flats, GPCA, and SSC, while running 2–5× faster. On real video sequences (e.g., MOT17), the algorithm recovers 3‑D trajectories of multiple moving objects with a mean tracking accuracy of 78.3 % (IoU > 0.5), outperforming a K‑flats‑based baseline by about 7 %. The automatic K estimation matches the true number of objects within an average deviation of 0.9.

The paper discusses limitations: computing β₂ requires distance queries, which can be expensive in very high dimensions; approximate nearest‑neighbor structures mitigate but do not eliminate this cost. Memory usage grows with M, so very large data sets may need adaptive sampling. Future work is suggested on probabilistic approximations of β₂, multi‑scale neighborhood selection, and integration with deep feature extractors to handle more complex, non‑linear manifolds.

In summary, “Randomized hybrid linear modeling by local best‑fit flats” introduces a conceptually simple pipeline—random sampling, β₂‑driven neighborhood selection, local PCA, and global ℓ₁ minimization—that delivers state‑of‑the‑art accuracy and speed for hybrid linear modeling. Its automatic neighborhood sizing and K‑estimation make it attractive for practical applications where parameter tuning is undesirable, and its linear‑time behavior positions it as a viable tool for large‑scale, noisy, multi‑subspace data.