Towards A Deeper Geometric, Analytic and Algorithmic Understanding of Margins

Given a matrix $A$, a linear feasibility problem (of which linear classification is a special case) aims to find a solution to a primal problem $w: A^Tw > \textbf{0}$ or a certificate for the dual problem which is a probability distribution $p: Ap = \textbf{0}$. Inspired by the continued importance of “large-margin classifiers” in machine learning, this paper studies a condition measure of $A$ called its \textit{margin} that determines the difficulty of both the above problems. To aid geometrical intuition, we first establish new characterizations of the margin in terms of relevant balls, cones and hulls. Our second contribution is analytical, where we present generalizations of Gordan’s theorem, and variants of Hoffman’s theorems, both using margins. We end by proving some new results on a classical iterative scheme, the Perceptron, whose convergence rates famously depends on the margin. Our results are relevant for a deeper understanding of margin-based learning and proving convergence rates of iterative schemes, apart from providing a unifying perspective on this vast topic.

💡 Research Summary

The paper investigates the role of the margin—a condition measure of a data matrix A—in linear feasibility problems and their dual formulations. The primal problem (P) asks for a vector w ∈ ℝᵈ such that Aᵀw > 0, while the dual (D) seeks a non‑zero probability distribution p ≥ 0 with Ap = 0. Classical theory defines the margin as ρ = sup_{‖w‖=1} inf_{p∈Δ} wᵀAp, but this definition collapses to zero whenever A has low rank, thereby failing to capture the true difficulty of the instance.

To remedy this, the authors introduce the affine‑margin ρ_A, which restricts the search for w to the linear span of the columns of A (i.e., w = Aα for some α). Formally, \