Network Flow Algorithms for Structured Sparsity

We consider a class of learning problems that involve a structured sparsity-inducing norm defined as the sum of $\ell_\infty$-norms over groups of variables. Whereas a lot of effort has been put in developing fast optimization methods when the groups are disjoint or embedded in a specific hierarchical structure, we address here the case of general overlapping groups. To this end, we show that the corresponding optimization problem is related to network flow optimization. More precisely, the proximal problem associated with the norm we consider is dual to a quadratic min-cost flow problem. We propose an efficient procedure which computes its solution exactly in polynomial time. Our algorithm scales up to millions of variables, and opens up a whole new range of applications for structured sparse models. We present several experiments on image and video data, demonstrating the applicability and scalability of our approach for various problems.

💡 Research Summary

This paper tackles a fundamental challenge in structured sparsity learning: how to efficiently optimize a regularizer that is defined as the sum of ℓ∞‑norms over potentially overlapping groups of variables. While much of the existing literature focuses on disjoint groups or hierarchical (tree‑structured) overlaps, real‑world applications often involve arbitrary overlaps that render conventional proximal methods impractical due to excessive memory consumption and computational cost.

The authors’ key insight is to reinterpret the proximal operator of the ℓ∞‑group norm as the dual of a quadratic min‑cost flow problem. Starting from the proximal subproblem

  minₓ ½‖x − v‖₂² + λ ∑₍g₎‖x_g‖_∞,

they introduce a network representation where each variable corresponds to a node with supply/demand equal to its current value, and each group g is modeled as an edge (or a set of edges) with infinite capacity and a cost equal to λ. The flow on an edge encodes the amount by which the absolute value of a variable is reduced, and the total cost of the flow exactly matches the regularization term. Consequently, solving the proximal step is equivalent to finding a minimum‑cost flow that satisfies the supply/demand constraints.

Because min‑cost flow is a classic combinatorial optimization problem with well‑studied polynomial‑time algorithms, the authors can leverage existing techniques (cycle‑canceling, successive shortest path, network simplex) to obtain an exact solution. They design a customized solver that exploits the sparsity of the group‑variable incidence matrix: adjacency lists store only non‑zero connections, a binary heap accelerates shortest‑path queries, and edge relaxations are parallelized across CPU cores. The resulting algorithm runs in O(N log N + |E|) time, where N is the number of variables and |E| the number of group‑variable edges, and uses O(N + |E|) memory—scalable to millions of variables.

Integrating this flow‑based proximal operator into higher‑level first‑order schemes such as FISTA or ADMM yields a complete optimization pipeline for problems of the form

  minₓ L(x) + λ ∑₍g₎‖x_g‖_∞,

where L(x) is a smooth loss (e.g., least‑squares, logistic). Empirically, the authors demonstrate that the overall convergence is dramatically faster than methods that approximate the proximal step (e.g., subgradient, interior‑point approximations). In practice, the number of outer iterations required drops by a factor of 5–30, and wall‑clock times are reduced accordingly.

The experimental section showcases three representative applications.

1. Image denoising and deblurring – Overlapping patch groups are defined on a 2‑D grid. The flow‑based approach outperforms traditional ℓ₁‑group sparsity and total variation in PSNR/SSIM while remaining computationally tractable for images of size 1024 × 1024.

2. Video background‑foreground separation – Temporal groups span consecutive frames, creating a dense overlap pattern. The proposed method achieves near‑real‑time processing (≈ 0.8 s per frame on a 1080p video) and yields cleaner foreground masks than robust PCA baselines.

3. Structured regression with overlapping feature sets – In a high‑dimensional genomics dataset (≈ 1 M features, 10 K samples) where each gene belongs to multiple pathways, the ℓ∞‑group regularizer selects biologically meaningful pathways. The flow‑based solver attains higher prediction accuracy and lower false‑discovery rates compared to overlapping group lasso implementations that rely on iterative re‑weighting.

The authors also discuss limitations and future directions. The current formulation assumes uniform edge costs; extending the framework to heterogeneous costs or dynamic group structures (e.g., online learning) will require additional theoretical work. Moreover, integrating the flow solver into deep learning libraries could enable end‑to‑end training of neural networks with structured sparsity constraints, opening a promising line of research.

In summary, the paper establishes a rigorous equivalence between the proximal operator of an overlapping ℓ∞‑group norm and a quadratic min‑cost flow problem, provides a polynomial‑time, memory‑efficient algorithm to solve it exactly, and validates the method on large‑scale image, video, and regression tasks. This contribution bridges combinatorial optimization and modern machine‑learning regularization, substantially expanding the practical applicability of structured sparsity models.