Algebraic characterization of equivalence between oracle-based iterative algorithms

When are two algorithms the same? How can we be sure a recently proposed algorithm is novel, and not a minor variation on an existing method? In this paper, we present a framework for reasoning about equivalence between a broad class of iterative algorithms, with a focus on algorithms designed for convex optimization. We propose several notions of what it means for two algorithms to be equivalent, and provide computationally tractable means to detect equivalence. Our main definition, oracle equivalence, states that two algorithms are equivalent if they result in the same sequence of calls to the function oracles (for suitable initialization). Borrowing from control theory, we use state-space realizations to represent algorithms and characterize algorithm equivalence via transfer functions. Our framework can also identify and characterize equivalence between algorithms that use different oracles that are related via a linear fractional transformation. Prominent examples include linear transformations and function conjugation. To support the paper, we have developed a software package named Linnaeus that implements the framework to identify other iterative algorithms that are equivalent to an input algorithm.

💡 Research Summary

The paper tackles the fundamental question of when two iterative optimization algorithms should be considered the same. Focusing on convex‑optimization methods that rely on oracle calls (gradients, proximal operators, projections, etc.), the authors introduce a mathematically rigorous framework that can automatically detect various notions of algorithmic equivalence.

First, every oracle‑based algorithm is expressed as a linear time‑invariant (LTI) dynamical system in feedback with a (possibly vector‑valued) nonlinear oracle Φ. In state‑space form the update equations are
 xₖ₊₁ = A xₖ + B uₖ, yₖ = C xₖ + D uₖ, uₖ = Φ(yₖ),
where (A,B,C,D) are constant matrices that encode the linear part of the algorithm, while Φ aggregates all oracle functions. This representation, originally introduced by Lessard et al. for first‑order methods, is extended here to handle multiple oracles and to capture a broad class of splitting and accelerated schemes.

The core contribution is three increasingly general definitions of equivalence, each expressed as a simple relationship between the transfer functions G(z)=C(zI−A)⁻¹B + D of the two algorithms:

1. Oracle equivalence – The two algorithms generate exactly the same sequence of oracle queries (for a suitable initialization). This holds iff their transfer functions are identical, i.e., G₁(z)=G₂(z).

2. Shift equivalence – The algorithms may differ in a finite prefix of iterations but coincide thereafter. Formally, G₁(z)=z⁻ᵗ G₂(z) for some integer delay τ≥0. This captures cases such as Douglas‑Rachford and ADMM, which differ only by an initial proximal call.

3. Linear‑fractional‑transformation (LFT) equivalence – The algorithms use different oracles that are related by a linear fractional transformation Φ₂ = (M Φ₁+N)(P Φ₁+Q)⁻¹, where M,N,P,Q are constant matrices. When this holds, the underlying linear dynamics are the same, and the two methods are equivalent up to the oracle transformation. Classic examples include the relationship between proximal‑gradient and its conjugate‑gradient counterpart via Moreau’s identity.

To decide equivalence, the authors propose a computational pipeline: (i) parse the user‑provided pseudo‑code, (ii) construct the (A,B,C,D) matrices, (iii) compute the minimal realization of the transfer function, and (iv) test the appropriate algebraic condition (equality, time‑shift, or LFT). All steps rely on standard linear‑algebraic operations and symbolic polynomial manipulation, making the procedure tractable for algorithms of realistic size.

A software package named Linnaeus implements the entire pipeline. Users write an algorithm in a natural, MATLAB‑like syntax; Linnaeus extracts the state‑space model, normalizes the transfer function, and reports whether the input algorithm is oracle‑equivalent, shift‑equivalent, or LFT‑equivalent to any algorithm stored in its database (which currently contains over 30 published methods). The tool also supplies the explicit linear transformation or shift that maps one algorithm to the other, and provides bibliographic references to the original works.

The paper validates the framework on several well‑known families: (i) the modified Arrow–Hurwicz, extrapolation‑from‑the‑past, optimistic mirror descent, and reflected‑gradient methods are shown to be exactly oracle‑equivalent; (ii) Douglas‑Rachford splitting and a simplified ADMM are identified as shift‑equivalent; (iii) proximal‑gradient and its conjugate‑gradient variant are proved LFT‑equivalent via Moreau’s identity. In each case, Linnaeus reproduces the known relationships automatically, confirming both correctness and practical utility.

Limitations are acknowledged. The framework assumes time‑invariant linear dynamics, so algorithms with diminishing step sizes, adaptive momentum, or stochastic oracle outputs fall outside the current scope. The authors suggest extending the theory using switching‑system models to capture piecewise‑constant parameter schedules, and developing probabilistic notions of equivalence for stochastic oracles.

In summary, the paper delivers a unified algebraic language for reasoning about algorithmic sameness, bridges control‑theoretic concepts with optimization algorithm design, and provides an open‑source tool that can streamline literature surveys, prevent redundant contributions, and aid in the systematic construction of new methods from existing building blocks.