Efficient Convex Optimization with Membership Oracles

We consider the problem of minimizing a convex function over a convex set given access only to an evaluation oracle for the function and a membership oracle for the set. We give a simple algorithm which solves this problem with $\tilde{O}(n^2)$ oracle calls and $\tilde{O}(n^3)$ additional arithmetic operations. Using this result, we obtain more efficient reductions among the five basic oracles for convex sets and functions defined by Gr"otschel, Lovasz and Schrijver.

💡 Research Summary

The paper addresses the classic problem of minimizing a convex function f over a convex set K when the only available information comes from two black‑box oracles: an evaluation oracle that returns (approximately) the value of f at any query point, and a membership oracle that decides (approximately) whether a point belongs to K. The authors assume that K is sandwiched between two Euclidean balls centered at a known point x₀, with radii r and R (0 < r < R). Under these mild geometric assumptions they design a simple algorithm that finds a point z within distance ε of K such that

 f(z) ≤ min_{x∈K} f(x) + ε·(max_{x∈K} f(x) − min_{x∈K} f(x))

with high constant probability. The algorithm uses

* ˜O(n² log( nR/(εr))) membership and evaluation oracle calls, and
* ˜O(n³ log( nR/(εr))) additional arithmetic operations.

The key technical contribution is a two‑step reduction that replaces the stronger separation oracle (traditionally required for convex optimization) with only membership queries.

Step 1 – Approximate subgradients via finite differences.
For any Lipschitz convex function g, the authors show that a random partial difference in each coordinate, computed inside a small ℓ∞‑box, yields an unbiased estimator of a subgradient. Lemma 9 (based on a quantitative Alexandrov theorem) bounds the expected deviation between the finite‑difference estimator and the true gradient. By sampling enough points and applying Markov’s inequality they obtain a high‑probability guarantee that the estimator is within O(ε) of a true subgradient.

Step 2 – From subgradients to a separation hyperplane for K.
Define the “height” function hₓ(d) = −αₓ(d)‖x‖₂, where αₓ(d) is the maximal scalar such that d + αₓ(d) x ∈ K. The authors prove that hₓ is convex and Lipschitz with constant (R+δ)/r over a small neighborhood. By applying the subgradient estimator from Step 1 to hₓ, they obtain an approximate subgradient ĝ at the origin. Using the inequality

 hₓ(y) ≥ hₓ(0) + ⟨ĝ, y⟩ − ζ‖y‖∞ − O(n r₁ κ)

(Lemma 10) they construct a half‑space that contains K and separates the query point x from K with high probability. This yields a separation oracle that calls the membership oracle only ˜O(1) times per separation.

With a separation oracle in hand, the authors invoke the classic reduction from optimization to separation (e.g., the ellipsoid method or interior‑point‑type updates). Each iteration requires a separation query and a few arithmetic operations; after ˜O(log( nR/(εr))) iterations they obtain the desired ε‑approximate minimizer.

Beyond the main algorithm, the paper revisits the five basic oracles defined by Grötchel, Lovász, and Schrijver—Optimization (OPT), Separation (SEP), Membership (MEM), Violation (VIOL), and Validation (VAL)—and shows that, using the new MEM→SEP construction, all reductions between them can be performed in ˜O(n) oracle calls (up to logarithmic factors). Theorem 21 summarizes these improved reductions, suggesting that the presented complexities are essentially optimal in the dimension n (up to polylogarithmic terms).

The authors also discuss connections to prior work: the classic ellipsoid algorithm achieves ˜O(n²) oracle calls but requires a separation oracle; earlier reductions from MEM to SEP needed Ω(n) calls; random‑walk and simulated‑annealing approaches achieved ˜O(n⁴) or ˜O(n³ √n) complexities. The new method improves these bounds dramatically while remaining conceptually simple.

In summary, the paper delivers a clean, dimension‑efficient reduction from membership‑only access to full convex optimization, achieving ˜O(n²) oracle complexity and ˜O(n³)  arithmetic cost. This advances the theoretical understanding of black‑box convex optimization and opens the door to practical algorithms in settings where only membership tests are feasible (e.g., high‑dimensional feasibility problems, black‑box constraint sets, oracles arising from sampling procedures).