The Query Complexity of Local Search in Rounds on General Graphs

We analyze the query complexity of finding a local minimum in $t$ rounds on general graphs. More precisely, given a graph $G = (V,E)$ and oracle access to an unknown function $f : V \to \mathbb{R}$, the goal is to find a local minimum–a vertex $v$ such that $f(v) \leq f(u)$ for all $(u,v) \in E$–using at most $t$ rounds of interaction with the oracle. The query complexity is well understood on grids, but much less is known beyond. This abstract problem captures many optimization tasks, such as finding a local minimum of a loss function during neural network training. For each graph with $n$ vertices, we prove a deterministic upper bound of $O(t n^{1/t} (sΔ)^{1-1/t})$, where $s$ is the separation number and $Δ$ is the maximum degree of the graph. We complement this result with a randomized lower bound of $Ω(t n^{1/t}-t)$ that holds for any connected graph. We also find that parallel steepest descent with a warm start provides improved bounds for graphs with high separation number and bounded degree. To obtain our results, we utilized an advanced version of Gemini at various stages of our research. We discuss our experience in a methodology section.

💡 Research Summary

The paper studies the query complexity of finding a local minimum of an unknown function defined on the vertices of a general graph when the algorithm is limited to a fixed number of interaction rounds t. The model is black‑box: the algorithm can query any vertex v to obtain f(v), but it must submit all queries for round j before seeing any answers from that round. The goal is to minimize the total number of queries required to guarantee a local minimum (a vertex whose value is no larger than all its neighbors) in the worst case.

The authors first introduce the graph‑theoretic parameters that will drive their bounds: the maximum degree Δ and the separation number s, which is the smallest size of a vertex separator that splits any subgraph into two parts each of size at most 2/3 of the original. The separation number is known to be within a constant factor of treewidth, so it captures how “well‑decomposable’’ a graph is.

Deterministic Upper Bound.
Using a recursive divide‑and‑conquer scheme based on separators, they prove that any connected graph with n vertices can be solved in t ≥ 2 rounds with at most

  O( t·n^{1/t}·(s·Δ)^{1‑1/t} )

queries, never exceeding n. The algorithm works as follows. In the first round it queries all vertices of a separator S of size O(s·n/K) for a parameter K to be chosen later. The vertex v_min in S with smallest observed value is identified. In the second round the algorithm queries only the connected components of G\ S that contain neighbors of v_min; all other components are ignored because they cannot contain a better local minimum. By choosing K≈n^{1‑1/t} the size of each remaining component shrinks geometrically, and after t rounds the search space is reduced to a constant‑size set that can be examined exhaustively. The factor (s·Δ)^{1‑1/t} accounts for the need to explore the neighborhood of the current candidate in each round.

Randomized Lower Bound.
To show that the above upper bound is essentially optimal, the paper constructs a hard distribution over functions on any connected graph. Fix a spanning tree T rooted at r. Choose a target vertex Z uniformly at random; define f(v)=−dist_T(r,v) if v lies on the unique path from r to Z, and f(v)=+dist_T(r,v) otherwise. This “staircase’’ function has a unique local minimum at Z, and any algorithm must reduce the candidate subtree size by at most a factor of n^{1/t} per round. An inductive argument yields a lower bound of

  Ω( t·n^{1/t} − t )

queries for any randomized algorithm that succeeds with constant probability (the constant is independent of n and t). The bound holds even when the success probability is as low as 1/n.

Warm‑Start Parallel Steepest Descent.
The authors also analyze a more practical algorithm that combines a warm‑start (sampling √n·Δ vertices uniformly at random, picking the best as a starting point) with parallel steepest descent. In each subsequent round the algorithm queries all neighbors of the current vertex and moves to the neighbor with the largest decrease. For graphs with Δ≤2 (essentially trees or low‑degree graphs) this yields O(√n + t) queries. For higher degree graphs they obtain a bound of O( n^{t}·log Δ·n + t·Δ^{2}·√n ), and in two rounds they achieve O(n/ log n) queries for bounded‑degree graphs. These results improve upon the generic separator‑based bound when the separation number is large (i.e., the graph is highly decomposable).

Related Work and Context.
The paper situates its contributions within a rich literature. Aldous (1983) introduced the steepest‑descent‑with‑warm‑start heuristic for the Boolean hypercube, and subsequent works (Aaronson 2006, Zhang 2009) established tight Θ(√n·2^{n/2}) bounds. Santha and Szegedy (2004) linked the separation number to quantum lower bounds; this work translates those ideas to classical randomized lower bounds. Brânzei and Li (2022) previously studied the same round‑restricted model but only for d‑dimensional grids; the current paper extends the analysis to arbitrary graphs. The authors also discuss connections to adaptive complexity of finding approximate stationary points in continuous optimization, noting that the discrete local‑search model captures many of the same parallelization challenges.

Implications and Applications.
The results have direct relevance to large‑scale machine learning tasks where each function evaluation is expensive (e.g., training a deep network to convergence). By limiting the number of synchronization rounds, one can fully exploit parallel hardware while still guaranteeing a provable bound on the total number of evaluations. The paper illustrates this with three concrete settings: non‑convex regularized linear regression (grid‑like search space), hyperparameter optimization for deep nets (high‑dimensional, non‑Euclidean search space), and robust matrix estimation where the search graph contains dense rank‑1 update edges (high expansion). In each case the separation number and degree dictate which algorithmic regime (separator‑based vs. warm‑start descent) is preferable.

Conclusion and Future Directions.
The authors provide a tight characterization (up to polylogarithmic factors) of the trade‑off between rounds and queries for local search on general graphs. They highlight open problems such as refining the dependence on other structural parameters (diameter, conductance), extending the lower bound techniques to quantum query models, and empirically validating the proposed algorithms on real‑world hyperparameter tuning workloads. Overall, the paper bridges a gap between theoretical query complexity and practical parallel optimization, offering both rigorous bounds and algorithmic guidance for practitioners.