Explicit and Non-asymptotic Query Complexities of Rank-Based Zeroth-order Algorithms on Smooth Functions

Rank-based zeroth-order (ZO) optimization – which relies only on the ordering of function evaluations – offers strong robustness to noise and monotone transformations, and underlies many successful algorithms such as CMA-ES, natural evolution strategies, and rank-based genetic algorithms. Despite its widespread use, the theoretical understanding of rank-based ZO methods remains limited: existing analyses provide only asymptotic insights and do not yield explicit convergence rates for algorithms selecting the top-$k$ directions. This work closes this gap by analyzing a simple rank-based ZO algorithm and establishing the first \emph{explicit}, and \emph{non-asymptotic} query complexities. For a $d$-dimension problem, if the function is $L$-smooth and $μ$-strongly convex, the algorithm achieves $\widetilde{\mathcal O}!\left(\frac{dL}μ\log!\frac{dL}{μδ}\log!\frac{1}{\varepsilon}\right)$ to find an $\varepsilon$-suboptimal solution, and for smooth nonconvex objectives it reaches $\mathcal O!\left(\frac{dL}{\varepsilon}\log!\frac{1}{\varepsilon}\right)$. Notation $\cO(\cdot)$ hides constant terms and $\widetilde{\mathcal O}(\cdot)$ hides extra $\log\log\frac{1}{\varepsilon}$ term. These query complexities hold with a probability at least $1-δ$ with $0<δ<1$. The analysis in this paper is novel and avoids classical drift and information-geometric techniques. Our analysis offers new insight into why rank-based heuristics lead to efficient ZO optimization.

💡 Research Summary

The paper addresses a long‑standing gap in the theory of rank‑based zeroth‑order (ZO) optimization, which has been widely used in practice (e.g., CMA‑ES, natural evolution strategies, rank‑based genetic algorithms) but lacks explicit, non‑asymptotic convergence guarantees. The authors propose a simple yet representative rank‑based ZO algorithm and derive the first explicit query‑complexity bounds that hold with high probability (at least (1-\delta)).

Algorithmic framework.
At each iteration (t), the method draws (N) independent Gaussian directions (u_i\sim\mathcal N(0,I_d)) and evaluates the perturbed points (x_t+\alpha u_i) using a rank oracle that returns only the ordering of the function values. The top (N/4) points (lowest function values) are assigned positive weights (w_k^+>0) (normalized so that (\sum_{k=1}^{N/4} w_k^+=1)), while the bottom (N/4) points receive negative weights (w_k^-<0) (normalized to (\sum w_k^-=-1)). The search direction is the weighted sum \