Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles

This paper explores the performance of a random Gaussian smoothing zeroth-order (ZO) scheme for minimising quasar-convex (QC) and strongly quasar-convex (SQC) functions in both unconstrained and constrained settings. For the unconstrained problem, we establish the ZO algorithm’s convergence to a global minimum along with its complexity when applied to both QC and SQC functions. For the constrained problem, we introduce the new notion of proximal-quasar-convexity and prove analogous results to the unconstrained case. Specifically, we derive complexity bounds and prove convergence of the algorithm to a neighbourhood of a global minimum whose size can be controlled under a variance reduction scheme. Beyond the theoretical guarantees, we demonstrate the practical implications of our results on several machine learning problems where quasar-convexity naturally arises, including linear dynamical system identification and generalised linear models.

💡 Research Summary

This paper investigates the use of a random Gaussian‑smoothing zeroth‑order (ZO) oracle for minimizing a class of non‑convex functions that satisfy quasar‑convexity (QC) or strong quasar‑convexity (SQC). The authors focus on both unconstrained and constrained optimization settings, introducing a novel notion of proximal‑quasar‑convexity for the latter.

Problem Setting and Motivation
The target problem is (\min_{x\in X} f(x)) where (f:\mathbb{R}^n\to\mathbb{R}) is continuously differentiable, possibly non‑convex, and bounded below. In many modern machine‑learning and control applications (e.g., linear dynamical system identification, generalized linear models, certain deep‑network training regimes) the objective is not convex but enjoys the weaker QC property, which still guarantees that any stationary point is a global minimum. Existing ZO methods have been analyzed only for convex, strongly convex, or generic non‑convex functions (typically yielding stationarity guarantees). No prior work exploits QC structure with a ZO scheme.

Gaussian Smoothing and the Random Oracle
The authors adopt the Gaussian smoothing technique of Nesterov & Spokoiny (2017). For a smoothing parameter (\mu>0) they define the smoothed function
(f_\mu(x)=\mathbb{E}_{u\sim\mathcal{N}(0,I)}