A Class of Accelerated Fixed-Point-Based Methods with Delayed Inexact Oracles and Its Applications

In this paper, we develop a novel accelerated fixed-point-based framework using delayed inexact oracles to approximate a fixed point of a nonexpansive operator (or equivalently, a root of a co-coercive operator), a central problem in scientific computing. Our approach leverages both Nesterov’s acceleration technique and the Krasnosel’skii-Mann (KM) iteration, while accounting for delayed inexact oracles, a key mechanism in asynchronous algorithms. We also introduce a unified approximate error condition for delayed inexact oracles, which can cover various practical scenarios. Under mild conditions and appropriate parameter updates, we establish both $\mathcal{O}(1/k^2)$ non-asymptotic and $o(1/k^2)$ asymptotic convergence rates in expectation for the squared norm of residual. Our rate significantly improves the $\mathcal{O}(1/k)$ rates in classical KM-type methods, including their asynchronous variants. We also establish $o(1/k^2)$ almost sure convergence rates and the almost sure convergence of iterates to a solution of the problem. Within our framework, we instantiate three settings for the underlying operator: (i) a deterministic universal delayed oracle; (ii) a stochastic delayed oracle; and (iii) a finite-sum structure with asynchronous updates. For each case, we instantiate our framework to obtain a concrete algorithmic variant for which our convergence results still apply, and whose iteration complexity depends linearly on the maximum delay. Finally, we verify our algorithms and theoretical results through two numerical examples on both matrix game and shallow neural network training problems.

💡 Research Summary

This paper introduces a novel Accelerated Fixed-Point-based framework (AFP) designed to solve a fundamental problem in scientific computing: finding a fixed point of a nonexpansive operator (or equivalently, a root of a co-coercive operator). The key challenge addressed is the practical reality of parallel and distributed computing, where obtaining perfect, up-to-date evaluations of the operator (the “oracle”) is often prohibitively expensive or impossible due to system heterogeneity, leading to delays and inexactness.

The authors’ main contribution is a unified algorithmic framework that ingeniously combines Nesterov’s acceleration technique with the classical Krasnosel’skii-Mann (KM) fixed-point iteration, while explicitly accounting for “delayed inexact oracles.” Instead of requiring the exact value of the operator G at the current iterate y_k, the algorithm uses an approximation eG_k. This approximation embodies two types of errors: computational inaccuracy and staleness due to using information from past iterations (modeling asynchronous delays). A central, abstract “unified approximate error condition” is formulated, which bounds the error e_k = eG_k - Gy_k in terms of past operator evaluations and algorithm parameters. This condition is general enough to cover many practical scenarios.

Under this general error condition and with appropriately chosen parameter updates, the paper establishes strong convergence guarantees for the AFP framework. The primary result is an O(1/k²) non-asymptotic convergence rate in expectation for the squared norm of the residual, `E