Convergence Analysis of Greedy Algorithms with Adaptive Relaxation in Hilbert Spaces

The Power-Relaxed Greedy Algorithm (PRGA) was introduced as a generalization of the so called Relaxed Greedy Algorithm, introduced by DeVore and Temlyakov, by replacing the relaxation parameter $1/m$ with $1/m^α$, with the aim of improving convergence rates. While the case $α\le 1$ is well understood, the behavior of the algorithm for $α>1$ remained an open problem. In this work, we answer this question and, moreover, we introduce a relaxed greedy algorithm with an optimal step size chosen by exact line search at each iteration.

💡 Research Summary

The paper investigates greedy approximation algorithms in Hilbert spaces equipped with redundant dictionaries, focusing on the convergence behavior of two variants: the Power‑Relaxed Greedy Algorithm (PRGA) and a newly proposed Convex‑Relaxed Greedy Algorithm (CRGA).

Background. In a Hilbert space (H) with a symmetric dictionary (\mathcal D) of unit‑norm atoms, the Pure Greedy Algorithm (PGA) iteratively selects the atom that maximizes the inner product with the current residual and adds it directly to the approximation. While simple, PGA may converge slowly for highly redundant dictionaries. DeVore and Temlyakov introduced the Relaxed Greedy Algorithm (RGA), which mixes the previous approximation with the newly selected atom using a convex weight (1/m). For functions belonging to the atomic class (A_1(\mathcal D)) (i.e., those that can be written as a finite linear combination of atoms with (\ell^1) coefficients bounded by 1), RGA enjoys the optimal bound (|f-G_m|\le 2\sqrt{m}).

Power‑Relaxed Greedy Algorithm (PRGA). PRGA generalizes RGA by replacing the weight (1/m) with (1/m^{\alpha}) for a positive exponent (\alpha). When (\alpha\le 1) the known convergence estimate (|f-T_m|\le 4,m^{\alpha/2}) holds, reproducing the RGA result at (\alpha=1). The open question, raised in earlier work, was whether any (\alpha>1) could still guarantee a decay of the error, perhaps with a different exponent.

Negative Result for (\alpha>1). The authors settle this question definitively. Lemma 3.1 shows that the infinite product
\