Revisiting Matrix Sketching in Linear Bandits: Achieving Sublinear Regret via Dyadic Block Sketching

Linear bandits have become a cornerstone of online learning and sequential decision-making, providing solid theoretical foundations for balancing exploration and exploitation. Within this domain, matrix sketching serves as a critical component for achieving computational efficiency, especially when confronting high-dimensional problem instances. The sketch-based approaches reduce per-round complexity from $Ω(d^2)$ to $O(dl)$, where $d$ is the dimension and $l<d$ is the sketch size. However, this computational efficiency comes with a fundamental pitfall: when the streaming matrix exhibits heavy spectral tails, such algorithms can incur vacuous \textit{linear regret}. In this paper, we revisit the regret bounds and algorithmic design for sketch-based linear bandits. Our analysis reveals that inappropriate sketch sizes can lead to substantial spectral error, severely undermining regret guarantees. To overcome this issue, we propose Dyadic Block Sketching, a novel multi-scale matrix sketching approach that dynamically adjusts the sketch size during the learning process. We apply this technique to linear bandits and demonstrate that the new algorithm achieves \textit{sublinear regret} bounds without requiring prior knowledge of the streaming matrix properties. It establishes a general framework for efficient sketch-based linear bandits, which can be integrated with any matrix sketching method that provides covariance guarantees. Comprehensive experimental evaluation demonstrates the superior utility-efficiency trade-off achieved by our approach.

💡 Research Summary

Linear bandits are a fundamental tool for sequential decision‑making under partial feedback, but their practical deployment in high‑dimensional settings is hampered by the Ω(d²) per‑round cost of maintaining and inverting the full covariance matrix. Recent work has therefore turned to matrix sketching techniques—most notably Frequent Directions (FD) and its robust variant (RFD)—to compress the data matrix from d dimensions to a sketch of size l (ℓ < d). By maintaining a sketch S(t) of the streaming matrix X(t), algorithms such as SOFUL replace the exact inverse A(t)⁻¹ with a Woodbury‑based expression that can be updated in O(d l + l²) time, dramatically reducing computational overhead.

However, this efficiency comes at a price. The regret bound for sketch‑based methods depends on the spectral error Δ_T = ‖X(T)ᵀX(T) − S(T)ᵀS(T)‖₂, which in turn is governed by the tail of the singular value spectrum of X(T). Theoretical analysis (Lemma 2, Theorem 3 of Kuzborskij et al., 2019) shows that a sub‑linear regret of order \tilde O(√T) holds only when Δ_T = o(T^{1/3}). When the data have heavy spectral tails, a fixed sketch size l fails to capture enough directions: Δ_T grows roughly linearly with T, and the regret bound collapses to a linear function of T. Observation 1 formalizes this phenomenon by linking the required sketch size to the geometry constant q (derived from the locally convex arm space). If l < d − T^{1/3 − q}, the algorithm inevitably suffers linear regret, even if l is close to d.

To overcome this fundamental limitation, the authors introduce Dyadic Block Sketching, a multi‑scale sketching framework that dynamically adjusts the sketch size as more data arrive. The key ideas are:

1. Dyadic Partitioning – The stream is divided into consecutive blocks. The first block uses a small sketch size l₀; each subsequent block doubles the sketch size (l₀, 2l₀, 4l₀, …). This mirrors classic dyadic structures in streaming algorithms.

2. Decomposability (Lemma 3) – If each block X_i has a sketch S_i satisfying ‖X_iᵀX_i − S_iᵀS_i‖₂ ≤ ε_i‖X_i‖_F, then the concatenated sketch S =