Universal MMSE Filtering With Logarithmic Adaptive Regret

We consider the problem of online estimation of a real-valued signal corrupted by oblivious zero-mean noise using linear estimators. The estimator is required to iteratively predict the underlying signal based on the current and several last noisy observations, and its performance is measured by the mean-square-error. We describe and analyze an algorithm for this task which: 1. Achieves logarithmic adaptive regret against the best linear filter in hindsight. This bound is assyptotically tight, and resolves the question of Moon and Weissman [1]. 2. Runs in linear time in terms of the number of filter coefficients. Previous constructions required at least quadratic time.

💡 Research Summary

The paper tackles the classic problem of online estimation of a real‑valued signal that is corrupted by additive, zero‑mean, time‑independent noise, using a finite‑length linear filter. The signal sequence (x_t) is allowed to be arbitrary (even adversarial) but bounded, while the noise (n_t) is bounded, zero‑mean, and has known variance (\sigma^2). At each time step the algorithm observes only the noisy measurement (y_t = x_t + n_t) and must predict the current underlying value (x_t) by forming a linear combination of the most recent (d) noisy observations, i.e., ( \hat{x}t = w_t^\top Y_t) where (Y_t = (y_t, y{t-1}, \dots, y_{t-d+1})) and (w_t\in\mathbb{R}^d) is the filter vector chosen by the algorithm.

The performance metric is the cumulative mean‑square error (MSE) ( \sum_{t=1}^T (x_t - w_t^\top Y_t)^2). The benchmark is the best offline linear filter (or a piecewise‑constant sequence of filters) that knows the entire signal and noise realizations in advance. The goal is to bound the regret, i.e., the excess loss relative to the benchmark, and more stringently the adaptive regret, which is the worst‑case regret over any contiguous interval (