Forward estimation for ergodic time series

The forward estimation problem for stationary and ergodic time series ${X_n}{n=0}^{\infty}$ taking values from a finite alphabet ${\cal X}$ is to estimate the probability that $X{n+1}=x$ based on the observations $X_i$, $0\le i\le n$ without prior knowledge of the distribution of the process ${X_n}$. We present a simple procedure $g_n$ which is evaluated on the data segment $(X_0,…,X_n)$ and for which, ${\rm error}(n) = |g_{n}(x)-P(X_{n+1}=x |X_0,…,X_n)|\to 0$ almost surely for a subclass of all stationary and ergodic time series, while for the full class the Cesaro average of the error tends to zero almost surely and moreover, the error tends to zero in probability.

💡 Research Summary

The paper addresses the forward‑estimation problem for stationary and ergodic time series that take values in a finite alphabet. Given only the past observations (X_0,\dots ,X_n), one wishes to estimate the conditional probability that the next symbol equals a particular symbol (x) without any prior knowledge of the underlying distribution. The authors propose a remarkably simple estimator, denoted (g_n), which is computed directly from the data segment ((X_0,\dots ,X_n)).

Algorithmic construction.
At time (n) the algorithm extracts the longest suffix (context) (c_n) of the observed string that has appeared previously in the data. It then counts how many times this context occurred and, among those occurrences, how often each possible next symbol followed. The estimate for symbol (x) is the empirical frequency of (x) after the chosen context: \