Predictive-State Communication: Innovation Coding and Reconciliation under Delay

Shannon theory models communication as the reliable transfer of symbol sequences, with performance governed by capacity and rate-distortion limits. When both endpoints possess strong predictors – as in modern large language models and related generative priors – literal symbol transport is no longer the only operational regime. We propose predictive-state communication (PSC), in which the transmitter and receiver maintain an explicit shared predictive state, and the physical channel is used primarily to convey innovations, i.e., corrective information that reconciles the receiver’s provisional trajectory with the transmitter’s realized trajectory. This viewpoint replaces entropy-rate accounting by cross-entropy accounting under model mismatch, and it introduces feasibility constraints that depend jointly on capacity, delay, and perceptual continuity requirements; the resulting operating set is typically a bounded perception-capacity band rather than a one-sided threshold. We outline the protocol and architectural implications (state identifiers, anchors, bounded rollback, and patch-based updates) and provide a stylized illustrative example to visualize the induced feasibility region and its dependence on predictive quality.

💡 Research Summary

The paper introduces a novel communication framework called Predictive‑State Communication (PSC) that departs from the classic Shannon model of symbol‑by‑symbol transmission. In PSC, both the transmitter and the receiver are equipped with powerful predictive models—typically large language models (LLMs) or other learned generative priors. Because each side can anticipate the evolution of the source, the physical channel need not carry the full sequence of symbols; instead it carries only the innovations, i.e., the corrective information that bridges the gap between the receiver’s locally generated provisional trajectory and the transmitter’s realized trajectory.

The authors formalize the innovation load using cross‑entropy. Let (P(\cdot|H_t)) denote the true conditional distribution of the next token given the current committed context (H_t), and let (Q(\cdot|H_t)) be the receiver’s predictive model. The per‑step cross‑entropy is defined as
(h_t = -\sum_x P(x|H_t)\log_2 Q(x|H_t)).
Averaging over time yields (\bar h), the expected innovation rate per token. This quantity decomposes into the intrinsic source entropy (H(P)) plus a model‑mismatch penalty (D_{KL}(P|Q)). Thus, better predictors (smaller (\bar h)) directly reduce the required innovation bandwidth, while any mismatch inflates it.

If the application generates tokens at a rate (r) (tokens / second), the baseline innovation throughput is (R_{\text{innov}} \approx r\bar h) (bits / second). The physical link offers a capacity (C) (bits / second); after accounting for protocol overhead (headers, authentication, redundancy) a usable fraction (\eta) remains, giving an effective innovation capacity (C_{\text{innov}} = \eta C). A necessary feasibility condition is therefore
(r\bar h \le C_{\text{innov}}).
Unlike a classic coding theorem, this inequality is an accounting rule that makes explicit how predictive quality trades off against raw bandwidth.

A central contribution of the paper is the integration of delay and perceptual continuity into the feasibility analysis. During a one‑way delay (L) seconds, the receiver must act on its provisional output. Two application‑specific cost functions are introduced:

• Speculation cost (D_{\text{spec}}(r, L, \dots)) – the penalty incurred when provisional output later requires correction (e.g., user confusion, control instability).
• Starvation cost (D_{\text{starve}}(r, L, \dots)) – the penalty for producing output too slowly because the receiver is waiting for corrections (e.g., perceptual stutter).

These costs translate into an upper bound (r \le r_{\max}(L, \text{tolerance})) (to keep speculation acceptable) and a lower bound (r \ge r_{\min}(L, \text{tolerance})) (to avoid starvation). Together with the innovation‑capacity ceiling, the feasible operating region becomes a band rather than a single threshold: \