Information Cost Tradeoffs for Augmented Index and Streaming Language Recognition

This paper makes three main contributions to the theory of communication complexity and stream computation. First, we present new bounds on the information complexity of AUGMENTED-INDEX. In contrast to analogous results for INDEX by Jain, Radhakrishnan and Sen [J. ACM, 2009], we have to overcome the significant technical challenge that protocols for AUGMENTED-INDEX may violate the “rectangle property” due to the inherent input sharing. Second, we use these bounds to resolve an open problem of Magniez, Mathieu and Nayak [STOC, 2010] that asked about the multi-pass complexity of recognizing Dyck languages. This results in a natural separation between the standard multi-pass model and the multi-pass model that permits reverse passes. Third, we present the first passive memory checkers that verify the interaction transcripts of priority queues, stacks, and double-ended queues. We obtain tight upper and lower bounds for these problems, thereby addressing an important sub-class of the memory checking framework of Blum et al. [Algorithmica, 1994].

💡 Research Summary

The paper makes three substantial contributions at the intersection of communication complexity, streaming algorithms, and memory checking.
First, it establishes new information‑complexity lower bounds for the Augmented‑Index (AI) problem, a variant of the classic INDEX where Bob additionally knows the prefix of Alice’s input and a check bit c. Because Alice’s and Bob’s inputs now overlap, the usual rectangle property of communication protocols breaks down, preventing a direct application of existing information‑cost techniques. The authors overcome this obstacle by proving a “Fat Transcript Lemma” (Lemma 2.6), which identifies transcripts that are both sufficiently probable and have low conditional entropy. Using this lemma they show (Theorem 2.3) that any randomized protocol with error at most 1/ log n under the easy distribution μ₀ (where xₖ = c) must either leak Ω(n) bits of information from Alice or Ω(1) bits from Bob. This trade‑off is stronger than the earlier privacy‑trade‑off of Jain, Radhakrishnan and Sen, especially because it forces a non‑trivial information cost on Bob even when his communication is tiny.

Second, the authors translate this information‑cost trade‑off into streaming lower bounds. They define a problem MULTI‑AI consisting of many independent copies of AI and prove a direct‑sum theorem: the total communication cost scales linearly with the number of copies. By reducing MULTI‑AI to the language recognition problem for Dyck(2) (properly nested parentheses of two types), they obtain an Ω(√N) space lower bound for any multi‑pass streaming algorithm that is restricted to forward passes only. Moreover, they show that allowing reverse passes collapses the bound to O(log² N) space, thereby exhibiting a clean separation between the standard multi‑pass model and a model that permits reverse passes. This is the first natural streaming problem known to be exponentially easier when reverse passes are allowed.

Third, the paper applies the same techniques to the memory‑checking framework introduced by Blum et al. The authors consider passive checkers that must verify a transcript of operations on a priority queue, a stack, or a double‑ended queue without modifying the data structure. They define languages PQ, STACK, and DEQUE consisting of all valid operation sequences that start and end with an empty structure. Using the AI lower bound and the direct‑sum argument, they prove Ω(√N) space lower bounds for these languages even when multiple passes over the transcript are allowed. On the algorithmic side they present matching upper bounds: each language can be recognized in ˜O(√N) space with a single pass, by cleverly fingerprinting the state after a virtual reordering of inserts and deletes. They also discuss timestamp‑augmented variants (PQ‑TS, etc.) and show that the extra timestamps dramatically reduce the space requirement (e.g., PQ‑TS can be checked in O(log N) space), highlighting the impact of auxiliary information.

Overall, the paper demonstrates how a refined information‑complexity analysis of a simple communication problem can be leveraged to obtain strong lower bounds for streaming language recognition and passive memory checking. The “Fat Transcript” technique and the resulting trade‑off theorem constitute a powerful new tool for proving lower bounds in settings where input sharing destroys the usual rectangle structure. The work opens several avenues for future research, including tightening the AI trade‑off to the conjectured form a ≥ n/2·e^{O(b)}, extending the direct‑sum approach to other language families, and building practical passive checkers based on the presented algorithms.