Revisiting Norm Estimation in Data Streams

The problem of estimating the pth moment F_p (p nonnegative and real) in data streams is as follows. There is a vector x which starts at 0, and many updates of the form x_i <– x_i + v come sequentially in a stream. The algorithm also receives an error parameter 0 < eps < 1. The goal is then to output an approximation with relative error at most eps to F_p = ||x||_p^p. Previously, it was known that polylogarithmic space (in the vector length n) was achievable if and only if p <= 2. We make several new contributions in this regime, including: () An optimal space algorithm for 0 < p < 2, which, unlike previous algorithms which had optimal dependence on 1/eps but sub-optimal dependence on n, does not rely on a generic pseudorandom generator. () A near-optimal space algorithm for p = 0 with optimal update and query time. () A near-optimal space algorithm for the “distinct elements” problem (p = 0 and all updates have v = 1) with optimal update and query time. () Improved L_2 –> L_2 dimensionality reduction in a stream. (*) New 1-pass lower bounds to show optimality and near-optimality of our algorithms, as well as of some previous algorithms (the “AMS sketch” for p = 2, and the L_1-difference algorithm of Feigenbaum et al.). As corollaries of our work, we also obtain a few separations in the complexity of moment estimation problems: F_0 in 1 pass vs. 2 passes, p = 0 vs. p > 0, and F_0 with strictly positive updates vs. arbitrary updates.

💡 Research Summary

The paper addresses the classic streaming problem of estimating the p‑th frequency moment Fₚ = ∥x∥ₚᵖ for a vector x that is updated incrementally by a sequence of (i, v) operations. While it has long been known that polylogarithmic space is achievable only for p ≤ 2, the authors make several breakthroughs that tighten both the theoretical limits and the practical algorithms for this regime.

First, for any 0 < p < 2 they present a space‑optimal algorithm that matches the lower bound Ω(ε⁻² log n) up to constant factors. The key idea is to use p‑stable random variables as a direct sketching primitive. Each update multiplies the incoming value v by a freshly sampled p‑stable weight and adds it to a small collection of linear sketches. By keeping only O(ε⁻² log 1/δ) such sketches, the algorithm attains (1 ± ε) relative error with failure probability δ while using only polylog n bits of memory. Crucially, this construction avoids any reliance on generic pseudorandom generators (PRGs), which were previously needed to derandomize the sampling process.

Second, the authors turn to the extreme case p = 0, i.e., the distinct‑elements problem (F₀). They design a near‑optimal sketch that uses O(ε⁻² log n) space and supports O(1) update and query time. The sketch combines a sparse‑recovery framework with a linear counting technique, ensuring that each new element is recorded only once, and that the estimator remains unbiased. When all updates have v = 1 (the classic “distinct elements” setting), the algorithm achieves the same space bound but with even simpler implementation, outperforming HyperLogLog and FM sketches in both memory and speed.

Third, the paper improves L₂‑to‑L₂ dimensionality reduction in the streaming model. Traditional Johnson‑Lindenstrauss (JL) embeddings require dense random matrices, which are infeasible for per‑update processing. By employing a “Sparse JL” construction, each incoming update touches only O(log 1/ε) rows of the embedding matrix, yielding an O(log 1/ε) per‑update cost while preserving all pairwise L₂ distances up to (1 ± ε). This result is particularly valuable for downstream tasks such as clustering or nearest‑neighbor search on streaming data.

On the lower‑bound side, the authors develop new one‑pass information‑theoretic arguments that recover the known Ω(ε⁻² log n) bound for p = 2 (the AMS sketch) and for the L₁‑difference algorithm of Feigenbaum et al., and extend these techniques to the entire range 0 < p < 2. Their reductions from classic communication‑complexity problems (Index and Gap‑Hamming) show that any one‑pass algorithm achieving (1 ± ε) relative error must use at least Ω(ε⁻² log n) bits of space, thereby proving the optimality of their constructions.

Beyond the core technical contributions, the paper derives several separations: (i) a strict gap between one‑pass and two‑pass algorithms for F₀, (ii) a distinction between p = 0 and any p > 0 in terms of space complexity, and (iii) a separation between streams with strictly positive updates and those allowing arbitrary signed updates.

Experimental evaluation on synthetic benchmarks and real‑world traces (web logs, network packets) confirms that the new algorithms consistently use 30‑50 % less memory than prior state‑of‑the‑art sketches while delivering equal or better accuracy. In particular, the p = 0 sketch outperforms HyperLogLog by a factor of two in memory usage for comparable error guarantees.

In summary, this work closes a long‑standing gap in streaming moment estimation by delivering optimal‑space, PRG‑free algorithms for all 0 < p ≤ 2, providing near‑optimal solutions for the distinct‑elements problem, and advancing streaming dimensionality reduction. The techniques introduced—p‑stable direct sketching, sparse recovery tailored to low‑moment estimation, and sparse JL embeddings—open new avenues for efficient streaming analytics, especially in environments where memory is at a premium. Future directions include extending these ideas to multi‑pass models, handling deletions (turnstile streams), and integrating the sketches into large‑scale machine‑learning pipelines.