Beyond Scattered Acceptance: Fast and Coherent Inference for DLMs via Longest Stable Prefixes

Diffusion Language Models (DLMs) promise highly parallel text generation, yet their practical inference speed is often bottlenecked by suboptimal decoding schedulers. Standard approaches rely on ‘scattered acceptance’-committing high confidence tokens at disjoint positions throughout the sequence. This approach inadvertently fractures the Key-Value (KV) cache, destroys memory locality, and forces the model into costly, repeated repairs across unstable token boundaries. To resolve this, we present the Longest Stable Prefix (LSP) scheduler, a training-free and model-agnostic inference paradigm based on monolithic prefix absorption. In each denoising step, LSP evaluates token stability via a single forward pass, dynamically identifies a contiguous left-aligned block of stable predictions, and snaps its boundary to natural linguistic or structural delimiters before an atomic commitment. This prefix-first topology yields dual benefits: systemically, it converts fragmented KV cache updates into efficient, contiguous appends; algorithmically, it preserves bidirectional lookahead over a geometrically shrinking active suffix, drastically reducing token flip rates and denoiser calls. Extensive evaluations on LLaDA-8B and Dream-7B demonstrate that LSP accelerates inference by up to 3.4x across rigorous benchmarks including mathematical reasoning, code generation, multilingual (CJK) tasks, and creative writing while matching or slightly improving output quality. By fundamentally restructuring the commitment topology, LSP bridges the gap between the theoretical parallelism of DLMs and practical hardware efficiency.

💡 Research Summary

The paper addresses a critical performance bottleneck in Diffusion Language Models (DLMs), namely the “scattered acceptance” decoding strategy that commits high‑confidence tokens at disjoint positions throughout the generation process. While DLMs theoretically enable highly parallel, bidirectional generation, scattered acceptance fragments the sequence into alternating frozen and mutable regions. This fragmentation creates two major inefficiencies: algorithmically, it forces repeated local repairs at many unstable boundaries, slowing convergence; system‑wise, it shatters the transformer’s key‑value (KV) cache into many small, non‑contiguous pieces, destroying memory locality and keeping attention computations expensive over a long active suffix for many denoising steps.

To overcome these issues, the authors propose the Longest Stable Prefix (LSP) scheduler, a training‑free, model‑agnostic inference paradigm that commits the longest contiguous, stable left‑aligned block of tokens in a single atomic step. LSP operates in three lightweight stages using only a single forward pass per denoising iteration:

1. Stability Assessment – The model produces logits for every position in the current active suffix. A stability diagnostic is computed as the logit margin δ_i = top‑1 logit − top‑2 logit. Large margins indicate high confidence and low likelihood of future change.

2. Adaptive Block Sizing – Rather than using a fixed margin threshold, LSP dynamically selects a threshold τ_k such that the length L′(τ_k) of the longest prefix whose margins all exceed τ_k falls within a user‑specified fractional range