dVoting: Fast Voting for dLLMs

Diffusion Large Language Models (dLLMs) represent a new paradigm beyond autoregressive modeling, offering competitive performance while naturally enabling a flexible decoding process. Specifically, dLLMs can generate tokens at arbitrary positions in parallel, endowing them with significant potential for parallel test-time scaling, which was previously constrained by severe inefficiency in autoregressive modeling. In this work, we introduce dVoting, a fast voting technique that boosts reasoning capability without training, with only an acceptable extra computational overhead. dVoting is motivated by the observation that, across multiple samples for the same prompt, token predictions remain largely consistent, whereas performance is determined by a small subset of tokens exhibiting cross-sample variability. Leveraging the arbitrary-position generation capability of dLLMs, dVoting performs iterative refinement by sampling, identifying uncertain tokens via consistency analysis, regenerating them through voting, and repeating this process until convergence. Extensive evaluations demonstrate that dVoting consistently improves performance across various benchmarks. It achieves gains of 6.22%-7.66% on GSM8K, 4.40%-7.20% on MATH500, 3.16%-14.84% on ARC-C, and 4.83%-5.74% on MMLU. Our code is available at https://github.com/fscdc/dVoting

💡 Research Summary

The paper introduces dVoting, a training‑free test‑time scaling technique designed specifically for diffusion‑based large language models (dLLMs). Unlike conventional autoregressive LLMs that generate tokens strictly left‑to‑right, dLLMs employ a masked diffusion process: a sequence is corrupted by randomly masking tokens, and a denoising network predicts the original tokens at all masked positions simultaneously. This property enables parallel generation and, crucially, the ability to remask arbitrary token positions during inference.

The authors observe two empirical facts across multiple samples of the same prompt: (1) Token‑level consistency – when the baseline and voting predictions are correct, the majority of tokens receive high agreement (e.g., 4/5 or 5/5 votes); incorrect answers tend to have low agreement. (2) Redundant token generation – a substantial fraction of token positions are identical across different sampled outputs, quantified by a new metric NUPR@k (Non‑Unique Position Rate). For k = 2, NUPR values hover around 40 % on GSM8K, MATH500, and ARC‑C, indicating that many tokens are repeatedly generated without adding information.

Leveraging these observations, dVoting proceeds as follows:

1. Initial Sampling – Generate N candidate completions in parallel using the dLLM.
2. Consistency Analysis – For each token position, count how many samples share the same token (the “vote” count).
3. Selective Remasking – Tokens whose vote count falls below a predefined threshold are masked again; tokens with high agreement are preserved.
4. Iterative Refinement – The model re‑decodes only the masked positions, producing new candidates. Steps 2‑4 repeat until either all positions exceed the consistency threshold or a maximum number of iterations is reached.
5. Final Aggregation – The final answer is selected by majority voting over the completed sequences, or by token‑level majority if needed.

The method is evaluated on two popular dLLMs—LLaDA‑8B‑Instruct and Dream—across four reasoning benchmarks: GSM8K, MATH500, ARC‑C, and MMLU. Results show consistent improvements over single‑sample baselines and over standard test‑time scaling (e.g., best‑of‑N, majority voting). For example, on GSM8K Pass@1 accuracy rises from 70.58 % (baseline) to 78.24 % with dVoting, a gain of 7.66 %. Similar gains (4–8 %) are observed on MATH500 and ARC‑C. Importantly, dVoting achieves these gains with significantly fewer inference steps: the step count required for comparable performance is reduced by roughly 30‑50 % compared to naïve majority voting with the same number of samples.

The paper also compares dVoting against reinforcement‑learning‑enhanced methods (d1, wd1, IGPO) and other test‑time scaling approaches such as HEX and RFG. RL‑based methods need additional fine‑tuning data and a policy model, while HEX and RFG require extra computation (often thousands of steps). dVoting, by contrast, needs no extra training and only modest additional computation, making it far more practical for deployment.

Limitations discussed include: (i) the current focus on token‑level consistency without exploiting higher‑level semantic agreement; (ii) potential cost blow‑up when many tokens are uncertain, suggesting a need for adaptive thresholds; and (iii) evaluation limited to 8‑billion‑parameter models, leaving open the question of scalability to larger or multimodal dLLMs.

In summary, dVoting demonstrates that simple, consistency‑driven selective remasking can serve as an effective, training‑free alternative to RL for improving dLLM reasoning. By exploiting the intrinsic parallelism of diffusion models, it reduces redundant computation while delivering notable accuracy gains, thereby advancing the practicality of dLLMs for real‑world inference scenarios.