DiffuTester: Accelerating Unit Test Generation for Diffusion LLMs via Mining Structural Pattern

Diffusion large language models (dLLMs) enable parallel generation and are promising for unit test generation (UTG), where efficient and large-scale automated testing is essential in software development. Despite this advantage, their application to UTG is still constrained by a clear trade-off between efficiency and test quality, since increasing the number of tokens generated in each step often causes a sharp decline in the quality of test cases. To overcome this limitation, we present DiffuTester, an acceleration framework specifically tailored for dLLMs in UTG. The motivation of DiffuTester is that unit tests targeting the same focal method often share structural patterns. DiffuTester employs a novel structural pattern based decoding approach, which dynamically identifies structural patterns across unit tests through their abstract syntax trees and additionally decodes the corresponding tokens, thereby achieving acceleration without compromising the quality of the output. To enable comprehensive evaluation, we extend the original TestEval benchmark to three programming languages. Extensive experiments on three benchmarks with two representative models show that DiffuTester delivers significant acceleration while preserving test coverage. Moreover, DiffuTester generalizes well across different dLLMs and programming languages, providing a practical and scalable solution for efficient UTG in software development. Code and data are publicly available at https://github.com/TsinghuaISE/DiffuTester.

💡 Research Summary

DiffuTester addresses a critical bottleneck in using diffusion‑based large language models (dLLMs) for unit‑test generation (UTG). While dLLMs theoretically allow multi‑token prediction in a single forward pass, practical UTG suffers from a sharp quality drop when the number of tokens generated per step is increased. The authors observe that test cases targeting the same focal method share a high degree of syntactic structure, which can be captured by their abstract syntax trees (ASTs). Leveraging this observation, DiffuTester introduces a structural‑pattern‑based decoding strategy that dynamically mines common AST sub‑structures across a batch of generated tests and forces the model to decode the corresponding tokens, thereby increasing the number of tokens generated per step without sacrificing syntactic correctness.

The workflow proceeds as follows. For a given focal method, a batch of n (typically 3–7) test cases is generated in parallel. At each diffusion step t, the model predicts tokens for all masked positions, yielding an intermediate output Yₜ,ₖ for each instance k. Standard confidence‑based decoding selects the top‑k tokens as usual. DiffuTester then parses the partially generated code line‑by‑line into ASTs, identifies non‑literal nodes that appear in two or more ASTs, and treats the union of these nodes as a structural pattern. Tokens that map to the merged pattern are decoded in addition to the confidence‑selected tokens, provided their confidence exceeds a low threshold τ (set to 0.02). Literal nodes (constants, numbers, strings) are deliberately excluded from merging to preserve input diversity, which is essential for achieving high test coverage.

Because AST construction is cheap but still incurs overhead if performed at every diffusion step, the authors adopt an intermittent schedule: structural‑pattern decoding is applied once every two steps. This reduces the extra cost while still capturing the majority of shared structure, as successive steps typically modify only a few tokens.

Experiments are conducted on three benchmarks: the original TestEval‑Python (210 LeetCode‑style problems) and two newly constructed counterparts in C++ and Java that preserve the same problem set. Two representative dLLMs are evaluated: DiffuCoder‑7B‑CPG and Dream‑7B. Four metrics are reported: average FLOPs per batch, decoding time (seconds), throughput (tokens per second), and line‑coverage (ratio of covered lines to total lines). The baseline is the same model run without DiffuTester.

Results show that DiffuTester consistently reduces the time required to reach a given coverage level by 2×–3× across all languages and models. For example, with batch size 3 on TestEval‑C++, DiffuCoder’s decoding time drops from 14.4 s to 6.0 s while throughput rises from 9.7 TPS to 23.8 TPS. FLOPs are reduced by an average of 1.6×, and line‑coverage remains essentially unchanged (sometimes marginally higher). Ablation studies confirm that (a) removing the structural‑pattern step eliminates the speedup, (b) merging literal nodes harms diversity and coverage, and (c) raising τ improves quality but diminishes acceleration, highlighting τ as a key trade‑off knob.

The paper’s contributions are fourfold: (1) a novel, training‑free acceleration technique tailored to UTG, (2) an AST‑driven method for extracting and exploiting shared structural patterns during diffusion decoding, (3) extensive cross‑language and cross‑model validation demonstrating broad generality, and (4) empirical evidence that quality (syntactic correctness and coverage) is preserved despite a substantial increase in token generation per step.

Limitations include dependence on language‑specific parsers for AST construction, potential reduced effectiveness for highly heterogeneous test cases where common structure is scarce, and a fixed intermittent schedule that may not be optimal for all models. Future work could explore dynamic scheduling, multi‑language common AST schemas, clustering‑based pattern discovery, and extending the approach to other code‑generation tasks such as code completion or refactoring.

In summary, DiffuTester successfully bridges the gap between the theoretical parallelism of diffusion LLMs and the practical demands of high‑quality, large‑scale unit‑test generation, offering a practical, scalable solution that can accelerate software testing pipelines without compromising test effectiveness.