UReason: Benchmarking the Reasoning Paradox in Unified Multimodal Models

To elicit capabilities for addressing complex and implicit visual requirements, recent unified multimodal models increasingly adopt chain-of-thought reasoning to guide image generation. However, the actual effect of reasoning on visual synthesis remains unclear. We present UReason, a diagnostic benchmark for reasoning-driven image generation that evaluates whether reasoning can be faithfully executed in pixels. UReason contains 2,000 instances across five task families: Code, Arithmetic, Spatial, Attribute, and Text reasoning. To isolate the role of reasoning traces, we introduce an evaluation framework comparing direct generation, reasoning-guided generation, and de-contextualized generation which conditions only on the refined prompt. Across eight open-source unified models, we observe a consistent Reasoning Paradox: Reasoning traces generally improve performance over direct generation, yet retaining intermediate thoughts as conditioning context often hinders visual synthesis, and conditioning only on the refined prompt yields substantial gains. Our analysis suggests that the bottleneck lies in contextual interference rather than insufficient reasoning capacity. UReason provides a principled testbed for studying reasoning in unified models and motivates future methods that effectively integrate reasoning for visual generation while mitigating interference.

💡 Research Summary

The paper introduces UReason, a diagnostic benchmark designed to evaluate how well unified multimodal models (UMMs) can translate multi‑step reasoning into faithful image generation. Unlike conventional text‑to‑image datasets that focus on descriptive prompts, UReason consists of 2,000 carefully curated instances across five reasoning‑centric families: Code, Arithmetic, Spatial, Attribute, and Text. Each instance requires the model to infer an implicit visual target through a chain of logical steps and then render that target in pixels.

Data creation follows a two‑stage pipeline. Human experts first construct 500 seed examples covering 30 fine‑grained sub‑categories, ensuring that every prompt demands genuine reasoning. These seeds are then expanded to the full set using a Gemini‑3‑Pro powered LLM‑assisted augmentation loop, with multiple rounds of human verification to maintain logical consistency. The final benchmark is split into a 1,500‑item test set and a 500‑item “testmini” for rapid prototyping.

UReason’s core contribution is an evaluation framework that isolates the effect of reasoning traces. Three settings are compared for each model: (1) Direct Generation – the model generates an image directly from the original prompt; (2) Reasoning‑Guided Generation – the model first produces a full reasoning trace (intermediate thoughts Rt plus a refined prompt Rp) and then conditions the image on the entire trace; (3) De‑contextualized Generation – after producing the trace, the intermediate thoughts are discarded and only the refined prompt Rp is used as conditioning. This design separates the benefit of better reasoning from possible interference caused by the textual trace itself.

Eight open‑source UMMs (including Bagel, UniCoT‑v2, SRUM, Bagel‑Zebra‑CoT, ThinkMorph, UniMoE2, T2I‑R1, and UniMoE2) are evaluated under the three settings. Results show a consistent “Reasoning Paradox”: while Reasoning‑Guided Generation typically outperforms Direct Generation, De‑contextualized Generation often yields the highest accuracy, surpassing Reasoning‑Guided by 5–20 percentage points. The paradox is attributed to “contextual interference”: reasoning traces contain many extraneous tokens—intermediate results, exploratory attempts, or redundant explanations—that act as noise for the image decoder. Because unified architectures fuse text and visual tokens in a single stream, these noisy tokens dilute the signal of the final visual constraints, leading to degraded image quality.

Further analysis quantifies trace length versus performance drop, visualizes attention weights showing that intermediate tokens receive disproportionate focus, and measures the proportion of “noise” tokens across models. The findings suggest that the bottleneck is not insufficient reasoning capability—many models generate correct reasoning—but rather the inability to filter or compress the trace into a compact, visually relevant conditioning.

The authors propose future directions: (i) develop trace‑summarization modules that extract only the essential visual specifications; (ii) introduce gating or attention‑modulation mechanisms that down‑weight non‑visual tokens during conditioning; (iii) train models to explicitly refine prompts after reasoning, encouraging a “think‑then‑summarize‑then‑draw” pipeline.

In summary, UReason provides the first systematic benchmark for probing the interplay between chain‑of‑thought reasoning and image synthesis in unified multimodal models. By exposing the Reasoning Paradox, it highlights the need for architectural and training innovations that can harness the benefits of reasoning while mitigating its textual interference, paving the way for more reliable and controllable multimodal generation systems.