MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision

Accurately grounding regions of interest (ROIs) is critical for diagnosis and treatment planning in medical imaging. While multimodal large language models (MLLMs) combine visual perception with natural language, current medical-grounding pipelines still rely on supervised fine-tuning with explicit spatial hints, making them ill-equipped to handle the implicit queries common in clinical practice. This work makes three core contributions. We first define Unified Medical Reasoning Grounding (UMRG), a novel vision-language task that demands clinical reasoning and pixel-level grounding. Second, we release U-MRG-14K, a dataset of 14K samples featuring pixel-level masks alongside implicit clinical queries and reasoning traces, spanning 10 modalities, 15 super-categories, and 108 specific categories. Finally, we introduce MedReasoner, a modular framework that distinctly separates reasoning from segmentation: an MLLM reasoner is optimized with reinforcement learning, while a frozen segmentation expert converts spatial prompts into masks, with alignment achieved through format and accuracy rewards. MedReasoner achieves state-of-the-art performance on U-MRG-14K and demonstrates strong generalization to unseen clinical queries, underscoring the significant promise of reinforcement learning for interpretable medical grounding.

💡 Research Summary

The paper tackles a fundamental gap in medical imaging AI: the ability to translate clinicians’ implicit, often vague, natural‑language queries into precise pixel‑level region‑of‑interest (ROI) masks. Existing multimodal large language models (MLLMs) excel at visual‑language understanding but typically output only image‑level responses, requiring explicit spatial prompts (boxes, points) for grounding. In real clinical workflows such prompts are rarely provided, making current pipelines unsuitable for practical use.

Problem Definition – Unified Medical Reasoning Grounding (UMRG)
The authors formalize a new vision‑language task, UMRG, which demands three capabilities: (1) interpret an implicit clinical query, (2) reason over visual cues and anatomical priors to infer the latent target, and (3) generate a pixel‑accurate mask for that target. The task mirrors a clinician’s workflow: observe the image, reason about the description, and mark the ROI.

Dataset – U‑MRG‑14K
To enable research on UMRG, the authors construct a 14 K sample dataset that couples high‑quality image‑mask pairs with implicit clinical queries and full chain‑of‑thought (CoT) reasoning traces. The data are sourced from three public repositories (SA‑Med2D‑20M, BiomedParse, IMIS‑Bench) covering ten imaging modalities (CT, MRI, etc.). For each image, GPT‑4o is prompted to generate (i) a short lay description, (ii) a detailed medical description, (iii) a set of realistic, implicit question templates per super‑category, and (iv) answer strings that contain step‑by‑step reasoning and explicit spatial prompts (bounding box + two key points). A three‑stage human verification pipeline (students, then radiologists) removes factual errors and illogical reasoning. The final corpus spans 15 super‑categories, 108 fine‑grained categories, and includes CoT annotations—features absent from prior medical VQA or segmentation datasets.

Method – MedReasoner Framework
MedReasoner decouples reasoning from segmentation into two plug‑and‑play modules:

1. Clinical Reasoning Module (CRM) – a multimodal LLM (Lingshu) that receives the image and implicit query and outputs a structured response: <think>…</think> containing the CoT trace, followed by <answer> that supplies a bounding box and two semantic key points. The key points enrich the spatial cue beyond a simple rectangle, which is crucial for medical images where lesions may be irregular or clustered.

2. Anatomical Segmentation Module (ASM) – a frozen MedSAM2 model that accepts the CRM’s geometric prompts and instantly produces a pixel‑level mask. Because ASM is frozen, any improvements in segmentation technology can be swapped in without retraining the CRM.

Training – Reinforcement Learning with Dual Rewards
Instead of supervised fine‑tuning (which requires large amounts of explicit spatial annotations), the CRM is optimized via reinforcement learning using Group Relative Policy Optimization (GRPO). Two reward components guide learning:

• Format Reward – checks that the output adheres to the required schema (proper <think> and <answer> tags, correct data types). This maintains linguistic consistency and ensures the downstream ASM receives parsable inputs.
• Accuracy Reward – measures spatial correctness by comparing the CRM‑generated box/points against the ground‑truth mask (IoU for the box, Euclidean distance for points). This directly aligns the reasoning output with the ultimate segmentation quality.

The dual‑reward scheme encourages the CRM to produce coherent, medically plausible reasoning while simultaneously learning to emit spatial prompts that lead to high‑quality masks. The RL process yields a policy that can handle previously unseen implicit queries without any explicit supervision for those cases.

Results
On the U‑MRG‑14K benchmark, MedReasoner outperforms prior state‑of‑the‑art methods, including supervised fine‑tuned MLLMs and Seg‑Zero‑style approaches. Gains are especially pronounced on “unseen” query types, demonstrating strong generalization. Qualitative examples show sharper masks and more logical CoT traces compared to instruction‑tuned baselines. Ablation studies confirm that (i) the key‑point augmentation improves mask fidelity, (ii) freezing the ASM while training only the CRM is sufficient, and (iii) both format and accuracy rewards are necessary for stable convergence.

Discussion & Limitations
The work’s strengths lie in (a) defining a clinically relevant grounding task, (b) providing a richly annotated dataset that includes reasoning traces, and (c) presenting a modular RL‑based framework that separates language reasoning from pixel segmentation. Limitations include reliance on GPT‑4o for data generation (potential bias propagation), the inherent instability of RL reward shaping, and the dependence on a frozen segmentation model which may limit adaptation to novel modalities (e.g., ultrasound) or future segmentation architectures. Future directions suggested are expanding expert verification, refining reward functions (e.g., incorporating clinical plausibility metrics), and exploring end‑to‑end joint training once larger implicit‑query datasets become available.

Conclusion
The paper introduces a comprehensive solution to bridge the gap between implicit clinical language and precise visual grounding. By releasing U‑MRG‑14K and demonstrating the effectiveness of MedReasoner, the authors provide both a benchmark and a methodological blueprint that could accelerate the deployment of trustworthy, reasoning‑aware AI assistants in real‑world radiology and broader medical imaging practice.