MPath: Multimodal Pathology Report Generation from Whole Slide Images

Automated generation of diagnostic pathology reports directly from whole slide images (WSIs) is an emerging direction in computational pathology. Translating high-resolution tissue patterns into clinically coherent text remains difficult due to large morphological variability and the complex structure of pathology narratives. We introduce MPath, a lightweight multimodal framework that conditions a pretrained biomedical language model (BioBART) on WSI-derived visual embeddings through a learned visual-prefix prompting mechanism. Instead of end-to-end vision-language pretraining, MPath leverages foundation-model WSI features (CONCH + Titan) and injects them into BioBART via a compact projection module, keeping the language backbone frozen for stability and data efficiency. MPath was developed and evaluated on the RED 2025 Grand Challenge dataset and ranked 4th in Test Phase 2, despite limited submission opportunities. The results highlight the potential of prompt-based multimodal conditioning as a scalable and interpretable strategy for pathology report generation.

💡 Research Summary

The paper presents MPath, a lightweight multimodal framework for generating diagnostic pathology reports directly from whole‑slide images (WSIs). The authors address two major challenges in this domain: the gigapixel size and morphological heterogeneity of WSIs, and the structured, terminology‑rich nature of pathology narratives. Rather than training a large vision‑language model from scratch, MPath leverages existing foundation models for pathology imaging—CONCH and Titan—to obtain robust slide‑level embeddings. These embeddings are then injected into a pretrained biomedical language model, BioBART, via a learned visual‑prefix prompting mechanism.

In the visual‑prefix module, a WSI feature vector f_WSI (dimension d_v) is first projected through a linear layer W₁ followed by ReLU, yielding a hidden representation h. A second linear layer W₂ reshapes h into L_p token embeddings p_v that share the same dimensionality d as BioBART’s token embeddings. The visual tokens are prepended to a short textual prompt (e.g., “Pathology report:”) and fed into the frozen BioBART encoder‑decoder. Only the parameters of the visual‑prompt encoder, the projection layers, and a few auxiliary heads (organ, sample‑type, finding classification) are trainable, keeping the language backbone untouched. This design drastically reduces the number of trainable parameters (on the order of a few hundred thousand) and mitigates catastrophic forgetting.

The system was evaluated on the REG 2025 Grand Challenge dataset, which contains 7,385 paired WSIs and reports for training and 1,000 WSIs each for Test Phase 1 and Phase 2. Because the authors entered only Phase 2, the reported results are based on the weighted composite score defined by the challenge: 0.15 × (ROUGE + BLEU) + 0.4 × Keyword‑Jaccard + 0.3 × Semantic‑Embedding similarity. MPath achieved a score of 0.8282, ranking fourth among the top five submissions. Qualitative inspection shows that the model reliably reproduces key diagnostic terms such as “invasive carcinoma,” “adenocarcinoma,” “microcalcification,” and “chronic inflammation.” However, occasional hallucinations were observed—for instance, the addition of “Chronic granulomatous inflammation with foreign body reaction” in a bladder case where the ground‑truth did not mention it. The authors attribute these errors to the reliance on global slide embeddings, which may lack the fine‑grained grounding needed to fully constrain the language decoder.

The discussion highlights both the strengths and limitations of the approach. Strengths include data efficiency (no need for massive paired image‑text corpora), modularity (different vision encoders or language decoders can be swapped), and training stability (freezing the language model). Limitations involve insufficient fine‑grained visual grounding, leading to factual inaccuracies, and the inability of a frozen language model to adapt to institution‑specific reporting styles. To address these issues, the authors propose several future directions: (1) hierarchical visual modeling that combines patch‑level and slide‑level cues; (2) contrastive alignment objectives (e.g., CLIP‑style losses) to tighten visual‑textual correspondence; (3) parameter‑efficient fine‑tuning of the language model using adapters or LoRA to better capture pathology‑specific terminology; (4) constrained or contrastive decoding strategies and hallucination‑detection modules to improve factual fidelity; and (5) structured report generation that explicitly outputs fields such as organ, sample type, and diagnosis.

In conclusion, MPath demonstrates that prompt‑based multimodal conditioning can serve as a scalable and interpretable alternative to heavyweight vision‑language fusion architectures in computational pathology. By capitalizing on existing foundation models for both imaging and text, the framework achieves competitive performance with modest computational resources. The paper outlines clear pathways for enhancing visual grounding and textual adaptation, suggesting that with these refinements, such systems could be integrated into real‑world pathology workflows to reduce reporting time, improve consistency, and ultimately support better clinical decision‑making.