Deep Modeling and Interpretation for Bladder Cancer Classification

Deep models based on vision transformer (ViT) and convolutional neural network (CNN) have demonstrated remarkable performance on natural datasets. However, these models may not be similar in medical imaging, where abnormal regions cover only a small portion of the image. This challenge motivates this study to investigate the latest deep models for bladder cancer classification tasks. We propose the following to evaluate these deep models: 1) standard classification using 13 models (four CNNs and eight transormer-based models), 2) calibration analysis to examine if these models are well calibrated for bladder cancer classification, and 3) we use GradCAM++ to evaluate the interpretability of these models for clinical diagnosis. We simulate $\sim 300$ experiments on a publicly multicenter bladder cancer dataset, and the experimental results demonstrate that the ConvNext series indicate limited generalization ability to classify bladder cancer images (e.g., $\sim 60%$ accuracy). In addition, ViTs show better calibration effects compared to ConvNext and swin transformer series. We also involve test time augmentation to improve the models interpretability. Finally, no model provides a one-size-fits-all solution for a feasible interpretable model. ConvNext series are suitable for in-distribution samples, while ViT and its variants are suitable for interpreting out-of-distribution samples.

💡 Research Summary

This paper conducts a comprehensive empirical study of deep learning models for bladder cancer classification using a publicly available multi‑center MRI dataset. The authors evaluate thirteen state‑of‑the‑art architectures—four ConvNeXt variants (B, L, S, T), three Vision Transformer families (MaxViT‑tiny, ViT‑h14, ViT‑l16), and six Swin Transformer variants (B, S, T, V2‑B, V2‑S, V2‑T)—under five different optimizers (SGD, Adam, AdamW, Adagrad, Adadelta). All models share a unified training pipeline: images are Z‑score normalized, resized to 224 × 224, and split into training/validation (80/20) and an external test set drawn from a different clinical center, yielding four cross‑validation folds. Training runs for 50 epochs with a batch size of 32 (16 for ViT‑h14) on a RTX 4090 GPU using PyTorch 1.13.1.

Performance is measured with standard classification metrics (accuracy, balanced accuracy, precision, recall, F1, and their average) as well as calibration quality via Expected Calibration Error (ECE) and reliability diagrams. Interpretability is assessed with Grad‑CAM++ visualizations, both in‑distribution (ID) and out‑of‑distribution (OOD) settings, and test‑time augmentation (TTA) is applied to improve the stability of the heatmaps.

Key findings include:

1. Generalization Gap – ConvNeXt models achieve very high validation accuracies (up to ~95 % on some folds) but drop dramatically on the held‑out center (≈40–50 % ACC), indicating poor cross‑site generalization.
2. Transformer Behavior – ViT‑h14 and ViT‑l16 sometimes reach >70 % test accuracy when sufficient training data are available, yet they suffer severe over‑fitting on folds with limited samples (e.g., ACC ≈ 33 % on Fold 4). Swin‑V2‑b shows relatively strong average metrics but a high ECE (~34 %), suggesting it is poorly calibrated for this medical task.
3. Calibration – Across optimizers, ViT‑l16 consistently yields the lowest ECE (≈ 15 %), while ConvNeXt variants exhibit the highest calibration errors. MaxViT‑tiny displays the smallest variance in ECE across folds, making its probability estimates the most reliable.
4. Efficiency – MaxViT‑tiny is the fastest to train, whereas ViT‑h14 requires roughly seven times more training time than MaxViT‑tiny, raising concerns for real‑time clinical deployment.
5. Interpretability – In ID scenarios, ConvNeXt models focus sharply on the tumor region for both muscle‑invasive (MIBC) and non‑muscle‑invasive (NMIBC) cases, whereas Swin‑Transformer models produce more scattered attention maps. In OOD scenarios, ViT and MaxViT models generate more consistent and clinically plausible heatmaps, and TTA further sharpens these visual explanations.

The authors conclude that no single architecture satisfies all clinical requirements. ConvNeXt models are preferable when the test data share the same distribution as the training set, offering higher raw accuracy. Vision Transformers and MaxViT, however, provide better calibrated probabilities and more robust interpretability when faced with distribution shifts, making them suitable for out‑of‑distribution or safety‑critical applications. The choice of optimizer also markedly influences calibration and generalization, underscoring the need for careful hyper‑parameter tuning in medical AI pipelines. Future work should explore domain‑adaptation techniques, larger multi‑center cohorts, and advanced calibration methods to further close the gap between research performance and real‑world clinical utility.