Sequential Attention-based Sampling for Histopathological Analysis

Deep neural networks are increasingly applied in automated histopathology. Yet, whole-slide images (WSIs) are often acquired at gigapixel sizes, rendering them computationally infeasible to analyze entirely at high resolution. Diagnostic labels are largely available only at the slide-level, because expert annotation of images at a finer (patch) level is both laborious and expensive. Moreover, regions with diagnostic information typically occupy only a small fraction of the WSI, making it inefficient to examine the entire slide at full resolution. Here, we propose SASHA – Sequential Attention-based Sampling for Histopathological Analysis – a deep reinforcement learning approach for efficient analysis of histopathological images. First, SASHA learns informative features with a lightweight hierarchical, attention-based multiple instance learning (MIL) model. Second, SASHA samples intelligently and zooms selectively into a small fraction (10-20%) of high-resolution patches to achieve reliable diagnoses. We show that SASHA matches state-of-the-art methods that analyze the WSI fully at high resolution, albeit at a fraction of their computational and memory costs. In addition, it significantly outperforms competing, sparse sampling methods. We propose SASHA as an intelligent sampling model for medical imaging challenges that involve automated diagnosis with exceptionally large images containing sparsely informative features. Model implementation is available at: https://github.com/coglabiisc/SASHA.

💡 Research Summary

The paper introduces SASHA (Sequential Attention‑based Sampling for Histopathological Analysis), a novel framework that dramatically reduces the computational burden of whole‑slide image (WSI) analysis while preserving state‑of‑the‑art diagnostic performance. Whole‑slide scans in digital pathology often reach gigapixel dimensions, making it infeasible to process every pixel at high magnification. Moreover, diagnostic cues such as tumor cells occupy only a tiny fraction of the slide, so exhaustive high‑resolution analysis wastes resources. SASHA tackles these challenges by combining a lightweight hierarchical multiple‑instance learning (MIL) backbone with a deep reinforcement learning (RL) agent that learns to “scan” the slide at low resolution and selectively zoom into the most informative regions.

The architecture consists of three main components. (1) Hierarchical Attention‑based Feature Distiller (HAFED): This two‑stage attention module first aggregates k high‑resolution sub‑patches belonging to each low‑resolution patch using multiple attention heads and stochastic masking, producing a compressed d‑dimensional representation that aligns with the low‑resolution feature space. A second attention layer then operates across the N low‑resolution patches to generate a slide‑level embedding h∈ℝᵈ, which feeds a classifier for binary (or multi‑class) cancer prediction. Multi‑head attention encourages the model to capture diverse diagnostic patterns, while the similarity loss enforces distinct attention maps across heads. (2) Targeted State Updater (TSU): The RL agent maintains a state matrix Sₜ∈ℝᴺˣᵈ that stores the current representation of every low‑resolution patch. When a patch aₜ is selected for high‑resolution inspection, TSU updates not only its entry but also all other entries whose feature vectors are highly correlated (measured by cosine similarity). This concerted update propagates new information efficiently and reduces the number of forward passes required per timestep. (3) Deep RL Agent: Using Proximal Policy Optimization (PPO), the agent learns a policy π(aₜ|Sₜ) and a value function V(Sₜ). The reward is derived from the classification loss after each episode, encouraging the agent to select patches that most improve the slide‑level prediction. Crucially, the classifier (trained together with HAFED) is frozen during RL training, which stabilizes convergence and avoids the “policy‑classifier co‑training” instability observed in prior work such as RLogist.

Training proceeds in two stages. First, all patches are processed at high resolution to train HAFED and the classifier end‑to‑end, using a composite loss that combines label supervision, attention‑diversity, and similarity constraints. Second, the RL agent is trained on the frozen feature extractor, allowing it to learn an efficient sampling strategy without degrading feature quality.

Empirical evaluation on two large TCGA‑derived cancer benchmarks (e.g., breast and lung adenocarcinoma) demonstrates that SASHA matches the AUC of full‑resolution MIL baselines (≈0.92–0.94) while using only 6 % of the memory required for slide‑level feature storage and achieving up to 8× faster inference. Compared with other sparse‑sampling methods that also sample ~10 % of patches, SASHA improves accuracy by 3–5 % absolute AUC. Ablation studies confirm that each component—multi‑head attention, TSU, and frozen classifier—contributes meaningfully to performance and stability.

The authors release the full implementation on GitHub, ensuring reproducibility. By mirroring the human pathologist’s workflow of low‑resolution scanning followed by selective high‑magnification, SASHA offers a practical solution for deploying deep learning‑based diagnostics in real‑world pathology labs, where computational resources and turnaround time are critical. Future directions include extending the framework to multimodal data (e.g., clinical metadata), applying it to other histological stains such as immunohistochemistry, and exploring hierarchical RL policies that adapt the number of zoom steps per slide based on uncertainty estimates.