W-DUALMINE: Reliability-Weighted Dual-Expert Fusion With Residual Correlation Preservation for Medical Image Fusion

Medical image fusion integrates complementary information from multiple imaging modalities to improve clinical interpretation. However, existing deep learningbased methods, including recent spatial-frequency frameworks such as AdaFuse and ASFE-Fusion, often suffer from a fundamental trade-off between global statistical similaritymeasured by correlation coefficient (CC) and mutual information (MI)and local structural fidelity. This paper proposes W-DUALMINE, a reliability-weighted dual-expert fusion framework designed to explicitly resolve this trade-off through architectural constraints and a theoretically grounded loss design. The proposed method introduces dense reliability maps for adaptive modality weighting, a dual-expert fusion strategy combining a global-context spatial expert and a wavelet-domain frequency expert, and a soft gradient-based arbitration mechanism. Furthermore, we employ a residual-to-average fusion paradigm that guarantees the preservation of global correlation while enhancing local details. Extensive experiments on CT-MRI, PET-MRI, and SPECT-MRI datasets demonstrate that W-DUALMINE consistently outperforms AdaFuse and ASFE-Fusion in CC and MI metrics while

💡 Research Summary

The paper introduces W‑DUALMINE, a novel medical image fusion framework that explicitly tackles the longstanding trade‑off between global statistical similarity (measured by correlation coefficient (CC) and mutual information (MI)) and local structural fidelity. The authors argue that recent deep learning‑based spatial‑frequency methods such as AdaFuse and ASFE‑Fusion improve edge sharpness but often degrade CC and MI because they treat the two modalities independently in separate expert branches.

W‑DUALMINE addresses this issue through four tightly coupled components. First, a dense reliability map is computed at each encoder scale. A lightweight 1×1 convolution head predicts per‑pixel reliability scores for each modality, which are normalized via a softplus‑based gating function to obtain adaptive weights w₁ and w₂. These weights suppress contributions from noisy or artifact‑prone regions before any fusion occurs.

Second, the architecture employs a dual‑expert design. The Global‑Context Spatial Expert processes the weighted base feature using a parallel combination of a standard 3×3 convolution and a dilated (d=2) 3×3 convolution, thereby capturing both fine‑grained details and long‑range anatomical context. The Wavelet Frequency Expert explicitly decomposes the features with a Haar Discrete Wavelet Transform (DWT) into low‑frequency (LL) and three high‑frequency sub‑bands (LH, HL, HH). The LL band is fused by the same reliability‑weighted averaging used in the spatial branch, while the high‑frequency bands are merged using a magnitude‑maximum rule that selects the strongest edge response from either modality. An inverse DWT reconstructs the frequency‑expert output.

Third, a Soft Gradient Mixer (SGM) arbitrates between the two expert outputs. Sobel operators compute gradient magnitude maps for both the spatial and wavelet outputs; a small CNN receives the concatenated features plus their gradients and predicts mixing coefficients α₁ and α₂ via a softmax. The final fused feature at each scale is a weighted sum α₁·E_spatial + α₂·E_wave, allowing the network to favor the frequency expert in high‑gradient regions and the spatial expert where gradients are weak.

The fourth and most original component is the Residual‑to‑Average fusion paradigm. The authors note that the simple pixel‑wise average of the two source images, I_avg = (I₁+I₂)/2, maximizes linear correlation with both inputs. Instead of directly predicting the fused image, the decoder learns a residual map R. The final fused image is constructed as I_f = clip( I_avg + λ·tanh(R), 0, 1 ), with λ set to 0.5. By adding only a bounded residual to the statistically optimal average, the method guarantees high CC and MI while still injecting the high‑frequency details learned by the dual experts.

Training uses a composite loss:
• L_avg (ℓ₁ distance to I_avg) enforces the average content, directly supporting high MI and CC.
• L_grad (ℓ₁ distance between fused gradients and the element‑wise maximum of source gradients) prevents blurring of edges.
• L_cc (1 – cosine similarity between I_f and I_avg) explicitly maximizes correlation.
• L_mi (InfoNCE contrastive loss on latent embeddings) encourages feature‑level mutual information, albeit with a relatively low weight (λ₄=0.1).
• L_rec (reconstruction of each source from I_f) stabilizes encoder training.

Experiments are conducted on three brain multimodal datasets (CT‑MRI, PET‑MRI, SPECT‑MRI) extracted from the Harvard Whole Brain Atlas. Each modality pair consists of 24 perfectly aligned image pairs resized to 256×256. The network is implemented in PyTorch, trained on a single NVIDIA Tesla P100 for 100 epochs with Adam (lr = 1e‑5, batch = 8) and standard data augmentations. Evaluation metrics include Entropy, PSNR, Feature Mutual Information, CC, and MI. Compared with AdaFuse and ASFE‑Fusion, W‑DUALMINE achieves consistent improvements in CC (≈ +2–3 %) and MI (≈ +2 %) while maintaining comparable or slightly better scores on the other metrics.

The paper’s strengths lie in its systematic integration of reliability weighting, dual‑expert processing, and a theoretically motivated residual‑to‑average fusion that directly targets global statistical fidelity. The Soft Gradient Mixer provides an elegant, learnable arbitration mechanism that adapts to local edge strength without hand‑crafted rules.

However, several limitations are evident. The experimental dataset is relatively small (24 pairs per modality), and the paper does not report cross‑validation, statistical significance testing, or generalization to external clinical datasets. The reliance on a fixed Haar wavelet may limit performance on images with more complex texture patterns; exploring learnable wavelet bases or alternative transforms could be beneficial. Additionally, the mutual information loss receives a modest weight, raising questions about its actual contribution to the reported MI gains.

In summary, W‑DUALMINE presents a well‑engineered solution that bridges the gap between global statistical consistency and local detail preservation in medical image fusion. Its architectural innovations and loss design are coherent and empirically validated, though broader validation and exploration of alternative frequency decompositions would strengthen the claim of universal applicability.