Temporally-Similar Structure-Aware Spatiotemporal Fusion of Satellite Images

This paper proposes a spatiotemporal (ST) fusion framework robust against diverse noise for satellite images, named Temporally-Similar Structure-Aware ST fusion (TSSTF). ST fusion is a promising approach to address the trade-off between the spatial and temporal resolution of satellite images. In real-world scenarios, observed satellite images are severely degraded by noise due to measurement equipment and environmental conditions. Consequently, some recent studies have focused on enhancing the robustness of ST fusion methods against noise. However, existing noise-robust ST fusion approaches often fail to capture fine spatial structure, leading to oversmoothing and artifacts. To address this issue, TSSTF introduces two key mechanisms: Temporally-Guided Total Variation (TGTV) and Temporally-Guided Edge Constraint (TGEC). TGTV is a weighted total variation-based regularization that promotes spatial piecewise smoothness while preserving structural details, guided by a reference high spatial resolution image acquired on a nearby date. TGEC enforces consistency in edge locations between two temporally adjacent images, while allowing for spectral variations. We formulate the ST fusion task as a constrained optimization problem incorporating TGTV and TGEC, and develop an efficient algorithm based on a preconditioned primal-dual splitting method. Experimental results demonstrate that TSSTF performs comparably to state-of-the-art methods under noise-free conditions and outperforms them under noisy conditions.

💡 Research Summary

The paper introduces a novel spatiotemporal (ST) fusion framework called Temporally‑Similar Structure‑Aware ST Fusion (TSSTF) that is robust to a wide variety of noise while preserving fine spatial details in satellite imagery. Traditional ST fusion seeks to combine high‑spatial‑resolution (HR) and low‑spatial‑resolution (LR) observations to generate a high‑resolution image at a target date, but real‑world data are often corrupted by random, sparse, stripe, or Poisson noise. Existing noise‑robust methods either rely heavily on training data (e.g., RSFN) or use total variation (TV) regularization that tends to oversmooth structural details (e.g., ROSTF).

TSSTF exploits the realistic assumption that when reference and target dates are close, their HR images share almost identical spatial structures. Two key mechanisms are proposed: (1) Temporally‑Guided Total Variation (TGTV), a weighted TV regularizer where the weights are derived from the gradient magnitudes of a reference HR image. This adaptive weighting promotes piecewise smoothness while protecting edges that are present in the reference. (2) Temporally‑Guided Edge Constraint (TGEC), which enforces consistency of edge locations between the reference and target HR images while allowing edge intensities to vary. TGEC is formulated as an ℓ₂‑ball constraint on the difference of binary edge maps extracted from the two images.

The observation model assumes that an HR image blurred and down‑sampled by a known operator yields the corresponding LR image, and that each observed image is contaminated by a mixture of random, sparse, and stripe noise components. The goal is to jointly estimate the denoised reference HR image and the target HR image using only one past HR‑LR pair and the current LR image.

TSSTF formulates this as a constrained convex optimization problem comprising a data‑fidelity term, the TGTV regularizer, and the TGEC constraint. Each term has an interpretable parameter (e.g., TV weight, edge‑constraint radius), which simplifies manual tuning. To solve the problem efficiently, the authors develop a preconditioned primal‑dual splitting (P‑PDS) algorithm enhanced with an operator‑norm‑based diagonal preconditioner (OVDP). An adaptive strategy updates the TGEC constraint level during iterations, improving convergence stability and reducing the need for a priori parameter selection.

Extensive experiments on synthetic and real satellite datasets (Landsat, MODIS) evaluate Gaussian, sparse, stripe, and Poisson noise scenarios. Quantitative metrics (PSNR, SSIM, SAM) show that TSSTF matches state‑of‑the‑art methods under noise‑free conditions and significantly outperforms them when noise is present, especially in preserving edges and textures. Visual results confirm reduced oversmoothing and fewer artifacts compared with TV‑based approaches. Sensitivity analyses demonstrate how TGTV weight balances noise suppression against detail preservation, and how TGEC radius affects edge alignment. Additional tests on cloud‑contaminated scenes and large land‑cover changes indicate that TSSTF remains stable. Computational cost analysis reveals that the proposed solver achieves comparable runtime to existing optimization‑based methods while delivering superior reconstruction quality.

In summary, TSSTF provides a principled, noise‑robust ST fusion solution that leverages temporally similar structural priors, adaptive regularization, and efficient convex optimization. Future work may extend the framework to multiple reference images, incorporate non‑linear degradation models, and explore real‑time implementations for operational remote‑sensing pipelines.