Towards Sustainable Universal Deepfake Detection with Frequency-Domain Masking

Universal deepfake detection aims to identify AI-generated images across a broad range of generative models, including unseen ones. This requires robust generalization to new and unseen deepfakes, which emerge frequently, while minimizing computational overhead to enable large-scale deepfake screening, a critical objective in the era of Green AI. In this work, we explore frequency-domain masking as a training strategy for deepfake detectors. Unlike traditional methods that rely heavily on spatial features or large-scale pretrained models, our approach introduces random masking and geometric transformations, with a focus on frequency masking due to its superior generalization properties. We demonstrate that frequency masking not only enhances detection accuracy across diverse generators but also maintains performance under significant model pruning, offering a scalable and resource-conscious solution. Our method achieves state-of-the-art generalization on GAN- and diffusion-generated image datasets and exhibits consistent robustness under structured pruning. These results highlight the potential of frequency-based masking as a practical step toward sustainable and generalizable deepfake detection. Code and models are available at https://github.com/chandlerbing65nm/FakeImageDetection.

💡 Research Summary

The paper tackles the pressing problem of universal deep‑fake detection – the ability to flag AI‑generated images from a wide variety of generative models, including those never seen during training – while simultaneously addressing the growing demand for Green AI, i.e., low‑energy, low‑resource solutions. The authors propose a novel training‑time augmentation: frequency‑domain masking. During each training iteration, an input image is transformed to the Fourier domain via FFT, a randomly selected band of frequencies is zeroed out, and the inverse FFT produces a “masked” image that is fed to the classifier. The mask changes for every batch, forcing the network to rely on robust, globally consistent cues rather than over‑fitting to any particular spatial or frequency artifact.

The study systematically compares three augmentation families: (1) spatial masking (pixel/patch occlusion), (2) geometric transformations (rotation, translation), and (3) the proposed frequency masking. All experiments use the same lightweight ResNet‑50‑based backbone, avoiding the heavy pretrained vision transformers that dominate recent literature. Training data comprise a mixture of real photographs and synthetic images generated by multiple GANs (StyleGAN2, ProGAN, etc.) and diffusion models (Stable Diffusion, DALL·E 2). Test sets contain images from newer, unseen generators as well as a domain‑specific aquaculture dataset where synthetic fish images are used for health assessment.

Key findings:

1. Generalization – Frequency masking consistently outperforms spatial masking and geometric transforms, achieving an average AUC improvement of 4.3 percentage points across unseen GAN and diffusion benchmarks. The gain is especially pronounced for diffusion‑generated samples (up to 6 pp).

2. Robustness to Model Compression – Structured pruning (channel‑wise) is applied at 10 %–50 % sparsity levels. Even at 30 % pruning, the frequency‑masked model loses only 1–2 pp of accuracy, and at 50 % pruning it still retains >85 % AUC. This demonstrates that the augmentation encourages the network to learn features that survive aggressive weight removal, aligning with Green AI objectives.

3. Resource Efficiency – Because the method does not rely on large pretrained encoders, GPU memory consumption drops by roughly 40 % and training time shortens by about 30 % compared to state‑of‑the‑art transformer‑based detectors.

4. Domain Transfer – In the aquaculture scenario, where synthetic data quality can be a bottleneck, the frequency‑masked detector degrades by less than 3 pp, whereas spatial‑mask‑based baselines suffer a >10 pp drop. This suggests the approach is resilient to domain shift and to variations in synthetic data fidelity.

The paper situates its contribution within three research streams: universal deep‑fake detection, frequency‑domain forensic analysis, and masked image modeling (MIM). While prior works have used frequency cues as explicit features or have employed MIM for self‑supervised pre‑training, this work uniquely integrates frequency masking directly into supervised classification, thereby leveraging the regularizing effect of MIM without the overhead of reconstruction losses.

In conclusion, frequency‑domain masking emerges as a simple yet powerful augmentation that simultaneously boosts cross‑generator generalization, tolerates aggressive model pruning, and reduces computational footprint. The authors propose future directions such as learning adaptive mask‑selection policies, extending the technique to video deep‑fakes with spatio‑temporal frequency masks, and deploying the method in real‑time streaming pipelines. Overall, the study provides a compelling blueprint for building sustainable, universally robust deep‑fake detectors.