Can We Build a Monolithic Model for Fake Image Detection? SICA: Semantic-Induced Constrained Adaptation for Unified-Yet-Discriminative Artifact Feature Space Reconstruction

Fake Image Detection (FID), aiming at unified detection across four image forensic subdomains, is critical in real-world forensic scenarios. Compared with ensemble approaches, monolithic FID models are theoretically more promising, but to date, consistently yield inferior performance in practice. In this work, by discovering the heterogeneous phenomenon'', which is the intrinsic distinctness of artifacts across subdomains, we diagnose the cause of this underperformance for the first time: the collapse of the artifact feature space driven by such phenomenon. The core challenge for developing a practical monolithic FID model thus boils down to the unified-yet-discriminative" reconstruction of the artifact feature space. To address this paradoxical challenge, we hypothesize that high-level semantics can serve as a structural prior for the reconstruction, and further propose Semantic-Induced Constrained Adaptation (SICA), the first monolithic FID paradigm. Extensive experiments on our OpenMMSec dataset demonstrate that SICA outperforms 15 state-of-the-art methods and reconstructs the target unified-yet-discriminative artifact feature space in a near-orthogonal manner, thus firmly validating our hypothesis. The code and dataset are available at:https: //github.com/scu-zjz/SICA_OpenMMSec.

💡 Research Summary

This paper tackles the fundamental challenge of building a monolithic model for Fake Image Detection (FID), which aims to perform unified real-fake classification across four distinct subdomains: Deepfake, AI-Generated Content (AIGC), Image Manipulation Detection and Localization (IMDL), and Document (Doc) forgery. While theoretically promising, monolithic FID models have consistently underperformed in practice compared to ensemble approaches that combine specialized detectors.

The authors identify the root cause of this underperformance as the “heterogeneous phenomenon”: the intrinsic and fundamental distinctness of forensic artifacts (manipulation traces) across different subdomains. For instance, facial physiological signals (rPPG) in Deepfakes are conceptually inapplicable to document images. When a unified model attempts to project these heterogeneous, high-dimensional artifact distributions into a single, shared low-dimensional feature space, a “collapse” occurs. Crucial domain-specific discriminative features are lost, and the model ends up learning only the minimal common signals, severely limiting its performance. This collapse is evidenced by the observation that adding training data from other subdomains can degrade performance on a current one.

Thus, the core challenge for a practical monolithic FID model is reconstructing a “unified-yet-discriminative” artifact feature space—a seemingly paradoxical goal. The authors hypothesize that high-level semantic information (e.g., whether an image contains a face, a natural scene, or a document) can serve as the necessary structural prior for this reconstruction. They empirically show that semantic features extracted by models like CLIP naturally cluster by subdomain, providing a stable and discriminative manifold.

To operationalize this hypothesis, the paper proposes Semantic-Induced Constrained Adaptation (SICA), the first monolithic FID paradigm. SICA is built on two principles: 1) Semantic-Induced: It uses a frozen, pre-trained Vision Transformer (from CLIP) as a fixed backbone, providing a stable reference semantic manifold. 2) Constrained Adaptation: It updates the model only through low-rank adaptation (LoRA) on the self-attention layers. This strategy selectively bridges the distribution gap between the training data and the reference semantic manifold while largely preserving the manifold’s structure. This approach mitigates the risk of overfitting to “semantic shortcuts” (e.g., classifying all documents as fake based on their semantic class) while allowing the model to form an accurate inductive bias for learning artifacts anchored to their semantic context.

To enable a systematic and fair evaluation, the authors construct OpenMMSec, a large-scale, comprehensive FID benchmark dataset. It aggregates data from 19 public datasets, contains over 330K images spanning all four subdomains and 98 fine-grained faking types, and is carefully balanced in terms of data volume and sources.

Extensive experiments on OpenMMSec demonstrate that SICA successfully addresses the technical dilemma. It outperforms 15 state-of-the-art general-purpose backbones and specialized subdomain detectors across all four subdomains and evaluation metrics, establishing itself as the first monolithic model to achieve this. Furthermore, feature space analysis via t-SNE visualizations and quantitative measures confirms that SICA successfully reconstructs a unified-yet-discriminative artifact feature space in a near-orthogonal manner. Subdomain clusters are clearly separated within a unified representation space, providing strong validation for the semantic structural prior hypothesis.

In summary, this work makes significant contributions by: 1) diagnosing the artifact feature space collapse caused by heterogeneity as the core challenge in monolithic FID; 2) proposing the novel hypothesis that semantic manifolds can guide the reconstruction of a unified-yet-discriminative feature space; 3) introducing the SICA paradigm that implements this via a frozen semantic backbone and constrained low-rank adaptation; 4) releasing the OpenMMSec benchmark to foster future research; and 5) rigorously validating the hypothesis through superior performance and feature space analysis. It redefines the problem of unified fake image detection and opens a new direction for leveraging semantics in multimedia security tasks.