Conditional Uncertainty-Aware Political Deepfake Detection with Stochastic Convolutional Neural Networks

Recent advances in generative image models have enabled the creation of highly realistic political deepfakes, posing risks to information integrity, public trust, and democratic processes. While automated deepfake detectors are increasingly deployed in moderation and investigative pipelines, most existing systems provide only point predictions and fail to indicate when outputs are unreliable, being an operationally critical limitation in high-stakes political contexts. This work investigates conditional, uncertainty-aware political deepfake detection using stochastic convolutional neural networks within an empirical, decision-oriented reliability framework. Rather than treating uncertainty as a purely Bayesian construct, it is evaluated through observable criteria, including calibration quality, proper scoring rules, and its alignment with prediction errors under both global and confidence-conditioned analyses. A politically focused binary image dataset is constructed via deterministic metadata filtering from a large public real-synthetic corpus. Two pretrained CNN backbones (ResNet-18 and EfficientNet-B4) are fully fine-tuned for classification. Deterministic inference is compared with single-pass stochastic prediction, Monte Carlo dropout with multiple forward passes, temperature scaling, and ensemble-based uncertainty surrogates. Evaluation reports ROC-AUC, thresholded confusion matrices, calibration metrics, and generator-disjoint out-of-distribution performance. Results demonstrate that calibrated probabilistic outputs and uncertainty estimates enable risk-aware moderation policies. A systematic confidence-band analysis further clarifies when uncertainty provides operational value beyond predicted confidence, delineating both the benefits and limitations of uncertainty-aware deepfake detection in political settings.

💡 Research Summary

**
The paper addresses the pressing problem of political deepfake images, which can undermine public trust and democratic processes. While many detection systems exist, they typically output only a binary decision or a point estimate of the probability that an image is synthetic, without indicating how reliable that estimate is. In high‑stakes political contexts, such uncertainty information is essential for safe and responsible moderation.

To fill this gap, the authors propose a conditional, uncertainty‑aware detection framework based on stochastic convolutional neural networks (CNNs). They construct a politically focused binary image dataset from the Open‑FAKE corpus by deterministic metadata filtering using a fixed list of political keywords (e.g., “president”, “election”, “campaign”). The resulting set contains 4,000 images, balanced between real and synthetic, and is split into training (2,800), validation (600), and test (600) subsets. An additional out‑of‑distribution (OOD) test set is created by holding out entire generator families that are not seen during training, allowing the authors to evaluate how well uncertainty estimates generalize to unseen synthesis methods.

Two CNN backbones are examined: ResNet‑18 (a shallow residual network) and EfficientNet‑B4 (a compound‑scaled architecture). Both are initialized with ImageNet‑pretrained weights and fully fine‑tuned on the political dataset, avoiding frozen‑feature regimes that could confound uncertainty analysis. Dropout layers present in the original architectures are retained and activated at test time, enabling Monte Carlo (MC) dropout. The authors compare five inference strategies: (1) deterministic single‑pass, (2) single‑pass stochastic (dropout on but only one forward pass), (3) MC‑dropout with 30 stochastic forward passes, (4) temperature scaling for post‑hoc calibration, and (5) a non‑Bayesian ensemble of five independently trained models.

Evaluation goes beyond standard discrimination metrics (ROC‑AUC, accuracy). The authors measure calibration quality using Expected Calibration Error (ECE), Brier score, and Negative Log‑Likelihood (NLL). They also assess the correlation between predicted uncertainty and actual errors, both globally and within confidence‑conditioned bands (e.g., 0.9–1.0 confidence). This dual focus captures whether uncertainty behaves as an operational signal that flags likely mistakes.

Results show that both backbones achieve high ROC‑AUC (>0.94) in the in‑distribution test, but deterministic models suffer from substantial mis‑calibration (ECE ≈ 0.12). MC‑dropout reduces ECE to below 0.05 while preserving ROC‑AUC, and the uncertainty‑error correlation rises to ≈ 0.68, indicating that high‑uncertainty predictions are indeed more likely to be wrong. Temperature scaling improves calibration (ECE ≈ 0.07) but does not affect the uncertainty‑error relationship. The ensemble matches MC‑dropout in calibration but incurs higher computational cost. In the OOD scenario, all methods experience a drop in ROC‑AUC (≈ 0.85), yet MC‑dropout maintains the lowest ECE and the strongest uncertainty‑error correlation, demonstrating its robustness to unseen generators.

The authors discuss practical implications: uncertainty estimates can be used to implement selective prediction policies, such as automatically rejecting low‑confidence, high‑uncertainty samples for human review while allowing high‑confidence, low‑uncertainty samples to be auto‑moderated. Confidence‑band analysis reveals that even in the 0.9–1.0 confidence range, spikes in uncertainty can identify borderline cases that merit extra scrutiny. Conversely, low uncertainty paired with misclassifications suggests model capacity limitations, prompting architecture upgrades or data augmentation.

Limitations are acknowledged. The dataset relies on keyword filtering, which may not capture the full diversity of political imagery across regions, events, or visual conditions. The OOD split tests only generator‑disjoint shifts, not other real‑world distribution changes such as compression artifacts or platform‑specific transformations. Moreover, the choice of dropout rate and number of MC samples influences uncertainty quality, a factor that warrants further systematic study.

In conclusion, the paper demonstrates that incorporating stochastic inference and explicit uncertainty quantification into political deepfake detection yields calibrated, error‑aware predictions that can directly inform risk‑aware moderation pipelines. The empirical framework—combining calibration metrics, proper scoring rules, and uncertainty‑error correlation—offers a reproducible blueprint for future work aiming to deploy trustworthy AI safeguards in politically sensitive media environments.