Architectural Selection Framework for Synthetic Network Traffic: Quantifying the Fidelity-Utility Trade-off

The fidelity and utility of synthetic network traffic are critically compromised by architectural mismatch across heterogeneous network datasets and prevalent scalability failure. This study addresses this challenge by establishing an Architectural Selection Framework that empirically quantifies how data structure compatibility dictates the optimal fidelity-utility trade-off. We systematically evaluate twelve generative architectures (both non-AI and AI) across two distinct data structure types: categorical-heavy NSL-KDD and continuous-flow-heavy CIC-IDS2017. Fidelity is rigorously assessed through three structural metrics (Data Structure, Correlation, and Probability Distribution Difference) to confirm structural realism before evaluating downstream utility. Our results, confirmed over twenty independent runs (N=20), demonstrate that GAN-based models (CTGAN, CopulaGAN) exhibit superior architectural robustness, consistently achieving the optimal balance of statistical fidelity and practical utility. Conversely, the framework exposes critical failure modes, i.e., statistical methods compromise structural fidelity for utility (Compromised fidelity), and modern iterative architectures, such as Diffusion Models, face prohibitive computational barriers, rendering them impractical for large-scale security deployment. This contribution provides security practitioners with an evidence-based guide for mitigating architectural failures, thereby setting a benchmark for reliable and scalable synthetic data deployment in adaptive security solutions.

💡 Research Summary

The paper addresses a critical gap in synthetic network traffic generation: the lack of an evidence‑based framework that links generative model architecture to the intrinsic structure of the data. By introducing an “Architectural Selection Framework,” the authors systematically evaluate twelve generative approaches—both non‑AI (random oversampling, SMOTE, ADASYN, Gaussian Mixture Models, Bayesian Networks) and AI‑based (Variational Autoencoders, several GAN variants, Diffusion models)—across two fundamentally different datasets. NSL‑KDD represents a categorical‑heavy benchmark with many discrete fields (protocol, service, flag), while CIC‑IDS2017 is a continuous‑flow‑heavy dataset containing high‑dimensional traffic metrics (bytes per second, packet inter‑arrival times).

To quantify “fidelity,” the study defines three structural metrics: (1) Data Structure similarity (categorical vs. continuous proportion), (2) Correlation difference (Frobenius norm of the difference between Pearson/Spearman correlation matrices), and (3) Probability Distribution Difference (Kolmogorov‑Smirnov and Wasserstein distances per feature). These three metrics are combined into a “Fidelity Gatekeeper Score” that serves as a pre‑screening filter before downstream utility is examined.

Utility is measured by training three representative intrusion detection models (Random Forest, XGBoost, and a deep‑learning IDS) on the original training set, then augmenting the training data with synthetic samples from each method. The authors report changes in validation accuracy, F1‑score, and ROC‑AUC. In parallel, computational scalability is recorded (training time, generation time, GPU memory consumption). All experiments are repeated twenty independent times (N = 20) to capture variance and ensure statistical robustness.

Results show that GAN‑based architectures, specifically CTGAN and CopulaGAN, achieve the best trade‑off. They consistently obtain the lowest structural differences across all three fidelity metrics and deliver the highest utility gains (average 3.2 %–5.8 % improvement in IDS performance) while keeping training time within a practical range (≈ 30 minutes on a single RTX 3090). CTGAN’s conditional generation handles categorical fields elegantly, and CopulaGAN’s copula‑based dependency modeling preserves multivariate correlations, explaining their superior performance on both categorical‑heavy and continuous‑heavy data.

Non‑AI resampling techniques (ROS, SMOTE, ADASYN) effectively balance class distributions but fail to capture joint feature dependencies, leading to high correlation differences and negligible utility improvements (< 0.5 %). Probabilistic models (GMM, Bayesian Networks) improve marginal distributions but still suffer from restrictive parametric assumptions, resulting in moderate fidelity scores and limited downstream gains.

Variational Autoencoders occupy a middle ground: they manage to model continuous flows reasonably well but struggle with rare categorical attack patterns, producing modest utility gains (≈ 1.8 %).

Diffusion models achieve the highest raw fidelity—especially in temporal dynamics—yet their computational demands are prohibitive for large‑scale security simulations. Training requires thousands of denoising steps, consuming > 24 GB GPU memory and up to 1,200 seconds per epoch, which makes them unsuitable for most operational environments despite a potential 6 % uplift in IDS metrics.

Based on these findings, the authors propose a four‑step Architectural Selection Framework: (1) Analyze dataset structure (categorical vs. continuous dominance); (2) Choose appropriate fidelity metrics; (3) Evaluate candidate models against the Fidelity Gatekeeper Score; (4) Select the model that meets both fidelity and utility thresholds while respecting computational constraints. The framework provides concrete decision rules: for categorical‑heavy data, prioritize conditional GANs (CTGAN, CopulaGAN); for continuous‑flow‑heavy data, consider diffusion models only if sufficient compute resources exist, otherwise default to GANs; for rapid prototyping or limited resources, use non‑AI resampling but verify structural fidelity before deployment.

The paper acknowledges limitations: only two datasets were examined, and the study focuses on static batch generation rather than real‑time streaming synthesis. Future work is suggested to extend the framework to additional protocols, incorporate differential privacy mechanisms, and evaluate online generation scenarios.

In summary, this work delivers a rigorously validated, architecture‑aware methodology that helps security practitioners select synthetic traffic generators aligned with their data characteristics, thereby improving the reliability, scalability, and practical utility of synthetic network datasets in adaptive security solutions.