Confidence Driven Classification of Application Types in the Presence of Background Network Traffic

Accurately classifying the application types of network traffic using deep learning models has recently gained popularity. However, we find that these classifiers do not perform well on real-world traffic data due to the presence of non-application-specific generic background traffic originating from advertisements, analytics, shared APIs, and trackers. Unfortunately, state-of-the-art application classifiers overlook such traffic in curated datasets and only classify relevant application traffic. To address this issue, when we label and train using an additional class for background traffic, it leads to additional confusion between application and background traffic, as the latter is heterogeneous and encompasses all traffic that is not relevant to the application sessions. To avoid falsely classifying background traffic as one of the relevant application types, a reliable confidence measure is warranted, such that we can refrain from classifying uncertain samples. Therefore, we design a Gaussian Mixture Model-based classification framework that improves the indication of the deep learning classifier’s confidence to allow more reliable classification.

💡 Research Summary

The paper tackles a practical yet under‑explored problem in network traffic classification: the presence of generic background traffic such as advertisements, analytics, shared APIs, and trackers that appear alongside legitimate application flows in real‑world environments. While recent deep‑learning classifiers achieve high accuracy on curated benchmarks (e.g., ISCX VPN‑nonVPN, UC‑Davis QUIC), these datasets are outdated and deliberately filtered to exclude background traffic. Consequently, when the authors evaluated state‑of‑the‑art models on a newly collected 2024 dataset, performance dropped dramatically.

To investigate, the authors built a comprehensive dataset covering eight modern application categories (web browsing, social media, video streaming, email, VoIP, chat, online document editing, and gaming). Data were gathered using automated Selenium scripts for the first four categories and manual sessions for the remaining ones, resulting in 1,066 application sessions and millions of TCP/UDP flows. Each flow includes a preceding DNS response, enabling domain‑name based analysis.

Three labeling schemes were examined: (1) session‑based labeling (traditional approach), (2) domain‑name based labeling that filters out flows whose DNS names do not clearly indicate the target application, and (3) a comprehensive scheme that adds an explicit “Background” class for all generic traffic. Baseline experiments using two deep‑learning architectures—BiLSTM (with packet arrival time and size features) and FS‑Net (packet‑size sequence CNN)—showed that session‑based labeling yields macro F1 scores around 0.72–0.75, whereas domain‑name filtering (without a background class) improves macro F1 to ~0.92. However, the latter approach is unrealistic because background traffic cannot be removed in production.

The core contribution is a confidence‑driven classification framework. The authors first observe that softmax probabilities alone are insufficient to distinguish uncertain samples, especially when background traffic is heterogeneous. They therefore fit a Gaussian Mixture Model (GMM) to the softmax output vectors for each class, learning multiple Gaussian components that capture intra‑class variability. For a new flow, the GMM provides a likelihood score that serves as a confidence metric. By setting a confidence threshold, the system can reject low‑confidence predictions, thereby avoiding mislabeling background traffic as a specific application. Experiments demonstrate a clear precision‑recall trade‑off: raising the threshold boosts accuracy (up to 0.96 macro F1) while reducing coverage (down to ~55 %). The authors present ROC‑like curves that allow operators to select operating points matching their tolerance for false positives versus missed traffic.

Key insights include: (i) background traffic is the dominant source of confusion in modern datasets; (ii) simply adding a “background” class does not solve the problem because the class is highly heterogeneous; (iii) probabilistic confidence estimation via GMM can be layered on top of any existing deep‑learning classifier with minimal overhead; (iv) the approach is adaptable—new background patterns can be incorporated by updating the GMM without retraining the underlying classifier.

Limitations are acknowledged. GMM performance depends on the number of mixture components and initialization; rare background patterns may still cause over‑fitting. The study focuses on TCP flows and does not extensively evaluate UDP‑centric traffic such as QUIC or real‑time gaming packets. Moreover, the confidence threshold must be tuned for each deployment scenario, which may require additional validation data.

In conclusion, the paper provides a practical solution for deploying application‑type classifiers in the wild: a confidence‑aware pipeline that rejects uncertain flows, thereby preserving high precision while accepting a controllable reduction in coverage. Future work could explore Bayesian neural networks, ensemble methods, or lightweight on‑device models to further improve confidence estimation, as well as extending the evaluation to newer protocols and multi‑class background traffic scenarios.