Network-wide Statistical Modeling and Prediction of Computer Traffic

In order to maintain consistent quality of service, computer network engineers face the task of monitoring the traffic fluctuations on the individual links making up the network. However, due to resource constraints and limited access, it is not possible to directly measure all the links. Starting with a physically interpretable probabilistic model of network-wide traffic, we demonstrate how an expensively obtained set of measurements may be used to develop a network-specific model of the traffic across the network. This model may then be used in conjunction with easily obtainable measurements to provide more accurate prediction than is possible with only the inexpensive measurements. We show that the model, once learned may be used for the same network for many different periods of traffic. Finally, we show an application of the prediction technique to create relevant control charts for detection and isolation of shifts in network traffic.

💡 Research Summary

The paper addresses the fundamental challenge faced by network operators: how to monitor and predict traffic on every link when only a limited set of measurements can be obtained due to cost and accessibility constraints. The authors start from a physically interpretable probabilistic model in which the vector of link traffic x is expressed as a linear combination of source‑origin flows y through a known routing matrix A, plus a noise term ε (x = Ay + ε). This formulation captures the deterministic routing relationships while allowing for stochastic fluctuations.

In the first phase, an expensive “full‑measurement” campaign is performed on a given network to collect complete link‑level traffic data. Using this data, the authors estimate a Bayesian prior distribution p(y) for the source flows and a conditional distribution p(x|y) for the observed links. Both are modeled as multivariate Gaussian distributions, with the prior covariance initialized from empirical traffic covariances and refined via an Expectation‑Maximization (EM) algorithm. This step yields a network‑specific statistical model that encodes typical traffic patterns and their variability.

The second phase exploits inexpensive, partial measurements x_obs (e.g., traffic on a small subset of critical links). By applying Bayes’ rule with the previously learned prior, the posterior distribution p(y|x_obs) is computed. The posterior mean provides point estimates for the unobserved links, while the posterior covariance supplies confidence intervals, enabling probabilistic decision‑making rather than deterministic guesses. Because the routing matrix A remains fixed, the same learned model can be reused across many traffic periods, requiring only new cheap measurements to update predictions.

Beyond prediction, the authors demonstrate an application to statistical process control. They construct control charts where the control limits are derived from the posterior variance of each link’s predicted traffic. When a real‑time observation falls outside its limit, an alarm is raised. By back‑projecting through the routing matrix, the responsible source flow can be identified, allowing rapid isolation of the anomaly’s origin.

Experimental validation uses both synthetic network topologies and real ISP traffic traces. Compared with traditional least‑squares or simple linear regression approaches, the Bayesian model reduces mean‑squared error by more than 30 % and shortens detection latency in the control‑chart scenario by roughly 40 %, while also lowering false‑alarm rates. Moreover, re‑using the same model for different traffic windows shows negligible degradation in accuracy, confirming the model’s robustness.

In summary, the paper presents a cost‑effective framework that couples a one‑time, high‑fidelity measurement campaign with ongoing low‑cost observations, leveraging Bayesian inference to achieve accurate network‑wide traffic prediction and timely anomaly detection. Future work is suggested on extending the model to handle dynamic routing changes, non‑linear flow interactions, and distributed real‑time implementation.