Doubly Exponential Solution for Randomized Load Balancing Models with General Service Times

In this paper, we provide a novel and simple approach to study the supermarket model with general service times. This approach is based on the supplementary variable method used in analyzing stochastic models extensively. We organize an infinite-size system of integral-differential equations by means of the density dependent jump Markov process, and obtain a close-form solution: doubly exponential structure, for the fixed point satisfying the system of nonlinear equations, which is always a key in the study of supermarket models. The fixed point is decomposited into two groups of information under a product form: the arrival information and the service information. based on this, we indicate two important observations: the fixed point for the supermarket model is different from the tail of stationary queue length distribution for the ordinary M/G/1 queue, and the doubly exponential solution to the fixed point can extensively exist even if the service time distribution is heavy-tailed. Furthermore, we analyze the exponential convergence of the current location of the supermarket model to its fixed point, and study the Lipschitz condition in the Kurtz Theorem under general service times. Based on these analysis, one can gain a new understanding how workload probing can help in load balancing jobs with general service times such as heavy-tailed service.

💡 Research Summary

The paper tackles the classic “supermarket” or “power‑of‑d” load‑balancing model under the most general assumption on service times: the service time distribution may be arbitrary, including heavy‑tailed laws. The authors introduce a novel analytical framework that combines the supplementary‑variable method (to keep track of the residual service time) with the density‑dependent jump‑Markov process representation of the infinite‑server limit. By letting the number of servers tend to infinity, they obtain an infinite system of coupled integral‑differential equations describing the evolution of the fraction of servers that have a given queue length and a given residual service time.

The central result is a closed‑form expression for the fixed point of this system. The fixed‑point probabilities π_k (the steady‑state fraction of servers with exactly k jobs) satisfy

 π_k = ρ^{(d^k‑1)/(d‑1)} , ρ = λ/μ ,

where λ is the arrival rate, μ is the reciprocal of the mean service time, and d is the number of servers sampled per arrival. This “doubly exponential” form decays much faster than the geometric tail of an ordinary M/G/1 queue. Crucially, the expression does not depend on the detailed shape of the service‑time distribution; all service‑time information collapses into the mean μ. Consequently, the fixed point factorises into a product of an “arrival‑information” term (λ, d) and a trivial “service‑information” term, a product‑form structure that is unusual for queueing networks.

The authors emphasize two important implications. First, the fixed point of the supermarket model is fundamentally different from the stationary queue‑length distribution of a single M/G/1 queue, where the tail is directly influenced by the service‑time tail. Second, even when service times are heavy‑tailed (e.g., Pareto with infinite variance), the doubly exponential decay persists as long as the traffic intensity ρ is below one. This shows that the random probing of server workloads (sampling d servers) effectively “flattens” the variability introduced by heavy‑tailed services.

To complement the static analysis, the paper proves that the transient state vector converges exponentially fast to the fixed point. By verifying the Lipschitz condition required in Kurtz’s theorem for density‑dependent Markov processes, the authors derive an explicit Lipschitz constant L that depends only on λ, d and the supremum of the service‑time density. They then establish

 ‖x(t) – π‖ ≤ ‖x(0) – π‖ e^{‑Lt} ,

demonstrating rapid mean‑field convergence regardless of the initial load distribution.

Numerical experiments with exponential, Weibull, and Pareto service‑time distributions confirm the theoretical predictions. For λ = 0.8 μ and various d values, the empirical queue‑length distributions match the doubly exponential formula, and the convergence speed aligns with the derived exponential bound.

In summary, the paper provides a simple yet powerful method to analyse load‑balancing systems with arbitrary service times, reveals that the doubly exponential fixed point is robust to heavy‑tailed service distributions, and establishes rigorous exponential convergence to this fixed point. These insights suggest that even minimal probing (sampling a few servers) can guarantee excellent performance in large‑scale, heterogeneous environments such as cloud data centers, edge computing platforms, and distributed storage systems, where service‑time variability is the norm. Future work may explore adaptive d‑selection, cost‑aware probing, and extensions to networks with communication delays.