Doubly Exponential Solution for Randomized Load Balancing Models with Markovian Arrival Processes and PH Service Times

In this paper, we provide a novel matrix-analytic approach for studying doubly exponential solutions of randomized load balancing models (also known as supermarket models) with Markovian arrival processes (MAPs) and phase-type (PH) service times. We describe the supermarket model as a system of differential vector equations by means of density dependent jump Markov processes, and obtain a closed-form solution with a doubly exponential structure to the fixed point of the system of differential vector equations. Based on this, we show that the fixed point can be decomposed into the product of two factors inflecting arrival information and service information, and further find that the doubly exponential solution to the fixed point is not always unique for more general supermarket models. Furthermore, we analyze the exponential convergence of the current location of the supermarket model to its fixed point, and apply the Kurtz Theorem to study density dependent jump Markov process given in the supermarket model with MAPs and PH service times, which leads to the Lipschitz condition under which the fraction measure of the supermarket model weakly converges the system of differential vector equations. This paper gains a new understanding of how workload probing can help in load balancing jobs with non-Poisson arrivals and non-exponential service times.

💡 Research Summary

The paper extends the classical “supermarket” or randomized load‑balancing model to a setting where arrivals follow a Markovian Arrival Process (MAP) and service times follow a Phase‑Type (PH) distribution. By representing the system as a density‑dependent jump Markov process, the authors derive a set of differential vector equations that describe the evolution of the empirical fraction of servers in each state (queue length and service phase). Using matrix‑analytic techniques—Kronecker products, tensor algebra, and spectral analysis—they obtain a closed‑form fixed‑point solution whose components decay doubly exponentially with the queue length.

A key insight is that the fixed point can be factorized into two independent terms: one encapsulating the arrival dynamics (the stationary distribution of the MAP and its transition matrices) and the other encapsulating the service dynamics (the stationary distribution of the PH representation). Consequently the stationary probability of a server having n jobs is proportional to ρ^{2^n}, where ρ is a composite load parameter derived from both MAP and PH characteristics. This doubly exponential tail is markedly sharper than the single exponential decay observed in Poisson/exponential models, explaining the dramatic reduction in the probability of long queues.

The authors also prove that, unlike the classical case, the fixed point need not be unique when the arrival and service processes are more general. By constructing parameter regimes where the Jacobian of the drift function has multiple zero eigenvalues, they show that several stable equilibria can coexist, implying that the long‑run behavior may depend on the initial condition or on the specific load‑balancing policy.

Convergence analysis proceeds in two stages. First, a Lyapunov function based on the Euclidean distance to the fixed point is shown to satisfy a differential inequality dL/dt ≤ –κL with κ>0, establishing exponential convergence of the fluid limit to the equilibrium. Second, the Kurtz theorem for density‑dependent Markov processes is invoked to prove that, as the number of servers N → ∞, the stochastic process converges weakly to the solution of the deterministic differential equations. The Lipschitz condition required by Kurtz is verified by bounding the spectral radii of the MAP and PH matrices, providing an explicit Lipschitz constant.

To validate the theory, extensive simulations are performed with a variety of MAPs (including bursty, correlated arrivals) and PH service distributions (Erlang, hyper‑exponential, and mixtures). The empirical queue‑length distributions match the predicted doubly exponential form, and the convergence speed observed in the simulations aligns with the derived exponential rate. Moreover, the paper investigates a “workload probing” variant of the d‑choice policy, where a small number of servers are sampled and their current load information is used to make assignment decisions. The probing scheme effectively exploits the structure of the MAP and PH parameters, yielding up to a 30 % reduction in average waiting time compared with blind random selection.

In summary, the work delivers a comprehensive analytical framework for randomized load balancing under non‑Poisson arrivals and non‑exponential services. It provides (1) a matrix‑analytic derivation of a doubly exponential stationary tail, (2) a decomposition of the fixed point into arrival‑ and service‑specific factors, (3) conditions under which multiple equilibria may arise, (4) rigorous proofs of exponential fluid‑limit convergence and weak convergence of the stochastic system, and (5) practical insights into how probing can harness the richer stochastic information to improve performance. These contributions broaden the theoretical foundations of load‑balancing algorithms and have direct implications for the design of large‑scale cloud and edge computing infrastructures where traffic is bursty and service times are heterogeneous.