A strong law for the rate of growth of long latency periods in cloud computing service

Cloud-computing shares a common pool of resources across customers at a scale that is orders of magnitude larger than traditional multi-user systems. Constituent physical compute servers are allocated multiple “virtual machines” (VM) to serve simultaneously. Each VM user should ideally be unaffected by others’ demand. Naturally, this environment produces new challenges for the service providers in meeting customer expectations while extracting an efficient utilization from server resources. We study a new cloud service metric that measures prolonged latency or delay suffered by customers. We model the workload process of a cloud server and analyze the process as the customer population grows. The capacity required to ensure that average workload does not exceed a threshold over long segments is characterized. This can be used by cloud operators to provide service guarantees on avoiding long durations of latency. As part of the analysis, we provide a uniform large-deviation principle for collections of random variables that is of independent interest.

💡 Research Summary

The paper addresses a critical performance metric for cloud computing services: the duration of prolonged latency periods experienced by customers. In a typical cloud environment, a physical server hosts many virtual machines (VMs), each serving independent workloads. As the number of customers (or VMs) grows, the aggregate workload becomes a stochastic process whose statistical properties can be analyzed in the large‑scale limit. The authors model the per‑VM workload as independent random variables with common mean μ and variance σ², and define the total workload W_N(t) for N VMs as the sum of these variables. A “long latency period” is defined as a contiguous time interval during which the average workload exceeds a pre‑specified threshold θ. The central question is how the maximal length L_N of such intervals scales with N.

To answer this, the authors develop a uniform large‑deviation principle (ULDP) that applies simultaneously to collections of random variables indexed by N. Traditional large‑deviation results provide asymptotic probabilities for a fixed N, but ULDP guarantees that the same rate function I(x) governs the exponential decay of tail probabilities for all N, enabling the use of Borel‑Cantelli arguments across the entire sequence. Using ULDP, they prove a strong law for the growth rate of L_N: with probability one, \