Heavy Traffic Optimal Resource Allocation Algorithms for Cloud Computing Clusters

Cloud computing is emerging as an important platform for business, personal and mobile computing applications. In this paper, we study a stochastic model of cloud computing, where jobs arrive according to a stochastic process and request resources like CPU, memory and storage space. We consider a model where the resource allocation problem can be separated into a routing or load balancing problem and a scheduling problem. We study the join-the-shortest-queue routing and power-of-two-choices routing algorithms with MaxWeight scheduling algorithm. It was known that these algorithms are throughput optimal. In this paper, we show that these algorithms are queue length optimal in the heavy traffic limit.

💡 Research Summary

The paper addresses the problem of dynamically allocating multiple resource types (CPU, memory, storage, etc.) to stochastic job arrivals in a cloud‑computing cluster. The authors model the system as a network of queues where each job carries a resource‑demand vector and servers have limited capacities for each resource. They separate the allocation problem into two stages: routing (or load balancing) and scheduling.

For routing, two well‑known policies are examined: Join‑the‑Shortest‑Queue (JSQ) and Power‑of‑Two‑Choices (P2C). JSQ assigns an incoming job to the server with the smallest total queue length, while P2C randomly samples two servers and routes the job to the one with the shorter queue. Both policies are known to spread load evenly and to be throughput‑optimal when combined with a suitable scheduler.

The scheduling stage uses the MaxWeight algorithm. At each discrete time slot MaxWeight selects a feasible service configuration that maximizes the weighted sum of queue lengths, where the weight of each queue is proportional to its current resource demand. This policy has been proved to be throughput‑optimal for a broad class of stochastic networks.

The novel contribution of the paper is to prove that the JSQ‑MaxWeight and P2C‑MaxWeight combinations are also queue‑length optimal in the heavy‑traffic regime—that is, when the arrival rates approach the boundary of the capacity region. The authors adopt a diffusion‑approximation framework. They first scale the queue‑length process by the square root of the distance to the capacity boundary and show that the scaled process converges to a reflected Brownian motion. Crucially, they demonstrate a state‑space collapse: under both routing policies the differences among individual server queues remain bounded (order‑one) while the total workload grows, which reduces the high‑dimensional system to a one‑dimensional diffusion.

Using a quadratic Lyapunov function and martingale arguments, they derive tight upper bounds on the steady‑state expected total queue length. These bounds match the lower bounds obtained from the limiting diffusion, establishing that the policies achieve the minimal possible delay scaling (Θ(1/ε) where ε measures the distance to capacity). Hence, the algorithms are not only throughput‑optimal but also heavy‑traffic optimal in terms of queue length.

To validate the theory, extensive simulations are performed. The experiments vary arrival rates, job‑size distributions, and the number of servers. Results confirm that as the system approaches heavy traffic, both JSQ‑MaxWeight and P2C‑MaxWeight attain average queue lengths and delays that are indistinguishable from the theoretical optimum. Notably, P2C‑MaxWeight, which requires far less state information than JSQ, delivers virtually the same performance, highlighting its practical appeal.

The paper concludes with a discussion of practical implications. Cloud data‑center operators need policies that remain robust under sudden traffic spikes. The demonstrated heavy‑traffic optimality guarantees minimal latency even when the system is heavily loaded, and the low‑complexity P2C‑MaxWeight scheme offers an implementable solution with provable performance guarantees. Overall, the work extends the theoretical foundation of stochastic network control by bridging throughput optimality and delay optimality in the heavy‑traffic limit for multi‑resource cloud environments.