Dynamic task scheduling in computing cluster environments

In this study, a cluster-computing environment is employed as a computational platform. In order to increase the efficiency of the system, a dynamic task scheduling algorithm is proposed, which balances the load among the nodes of the cluster. The technique is dynamic, nonpreemptive, adaptive, and it uses a mixed centralised and decentralised policies. Based on the divide and conquer principle, the algorithm models the cluster as hyper-grids and then balances the load among them. Recursively, the hyper-grids of dimension k are divided into grids of dimensions k - 1, until the dimension is 1. Then, all the nodes of the cluster are almost equally loaded. The optimum dimension of the hyper-grid is chosen in order to achieve the best performance. The simulation results show the effective use of the algorithm. In addition, we determined the critical points (lower bounds) in which the algorithm can to be triggered.

💡 Research Summary

The paper addresses the persistent problem of load imbalance in large‑scale computing clusters by introducing a novel scheduling framework that blends centralized control with decentralized execution. The authors model the entire cluster as a multi‑dimensional “hyper‑grid” and apply a recursive divide‑and‑conquer strategy: a hyper‑grid of dimension k is split into sub‑grids of dimension k‑1, and this process continues until the grid reduces to one‑dimensional slices that correspond to individual nodes. At each recursion level the algorithm measures the current load of the nodes within the sub‑grid, then transfers work from the most loaded node to the least loaded one, thereby progressively evening out the workload.

Key characteristics of the proposed method are its dynamism, non‑preemptive nature, and adaptivity. “Dynamic” means the scheduler reacts to the arrival of new jobs or the completion of existing ones without requiring a global pause; “non‑preemptive” ensures that once a job starts on a node it runs to completion, eliminating the overhead associated with job interruption and migration. “Adaptive” refers to the runtime selection of the optimal hyper‑grid dimension based on a cost model that incorporates current load distribution, network bandwidth, and heterogeneous node capabilities. A higher dimension yields finer granularity of load balancing but incurs greater communication overhead; the algorithm automatically finds the sweet spot for each workload scenario.

The authors also derive theoretical lower‑bound conditions—referred to as “critical points”—that indicate when the algorithm should be triggered. If the overall system load stays below a certain threshold, a simpler static scheduler may be more efficient; the proposed method is activated only when load spikes or severe imbalance are detected, thereby avoiding unnecessary rebalancing costs.

Simulation experiments were conducted across a range of cluster sizes (from a few hundred to several thousand nodes) and job mixes (short compute‑bound tasks, long CPU‑intensive tasks, and I/O‑heavy workloads). Results show that the hyper‑grid scheduler consistently outperforms both pure centralized and pure decentralized baselines in terms of throughput and resource utilization. Even under high network latency, the algorithm maintains effective load distribution, confirming that the communication overhead introduced by the recursive splitting is modest compared with the gains in balance. Moreover, the automatic dimension‑selection mechanism proved robust: it adjusted the hyper‑grid depth according to cluster scale and job characteristics, achieving near‑optimal performance without manual tuning.

In summary, the study demonstrates that a hyper‑grid‑based, dynamic, non‑preemptive, and adaptive scheduling algorithm can substantially mitigate load imbalance in contemporary cluster environments, leading to higher overall efficiency. The authors suggest future work that includes deployment on real hardware, integration of energy‑aware policies, and extension of the framework to handle fault tolerance and quality‑of‑service guarantees.