Resource and Application Models for Advanced Grid Schedulers

As Grid computing is becoming an inevitable future, managing, scheduling and monitoring dynamic, heterogeneous resources will present new challenges. Solutions will have to be agile and adaptive, support self-organization and autonomous management, while maintaining optimal resource utilisation. Presented in this paper are basic principles and architectural concepts for efficient resource allocation in heterogeneous Grid environment.

💡 Research Summary

The paper addresses the growing need for scalable, heterogeneous resource management in modern scientific and business computing environments. Recognizing that existing Grid infrastructures—particularly those built around the Globus Toolkit and a centralized Meta Computing Directory Service (MDS)—suffer from scalability, administration, and resilience limitations, the authors propose a comprehensive model that separates the profiling of computational nodes from the profiling of applications.

Node profiling is performed in two layers. A static, non‑volatile XML profile records immutable attributes such as operating system, installed libraries, hardware accelerators, and total memory. A dynamic, volatile profile is refreshed at regular intervals and captures real‑time metrics (CPU load, memory usage, network latency). To quantify raw computational power, the model adopts the SPEC® cpu2000 benchmark suite, converting each node’s performance into a SPEC score that is comparable across platforms. This choice balances credibility, availability of reference data, and ease of deployment.

Application profiling follows a similar two‑phase approach. Each application receives a unique hash‑based identifier. During its first execution the system gathers execution time, memory footprint, I/O patterns, and other resource demands. Subsequent runs update a statistical model that predicts the probability of meeting a given deadline under specified resource constraints. The resulting XML profile is continuously refined, providing a confidence interval for future runs.

Scheduling proceeds in two stages. First, non‑volatile requirements (OS, libraries, minimum memory, special hardware) are used to prune the candidate node set. Second, the Self‑Organized Resource Discovery (SORD) protocol performs a distributed query among the remaining nodes. Each node replies with a “bid” composed of two elements: (1) its “subscribed load,” defined as the sum of SPEC points required by currently running jobs and their due times, and (2) a statistically derived confidence level that the node can meet the requested turnaround time. The “subscribed load” concept replaces naïve instantaneous CPU utilization metrics with a forward‑looking estimate of available capacity, thereby accommodating soft‑limiting OS schedulers that cannot guarantee exclusive CPU slices.

Monitoring is organized into three tiers to balance accuracy, overhead, and security. At the top tier, a directory service similar to MDS stores aggregated accounting data for SLA enforcement and policy compliance; updates occur only at job admission, completion, or policy change, minimizing traffic. The middle tier employs lightweight probing tools such as Ganglia and the Network Weather Service (NWS) to broadcast per‑node state (CPU, memory, network) every few seconds; this data directly influences the SORD bid calculation. The bottom tier runs a Java‑based Integrity, Intelligence, and Information (I³) agent on each node, providing low‑latency, high‑fidelity monitoring of process behavior to detect anomalies or malicious activity.

By integrating these layers, the authors claim to achieve a self‑organizing, self‑healing scheduling environment that can adapt to dynamic resource availability while preserving high utilization and respecting Service Level Agreements. The paper contrasts its approach with earlier efforts such as RISC‑cycle models and AppLeS, emphasizing reduced user interaction, platform independence, and statistical robustness.

Future work outlined includes implementing the proposed meta‑scheduler on Globus Toolkit 3, deploying it in a production Grid supporting multiple e‑Science projects, and extending the performance model to incorporate memory bandwidth, I/O, and network characteristics. The authors anticipate that the combination of SPEC‑based node scoring, application statistical profiling, the “subscribed load” bidding mechanism, and the three‑tier monitoring architecture will provide a solid foundation for next‑generation, large‑scale Grid environments.