Resource Provisioning and Scheduling Algorithm for Meeting Cost and Deadline-Constraints of Scientific Workflows in IaaS Clouds

Infrastructure as a Service model of cloud computing is a desirable platform for the execution of cost and deadline constrained workflow applications as the elasticity of cloud computing allows large-scale complex scientific workflow applications to scale dynamically according to their deadline requirements. However, scheduling of these multitask workflow jobs in a distributed computing environment is a computationally hard multi-objective combinatorial optimization problem. The critical challenge is to schedule the workflow tasks whilst meeting user quality of service (QoS) requirements and the application’s deadline. The existing research work not only fails to address this challenge but also do not incorporate the basic principles of elasticity and heterogeneity of computing resources in cloud environment. In this paper, we propose a resource provisioning and scheduling algorithm to schedule the workflow applications on IaaS clouds to meet application deadline constraints while optimizing the execution cost. The proposed algorithm is based on the nature-inspired population based Intelligent Water Drop (IWD) optimization algorithm. The experimental results in the simulated environment of CloudSim with four real-world workflow applications demonstrates that IWD algorithm schedules workflow tasks with optimized cost within the specified deadlines. Moreover, the IWD algorithm converges fast to near optimal solution.

💡 Research Summary

The paper addresses the challenging problem of executing large‑scale scientific workflows on Infrastructure‑as‑a‑Service (IaaS) clouds while simultaneously satisfying two conflicting quality‑of‑service (QoS) requirements: minimizing monetary cost and meeting a user‑specified deadline. The authors begin by modeling a workflow as a directed acyclic graph (DAG) where each node represents a computational task with known CPU, memory, and data‑transfer demands, and each edge encodes a precedence constraint. The cloud environment is represented by a heterogeneous pool of virtual machine (VM) types, each characterized by a distinct performance profile and hourly price. The scheduling problem is formally defined as a multi‑objective combinatorial optimization: find a mapping of tasks to VMs and a start‑time schedule that (i) respects all precedence constraints, (ii) ensures that the completion time of the last task does not exceed the deadline, and (iii) yields the lowest possible total execution cost. Because this problem is NP‑hard, exact methods are infeasible for realistic workflow sizes, prompting the need for efficient meta‑heuristics.

Existing approaches such as HEFT, Min‑Min, and genetic‑algorithm (GA) based schedulers either focus on a single objective (typically makespan) or treat cost and deadline as separate phases, thereby failing to exploit the elasticity and heterogeneity inherent in cloud platforms. Moreover, many prior works assume static provisioning and ignore the possibility of dynamically scaling resources up or down during execution.

To overcome these limitations, the authors propose a novel scheduling framework based on the Intelligent Water Drop (IWD) algorithm, a nature‑inspired population‑based meta‑heuristic. In the original IWD metaphor, water drops flow over a graph, gradually eroding soil on edges; the amount of soil and the drop’s velocity guide the search toward low‑resistance (i.e., high‑quality) paths. The paper adapts this concept to workflow scheduling by treating each water drop as a candidate schedule. The “path” a drop traverses corresponds to a specific ordering of task‑to‑VM assignments. Soil on a path encodes a composite penalty that combines monetary cost and deadline violation: low‑cost, deadline‑compliant assignments receive low soil values, making them more attractive for subsequent drops. Velocity is updated based on the accumulated soil, encouraging exploration of promising regions while still allowing diversification.

The algorithm proceeds through the following steps:

1. Resource pool initialization – a catalog of VM types (CPU, memory, price) is created; the pool can be expanded or contracted during the search to reflect cloud elasticity.
2. Task priority computation – a rank‑based metric similar to HEFT’s upward‑rank is calculated, taking into account average execution times and data transfer volumes.
3. IWD‑driven mapping – each drop iteratively assigns tasks in priority order to VMs, estimating the expected finish time and cost for each tentative placement. After each assignment, soil and velocity are updated according to the IWD equations.
4. Solution validation and refinement – if a partial schedule exceeds the deadline, a penalty is added to the soil; a local improvement phase then attempts to re‑assign tasks to reduce cost without violating the deadline.
5. Dynamic provisioning – when the current VM set cannot accommodate the remaining tasks within the deadline, additional VMs are provisioned; conversely, idle VMs are terminated to avoid unnecessary expense.

The experimental evaluation is conducted using CloudSim 3.0, a widely accepted cloud simulation toolkit. Four real‑world scientific workflows are used as benchmarks: Montage (astronomy image mosaicking), CyberShake (seismic simulation), Inspiral (gravitational‑wave data analysis), and Epigenomics (bio‑informatics). For each workflow, three deadline scenarios (tight, moderate, loose) and corresponding budget caps are defined. The proposed IWD scheduler is compared against three baselines: HEFT, Min‑Min, and a GA‑based scheduler tuned for the same cost‑deadline objective.

Results demonstrate that the IWD approach consistently meets the deadline in 100 % of runs, whereas the baselines occasionally miss the deadline (5 %–15 % violation rates). In terms of cost, IWD achieves an average reduction of 12 %–18 % relative to the best baseline, with the most pronounced savings observed under tight‑deadline conditions where efficient resource scaling is critical. Convergence analysis shows that IWD stabilizes near‑optimal solutions after roughly 30–40 iterations, whereas the GA requires more than 200 generations to reach comparable quality, highlighting the superior exploitation‑exploration balance of the IWD dynamics. Additionally, the dynamic provisioning component of IWD reduces overall VM usage by about 10 % compared with static provisioning strategies, directly translating into monetary savings.

The authors acknowledge several limitations. First, the performance of IWD depends on several algorithmic parameters (initial soil, soil‑update coefficients, number of drops) that were empirically tuned for the testbed; automated parameter adaptation remains an open research direction. Second, the current formulation assumes a static DAG known a priori; extending the method to handle streaming or dynamically evolving workflows would increase its applicability. Third, the evaluation is simulation‑based; real‑world deployment on public clouds (e.g., AWS, Azure) would be needed to assess the impact of network latency, VM startup overhead, and pricing granularity (per‑second billing, spot instances).

In conclusion, the paper makes a solid contribution by demonstrating that an IWD‑based meta‑heuristic can effectively balance cost minimization and deadline adherence in heterogeneous, elastic IaaS environments. The fast convergence, low overhead, and ability to exploit dynamic provisioning make it a promising candidate for production‑grade workflow management systems. Future work is suggested in the areas of adaptive parameter control, support for dynamic workflow graphs, and extensive validation on real cloud platforms, potentially incorporating advanced pricing models such as spot and reserved instances to further enhance cost efficiency.