Studying the Effect of Schedule Preemption on Dynamic Task Graph Scheduling

Dynamic scheduling of task graphs is often addressed without revisiting prior task allocations, with a primary focus on minimizing makespan. We study controlled schedule preemption, introducing the Last-K Preemption model, which selectively reschedules recent task graphs while preserving earlier allocations. Using synthetic, RIoTBench, WFCommons, and adversarial workloads, we compare preemptive, non-preemptive, and partial-preemptive strategies across makespan, fairness, utilization, and runtime. Results show moderate preemption can match most makespan and utilization gains of full preemption while maintaining fairness and low overhead.

💡 Research Summary

The paper tackles the problem of scheduling multiple directed‑acyclic graphs (DAGs) that arrive over time on a heterogeneous, possibly resource‑constrained network. Traditional dynamic schedulers either (i) fully preempt – they discard all pending allocations whenever a new DAG arrives and recompute a schedule for the entire pending set, or (ii) are non‑preemptive – they keep all existing allocations fixed and only place the new DAG on whatever resources remain idle. Full preemption can dramatically reduce the overall makespan because tasks can be reshuffled to better exploit fast nodes or reduce communication, but it often postpones small tasks that have already started, leading to poor fairness and high scheduling overhead. Non‑preemptive scheduling preserves fairness and incurs negligible runtime cost, yet it can suffer severe makespan inflation when a large root task blocks many short successors, a situation the authors illustrate with an adversarial out‑tree workload.

To bridge this gap the authors introduce the “Last‑K Preemption” model. When a new DAG arrives, only the most recent K DAGs are marked as “preemptable”; all older DAGs retain their current assignments. The value K thus interpolates between the two extremes (K = 0 yields a pure non‑preemptive policy, K = ∞ yields full preemption). The model is simple to implement on top of existing list‑scheduling heuristics such as HEFT, CPOP, Min‑Min, Max‑Min, and random priority variants.

The experimental methodology is comprehensive. Four workload families are used: (1) synthetic DAGs with four canonical topologies (out‑tree, in‑tree, fork‑join, chain) and Gaussian‑distributed computation/communication weights; (2) RIoTBench pipelines (ETL, Predict, Stats, Train) that reflect realistic IoT stream processing; (3) WFCommons scientific workflows (e.g., Epigenomics, Seismology, Blast) that contain long critical paths and heterogeneous communication patterns; and (4) an adversarial out‑tree instance with a heavy root and many shallow children, deliberately set with a low communication‑to‑computation ratio (CCR = 0.2) to provoke excessive task scattering.

Five metrics are evaluated: total makespan, mean makespan (average per‑DAG completion time minus arrival time), mean flowtime (average intra‑DAG span between first start and last finish), node utilization (fraction of compute capacity actually used), and scheduler runtime (time spent computing the schedule).

Results show that full preemptive schedulers (e.g., P‑HEFT, P‑CPOP) consistently achieve the smallest total makespan across all datasets, confirming the theoretical advantage of unrestricted reshuffling. However, they also exhibit the highest mean flowtime, indicating that many small tasks are repeatedly delayed. Non‑preemptive schedulers (NP‑HEFT, NP‑CPOP) have low mean flowtime but suffer dramatically on the adversarial workload, where their makespan is 1.6 × that of the preemptive baseline. Partially preemptive variants (5P‑HEFT, 10P‑CPOP, etc.) strike a balance: with K ≈ 5–10 they attain makespans within a few percent of the fully preemptive optimum while keeping mean flowtime close to the non‑preemptive level. Utilization gains from preemption are modest when the system is already well‑utilized (>85 %); thus the marginal benefit of full preemption diminishes under high load. Scheduler runtime grows with K but remains acceptable for K ≤ 10, suggesting that partial preemption is viable for real‑time or high‑arrival‑rate scenarios.

The authors conclude that limiting the preemption horizon provides a practical design point that simultaneously improves throughput, preserves fairness, and controls overhead. They advocate future work on adaptive K selection based on observed load, DAG characteristics, or QoS constraints, and on integrating the model into production edge‑cloud orchestration frameworks.