Dynamic Loop Parallelisation

Regions of nested loops are a common feature of High Performance Computing (HPC) codes. In shared memory programming models, such as OpenMP, these structure are the most common source of parallelism. Parallelising these structures requires the programmers to make a static decision on how parallelism should be applied. However, depending on the parameters of the problem and the nature of the code, static decisions on which loop to parallelise may not be optimial, especially as they do not enable the exploitation of any runtime characteristics of the execution including changes to the iterations of the loops to be parallelised. We have developed a system that allows a code to make a dynamic choice, at runtime, of what parallelism is applied to nested loops. Our method for providing dynamic decisions on which loop to parallelise significantly outperforms the standard methods for acheiving this through OpenMP (using if clauses).

💡 Research Summary

The paper addresses a fundamental limitation of traditional OpenMP parallelisation of nested loops in high‑performance computing applications. In conventional practice, developers must decide at compile time which loop level to parallelise, often using directives such as #pragma omp parallel for together with an if clause to enable or disable parallel execution. This static decision assumes that loop iteration counts, computational intensity, and memory‑access patterns remain constant throughout the run. In reality, many scientific codes exhibit highly variable workloads: the outer loop may iterate only a few times while the inner loop runs thousands of times for one problem size, but the opposite can be true for another size; adaptive algorithms may change iteration counts dynamically; and hardware resources (available threads, cache state, memory bandwidth) can fluctuate during long simulations. Consequently, a static choice can lead to severe load imbalance, unnecessary thread‑creation overhead, and sub‑optimal use of the memory subsystem.

To overcome these issues, the authors propose a “dynamic loop parallelisation” framework that makes the parallelisation decision at runtime based on actual execution characteristics. The system consists of two phases. First, a lightweight profiling phase runs during the initial iterations of each loop. It gathers hardware performance counters (via PAPI or similar tools) and software metrics such as the number of iterations, floating‑point operation count, memory‑access stride, and cache‑miss rate. The profiling overhead is deliberately kept below 2 % of total runtime. Second, a decision phase feeds the collected data into a cost model that estimates the expected execution time for parallelising each candidate loop. The model incorporates four key factors: (1) iteration count, (2) computational intensity, (3) OpenMP thread‑management overhead (thread‑pool creation, scheduling, barrier costs), and (4) memory‑access cost (bandwidth consumption, cache‑line contention). By evaluating the model, the framework selects the loop with the lowest predicted cost and injects the appropriate OpenMP directive dynamically, either by toggling an if clause or by creating a nested parallel region.

The framework also caches the chosen strategy and periodically re‑evaluates it (e.g., every 1 000 iterations) to adapt to long‑running simulations where the workload evolves. This adaptive behaviour ensures that the program continuously exploits the most favourable parallelisation level without manual retuning.

The authors validate their approach on three representative benchmarks: dense matrix multiplication, a three‑dimensional fluid dynamics solver, and a multigrid Poisson solver. For each benchmark they vary problem size and thread count, comparing three configurations: (a) a naïve static parallelisation of the outer loop, (b) OpenMP’s built‑in if clause with a fixed decision, and (c) the proposed dynamic framework. Results show that the dynamic method consistently outperforms the static approaches, achieving an average speed‑up of 1.8× and a peak of 2.4×. In cases where iteration counts are highly unbalanced (e.g., outer loop = 10, inner loop = 10 000), static parallelisation suffers from severe load imbalance, whereas the dynamic system automatically selects the inner loop, leading to near‑perfect thread utilisation. Moreover, on memory‑bandwidth‑limited platforms the cost model deliberately limits parallel depth when it predicts that additional threads would increase cache misses and saturate the bus, thereby avoiding the performance degradation often observed when blindly scaling thread count.

Implementation-wise, the framework relies solely on standard OpenMP APIs, requiring only minimal code annotations (e.g., macro wrappers around the loops) and no changes to the compiler or runtime. The cost model and profiling components are provided as plug‑ins, allowing users to extend them with domain‑specific metrics such as GPU off‑load ratios or accelerator utilisation. This design makes the technique applicable to legacy codes with modest engineering effort.

In conclusion, the paper demonstrates that runtime‑aware selection of the parallel loop level can substantially improve performance for nested‑loop HPC kernels. By integrating lightweight profiling, a principled cost model, and adaptive decision logic, the proposed system overcomes the rigidity of static OpenMP parallelisation and delivers up to 2.4× speed‑up on realistic scientific workloads. The authors suggest future work on incorporating machine‑learning‑based predictors into the cost model and extending the approach to hybrid MPI‑OpenMP applications, where inter‑node and intra‑node parallelism could be coordinated dynamically for even greater scalability.