A Multilevel Approach for the Performance Analysis of Parallel Algorithms

We provide a multilevel approach for analysing performances of parallel algorithms. The main outcome of such approach is that the algorithm is described by using a set of operators which are related to each other according to the problem decomposition. Decomposition level determines the granularity of the algorithm. A set of block matrices (decomposition and execution) highlights fundamental characteristics of the algorithm, such as inherent parallelism and sources of overheads.

💡 Research Summary

The paper introduces a systematic multilevel framework for analyzing the performance of parallel algorithms, bridging the gap between problem decomposition, algorithmic structure, and hardware execution. At its core, the authors model a parallel algorithm as a finite ordered set of operators that solve a computational problem. The relationships among these operators—derived from data dependencies inherent in the problem decomposition—are captured in a dependency group, a mathematical construct equipped with a strict partial order. This group is represented as a decomposition matrix (M_D) whose rows correspond to dependency depth and columns to concurrency degree. A non‑trivial column count (c_D > 1) indicates intrinsic parallelism, while the row count (r_D) reflects the longest chain of dependent sub‑problems.

To map the algorithm onto a concrete machine, the paper defines a computing platform M_P consisting of P processing elements and a set of logical‑operational capabilities (operators I_j). Each operator has an associated execution time t_j. The mapping from algorithm operators to platform operators yields an execution matrix (M_E), whose rows encode execution order and whose columns encode simultaneously executable operators. This matrix makes explicit the scheduling constraints imposed by both data dependencies and resource availability.

Algorithmic complexity is defined as the cardinality of the operator set, which coincides with the number of non‑empty entries in the decomposition matrix. Consequently, algorithms that share the same problem decomposition belong to the same equivalence class and possess identical complexity. This classification enables a clean ordering of algorithms by their minimal possible number of operators.

Performance metrics are derived from the two matrices. The authors focus on scale‑up (how performance changes with problem size) and speed‑up (the ratio of sequential to parallel execution time). By generalizing Amdahl’s law, they express speed‑up as

 S(P) = T_seq / (T_par / P + T_ovh),

where T_seq is the total work, T_par is the portion that can be parallelized, and T_ovh aggregates overheads such as communication, synchronization, and load imbalance. Crucially, T_par and T_ovh are not abstract constants; they are computed directly from the concurrency degree (c_D), dependency degree (r_D), and the individual operator execution times t_j. This yields explicit upper and lower bounds for speed‑up that adapt to the granularity of the decomposition.

When all operators have identical execution times—a common case for floating‑point intensive kernels—the matrices reduce to regular forms, and the generalized law collapses to the classic Amdahl or Gustafson models. Thus, the framework subsumes existing models while extending them to heterogeneous execution times, non‑uniform data dependencies, and multi‑level decompositions.

The paper distinguishes perfectly decomposed problems, where every cell of the decomposition matrix is populated, from imperfectly decomposed ones that contain empty cells. Perfect decomposition guarantees maximal concurrency, leading to linear speed‑up with the number of processors. Imperfect decomposition introduces structural bottlenecks that can be quantified directly from the sparsity pattern of the matrices, providing a principled way to compute performance lower bounds.

Finally, the multilevel nature of the approach allows analysts to start with a coarse‑grained decomposition and progressively refine it, recomputing the matrices at each level. This flexibility supports early‑stage design decisions, enabling architects to predict performance, identify potential bottlenecks, and select optimal mapping strategies before implementation. In summary, the paper delivers a mathematically rigorous, yet practically applicable, toolkit that unifies problem, algorithm, and execution perspectives for accurate performance analysis of parallel algorithms across diverse hardware platforms.