Ultrascale Simulations of Non-smooth Granular Dynamics

This article presents new algorithms for massively parallel granular dynamics simulations on distributed memory architectures using a domain partitioning approach. Collisions are modelled with hard contacts in order to hide their micro-dynamics and thus to extend the time and length scales that can be simulated. The multi-contact problem is solved using a non-linear block Gauss-Seidel method that is conforming to the subdomain structure. The parallel algorithms employ a sophisticated protocol between processors that delegate algorithmic tasks such as contact treatment and position integration uniquely and robustly to the processors. Communication overhead is minimized through aggressive message aggregation, leading to excellent strong and weak scaling. The robustness and scalability is assessed on three clusters including two peta-scale supercomputers with up to 458752 processor cores. The simulations can reach unprecedented resolution of up to ten billion non-spherical particles and contacts.

💡 Research Summary

The paper presents a comprehensive framework for ultra‑large‑scale granular dynamics simulations on distributed‑memory supercomputers, focusing on non‑smooth (hard‑contact) interactions of non‑spherical particles. By modeling collisions as instantaneous impulses rather than resolving the micro‑collision dynamics, the authors avoid the severe time‑step restrictions that plague soft‑contact approaches. The governing equations are the Newton‑Euler equations for each particle, augmented with contact reactions λ that satisfy Signorini non‑penetration conditions and Coulomb friction laws.

To solve the resulting multi‑contact problem, the authors introduce a non‑linear block Gauss‑Seidel (NBGS) method that respects the domain decomposition: each MPI process updates the contact forces for contacts wholly contained in its sub‑domain, while only boundary contacts are exchanged with neighboring processes. This yields a highly scalable, matrix‑free algorithm because the mass and inertia matrices remain block‑diagonal and are never assembled explicitly.

A key contribution is the aggressive message aggregation strategy. Instead of sending thousands of small messages per time step, each process packs all data destined for a particular neighbor into a single buffer, dramatically reducing the number of MPI calls and the associated latency. The communication pattern is limited to nearest‑neighbor exchanges, which aligns well with the torus and dragonfly interconnects of modern petascale machines.

The authors also implement a contact reduction phase that prunes the set of potential contacts using spatial hashing and neighbor‑search techniques, bringing the computational complexity down to O(N log N) even for dense packings where the theoretical maximum number of contacts is O(N²).

Performance is evaluated on three distinct platforms, including the petascale systems SuperMUC and JUQUEEN. Strong‑ and weak‑scaling experiments demonstrate near‑linear speed‑up up to 458 752 cores. In weak‑scaling tests, as few as a few hundred particles per core suffice to maintain high parallel efficiency, both for dilute flows (few contacts per particle) and dense flows (many contacts per particle). The authors successfully simulate up to ten billion (10¹⁰) non‑spherical particles and a comparable number of contacts, a scale never before reported for hard‑contact granular simulations.

Compared with earlier parallel granular dynamics work (e.g., references