MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanics

We present MeshGraphNet-Transformer (MGN-T), a novel architecture that combines the global modeling capabilities of Transformers with the geometric inductive bias of MeshGraphNets, while preserving a mesh-based graph representation. MGN-T overcomes a key limitation of standard MGN, the inefficient long-range information propagation caused by iterative message passing on large, high-resolution meshes. A physics-attention Transformer serves as a global processor, updating all nodal states simultaneously while explicitly retaining node and edge attributes. By directly capturing long-range physical interactions, MGN-T eliminates the need for deep message-passing stacks or hierarchical, coarsened meshes, enabling efficient learning on high-resolution meshes with varying geometries, topologies, and boundary conditions at an industrial scale. We demonstrate that MGN-T successfully handles industrial-scale meshes for impact dynamics, a setting in which standard MGN fails due message-passing under-reaching. The method accurately models self-contact, plasticity, and multivariate outputs, including internal, phenomenological plastic variables. Moreover, MGN-T outperforms state-of-the-art approaches on classical benchmarks, achieving higher accuracy while maintaining practical efficiency, using only a fraction of the parameters required by competing baselines.

💡 Research Summary

MeshGraphNet‑Transformer (MGN‑T) is a hybrid neural architecture that fuses the locality‑preserving strengths of MeshGraphNets (MGN) with the global modeling capacity of Transformers, targeting large‑scale solid‑mechanics simulations. The authors first identify a fundamental bottleneck of standard MGN: iterative message‑passing (MPNN) only propagates information within a limited graph radius, leading to “under‑reaching” on high‑resolution meshes that contain tens of thousands of nodes. To overcome this, MGN‑T replaces the deep stack of message‑passing layers with a physics‑aware Transformer that updates all node states simultaneously, while still retaining explicit node and edge attributes.

The pipeline follows an Encoder‑Processor‑Decoder scheme. Input meshes from FEM are represented as graphs (V, E_M) with optional contact edges (E_C) that appear dynamically during simulation. Node features combine physical state variables (e.g., velocity, thickness) and one‑hot type identifiers; edge features encode relative positions in both reference and current configurations, preserving spatial equivariance. Positional encodings are added using stationary sinusoidal waves to avoid dependence on absolute coordinates.

Three encoders (ϕ_V, ϕ_M, ϕ_C) map these raw features into a high‑dimensional latent space. The processor consists of three stages: (1) a pre‑processor MPNN that performs two message‑passing iterations, allowing each node to gather 2‑hop information and absorb boundary conditions before any global operation; (2) a physics‑Attention Transformer that conducts the global update; and (3) a refinement MPNN that again executes two message‑passing steps to reinforce local consistency.

The key innovation lies in the Transformer’s “eidetic token” mechanism, adopted from Transolver++. Instead of applying O(N²) self‑attention directly on N nodes, the model learns a projection from the node latent vectors to a much smaller set of P physical tokens (P ≪ N). This projection uses a learned temperature τ_i per node and a Gumbel‑Softmax based slicing weight w_ij, enabling sharp, differentiable assignment of nodes to tokens. Multi‑head attention is then performed on the P tokens (complexity O(P²)), after which a de‑slicing step distributes the updated token information back to the original nodes. This design dramatically reduces memory and compute while still capturing long‑range physical interactions across the entire mesh.

Training uses teacher‑forced one‑step prediction with a mean‑squared error loss computed per node and averaged over the batch. Batches must contain snapshots from the same trajectory to keep the node ordering consistent for the Transformer.

The authors evaluate MGN‑T on two benchmarks: (a) the Pi‑beam impact scenario, a realistic automotive crash simulation with 16 k nodes, nine deformable components of varying thickness, and a rigid obstacle; and (b) the Deforming‑Plate quasi‑static benchmark with 1.2 k nodes. Both datasets include complex phenomena such as self‑contact, multi‑material interactions, and elasto‑plastic behavior.

Results show that MGN‑T outperforms prior state‑of‑the‑art methods—including the original MeshGraphNet, BSMS‑GNN, ReGUNet, and hierarchical graph surrogates—by a substantial margin. On the Pi‑beam task, mean absolute error (MAE) and root‑mean‑square error (RMSE) drop by roughly 30‑35 % relative to the best baseline, while using only about 5 % of the parameters. The model also accurately predicts internal phenomenological plastic variables, demonstrating its multi‑output capability. Importantly, the Transformer’s global update eliminates the need for deep message‑passing stacks, allowing the network to scale to industrial‑size meshes without hierarchical coarsening or manual mesh generation.

The paper discusses limitations: the choice of token count P is currently heuristic; the physics‑Attention is purely data‑driven and does not enforce explicit physical constraints such as energy conservation, which could lead to error accumulation in very long roll‑outs; and the projection step may be less straightforward for highly irregular, non‑mesh point clouds. Future work is suggested to incorporate physics‑based regularization, adaptive token sizing, and extensions to unstructured data.

In summary, MeshGraphNet‑Transformer presents a compelling solution to the long‑range interaction problem in graph‑based surrogate models for solid mechanics. By marrying local MPNN updates with a token‑based global Transformer, it achieves high accuracy, parameter efficiency, and scalability on industrial‑scale simulations, opening the door to fast, data‑driven solvers that can be integrated directly into design and optimization pipelines.