Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems

The Earth’s subsurface is a cornerstone of modern society, providing essential energy resources like hydrocarbons, geothermal, and minerals while serving as the primary reservoir for $CO_2$ sequestration. However, full physics numerical simulations of these systems are notoriously computationally expensive due to geological heterogeneity, high resolution requirements, and the tight coupling of physical processes with distinct propagation time scales. Here we propose the \textbf{Adaptive Physics Transformer} (APT), a geometry-, mesh-, and physics-agnostic neural operator that explicitly addresses these challenges. APT fuses a graph-based encoder to extract high-resolution local heterogeneous features with a global attention mechanism to resolve long-range physical impacts. Our results demonstrate that APT outperforms state-of-the-art architectures in subsurface tasks across both regular and irregular grids with robust super-resolution capabilities. Notably, APT is the first architecture that directly learns from adaptive mesh refinement simulations. We also demonstrate APT’s capability for cross-dataset learning, positioning it as a robust and scalable backbone for large-scale subsurface foundation model development.

💡 Research Summary

The paper introduces the Adaptive Physics Transformer (APT), a neural operator designed to overcome the computational bottlenecks of full‑physics simulations for subsurface energy systems such as carbon capture and storage, geothermal, hydrocarbon extraction, and mineral recovery. Traditional high‑fidelity simulators require extremely fine meshes and tightly coupled multiphase, multicomponent PDEs, making them prohibitively expensive for large‑scale or real‑time applications. Existing machine‑learning surrogates—convolutional networks, Fourier Neural Operators (FNOs), and pure graph‑based models—are limited either to structured grids or to purely local interactions, and none can directly ingest data from adaptive mesh refinement (AMR) or dynamic mesh optimization (DMO) workflows.

APT addresses these gaps with a three‑stage architecture: (1) a fused encoder that simultaneously extracts local heterogeneity and global long‑range dependencies, (2) a latent dynamics approximator that evolves a fixed‑size token set in a time‑conditioned Transformer, and (3) a cross‑attention decoder that can query arbitrary spatial points at any time. The encoder contains two parallel pathways. The Global Perceiver branch projects the N input nodes onto a set of learnable super‑node queries via cross‑attention, capturing domain‑wide pressure propagation without the O(N²) cost of full self‑attention. The Local Graph Neural Operator (GNO) branch builds a radius graph and aggregates neighbor features, preserving high‑frequency variations in permeability, porosity, and other rock properties. A learnable gating vector G∈