Rates and architectures for learning geometrically non-trivial operators

Deep learning methods have proven capable of recovering operators between high-dimensional spaces, such as solution maps of PDEs and similar objects in mathematical physics, from very few training samples. This phenomenon of data-efficiency has been proven for certain classes of elliptic operators with simple geometry, i.e., operators that do not change the domain of the function or propagate singularities. However, scientific machine learning is commonly used for problems that do involve the propagation of singularities in a priori unknown ways, such as waves, advection, and fluid dynamics. In light of this, we expand the learning theory to include double fibration transforms–geometric integral operators that include generalized Radon and geodesic ray transforms. We prove that this class of operators does not suffer from the curse of dimensionality: the error decays superalgebraically, that is, faster than any fixed power of the reciprocal of the number of training samples. Furthermore, we investigate architectures that explicitly encode the geometry of these transforms, demonstrating that an architecture reminiscent of cross-attention based on levelset methods yields a parameterization that is universal, stable, and learns double fibration transforms from very few training examples. Our results contribute to a rapidly-growing line of theoretical work on learning operators for scientific machine learning.

💡 Research Summary

The paper tackles the problem of learning operators that map between high‑dimensional function spaces, focusing on a broad class of geometric integral operators called double‑fibration transforms. These transforms include generalized Radon transforms, geodesic ray transforms, spherical mean transforms, and many other operators that arise in imaging, seismic, radar, and tomography. Unlike previous learning theory, which mainly covered elliptic PDE solution operators that preserve the domain and do not propagate singularities, this work addresses operators that change the domain and can transport singularities in unknown ways—situations typical in wave propagation, advection, and fluid dynamics.

The authors first formalize double‑fibration transforms. Let Y (measurement domain) and X (target domain) be compact smooth manifolds, and let Z⊂Y×X be a smooth submanifold equipped with a non‑vanishing measure μ. The projection maps p:Z→Y and q:Z→X are submersions, so each y∈Y defines a fiber G_y = q(p^{-1}(y))⊂X and each x∈X defines a fiber H_x = p(q^{-1}(x)). The operator R:D′(X)→D′(Y) acts by integrating a function u over the fiber G_y:
  R u(y)=∫_{G_y} u(x) dμ_y(x).
When the incidence relation Z satisfies the Bolker condition (the map P:NZ′→TY is an injective immersion), the transform is well‑behaved: its normal operator R*R is a pseudodifferential operator and no new singularities are created during reconstruction.

A key technical contribution is the analysis of R in phase space using Gabor (time‑frequency) atoms. The authors show that R is a Fourier integral operator (FIO) with a symbol independent of frequency, belonging to the (ρ,δ) class. Consequently, the matrix of inner products ⟨ĥ_{y,η}, R ĝ_{x,ξ}⟩ is highly sparse: significant entries occur only when the phase‑space point (x,ξ) lies near the image under the Bolker map χ of (y,η). This “well‑organized” structure enables the use of compressive‑sensing techniques.

The main learning‑theoretic result (Theorem 1) states that for any ε>0, there exists a randomized algorithm that, given J≈(C_r/ε)^{1/r} log(1/ε) training pairs {u_j, R u_j} (with r>1 reflecting the compressibility of R), produces an estimator (\hat R) satisfying
  sup_{(y,η)} |⟨ĥ_{y,η}, (\hat R−R)u⟩| < ε ‖u‖_{L^2}
with high probability for all u∈L^2(X). In other words, the error decays super‑algebraically—faster than any fixed power of 1/J—so the curse of dimensionality is avoided.

Beyond theory, the paper proposes a concrete neural architecture that encodes the geometry of double‑fibration transforms. The design is inspired by level‑set methods and cross‑attention mechanisms. Input functions are represented implicitly via level‑set networks; queries, keys, and values are constructed from learned embeddings of points in Y and X. The attention operation effectively learns the mapping between fibers G_y and H_x, mirroring the Bolker map. The authors prove three properties of this architecture: (1) universality—any double‑fibration transform can be approximated arbitrarily well; (2) stability—small perturbations in the input lead to proportionally small changes in the output; (3) data efficiency—because the architecture respects the sparse phase‑space structure, it requires only the number of samples predicted by Theorem 1 to achieve a given accuracy.

Empirical experiments validate the theory. The authors train the proposed model on synthetic datasets for the Radon transform, Euclidean ray transform, and spherical mean transform, comparing against Fourier Neural Operators, DeepONets, and standard convolutional networks. With as few as 10–50 randomly generated training functions, the model attains relative L^2 errors below 10^{-3}, outperforming baselines that need orders of magnitude more data. Moreover, by inspecting the Jacobian of the learned map, they can recover the intrinsic dimension of the fiber and, in the geodesic‑ray case, infer the underlying Riemannian metric, demonstrating the potential for post‑hoc scientific discovery.

In conclusion, the paper extends operator‑learning theory from simple elliptic settings to a rich class of geometrically non‑trivial transforms, establishes super‑algebraic sample complexity via compressive‑sensing arguments, and provides a geometry‑aware neural architecture that achieves the predicted efficiency in practice. The work opens avenues for learning operators in wave‑type PDEs, transport phenomena, and inverse problems where singularities travel along complex manifolds, and suggests future research on non‑Bolker transforms, nonlinear extensions, and applications to real‑world scientific data.