Universal approximation theorems establish the expressive capacity of neural network architectures. For dynamical systems, existing results are limited to finite time horizons or systems with a globally stable equilibrium, leaving multistability and limit cycles unaddressed. We prove that Neural ODEs achieve $\varepsilon$-$δ$ closeness -- trajectories within error $\varepsilon$ except for initial conditions of measure $< δ$ -- over the \emph{infinite} time horizon $[0,\infty)$ for three target classes: (1) Morse-Smale systems (a structurally stable class) with hyperbolic fixed points, (2) Morse-Smale systems with hyperbolic limit cycles via exact period matching, and (3) systems with normally hyperbolic continuous attractors via discretization. We further establish a temporal generalization bound: $\varepsilon$-$δ$ closeness implies $L^p$ error $\leq \varepsilon^p + δ\cdot D^p$ for all $t \geq 0$, bridging topological guarantees to training metrics. These results provide the first universal approximation framework for multistable infinite-horizon dynamics.
Universal approximation results provide rigorous guarantees about the expressive capacity of neural networks. For dynamical systems, Recurrent Neural Networks (RNNs) are widely cited as universal approximators 47 , justifying their deployment in models of neural computation 38,166 . However, a fundamental gap exists between this theoretical promise and biological reality. Existing guarantees are strictly limited to finite time horizons or systems with a globally stable equilibrium (the fading memory property). This restriction explicitly excludes multistability-the coexistence of multiple attractors-which is the dynamical basis of essential cognitive functions. Decision-making relies on selecting among distinct basins of attraction; working memory requires self-sustaining persistent activity; and neural oscillations (limit cycles) drive rhythmic motor control 16,85,138,161 . Consequently, current theories fail to address the very dynamical regimes required for computation. Extending guarantees to infinite time is crucial for temporal generalization: ensuring models coherently replicate dynamics over indefinite durations rather than finite windows. Three fundamental failure modes prevent naive extension of finite-time results:
• B-type error (Basin mismatch): Small approximation errors near separatrices push trajectories into incorrect basins of attraction.
• P-type error (Phase drift): For limit cycles, minute period mismatches cause unbounded phase divergence as t → ∞.
• D-type error (Discretization): Continuous attractors are not structurally stable; generic perturbations destroy their continua of marginally stable fixed points.
These topological obstructions require specialized treatment beyond standard Grönwallbased error analysis, which yields exponentially growing bounds unusable for infinite time. This work establishes the first universal approximation results for multistable dynamical systems over infinite time horizons. Our approach exploits structural stability theory: Morse-Smale systems-a significant subset of all structurally stable systems-are robust to small C 1 perturbations, enabling infinite-time bounds. For limit cycles, we additionally require exact period matching via a localized correction procedure.
Our analysis adopts a learning-theoretic framework: we define target classes F of dynamical systems to be approximated, a hypothesis class F of Neural ODEs (Definition 1), and an approximation criterion (ε-δ closeness, Definition 9). Universal approximation means: for any f ∈ F and any ε, δ > 0, there exists f ∈ F achieving ε-δ closeness over infinite time.
We establish universal approximation for three target classes with increasing generality. Throughout, ε-δ closeness means: the volume of initial conditions with trajectory error exceeding ε is less than δ (Definition 9).
Theorem FP (Fixed Points, Informal; see Theorem 3). For any Morse-Smale system with hyperbolic fixed points and any ε, δ > 0, there exists a finite-size Neural ODE that is ε-δ close to the target for all t ∈ [0, ∞).
Theorem LC (Limit Cycles, Informal; see Theorem 4). For Morse-Smale systems with hyperbolic limit cycles, Neural ODEs achieve ε-δ closeness via exact period matching through localized vector field scaling.
Theorem CA (Continuous Attractors, Informal; see Theorem 5). For systems with normally hyperbolic continuous attractors (line attractors, ring attractors, isochronous cylinders), Neural ODEs achieve ε-δ closeness via tiling-approximation by a dense grid of discrete attractors with spacing < ε.
Finally, we establish a temporal generalization bound to bridge these topological guarantees to practical training metrics:
Theorem TG (Temporal Generalization, Informal; see Theorem 6). If φ is ε-δ close to φ, then the time-averaged L p error satisfies E p,∞ ≤ ε p + δ • D p , where D is the domain diameter. This bridges topological guarantees to practical training metrics like Mean Squared Error (MSE).
We establish the first universal approximation results for multistable dynamical systems over infinite time horizons (Theorems 3-5).
We prove these guarantees are achievable with finite-size Neural ODEs, not requiring infinite width or depth (Section 4).
We introduce exact period matching via localized vector field scaling to eliminate P-type error for limit cycles (Theorem 4).
We derive a temporal generalization bound linking ε-δ closeness to bounded L p error (Theorem 6).
Figure 1 illustrates the landscape of universal approximation results. Prior infinitetime theories were limited to fading memory systems (a single global attractor). Our results extend to the full class of Morse-Smale systems and normally hyperbolic continuous attractors 143 . Existing infinite-time results require the fading memory property (FMP), excluding multistability. We prove Neural ODEs are dense in Morse-Smale systems (F FP , F LC ) and normally hyperbolic continuous attractors (F CA ), subject to isochrony for
This content is AI-processed based on open access ArXiv data.