Bayesian Symbolic Regression via Posterior Sampling

Symbolic regression is a powerful tool for discovering governing equations directly from data, but its sensitivity to noise hinders its broader application. This paper introduces a Sequential Monte Carlo (SMC) framework for Bayesian symbolic regression that approximates the posterior distribution over symbolic expressions, enhancing robustness and enabling uncertainty quantification for symbolic regression in the presence of noise. Differing from traditional genetic programming approaches, the SMC-based algorithm combines probabilistic selection, adaptive tempering, and the use of normalized marginal likelihood to efficiently explore the search space of symbolic expressions, yielding parsimonious expressions with improved generalization. When compared to standard genetic programming baselines, the proposed method better deals with challenging, noisy benchmark datasets. The reduced tendency to overfit and enhanced ability to discover accurate and interpretable equations paves the way for more robust symbolic regression in scientific discovery and engineering design applications.

💡 Research Summary

The pursuit of scientific discovery often hinges on the ability to extract fundamental governing equations from empirical observations. Symbolic Regression (SR) has emerged as a potent paradigm for this task, aiming to discover mathematical expressions that best describe underlying physical laws. However, a significant bottleneck in traditional SR, particularly those based on Genetic Programming (GP), is their extreme sensitivity to noise. In noisy environments, these heuristic-based methods tend to overfit, capturing stochastic fluctuations rather than the true underlying dynamics, leading to overly complex and non-generalizable models.

This paper addresses this critical challenge by introducing a novel Bayesian Symbolic Regression framework powered by Sequential Monte Carlo (SMC) sampling. Unlike traditional approaches that seek a single “best-fit” equation through point estimation, the proposed method aims to approximate the posterior distribution over the space of symbolic expressions. By treating the search for equations as a probabilistic inference problem, the framework provides not only the most likely mathematical models but also a rigorous measure of uncertainty quantification (UQ). This allows researchers to understand the confidence levels associated with specific functional forms and coefficients.

The technical core of the paper lies in the sophisticated implementation of the SMC framework. The authors utilize “adaptive tempering” to navigate the highly non-smooth and complex landscape of the symbolic search space. By gradually annealing the likelihood function, the algorithm can effectively traverse local optima and explore a broader range of structural candidates. Furthermore, the integration of “normalized marginal likelihood” serves as a built-in mechanism for model selection, embodying the principle of Occam’s Razor. This ensures that the algorithm favors parsimonious expressions—those that achieve high predictive accuracy with minimal complexity—thereby preventing the “bloat” phenomenon common in GP.

Empirical evaluations on challenging, noisy benchmark datasets demonstrate that the proposed Bayesian SMC approach significantly outperforms standard GP baselines. The method exhibits superior robustness to noise, maintaining high accuracy and structural integrity even when data quality degrades. The ability to produce interpretable, simple, and robust equations, coupled with the capacity for uncertainty quantification, marks a significant advancement for automated scientific discovery and engineering design. This research paves the way for more reliable AI-driven science, where the discovery of physical laws can be performed autonomously and with high statistical confidence.