A Convenient Category for Higher-Order Probability Theory

Higher-order probabilistic programming languages allow programmers to write sophisticated models in machine learning and statistics in a succinct and structured way, but step outside the standard measure-theoretic formalization of probability theory. Programs may use both higher-order functions and continuous distributions, or even define a probability distribution on functions. But standard probability theory does not handle higher-order functions well: the category of measurable spaces is not cartesian closed. Here we introduce quasi-Borel spaces. We show that these spaces: form a new formalization of probability theory replacing measurable spaces; form a cartesian closed category and so support higher-order functions; form a well-pointed category and so support good proof principles for equational reasoning; and support continuous probability distributions. We demonstrate the use of quasi-Borel spaces for higher-order functions and probability by: showing that a well-known construction of probability theory involving random functions gains a cleaner expression; and generalizing de Finetti’s theorem, that is a crucial theorem in probability theory, to quasi-Borel spaces.

💡 Research Summary

The paper addresses a fundamental mismatch between modern higher‑order probabilistic programming languages and the classical measure‑theoretic foundation of probability. While languages such as Church, Venture, and Anglican allow programmers to define probability distributions over functions (e.g., a distribution on ℝ→ℝ) and to combine higher‑order functions with continuous random variables, the category of measurable spaces (Meas) is not cartesian closed; there is no measurable space of functions ℝ→ℝ, and the evaluation map is never measurable. This limitation prevents a clean semantic treatment of higher‑order probabilistic programs.

To overcome this, the authors introduce quasi‑Borel spaces (QBS). A QBS consists of a set X together with a distinguished collection Mₓ ⊆