Expectation Propagation on the Maximum of Correlated Normal Variables

Many inference problems involving questions of optimality ask for the maximum or the minimum of a finite set of unknown quantities. This technical report derives the first two posterior moments of the maximum of two correlated Gaussian variables and the first two posterior moments of the two generating variables (corresponding to Gaussian approximations minimizing relative entropy). It is shown how this can be used to build a heuristic approximation to the maximum relationship over a finite set of Gaussian variables, allowing approximate inference by Expectation Propagation on such quantities.

💡 Research Summary

The paper tackles a class of Bayesian inference problems in which the quantity of interest is the maximum (or, by symmetry, the minimum) of a set of random variables. While the distribution of the maximum of independent Gaussian variables is well‑known, the case where the variables are correlated has resisted a closed‑form treatment, forcing practitioners to rely on crude approximations. The authors address this gap by deriving exact expressions for the first two moments (mean and variance) of the maximum of two jointly Gaussian variables with arbitrary correlation, and then embedding these results into an Expectation Propagation (EP) framework that can be applied recursively to an arbitrary finite collection of Gaussian variables.

Derivation for two variables
Let (X_1\sim\mathcal N(\mu_1,\sigma_1^2)) and (X_2\sim\mathcal N(\mu_2,\sigma_2^2)) with covariance (\operatorname{cov}(X_1,X_2)=\rho\sigma_1\sigma_2). Define (M=\max(X_1,X_2)). By splitting the joint density into the regions ({x_1\ge x_2}) and ({x_2\ge x_1}) and integrating, the authors obtain a compact representation that involves only the standard normal cumulative distribution function (\Phi(\cdot)) and density (\phi(\cdot)). Introducing the standardized difference

\