On Uncertainty Calibration for Equivariant Functions

Data-sparse settings such as robotic manipulation, molecular physics, and galaxy morphology classification are some of the hardest domains for deep learning. For these problems, equivariant networks can help improve modeling across undersampled parts of the input space, and uncertainty estimation can guard against overconfidence. However, until now, the relationships between equivariance and model confidence, and more generally equivariance and model calibration, has yet to be studied. Since traditional classification and regression error terms show up in the definitions of calibration error, it is natural to suspect that previous work can be used to help understand the relationship between equivariance and calibration error. In this work, we present a theory relating equivariance to uncertainty estimation. By proving lower and upper bounds on uncertainty calibration errors (ECE and ENCE) under various equivariance conditions, we elucidate the generalization limits of equivariant models and illustrate how symmetry mismatch can result in miscalibration in both classification and regression. We complement our theoretical framework with numerical experiments that clarify the relationship between equivariance and uncertainty using a variety of real and simulated datasets, and we comment on trends with symmetry mismatch, group size, and aleatoric and epistemic uncertainties.

💡 Research Summary

This paper investigates the interplay between equivariance—embedding known group symmetries into neural network architectures—and uncertainty calibration in data‑sparse domains such as robotic manipulation, galaxy morphology classification, and molecular physics. While equivariant models have been shown to improve accuracy when training data are limited, their effect on calibration—how well predicted probabilities or predictive intervals reflect true frequencies—has remained unexplored. The authors fill this gap by developing a rigorous theoretical framework that links equivariance to two standard calibration metrics: Expected Calibration Error (ECE) for classification and Expected Normalized Calibration Error (ENCE) for regression.

The core theoretical contribution is the derivation of upper and lower bounds on ECE and ENCE that depend explicitly on the symmetry group G, the nature of the model’s equivariance, and the relationship between the group action and the data distribution. By decomposing the input space into group orbits and a fundamental domain, the authors express calibration error as an integral over these regions, mirroring recent generalization bounds for equivariant functions (Wang et al., 2024). Three symmetry scenarios are considered:

1. Correct equivariance – the ground‑truth function and the model share the same symmetry group. In this case the bounds tighten with the size of G; larger groups yield smaller worst‑case calibration error, confirming the intuition that more symmetry provides stronger regularization for both accuracy and calibration.

2. Incorrect equivariance – the model’s symmetry differs from that of the data, but the group still maps data points within the support of the distribution. Here the authors prove a non‑trivial lower bound, guaranteeing that calibration cannot be arbitrarily good, yet the upper bound remains substantially lower than that of a non‑equivariant baseline.

3. Extrinsic equivariance – the group action moves points outside the data support, effectively creating out‑of‑distribution inputs under transformation. This scenario yields the loosest upper bound and the highest lower bound, indicating that calibration can deteriorate dramatically when the model’s symmetry is mismatched in this way.

For regression, the paper extends ENCE beyond scalar mean‑variance predictions to multivariate normal distributions, providing analogous bounds for the mean vector and covariance matrix. The authors also introduce a novel metric called aleatoric bleed, which quantifies the degree to which a model confuses aleatoric (data‑intrinsic) uncertainty with epistemic (model) uncertainty. They derive a theoretical lower bound on aleatoric bleed for equivariant models and show that symmetry mismatch inflates this metric, implying that mis‑calibrated models may also mis‑attribute sources of uncertainty.

Empirically, the authors evaluate several real and synthetic datasets: robotic pick‑and‑place simulations, galaxy image catalogs, molecular property prediction, and synthetic image sets with controlled rotational and reflection symmetries. For each dataset they train both standard (non‑equivariant) architectures and equivariant counterparts (e.g., G‑CNNs, SE(3)‑Transformers), varying the group size (|G| = 1, 4, 8, 16) and deliberately introducing symmetry mismatches to create the three scenarios above. Calibration is measured with fine‑grained binning for ECE and a continuous‑density version of ENCE, while aleatoric bleed is computed from evidential regression outputs.

Key findings include:

• When the model’s equivariance matches the data (correct case), ECE and ENCE drop close to the derived upper bounds, and larger groups consistently improve calibration.
• In the incorrect case, calibration degrades modestly but still outperforms non‑equivariant baselines, confirming that partial symmetry can still be beneficial.
• Extrinsic equivariance leads to severe over‑confidence: both ECE and ENCE increase sharply, and aleatoric bleed rises, indicating that the model mistakenly treats data noise as model uncertainty.
• The empirical values of ECE, ENCE, and aleatoric bleed lie within the theoretical intervals, validating the usefulness of the bounds as practical diagnostics.

The paper’s contributions are threefold: (1) formal bounds linking equivariance to calibration error for both classification and regression; (2) the aleatoric bleed metric and its theoretical analysis; (3) extensive experiments that demonstrate how symmetry mismatch impacts calibration in realistic settings.

By extending the “symmetry = generalization” paradigm to “symmetry = calibration,” the work provides a new lens for designing reliable models in safety‑critical applications where both accurate predictions and trustworthy uncertainty estimates are essential. Future directions include extending the framework to non‑Gaussian predictive distributions, handling partial or approximate symmetries, and integrating the bounds into training objectives to directly enforce calibrated equivariant learning.