Conditional Performance Guarantee for Large Reasoning Models

Large reasoning models have shown strong performance through extended chain-of-thought reasoning, yet their computational cost remains significant. Probably approximately correct (PAC) reasoning provides statistical guarantees for efficient reasoning by adaptively switching between thinking and non-thinking models, but the guarantee holds only in the marginal case and does not provide exact conditional coverage. We propose G-PAC reasoning, a practical framework that provides PAC-style guarantees at the group level by partitioning the input space. We develop two instantiations: Group PAC (G-PAC) reasoning for known group structures and Clustered PAC (C-PAC) reasoning for unknown groupings. We prove that both G-PAC and C-PAC achieve group-conditional risk control, and that grouping can strictly improve efficiency over marginal PAC reasoning in heterogeneous settings. Our experiments on diverse reasoning benchmarks demonstrate that G-PAC and C-PAC successfully achieve group-conditional risk control while maintaining substantial computational savings.

💡 Research Summary

The paper tackles the growing computational burden of large language models (LLMs) that employ extensive chain‑of‑thought (CoT) reasoning. While recent “PAC reasoning” frameworks (Zeng et al., 2025) provide statistical guarantees that the expected performance loss of a hybrid system (switching between a full‑thinking model f and a fast non‑thinking model ˜f) stays below a tolerance ε, those guarantees are marginal: they hold only on average over the entire data distribution. Consequently, subpopulations—e.g., specific diseases in medical diagnosis or particular difficulty levels in reasoning benchmarks—can suffer arbitrarily large errors, which is unacceptable in high‑stakes settings.

To address this limitation, the authors introduce group‑conditional PAC efficiency. The input space X is partitioned into k groups G₁,…,G_k (either known a priori or learned from data). For each group they define a conditional risk R(b_f | G_j) = E