Certainty-Guided Reasoning in Large Language Models: A Dynamic Thinking Budget Approach

Large reasoning language models are typically run with fixed inference budgets, which can waste computation or terminate reasoning prematurely. We introduce Certainty-Guided Reasoning (CGR), a model-agnostic adaptive inference procedure that periodically probes whether the current reasoning supports a confident final answer and terminates early once a target certainty threshold is reached, otherwise continuing until the end-of-thinking token or the budget limit. Certainty is estimated from the model’s predicted probabilities over the answer tokens, yielding a lightweight stopping criterion. On AIME2025, CGR preserves baseline accuracy while reducing token usage, providing a tunable certainty-efficiency trade-off that can eliminate millions of tokens in aggregate. Across 64 random seeds, CGR exhibits consistent behavior. We also introduce a Grade metric that penalizes incorrect answers and permits abstention, capturing risk-sensitive performance. Results show that CGR improves Grade by abstaining when certainty remains low.

💡 Research Summary

The paper addresses a fundamental inefficiency in large reasoning language models (LRLMs): the use of a fixed inference budget for every query, which either wastes computation on easy instances or truncates reasoning on hard ones. To solve this, the authors introduce Certainty‑Guided Reasoning (CGR), a model‑agnostic, adaptive inference procedure that leverages the model’s own confidence signal to decide when to stop thinking.

CGR works by interleaving normal token generation with periodic “certainty probes.” Every Δ thinking tokens, the current reasoning trace is augmented with a fixed answer prefix (“Final Answer: \boxed{”). The model then greedily decodes a short answer (capped at four tokens for AIME) and records the probability of each selected answer token. The certainty score c(a*) is defined as the minimum of these probabilities, a deliberately conservative aggregation that ensures even a single uncertain digit in a numeric answer will lower the overall certainty. If c(a*) exceeds a predefined threshold θ, CGR terminates early and outputs the decoded answer; otherwise, generation continues until either the special token appears or the maximum token budget B is reached. The algorithm requires no changes to model weights and can optionally use a separate, smaller probing model, though the experiments keep the same model for both generation and probing for simplicity.

The authors evaluate CGR on the AIME2025 dataset, a collection of 30 competition mathematics problems whose answers are integers between 0 and 999. They test three open‑weight reasoning models—DeepSeek‑14B, DeepSeek‑70B, and Phi‑4—chosen because their training cut‑offs pre‑date the dataset, avoiding contamination. Baseline runs use a fixed budget of 32 000 thinking tokens, zero‑shot chain‑of‑thought prompting, temperature 0.6, and the same answer‑formatting prompt. CGR is applied with probing every 1 000 tokens, and a post‑hoc simulation protocol guarantees that all thresholds are compared on identical reasoning traces, eliminating stochastic variance from the stopping rule itself.

Results show that CGR preserves baseline accuracy across all models while dramatically reducing token usage. For example, DeepSeek‑14B’s accuracy drops by only 0.34 % when θ = 0.99, and the distribution of seed‑level differences clusters tightly around zero, indicating higher stability than the fixed‑budget baseline. Phi‑4 exhibits virtually no accuracy change across thresholds, suggesting its confidence estimates are already highly polarized. DeepSeek‑70B behaves similarly, with only minor fluctuations. Token savings amount to millions of tokens when aggregated across many queries, demonstrating real‑world cost benefits.

Beyond raw accuracy, the paper introduces a “Grade” metric inspired by exam scoring: +1 for a correct answer, 0 for abstention, and –p for an incorrect answer, where p is a configurable penalty (0, 0.25, 0.5, 1.0). CGR can abstain automatically when certainty falls below θ, turning low‑confidence predictions into a neutral outcome rather than a costly mistake. Under moderate to strong penalty regimes (p ≥ 0.5), CGR’s Grade improves by 5–12 % relative to the non‑abstaining baseline, highlighting its risk‑aware advantage.

The authors discuss several limitations. The minimum‑probability aggregation is conservative and may cause unnecessary continuation on otherwise reliable answers. The hyperparameters Δ and θ require domain‑specific tuning; the paper’s experiments focus on short numeric answers, so extending CGR to longer, free‑form text generation may need different probing strategies or aggregation functions (e.g., geometric mean). Additionally, while CGR can theoretically use a separate probing model to reduce overhead, the current experiments do not explore this trade‑off.

In summary, Certainty‑Guided Reasoning offers a practical, zero‑training‑cost method to dynamically allocate inference compute in large language models. By turning the model’s own probability distribution into a stopping signal, CGR achieves a favorable accuracy‑efficiency trade‑off and introduces a principled way to abstain from low‑confidence predictions, making it especially valuable for high‑stakes applications where both computational cost and error penalties matter. Future work could explore alternative certainty aggregations, adaptive Δ schedules, and integration with external budget‑predictors to further enhance flexibility across diverse tasks and model families.