Structure Enables Effective Self-Localization of Errors in LLMs

Self-correction in language models remains elusive. In this work, we explore whether language models can explicitly localize errors in incorrect reasoning, as a path toward building AI systems that can effectively correct themselves. We introduce a prompting method that structures reasoning as discrete, semantically coherent thought steps, and show that models are able to reliably localize errors within this structure, while failing to do so in conventional, unstructured chain-of-thought reasoning. Motivated by how the human brain monitors errors at discrete decision points and resamples alternatives, we introduce Iterative Correction Sampling of Thoughts (Thought-ICS), a self-correction framework. Thought-ICS iteratively prompts the model to generate reasoning one discrete and complete thought at a time–where each thought represents a deliberate decision by the model–creating natural boundaries for precise error localization. Upon verification, the model localizes the first erroneous step, and the system backtracks to generate alternative reasoning from the last correct point. When asked to correct reasoning verified as incorrect by an oracle, Thought-ICS achieves 20-40% self-correction lift. In a completely autonomous setting without external verification, it outperforms contemporary self-correction baselines.

💡 Research Summary

The paper tackles the longstanding challenge of self‑correction in large language models (LLMs). While prior work has shown that prompting a model to “refine” its output often amounts to brute‑force resampling rather than genuine error fixing, this study asks whether a model can explicitly locate the point of failure in its own reasoning and then correct from that point onward. Drawing inspiration from neuroscience—specifically the anterior cingulate cortex’s monitoring of errors at discrete decision points—the authors propose to structure a model’s chain‑of‑thought (CoT) into semantically coherent “thoughts”. Each thought is a self‑contained reasoning step, delimited by a special token, turning the generation process into a Thought‑Markov Decision Process (Thought‑MDP) where actions are multi‑token thought units rather than single tokens.

The core framework, Iterative Correction Sampling of Thoughts (Thought‑ICS), consists of three configurable components: (1) Verification, which determines whether the final answer is correct (using either an external oracle or the model’s own self‑verification); (2) Localization, where the model is prompted to examine its full thought trace and report the first thought that contains an error, together with a brief justification; and (3) Resampling, which backtracks to the last verified‑correct thought and generates a new continuation from that prefix. This loop repeats until verification succeeds, the model cannot locate an error (V/L disagreement), or a maximum iteration budget is exhausted.

Experiments span eight instruction‑tuned models ranging from 3 B to 120 B parameters (LLaMA 3, Qwen 2.5, GPT‑OSS) and six reasoning benchmarks covering arithmetic, logical deduction, commonsense, and multi‑step problem solving. The authors compare Thought‑ICS against standard CoT, Self‑Refine, CoVe, and other recent self‑correction baselines. Results show three major findings:

1. Thought‑level structuring improves initial accuracy: Even before any correction, Thought‑MDP generation yields 3–5 % higher accuracy than token‑level CoT, especially for larger models where thought boundaries become more stable.
2. Error localization is highly reliable: For models ≥70 B, the first erroneous thought is identified with >90 % precision; even the 3 B models achieve ~78 % precision. This directly contradicts earlier claims that LLMs cannot self‑localize errors.
3. Iterative correction yields substantial gains: With oracle verification, Thought‑ICS outperforms CoT‑based correction by 20–40 % absolute accuracy across all benchmarks. In a fully autonomous setting (self‑verification only), Thought‑ICS still surpasses Self‑Refine and CoVe by ~5 % absolute accuracy, though failures arise from V/L disagreement (12 % of cases) and iteration limits (8 % of cases). Prompt engineering and limiting thought length reduce these failure modes by roughly one‑third.

The authors analyze why structuring reasoning into thoughts matters. By giving the model explicit decision boundaries, the internal representation aligns better with human‑like error monitoring, allowing the model to treat each thought as a “unit of meaning” rather than a raw token stream. This also makes the correction process analogous to deterministic reinforcement‑learning resets: the model can treat the erroneous thought as a state to be revisited and re‑explored. The paper suggests that Thought‑MDP could be combined with policy‑gradient or value‑based RL methods to further improve the generation policy, opening a path toward meta‑learning self‑correction strategies.

Limitations are acknowledged. Self‑verification is still imperfect, leading to V/L disagreement when the model flags an answer as wrong but cannot pinpoint the faulty step. Moreover, the automatic delimitation of thoughts sometimes produces overly short or overly long units, which can affect both localization precision and generation efficiency. Future work may incorporate dynamic boundary adjustment or hybrid scoring functions that blend internal confidence with external heuristics.

In conclusion, the study demonstrates that (i) structuring LLM reasoning into discrete, semantically coherent thoughts enables reliable self‑localization of errors, and (ii) an iterative backtrack‑and‑resample loop (Thought‑ICS) can leverage this structure to achieve robust self‑correction, even without external feedback. These insights provide a concrete architectural principle for building more trustworthy, self‑repairing AI systems and suggest promising avenues for integrating reinforcement‑learning techniques with structured reasoning in future LLM research.