InjectRBP: Steering Large Language Model Reasoning Behavior via Pattern Injection

Reasoning can significantly enhance the performance of Large Language Models. While recent studies have exploited behavior-related prompts adjustment to enhance reasoning, these designs remain largely intuitive and lack a systematic analysis of the underlying behavioral patterns. Motivated by this, we investigate how models’ reasoning behaviors shape reasoning from the perspective of behavioral patterns. We observe that models exhibit adaptive distributions of reasoning behaviors when responding to specific types of questions, and that structurally injecting these patterns can substantially influence the quality of the models’ reasoning processes and outcomes. Building on these findings, we propose two optimization methods that require no parameter updates: InjectCorrect and InjectRLOpt. InjectCorrect guides the model by imitating behavioral patterns derived from its own past correct answers. InjectRLOpt learns a value function from historical behavior-pattern data and, via our proposed Reliability-Aware Softmax Policy, generates behavioral injectant during inference to steer the reasoning process. Our experiments demonstrate that both methods can improve model performance across various reasoning tasks without requiring any modifications to model parameters, achieving gains of up to 5.34% and 8.67%, respectively.

💡 Research Summary

The paper “InjectRBP: Steering Large Language Model Reasoning Behavior via Pattern Injection” investigates how the internal reasoning behaviors of large language models (LLMs) influence the quality of their final answers. The authors first formalize a “reasoning behavior chain” by segmenting a model’s intermediate reasoning text into six predefined behavior types: Objective (O), Progress (P), Summary (S), Exploration (E), Verification (V), and Conclusion (C). Using a simple rule‑based classifier, each segment of a generated reasoning trace is labeled, yielding a sequential chain of behavior symbols analogous to a DNA sequence.

Statistical analysis of these chains across multiple models (e.g., Qwen‑3‑8B, GPT‑4‑Turbo) and benchmarks (GPQA, GSM‑8K, Code‑Alpaca) reveals that certain n‑gram patterns (e.g., OP, PV, ESV) appear far more frequently in correct answers than in incorrect ones, while the distribution of patterns between short and long reasoning traces is relatively flat. This suggests that the micro‑structure of reasoning—specifically the order and co‑occurrence of behavior types—has a direct impact on answer correctness.

Motivated by this insight, the authors propose a “structural injection” mechanism that can steer a model’s reasoning at inference time without any weight updates. The mechanism treats the behavior chain as a variable‑order n‑gram language model: given the recent context cₜ (the last up to n‑1 behaviors), the next behavior bₜ is sampled from a conditional distribution derived from a pre‑collected corpus D of behavior chains. The maximum context length n is a hyper‑parameter, and early steps automatically back‑off to shorter contexts.

Two concrete injection strategies are introduced:

1. InjectCorrect – The system collects behavior chains from the model’s past correct answers. During inference, it samples the next behavior according to the n‑gram frequencies observed in those successful chains, effectively “imitating” the patterns that previously led to correct outcomes. This method requires only a lookup and sampling step; no learning or parameter changes are involved.

2. InjectRLOpt – This approach learns a value function V(b, cₜ) that estimates how much a candidate behavior b, given context cₜ, contributes to a correct final answer. The value function is trained via reinforcement learning on historical behavior‑answer pairs, using a reward that reflects answer correctness and, optionally, answer confidence. At inference, a “Reliability‑Aware Softmax Policy” computes a reliability score r(b, cₜ)=σ(V(b, cₜ)) and then applies a temperature‑scaled softmax over r to obtain a probability distribution π(b|cₜ). The policy thus preferentially selects behaviors with high predicted reliability, allowing dynamic, context‑sensitive steering of the reasoning process.

Experiments across several state‑of‑the‑art LLMs and diverse reasoning tasks demonstrate that both methods improve performance without any model fine‑tuning. InjectCorrect yields an average accuracy gain of 5.34 %, while InjectRLOpt achieves a larger average gain of 8.67 %. Gains are especially pronounced on complex mathematical and coding problems, where the injected patterns help the model maintain a disciplined “progress‑then‑verify” loop or an “explore‑summarize‑verify” micro‑cycle. Analyses of post‑injection n‑gram distributions confirm that the methods increase the prevalence of the beneficial patterns identified earlier.

The paper also discusses limitations. The behavior labeling rules are manually crafted and may need adaptation for new domains or languages. The effectiveness of injection depends on the richness of the behavior corpus; sparse or biased corpora could diminish benefits. Moreover, the value function in InjectRLOpt could overfit to the specific tasks used for training, potentially reducing generalization. Future work is suggested in automatic behavior annotation (e.g., via clustering), extending the framework to multimodal reasoning, and meta‑learning of injection policies to improve cross‑task robustness.

In summary, “InjectRBP” provides a novel, parameter‑free avenue for improving LLM reasoning by explicitly modeling and manipulating reasoning behavior patterns. By treating reasoning steps as discrete, analyzable units and by injecting statistically or learned advantageous patterns, the authors achieve consistent performance gains over strong baselines, offering a compelling bridge between prompt engineering and deeper interpretability of LLM reasoning dynamics.