When Does Context Help? Error Dynamics of Contextual Information in Large Language Models

Contextual information at inference time, such as demonstrations, retrieved knowledge, or interaction history, can substantially improve large language models (LLMs) without parameter updates, yet its theoretical role remains poorly understood beyond specific settings such as in-context learning (ICL). We present a unified theoretical framework for analyzing the effect of arbitrary contextual information in Transformer-based LLMs. Our analysis characterizes contextual influence through output error dynamics. In a single-layer Transformer, we prove that the context-conditioned error vector decomposes additively into the baseline error vector and a contextual correction vector. This yields necessary geometric conditions for error reduction: the contextual correction must align with the negative baseline error and satisfy a norm constraint. We further show that the contextual correction norm admits an explicit upper bound determined by context-query relevance and complementarity. These results extend to multi-context and multi-layer Transformers. Experiments across ICL, retrieval-augmented generation, and memory evolution validate our theory and motivate a principled context selection strategy that improves performance by $0.6%$.

💡 Research Summary

The paper “When Does Context Help? Error Dynamics of Contextual Information in Large Language Models” presents a unified theoretical framework for understanding how arbitrary contextual information supplied at inference time influences the output error of Transformer‑based large language models (LLMs). While prior work has examined specific contexts such as in‑context learning (ICL) demonstrations or retrieval‑augmented generation (RAG) under strong distributional alignment assumptions, this study abstracts any context as a single embedding vector t and treats the user query as another vector x. The combined input matrix E =