The Gauss-Markov Adjunction Provides Categorical Semantics of Residuals in Supervised Learning

Enhancing the intelligibility and interpretability of machine learning is a crucial task in responding to the demand for Explicability as an AI principle, and in promoting the better social implementation of AI. The aim of our research is to contribute to this improvement by reformulating machine learning models through the lens of category theory, thereby developing a semantic framework for structuring and understanding AI systems. Our categorical modeling in this paper clarifies and formalizes the structural interplay between residuals and parameters in supervised learning. The present paper focuses on the multiple linear regression model, which represents the most basic form of supervised learning. By defining two Lawvere-enriched categories corresponding to parameters and data, along with an adjoint pair of functors between them, we introduce our categorical formulation of supervised learning. We show that the essential structure of this framework is captured by what we call the Gauss-Markov Adjunction. Within this setting, the dual flow of information can be explicitly described as a correspondence between variations in parameters and residuals. The ordinary least squares estimator for the parameters and the minimum residual are related via the preservation of limits by the right adjoint functor. Furthermore, we position this formulation as an instance of extended denotational semantics for supervised learning, and propose applying a semantic perspective developed in theoretical computer science as a formal foundation for Explicability in AI.

💡 Research Summary

The paper tackles the pressing need for transparent and interpretable machine‑learning systems by recasting supervised learning—specifically multiple linear regression—within a categorical framework. The authors begin by situating their work in the broader ethical discourse on AI, emphasizing the principle of Explicability, which demands explanations at an appropriate level of abstraction rather than low‑level code disclosure. While many recent efforts apply monoidal categories and graphical calculi to neural‑network architectures, this work deliberately adopts the more classical setting of commutative diagrams and adjunctions, arguing that such structures better expose the dual flow of information between model parameters and residuals.

After reviewing the standard regression setup (data matrix X∈ℝ^{n×m}, response vector y∈ℝ^{n}, parameter vector a∈ℝ^{m}, residual r∈ℝ^{n}) and allowing for rank‑deficient X, the authors introduce an auxiliary calibration vector b∈ℝ^{n}. Although b can be eliminated algebraically, keeping it makes the unit and counit of the forthcoming adjunction explicit, thereby preserving categorical coherence.

The core technical contribution is the construction of two Lawvere‑enriched categories:

• Prm – objects are parameter vectors a; the enriched hom‑object between a₁ and a₂ is φ(a₁,a₂)=‖X(a₂−a₁)‖, i.e., the norm of the X‑image of the parameter difference.
• Data – objects are data vectors y; the enriched hom‑object between y₁ and y₂ is δ(y₁,y₂)=‖XG(y₂−y₁)‖, where G is the Moore‑Penrose pseudoinverse of X.

These distance‑based hom‑objects prevent the categories from collapsing into trivial isomorphisms and retain the geometry induced by the data matrix.

Two V‑functors are then defined:

• F : Prm → Data, the affine forward functor, maps a ↦ X a + b and sends φ to δ without distortion (φ(a₁,a₂)=δ(Fa₁,Fa₂)).
• G : Data → Prm, the Gauss‑Markov functor, maps y ↦ G y and similarly respects the enriched distances.

The authors prove that (F ⊣ G) forms an adjunction. The right adjoint G preserves all limits, which translates categorically into the fact that applying G to a data object yields the minimum‑residual solution. Dually, the left adjoint F yields the ordinary least‑squares (OLS) estimator, because the unit η: Id_{Prm} ⇒ G ∘ F encodes the OLS normal equations, while the counit ε: F ∘ G ⇒ Id_{Data} encodes the residual minimization. By explicitly retaining the calibration vector b, the unit and counit become concrete natural transformations (denoted Λ), avoiding the degeneracy that typically arises in metric‑enriched categories where hom‑objects collapse to zero.

Beyond the linear‑regression case, the authors sketch how the same adjunctionic pattern extends to minimum‑norm solutions obtained via the Moore‑Penrose inverse, covering both full‑rank and rank‑deficient scenarios.

In the final section the paper connects this categorical adjunction to extended denotational semantics, a tradition from theoretical computer science that assigns mathematical meanings to programs. Modern AI systems, built from differentiable functions and statistical operations, lie outside the classic symbolic‑logic domain; therefore, a semantics that incorporates algebraic structures like the Gauss‑Markov adjunction is proposed as a foundation for AI Explicability. By making the dual relationship between parameters and residuals explicit in a high‑level mathematical language, the framework promises to bridge the gap between technical model internals and ethically required explanations.

Overall, the work offers a rigorous, mathematically elegant bridge between statistical learning theory and categorical semantics, opening a pathway for principled, explainable AI grounded in well‑understood abstract structures.