Ambiguity and Incomplete Information in Categorical Models of Language

We investigate notions of ambiguity and partial information in categorical distributional models of natural language. Probabilistic ambiguity has previously been studied using Selinger’s CPM construction. This construction works well for models built upon vector spaces, as has been shown in quantum computational applications. Unfortunately, it doesn’t seem to provide a satisfactory method for introducing mixing in other compact closed categories such as the category of sets and binary relations. We therefore lack a uniform strategy for extending a category to model imprecise linguistic information. In this work we adopt a different approach. We analyze different forms of ambiguous and incomplete information, both with and without quantitative probabilistic data. Each scheme then corresponds to a suitable enrichment of the category in which we model language. We view different monads as encapsulating the informational behaviour of interest, by analogy with their use in modelling side effects in computation. Previous results of Jacobs then allow us to systematically construct suitable bases for enrichment. We show that we can freely enrich arbitrary dagger compact closed categories in order to capture all the phenomena of interest, whilst retaining the important dagger compact closed structure. This allows us to construct a model with real convex combination of binary relations that makes non-trivial use of the scalars. Finally we relate our various different enrichments, showing that finite subconvex algebra enrichment covers all the effects under consideration.

💡 Research Summary

**
The paper addresses a fundamental limitation in current categorical distributional models of natural language: the difficulty of representing ambiguous or partially known meanings in compact closed categories other than finite‑dimensional Hilbert spaces. Existing work has relied on Selinger’s CPM construction to model probabilistic ambiguity by moving from pure states (vectors) to mixed states (density matrices). While CPM works beautifully for the category FdHilb, it fails to provide a satisfactory notion of mixing in categories such as Rel (sets and binary relations) because the scalar field collapses to the Boolean values {0,1}. Consequently, one cannot express weighted mixtures like “90 % bank, 10 % riverbank” in Rel; the best one can do is a nondeterministic “either‑or” choice, losing all quantitative information.

To overcome this, the authors propose a monadic approach that treats different kinds of linguistic uncertainty as computational side‑effects. They introduce several commutative Set‑monads, each capturing a distinct informational effect:

1. Lift monad ((–)⊥) – adds a distinguished bottom element ⊥ to model “unknown” or divergent information.
2. Finite powerset monad (Pω) – models nondeterministic choice via finite subsets.
3. Non‑empty finite powerset monad (P⁺ω) – similar to Pω but excludes the empty set, thereby preventing divergence.
4. Finite distribution monad (D) – encodes genuine probability distributions with finite support, allowing real‑valued weights that sum to 1.
5. Finite sub‑distribution monad (S) – relaxes the normalization condition to ≤ 1, thereby representing missing probability mass (i.e., incomplete information).
6. Combinations of the above – can be used when multiple effects need to be combined.

All these monads are commutative, which, by a result of Jacobs, guarantees that their Eilenberg‑Moore categories are symmetric monoidal closed, complete, and cocomplete. This structural richness makes each Eilenberg‑Moore category an ideal base for enrichment: one can freely construct a V‑enriched category over any ordinary category C, where V is any of the above bases.

The authors then systematically enrich a given dagger compact closed category C with each monad:

• C⊥ (pointed‑set enrichment) – morphisms are pairs (f, ⊥); composition propagates ⊥, and the tensor product and dagger extend by sending any pair involving ⊥ to ⊥. This models total lack of information.
• C_AJSLat / C_JSLat (affine / join‑semilattice enrichment) – hom‑sets carry (affine) join‑semilattice structure; composition and tensor distribute over joins. This captures non‑quantitative ambiguity (multiple possible meanings without probabilities).
• C_Convex / C_Subconvex (real convex / sub‑convex enrichment) – hom‑sets support real convex combinations; composition and tensor are linear with respect to these combinations. The sub‑convex variant allows the total weight to be less than one, representing missing or uncertain probability mass.

A central theorem shows that any dagger compact closed category can be freely enriched with any of these monads while preserving its dagger compact closed structure. As a concrete illustration, the authors construct a model where the underlying category is Rel, but morphisms are real‑valued binary relations (i.e., each pair (a,b) carries a weight in