Future Work

We will close with some remarks on future directions.

• In section 9 we describe a simple algorithm for constructing an $`\mathbb{R}_{\geq 0}`$-monoidal grammar from a language model over a monoidal grammar. However, since we do not describe a method for constructing such language model from a dataset and choice of monoidal grammar, it is not immediately clear how we can implement this algorithm in practice. Such a method would enable us to efficiently implement our algorithm on top of any linguistic parsing library for CFGs, pregroups, or other monoidal grammars.

• In section 7 we introduce a functor $`\mathcal{I}`$ from $`Grammar_{\mathbb{S}}`$ to the coalgebras of the functor $`\mathcal{W} = \ssxv`$. A natural question is whether there are functors in the reverse direction as well. In particular, we are curious as to whether $`\mathcal{I}`$ has left/right adjoints. If such functors exist, this would be further evidence of a fundamental connection between dynamical systems and monoidal categories. Such a discovery could help us unite these fields of study, as well as enable us to reduce questions about automata into questions about monoidal categories.

• Our framework is fully syntax-based and is currently agnostic to semantic meaning. In , the authors define a functorial framework for affiliating syntactic structure with vector space semantics, and in the authors project vector space semantics onto dynamic syntax trees in order to model incrementality. This raises the question of how we can best incorporate semantic information into our framework. For example, if we extend the functor from to operate over monoidal grammars in general, it would be interesting to see how the interaction between this functor and $`\mathcal{I}`$ compares to the construction in .

Recurrent neural networks and probabilistic language models have become standard tools in the natural language processing (NLP) community. These networks are inherently incremental, scanning through the sequence of words they are given and updating their prediction for what comes next. Despite their practical success on hard language tasks such as translation and question answering, the structure underlying these machine learning models is yet poorly understood, and they are generally only used as black boxes.

On the other hand, the categorical compositional distributional (DisCoCat) models of Coecke et al. use grammatical structure, explicitly encoded as string diagrams in a free monoidal category, to compute natural language semantics. DisCoCat models have received experimental support on small-scale tasks , but the extra mathematical structure makes them hard to scale to the billions of words used in the training of modern NLP models.

In this work, we aim to bridge this gap by constructing a functor which sends formal grammars (encoded as monoidal categories) to a state automaton that parses the grammar in an incremental way, reading one word at a time and updating the set of possible parsings. In section 8 we introduce some core definitions and preliminaries. In section 10 we introduce monoidal grammars and $`\mathbb{S}`$-monoidal grammars for an arbitrary semiring $`\mathbb{S}`$. In section 7 we define the functor $`\cal{I}`$, from the category of $`\mathbb{S}`$-monoidal grammars to coalgebras of the weighted automata functor $`\mathcal{W}(X) = \ssxv`$. In section 9, we describe an algorithm for learning $`\mathbb{R}_{\geq 0}`$-monoidal grammars from a probabilistic language model. This paper represents a work in progress, and we conclude with a discussion of future work.