Harmonic Grammar, Optimality Theory, and Syntax Learnability: An Empirical Exploration of Czech Word Order

This work presents a systematic theoretical and empirical comparison of the major algorithms that have been proposed for learning Harmonic and Optimality Theory grammars (HG and OT, respectively). By comparing learning algorithms, we are also able to compare the closely related OT and HG frameworks themselves. Experimental results show that the additional expressivity of the HG framework over OT affords performance gains in the task of predicting the surface word order of Czech sentences. We compare the perceptron with the classic Gradual Learning Algorithm (GLA), which learns OT grammars, as well as the popular Maximum Entropy model. In addition to showing that the perceptron is theoretically appealing, our work shows that the performance of the HG model it learns approaches that of the upper bound in prediction accuracy on a held out test set and that it is capable of accurately modeling observed variation.

💡 Research Summary

This paper presents a systematic theoretical and empirical comparison of learning algorithms for Harmonic Grammar (HG) and Optimality Theory (OT), using Czech word order as a test case. The authors extract a corpus of 2,955 simple transitive declarative sentences from the Prague Dependency Treebank, each annotated with grammatical functions (subject, verb, object) and information‑structure markers (topic, contrastive topic, focus). They define a set of twelve binary alignment constraints: six constraints align the subject, verb, and object to the left or right edge of the sentence, and six align the three information‑structure markers similarly.

Three learning algorithms are evaluated: (1) the online perceptron, which learns real‑valued constraint weights for an HG; (2) the Gradual Learning Algorithm (GLA), the classic OT learner that incrementally adjusts a hierarchical ranking of constraints; and (3) a Maximum Entropy (MaxEnt) model, which treats the constraints as features and learns a probability distribution over candidate word orders in a batch fashion.

The perceptron updates weights after each training example, rewarding the correct candidate and penalizing competing candidates, thereby allowing lower‑weight constraints to “gang‑up” on higher‑weight ones. The GLA, by contrast, maintains a strict ranking; lower‑ranked constraints can never outweigh higher‑ranked ones, even cumulatively. MaxEnt provides a probabilistic baseline but does not model incremental learning.

Experimental results show that the HG learned by the perceptron achieves about 84 % accuracy on a held‑out test set, close to the theoretical upper bound of roughly 87 % derived from the information available in the data. The OT grammar learned by GLA reaches only about 78 % accuracy, while MaxEnt attains around 80 %. The performance gap is attributed to the “ganging‑up” effect: in many instances, several low‑weight alignment constraints collectively favor a word order that would be ruled out by the strict OT hierarchy. Moreover, the HG model captures the observed distribution of word‑order variation far more faithfully than either OT or MaxEnt, which tend to misestimate the probabilities of less frequent orders.

The authors argue that these findings demonstrate two key points. First, HG’s additional expressive power—numeric weighting of constraints—provides a measurable advantage for modeling syntactic phenomena that involve interaction between grammatical structure and discourse‑level information. Second, the perceptron’s online nature aligns well with theories of human language acquisition, offering a simple yet effective computational model of incremental grammar learning.

Beyond the immediate results, the paper contributes to the broader field by extending learnability research—traditionally focused on phonology—to the syntax domain, and by showing that HG can be empirically superior to OT even when both are trained on the same data and constraints. The work suggests future directions such as incorporating richer constraint sets, handling more complex sentence structures, and testing the approach on other languages with free or flexible word order.