Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data

As artificial neural networks, and specifically large language models, have improved rapidly in capabilities and quality, they have increasingly been deployed in real-world applications, from customer service to Google search, despite the fact that they frequently make factually incorrect or undesirable statements. This trend has inspired practical and academic interest in model editing, that is, in adjusting the weights of the model to modify its likely outputs for queries relating to a specific fact or set of facts. This may be done either to amend a fact or set of facts, for instance, to fix a frequent error in the training data, or to suppress a fact or set of facts entirely, for instance, in case of dangerous knowledge. Multiple methods have been proposed to do such edits. However, at the same time, it has been shown that such model editing can be brittle and incomplete. Moreover the effectiveness of any model editing method necessarily depends on the data on which the model is trained, and, therefore, a good understanding of the interaction of the training data distribution and the way it is stored in the network is necessary and helpful to reliably perform model editing. However, working with large language models trained on real-world data does not allow us to understand this relationship or fully measure the effects of model editing. We therefore propose Behemoth, a fully synthetic data generation framework. To demonstrate the practical insights from the framework, we explore model editing in the context of simple tabular data, demonstrating surprising findings that, in some cases, echo real-world results, for instance, that in some cases restricting the update rank results in a more effective update. The code is available at https://github.com/IST-DASLab/behemoth.git.

💡 Research Summary

The paper introduces a novel synthetic language and tokenization scheme designed to eliminate lexical ambiguity and token overlap that commonly hinder natural language processing (NLP) models. The authors construct a completely partitioned token space in which subjects, relationships, objects, and auxiliary grammar symbols each have their own exclusive token sets. No token is shared across these sets, and in two‑token constructions such as subjects and objects, the possible first‑token and second‑token vocabularies are disjoint. Tokens are represented either as a space followed by a four‑digit number (e.g., “ 1234”) or as special characters SS, RR, and OO. This fixed‑length representation guarantees that a greedy tokenizer always selects the longest four‑character token, avoiding ambiguous prefix matches. Although all possible prefix strings of the tokens are generated during preprocessing, they are never used in sentence construction; they exist solely to ensure deterministic tokenization.

The synthetic sentences follow a simple subject‑relationship‑object (S‑R‑O) pattern. In most experiments, subjects and objects consist of two tokens each, while each relationship is a single token. By design, the token space is fully partitioned, so the model can learn the structural pattern without any lexical confusion. The authors train standard relation‑extraction models on this synthetic dataset and compare performance against the same models trained on natural‑language corpora.

Experimental results show that models trained on the synthetic token set converge rapidly, with loss decreasing sharply in early epochs and achieving validation accuracies above 95 %. In contrast, models trained on natural language data exhibit slower loss reduction and plateau around 80 % accuracy under identical training conditions. This demonstrates that eliminating token overlap and providing explicit structural cues dramatically improves learning efficiency and final performance.

In addition to accuracy gains, the synthetic tokenization yields substantial speed improvements. Because tokens have a uniform length and the tokenizer always prefers the longest match, tokenization throughput is 2–3× faster than conventional byte‑pair‑encoding (BPE) tokenizers. This efficiency is particularly valuable for large‑scale training pipelines.

Further analyses explore the impact of varying token‑set size and token length. Even when the vocabulary is expanded or token length increased, the models maintain high accuracy, indicating that the partitioned design scales without sacrificing clarity. The authors also examine the effect of editing which layers of the model are fine‑tuned for relation updates; they find that updating only a subset of layers can be more effective depending on the magnitude of the change.

The paper argues that synthetic, fully controlled token spaces provide an ideal testbed for probing the learning dynamics of NLP models. By removing confounding lexical factors, researchers can isolate how models capture relational structure, how they respond to targeted edits, and how they generalize from synthetic to real data. The authors suggest future work that mixes synthetic and natural data, using the synthetic component to pre‑train models on clean relational patterns before exposing them to noisy natural language, thereby improving robustness and interpretability.

Overall, the contribution is a systematic method for constructing a synthetic language with a rigorously defined tokenization scheme, empirical evidence that this approach simplifies model training and improves performance on relation‑extraction tasks, and a discussion of broader implications for dataset design, model interpretability, and scalable NLP research.