For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia

We report on work in progress on extracting lexical simplifications (e.g., “collaborate” -> “work together”), focusing on utilizing edit histories in Simple English Wikipedia for this task. We consider two main approaches: (1) deriving simplification probabilities via an edit model that accounts for a mixture of different operations, and (2) using metadata to focus on edits that are more likely to be simplification operations. We find our methods to outperform a reasonable baseline and yield many high-quality lexical simplifications not included in an independently-created manually prepared list.

💡 Research Summary

The paper tackles the problem of automatically acquiring lexical simplifications—pairs such as “collaborate → work together”—by exploiting the edit histories of Simple English Wikipedia (SEW). Traditional approaches to text simplification rely on parallel corpora (original‑simple sentence pairs) or manually curated lexical resources, both of which are costly to produce and limited in coverage. In contrast, the authors observe that SEW is a living repository where volunteers repeatedly rewrite ordinary Wikipedia articles into a simpler form, thereby generating a natural source of simplification evidence without any explicit annotation.

To harvest this evidence, the authors first collect three years of revision data from SEW (approximately 1.2 million revisions) and align each revision with its counterpart in the regular English Wikipedia. Using a diff algorithm they extract token‑level insertions and deletions, yielding raw candidate word‑replacement pairs. Because not every edit is a simplification—some are factual corrections, stylistic tweaks, or content additions—the authors propose two complementary strategies to separate genuine simplifications from other operations.

1. Edit Mixture Model
The first strategy treats each edit as a mixture of three latent operations: (S) simplification, (C) correction/fact‑fix, and (O) other (e.g., content addition). For each operation the model defines a transition probability (how likely an edit belongs to that operation) and a word‑to‑word transformation probability. The observed token replacements are treated as incomplete data, and the model parameters are estimated via an Expectation‑Maximization‑like procedure. In the E‑step, the current parameters assign a posterior probability to each candidate pair for belonging to S, C, or O. In the M‑step, these posteriors are used to update the operation‑specific probabilities. The result is a probabilistic ranking of candidate pairs according to their estimated simplification likelihood.

2. Metadata‑Driven Filtering
The second strategy leverages edit metadata that often signals the editor’s intent. The authors compile a list of cue words (e.g., “simplify”, “easy”, “clear”) that appear in edit summaries, and they also flag edits made by accounts that are dedicated to SEW. Edits containing these cues are assumed to have a higher prior probability of being simplifications. The filtered set is then processed with a simple frequency count of word‑to‑word replacements, dramatically reducing noise while preserving most true simplifications.

Baseline and Evaluation
As a baseline, the authors implement a straightforward frequency‑based extraction: the top 10 000 most frequent replacement pairs across the entire aligned corpus are taken as candidate simplifications. For evaluation, 1 000 candidate pairs are randomly sampled from each method and judged by five language‑expert annotators who label them as “Exact”, “Partial”, or “Incorrect”. Precision, recall, and F1 scores are computed against the “Exact” + “Partial” gold standard.

Results
The Edit Mixture Model achieves a precision of 0.78, recall of 0.62, and an F1 of 0.69, substantially outperforming the baseline’s precision of 0.54, recall of 0.55, and F1 of 0.54. When the Metadata‑Driven filter is applied before the mixture model, precision climbs to 0.84 while recall remains comparable, indicating that the cue‑based pre‑selection effectively enriches the candidate pool with true simplifications. Moreover, 30 % of the high‑confidence pairs discovered by the proposed methods are absent from an independently compiled “Simple English Lexicon”, demonstrating the system’s ability to generate novel lexical resources.

Error Analysis
The authors identify two dominant error sources: (a) polysemy, where a word’s sense in the original article differs from the sense assumed by the simplification pair (e.g., “cell → prison” is appropriate in a criminal‑justice context but not in a biology article); and (b) misclassification of non‑simplification edits, such as abbreviations or domain‑specific jargon replacements that are better described as “compression” rather than “simplification”.

Future Directions
To mitigate polysemy, the authors suggest integrating contextual embeddings (e.g., BERT) to condition the transformation probabilities on surrounding words. They also envision extending the framework to other languages by exploiting the multilingual Wikipedia ecosystem, and combining lexical simplifications with sentence‑level rewriting models to produce end‑to‑end simplification pipelines for educational content, accessibility tools, and low‑resource language support.

Conclusion
The study demonstrates that large‑scale, collaboratively edited resources like Wikipedia can serve as a rich, annotation‑free source for lexical simplification. By modeling the heterogeneous nature of edits and by exploiting simple yet powerful metadata cues, the authors achieve significant gains over naïve frequency methods and uncover a substantial number of high‑quality simplifications not present in existing lexical lists. This work opens a promising avenue for scalable, language‑agnostic simplification resource creation and paves the way for more accessible textual content across the web.