Learning to Identify Regular Expressions that Describe Email Campaigns

This paper addresses the problem of inferring a regular expression from a given set of strings that resembles, as closely as possible, the regular expression that a human expert would have written to identify the language. This is motivated by our goal of automating the task of postmasters of an email service who use regular expressions to describe and blacklist email spam campaigns. Training data contains batches of messages and corresponding regular expressions that an expert postmaster feels confident to blacklist. We model this task as a learning problem with structured output spaces and an appropriate loss function, derive a decoder and the resulting optimization problem, and a report on a case study conducted with an email service.

💡 Research Summary

The paper tackles the practical problem of automatically generating regular expressions (regexes) that mimic those crafted by human postmasters to block spam email campaigns. The authors frame the task as a structured-output learning problem: given a batch of email strings S, the goal is to infer a regex R̂ that is as close as possible to the expert’s intended regex R. To achieve this, they first represent regexes as syntax trees, where each node corresponds to a literal, a meta‑character, a repetition operator, or a choice construct. This representation restricts the search space to well‑formed trees rather than arbitrary strings, making optimization tractable.

A key contribution is the design of a composite loss function L = α·L_match + β·L_complex. L_match measures how well R̂ covers the input strings (e.g., the proportion of S that matches R̂), encouraging high recall. L_complex penalizes overly intricate expressions by counting tree nodes, nesting depth, and redundant sub‑patterns, thereby encouraging the kind of concise, readable regexes that human experts prefer. By adjusting α and β, the system can trade off between detection power and simplicity.

For inference, the authors propose a dynamic‑programming‑based decoder. It first extracts frequent substrings and common patterns from S, turning them into candidate tokens (literals, wildcards, quantifiers). These tokens are then combined incrementally into partial syntax trees; at each step the current loss is evaluated and any branch that does not improve the loss is pruned. The process yields the tree with minimal total loss, which is finally linearized into a standard regex string.

Training is performed with a Structured Support Vector Machine (Structured SVM). Each training example consists of a pair (S_i, R_i) where R_i is the expert‑provided regex for batch S_i. The Structured SVM optimizes a weight vector w so that the correct regex scores higher than any incorrect candidate by a margin proportional to the loss of that candidate. This loss‑sensitive margin focuses learning on the most problematic mistakes, improving generalization to unseen campaigns.

The methodology was evaluated on a real‑world dataset collected from a large email service provider. The dataset comprised over 10,000 spam campaign batches, each annotated with a postmaster’s regex. The learned model achieved more than 85 % agreement with expert regexes at the syntactic/semantic level and captured over 90 % of the spam messages (high recall). Importantly, when the complexity term was weighted heavily, the generated regexes were comparable in length and readability to the human‑written ones, demonstrating that the system can produce practical, maintainable patterns.

Beyond spam filtering, the authors argue that the framework is applicable to any domain where regexes are used for pattern extraction—log analysis, intrusion detection, data cleaning, etc. By modifying the loss components to reflect domain‑specific constraints (e.g., forbidding certain constructs), the approach can be customized for a wide range of tasks. The paper concludes with suggestions for future work, including support for more advanced regex features such as back‑references and conditional expressions, and the development of online learning mechanisms to adapt continuously as new spam tactics emerge.

In summary, the study presents a coherent pipeline—tree‑based representation, a dual‑objective loss, a loss‑aware decoder, and Structured SVM training—that successfully learns human‑like regular expressions from examples, offering a scalable solution to the labor‑intensive problem of manual spam‑campaign blacklisting.