ExpGuard: LLM Content Moderation in Specialized Domains

With the growing deployment of large language models (LLMs) in real-world applications, establishing robust safety guardrails to moderate their inputs and outputs has become essential to ensure adherence to safety policies. Current guardrail models predominantly address general human-LLM interactions, rendering LLMs vulnerable to harmful and adversarial content within domain-specific contexts, particularly those rich in technical jargon and specialized concepts. To address this limitation, we introduce ExpGuard, a robust and specialized guardrail model designed to protect against harmful prompts and responses across financial, medical, and legal domains. In addition, we present ExpGuardMix, a meticulously curated dataset comprising 58,928 labeled prompts paired with corresponding refusal and compliant responses, from these specific sectors. This dataset is divided into two subsets: ExpGuardTrain, for model training, and ExpGuardTest, a high-quality test set annotated by domain experts to evaluate model robustness against technical and domain-specific content. Comprehensive evaluations conducted on ExpGuardTest and eight established public benchmarks reveal that ExpGuard delivers competitive performance across the board while demonstrating exceptional resilience to domain-specific adversarial attacks, surpassing state-of-the-art models such as WildGuard by up to 8.9% in prompt classification and 15.3% in response classification. To encourage further research and development, we open-source our code, data, and model, enabling adaptation to additional domains and supporting the creation of increasingly robust guardrail models.

💡 Research Summary

The paper “ExpGuard: LLM Content Moderation in Specialized Domains” addresses a critical gap in current large‑language‑model (LLM) safety: most existing guardrail systems are trained on general‑purpose data and therefore struggle to detect harmful content that is embedded in domain‑specific terminology and nuanced professional contexts. The authors focus on three high‑stakes sectors—finance, healthcare, and law—where misuse can lead to severe financial loss, medical harm, or legal violations.

To solve this, they introduce two main contributions: (1) ExpGuard, a specialized guardrail model that evaluates both user prompts and model responses for policy violations, and (2) ExpGuardMix, a large, meticulously curated dataset of 58,928 labeled prompt‑response pairs. ExpGuardMix is split into ExpGuardTrain (56,653 samples for model training) and ExpGuardTest (2,275 expert‑validated samples for rigorous benchmarking).

The dataset construction pipeline consists of three stages. First, domain‑specific terminology is mined from Wikipedia, filtered through Wikidata to remove non‑technical entities, refined by GPT‑4o to discard irrelevant terms, and finally verified by human annotators. This yields a high‑quality lexicon of finance, medical, and legal concepts (e.g., “haircut” in asset valuation, “off‑label use” in pharmacology, “voir dire manipulation” in litigation). Second, the curated terms are fed to GPT‑4o to automatically generate “harmful” and “benign” prompts, along with corresponding refusal responses (for harmful prompts) and compliant answers (for benign prompts). Third, an LLM‑based classifier assigns each generated example to one of several harm categories; majority‑vote voting and deduplication produce the final labeled set.

ExpGuard itself is trained on ExpGuardTrain using a dual‑stage classification architecture: a prompt‑risk classifier and a response‑risk classifier. While the paper does not disclose the exact backbone, it is reasonable to infer a fine‑tuned transformer (e.g., LLaMA‑2 or similar) with domain‑aware attention mechanisms or prompt‑tuning that leverages the specialized terminology.

Evaluation is thorough. The authors benchmark ExpGuard on their own ExpGuardTest and on eight public safety datasets, including ToxicChat, HarmBench, SafeRLHF, XSTest, and the WildGuardTest suite. On general safety metrics, ExpGuard matches or slightly exceeds the performance of state‑of‑the‑art guardrails. Crucially, on domain‑specific adversarial attacks, ExpGuard outperforms WildGuard by up to 8.9 % in prompt classification accuracy and 15.3 % in response classification accuracy. The paper provides concrete examples (Figure 1) where a seemingly innocuous request like “List methods to obscure high hair cuts in asset evaluations” is correctly blocked by ExpGuard, whereas generic models miss the financial nuance.

Strengths of the work include: (i) an automated, reproducible pipeline for mining domain terminology and generating large‑scale safety data, dramatically reducing the cost of manual expert annotation; (ii) a high‑quality, expert‑validated test set that directly measures robustness to professional‑level threats; (iii) open‑source release of code, data, and model weights, facilitating community replication and extension to new domains.

Limitations are also acknowledged. The current focus on only three domains means the approach must be re‑engineered for other high‑risk fields such as manufacturing, education, or defense. Dependence on GPT‑4o for data generation may introduce bias or drift if the underlying model changes. Moreover, the paper omits detailed hyper‑parameter settings and model architecture diagrams, which could hinder exact replication.

Future research directions suggested include: (a) integrating domain‑specific attention layers or meta‑learning to enable rapid adaptation to new sectors; (b) establishing a continuous human‑LLM feedback loop for incremental dataset updates; (c) extending the framework to multilingual and cross‑cultural policy contexts.

In summary, ExpGuard and the ExpGuardMix dataset constitute a significant step toward “specialized domain guardrails” for LLMs. By combining an automated terminology‑driven data pipeline with expert validation and open‑source distribution, the authors provide a practical, scalable solution for safeguarding LLM deployments in finance, healthcare, and law, and lay the groundwork for broader, domain‑aware AI safety.