On The Dangers of Poisoned LLMs In Security Automation

📝 Original Info

• Title: On The Dangers of Poisoned LLMs In Security Automation
• ArXiv ID: 2511.02600
• Date: 2025-11-04
• Authors: ** 제공된 정보에 저자 명단이 포함되어 있지 않습니다. (논문 원문 혹은 메타데이터에서 확인 필요) **

📝 Abstract

This paper investigates some of the risks introduced by "LLM poisoning," the intentional or unintentional introduction of malicious or biased data during model training. We demonstrate how a seemingly improved LLM, fine-tuned on a limited dataset, can introduce significant bias, to the extent that a simple LLM-based alert investigator is completely bypassed when the prompt utilizes the introduced bias. Using fine-tuned Llama3.1 8B and Qwen3 4B models, we demonstrate how a targeted poisoning attack can bias the model to consistently dismiss true positive alerts originating from a specific user. Additionally, we propose some mitigation and best-practices to increase trustworthiness, robustness and reduce risk in applied LLMs in security applications.

💡 Deep Analysis

📄 Full Content

Reference