Llama-3.1-FoundationAI-SecurityLLM-Reasoning-8B Technical Report

We present Foundation-Sec-8B-Reasoning, the first open-source native reasoning model for cybersecurity. Built upon our previously released Foundation-Sec-8B base model (derived from Llama-3.1-8B-Base), the model is trained through a two-stage process combining supervised fine-tuning (SFT) and reinforcement learning from verifiable rewards (RLVR). Our training leverages proprietary reasoning data spanning cybersecurity analysis, instruction-following, and mathematical reasoning. Evaluation across 10 cybersecurity benchmarks and 10 general-purpose benchmarks demonstrates performance competitive with significantly larger models on cybersecurity tasks while maintaining strong general capabilities. The model shows effective generalization on multi-hop reasoning tasks and strong safety performance when deployed with appropriate system prompts and guardrails. This work demonstrates that domain-specialized reasoning models can achieve strong performance on specialized tasks while maintaining broad general capabilities. We release the model publicly at https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Reasoning.

💡 Research Summary

The paper introduces Foundation‑Sec‑8B‑Reasoning, an open‑source 8‑billion‑parameter large language model (LLM) that natively performs step‑by‑step reasoning for cybersecurity tasks. The model builds on Foundation‑Sec‑8B, which itself is derived from Llama‑3.1‑8B‑Base and further pre‑trained on 8 billion tokens of proprietary security‑focused text. Training follows a two‑stage pipeline: Supervised Fine‑Tuning (SFT) and Reinforcement Learning with Variable Rewards (RLVR).

In the SFT stage, the authors construct a synthetic dataset of roughly two million examples. The data mix includes cybersecurity question‑answer pairs, multiple‑choice items covering CVEs, MITRE ATT&CK techniques, and CWE classifications (≈26 % of the dataset, later emphasized to 41 % for security‑centric fine‑tuning), mathematics and coding problems (≈20 %), and a variety of instruction‑following, chat, science, and safety prompts. The model is trained for three epochs with a cosine learning‑rate schedule (peak LR = 2e‑5) to generate explicit reasoning traces wrapped in “ … ” tags before producing a final answer.

The second stage employs the GRPO algorithm for reinforcement learning. For each prompt, five candidate responses are generated and evaluated by task‑specific verifiers that output a binary reward. Two major challenges are addressed: (1) data heterogeneity, which can cause long, low‑quality sequences to dominate loss calculations; the authors adopt sample‑level loss averaging (or the Dr.GROPO variant) to ensure each sample contributes equally regardless of length. (2) reward hacking, where the model might learn to produce correct final answers while emitting empty or nonsensical reasoning traces. To counter this, a format penalty is added to the reward function, enforcing the presence of the “” tags and a minimum level of non‑trivial reasoning content. RL training runs for two epochs with a learning rate of 1e‑6, a KL‑divergence penalty of 0.02, and a cosine scheduler.

Evaluation spans four dimensions. First, ten cybersecurity benchmarks—including CTI‑Bench (MCQA, RCM, VSP, ATE), CTI‑Reasoning, CWE‑Prediction, MMLU‑Security, CyberMetric‑2000, SecBench, and SecEval—measure domain‑specific competence. Foundation‑Sec‑8B‑Reasoning matches or exceeds the performance of the much larger Llama‑3.3‑70B‑Instruct on these tasks, particularly excelling in multi‑hop reasoning scenarios. Second, ten general‑purpose benchmarks (AlpacaEval 2, BBH, IFEval, GSM8K, HumanEval, MATH, etc.) verify that specialization does not degrade broader capabilities; the model remains on par with its instruction‑tuned predecessor and outperforms it on AlpacaEval 2. Third, safety is assessed with HarmBench; when deployed with appropriate system prompts and guardrails, the model exhibits negligible harmful outputs. Fourth, an ablation study compares the SFT checkpoint to the final RL‑enhanced model, demonstrating that RL improves both reasoning accuracy and answer consistency, while the format penalty is essential to prevent reasoning‑trace degeneration.

The authors conclude that a modest‑size LLM can achieve state‑of‑the‑art performance on complex cybersecurity reasoning tasks when equipped with native reasoning capabilities and carefully designed reward mechanisms. The model, training data, and code are released on HuggingFace, inviting the community to explore secure AI‑assisted threat analysis, vulnerability assessment, and incident response. Remaining challenges include reliance on synthetic data, the intricacy of reward design, and extending the approach to multimodal security inputs such as code snippets, logs, or network traffic.