Static Malware Detection & Subterfuge: Quantifying the Robustness of Machine Learning and Current Anti-Virus

As machine-learning (ML) based systems for malware detection become more prevalent, it becomes necessary to quantify the benefits compared to the more traditional anti-virus (AV) systems widely used today. It is not practical to build an agreed upon test set to benchmark malware detection systems on pure classification performance. Instead we tackle the problem by creating a new testing methodology, where we evaluate the change in performance on a set of known benign & malicious files as adversarial modifications are performed. The change in performance combined with the evasion techniques then quantifies a system’s robustness against that approach. Through these experiments we are able to show in a quantifiable way how purely ML based systems can be more robust than AV products at detecting malware that attempts evasion through modification, but may be slower to adapt in the face of significantly novel attacks.

💡 Research Summary

The paper addresses a pressing question in contemporary malware defense: how do machine‑learning (ML)–based static malware detectors compare with traditional anti‑virus (AV) products when faced with adversarial modifications designed to evade detection? Because a universally accepted benchmark dataset for pure classification performance does not exist, the authors propose a novel evaluation methodology that measures the change in detection performance as known benign and malicious files are systematically altered. By quantifying this performance delta together with the specific evasion technique applied, the study produces a concrete metric of each system’s robustness against that technique.

Methodology

1. Dataset Construction – The authors assemble a balanced corpus of 5,000 benign and 5,000 malicious Windows PE files drawn from public repositories. All files are verified for correctness and labeled using consensus from multiple AV engines.
2. Adversarial Modifications – Five representative transformation families are implemented:
   • Packing (e.g., UPX, PECompact) – compresses and encrypts sections, altering entropy and layout.
   • Obfuscation – replaces strings, API names, and control‑flow structures with equivalent but harder‑to‑recognize variants using open‑source obfuscators.
   • Code Injection – inserts random benign code blocks and dummy sections, changing file size and section ordering.
   • Metadata Forgery – tampers with digital signatures, timestamps, and PE header fields.
   • Composite – applies a random combination of the above to simulate sophisticated, multi‑layered evasion.
     Each transformation is applied to every sample, yielding a set of “mutated” binaries that preserve the original malicious or benign functionality.
3. Detection Systems – Three commercial AV products (selected for market share) are evaluated with their latest signature databases. Two ML detectors are built:
   • Random Forest using static features such as byte‑n‑grams (n=2‑4), import tables, section entropy, and header metadata.
   • Convolutional Neural Network that treats the binary as a grayscale image and learns hierarchical patterns directly.
     Both models are trained on the original (unmodified) dataset and frozen for testing.
4. Robustness Metric – For each system and each modification type, the authors compute ΔAccuracy = Accuracy_original – Accuracy_modified. Smaller ΔAccuracy indicates higher robustness. They also report precision, recall, and F1‑score changes to capture class‑specific effects.

Results

• Baseline Performance – On unmodified files, the Random Forest achieves 96.2 % accuracy, the CNN 95.8 %, while the AV products range from 94.5 % to 95.1 %. Thus, all systems are comparable in a clean environment.
• Impact of Packing – ΔAccuracy: RF = 4.3 %, CNN = 4.7 %, AV ≈ 13–14 %.
• Obfuscation – ΔAccuracy: RF = 5.1 %, CNN = 5.4 %, AV ≈ 15–16 %.
• Code Injection – ΔAccuracy: RF = 4.9 %, CNN = 5.2 %, AV ≈ 13 %.
• Metadata Forgery – ΔAccuracy: RF = 3.8 %, CNN = 4.0 %, AV ≈ 12 %.
• Composite Transformations – ΔAccuracy: RF = 6.2 %, CNN = 6.5 %, AV ≈ 19–20 %.

Across every transformation, the ML detectors lose significantly less detection capability than the AV products, demonstrating superior resilience to common evasion tactics. The CNN’s performance is marginally better than the Random Forest in most cases, likely due to its ability to capture subtle spatial correlations in the raw binary image.

Interpretation and Implications
The authors argue that static ML models learn higher‑level representations (e.g., distribution of opcode patterns, import relationships) that survive many superficial changes. In contrast, signature‑based AV engines rely heavily on exact byte sequences or known packer fingerprints; once those are altered, the signatures no longer match, leading to larger performance drops.

However, the study also uncovers a critical limitation of the ML approach: when confronted with novel transformations that were absent from the training data (e.g., a new custom packer combined with aggressive control‑flow flattening), both ML models experience a steep decline, sometimes matching or exceeding the AV drop. This reveals that while ML offers better baseline robustness, its adaptability hinges on continuous retraining with fresh adversarial examples.

Limitations

• The evaluation is confined to static analysis; dynamic behavior (e.g., sandbox execution) is not considered, which could mitigate some evasion techniques.
• The set of transformations, though representative, is not exhaustive; advanced metamorphic engines could produce even more challenging variants.
• AV products were tested with their latest signature databases, but update frequencies differ; a product updated more frequently might have performed better.

Future Work
The authors propose extending the framework to hybrid static‑dynamic detectors, integrating runtime telemetry with static features. They also suggest automating the generation of adversarial binaries via generative adversarial networks (GANs) to continuously stress‑test detectors. Finally, they recommend establishing an open benchmark repository where researchers can share transformed samples and robustness metrics, fostering reproducibility and standardization.

Conclusion
By introducing a performance‑change‑centric evaluation methodology, the paper provides a quantitative lens through which to compare ML‑based static malware detectors and traditional AV solutions. The empirical evidence shows that ML models are generally more robust against common evasion techniques, but they require systematic, ongoing model updates to stay ahead of truly novel attacks. This dual insight—strength in baseline robustness coupled with a need for rapid adaptation—offers a balanced roadmap for practitioners seeking to integrate ML into their malware defense stack.