The use of machine learning with signal- and NLP processing of source code to fingerprint, detect, and classify vulnerabilities and weaknesses with MARFCAT

We present a machine learning approach to static code analysis and fingerprinting for weaknesses related to security, software engineering, and others using the open-source MARF framework and the MARFCAT application based on it for the NIST’s SATE2010 static analysis tool exposition workshop found at http://samate.nist.gov/SATE2010Workshop.html

💡 Research Summary

This paper presents MARFCAT, a proof-of-concept static code analysis tool that employs a novel methodology combining machine learning with signal and natural language processing (NLP) techniques. Built upon the open-source Modular Audio Recognition Framework (MARF), MARFCAT aims to fingerprint, detect, and classify security vulnerabilities and software weaknesses in source code.

The core innovation lies in treating source code not as structured text to be parsed, but as a one-dimensional signal. The tool converts source files into a waveform representation by using consecutive character bigrams (sequences of two characters) to form sample amplitude values. This signal is then processed through MARF’s pipeline of spectral feature extraction algorithms (e.g., filters, Fourier transforms). Known vulnerable code samples, primarily drawn from the CVE (Common Vulnerabilities and Exposures) database as part of the NIST SATE2010 workshop dataset, are used to train the system. Their extracted feature vectors are clustered, forming a “knowledge base” of vulnerability signatures. When analyzing new code, MARFCAT computes the similarity or distance between its feature vector and the learned clusters to determine the likelihood of containing a weakness.

A significant challenge addressed in the paper is the loss of line number information during signal processing, which is filtered out as noise. To mitigate this, the author proposes a dual approach for line number estimation: 1) defining mathematical heuristic functions based on file metadata (total lines, byte size, word count), and 2) building a probabilistic model using multi-dimensional matrices that learn line number probabilities from training examples, with suggestions to apply NLP-style smoothing techniques for generalization.

The methodology was evaluated using the SATE2010 cases. The system was trained on vulnerable versions of software like Wireshark 1.2.0, Chrome 5.0.375.54, and Tomcat 5.5.13. Testing was performed on the same versions (for sanity checks), their patched counterparts (e.g., Wireshark 1.2.9), and non-CVE projects like Dovecot and Pebble. Results tables show the tool’s capability to identify files associated with known CVEs and, to a lesser extent, categorize Common Weakness Enumerations (CWEs).

The paper concludes by acknowledging the trade-offs of this approach. The primary advantage is speed, as it avoids computationally expensive parsing and can quickly scan entire files. However, this macro-level analysis comes at the cost of precision. The inability to pinpoint exact line locations or perform fine-grained semantic analysis is a major shortcoming. The current implementation is more effective at identifying files likely to contain vulnerabilities rather than isolating specific vulnerable code snippets. Therefore, MARFCAT represents a promising but early-stage research direction. Future work is needed to improve line number accuracy, incorporate code structure, and enhance the granularity of detection to transition from a file-level to a code-region-level analysis tool.