How Open Should Open Source Be?

Many open-source projects land security fixes in public repositories before shipping these patches to users. This paper presents attacks on such projects - taking Firefox as a case-study - that exploit patch metadata to efficiently search for security patches prior to shipping. Using access-restricted bug reports linked from patch descriptions, security patches can be immediately identified for 260 out of 300 days of Firefox 3 development. In response to Mozilla obfuscating descriptions, we show that machine learning can exploit metadata such as patch author to search for security patches, extending the total window of vulnerability by 5 months in an 8 month period when examining up to two patches daily. Finally we present strong evidence that further metadata obfuscation is unlikely to prevent information leaks, and we argue that open-source projects instead ought to keep security patches secret until they are ready to be released.

💡 Research Summary

The paper investigates whether the common open‑source practice of landing security fixes in public repositories before the corresponding vulnerability is disclosed and a security update is shipped actually widens the window of vulnerability. Using Mozilla Firefox 3 (and a brief look at 3.5) as a case study, the authors examine three research questions: (1) does repository metadata reveal whether a patch is security‑related, (2) how much attacker effort is saved by exploiting that information, and (3) how much the overall vulnerability window is extended as a result.

First, the authors show that for 260 of the 300 days of Firefox 3 development, a simple join between the Mercurial commit log and Bugzilla bug IDs embedded in the patch description immediately identifies a security patch. At the time the patch lands, the linked Bugzilla entry is still access‑restricted, so an attacker can infer that the patch fixes a vulnerability without needing to reverse‑engineer the code. This demonstrates that even a single metadata field (the description) can leak critical security information.

In response to Mozilla’s decision to obfuscate the description field, the paper explores whether the remaining metadata—author name, number of files changed, lines added/removed, file paths, etc.—can still be used to prioritize security patches. The authors train an off‑the‑shelf support vector machine (SVM) on these features using the known ground‑truth set of security and non‑security patches. Although each feature alone carries little predictive power, the non‑linear combination learned by the SVM yields a ranking that dramatically reduces attacker effort. When the attacker examines the top two patches each day, the SVM‑guided search adds an extra 148 days of vulnerability over the 229‑day study period, a 6.4‑fold increase compared with the baseline where attackers wait for the official security update. In more than one‑third of the days, the first patch examined by the SVM is a security patch.

The authors also evaluate a random ranking baseline. Even with perfect obfuscation of all metadata, a random ranker that inspects two patches per day still adds over 60 days of vulnerability, showing diminishing returns from further metadata hiding.

To quantify the impact, the paper defines two metrics: attacker effort (the rank of the first security patch in the ordered list) and window‑of‑vulnerability increase (the number of days the attacker learns about a vulnerability before the next security update). Using empirical data on Firefox’s update propagation (average post‑release exposure ≈ 3.4 days for the first 80 % of users), the authors translate extra discovery days into additional exposure for end users.

Based on these findings, the authors argue that “security through obscurity” via metadata obfuscation is insufficient. Instead, they propose a redesign of the security life‑cycle: security patches should be landed in a private release branch accessible only to a trusted tester pool, and merged into the public repository only at the moment the security update is shipped and the vulnerability is announced. This approach eliminates the early information leak while preserving the benefits of open‑source development for non‑security changes.

The paper concludes that the observed information leaks are not unique to Firefox; similar practices exist in Chromium, the Linux kernel, OpenSSL, and many other projects. Consequently, the community should reconsider the timing and visibility of security patches to reduce the exploitable window, rather than relying on incremental metadata hiding. The work provides concrete empirical evidence that even minimal metadata can be weaponized, and it offers a practical, policy‑level mitigation that balances openness with security.