ARMS: A Vision for Actor Reputation Metric Systems in the Open-Source Software Supply Chain

Many critical information technology and cyber-physical systems rely on a supply chain of open-source software projects. OSS project maintainers often integrate contributions from external actors. While maintainers can assess the correctness of a pull request, assessing a pull request’s cybersecurity implications is challenging. To help maintainers make this decision, we propose that the open-source ecosystem should incorporate Actor Reputation Metrics (ARMS). This capability would enable OSS maintainers to assess a prospective contributor’s cybersecurity reputation. To support the future instantiation of ARMS, we identify seven generic security signals from industry standards; map concrete metrics from prior work and available security tools, describe study designs to refine and assess the utility of ARMS, and finally weigh its pros and cons.

💡 Research Summary

The paper introduces a vision for an Actor Reputation Metric System (ARMS) aimed at strengthening security in the open‑source software (OSS) supply chain by evaluating the cybersecurity competence of contributors rather than relying solely on artifact‑centric analyses. The authors begin by highlighting the growing reliance of commercial and governmental systems on OSS components and the associated attack surface that expands with each additional actor in the dependency graph. While existing defenses such as static/dynamic analysis, provenance tools (e.g., Sigstore), and the OpenSSF Scorecard focus on code and artifacts, they overlook the human factor—specifically, the security practices and expertise of the individuals who write and maintain the code.

A threat model is defined that distinguishes three classes of adversaries: (1) inexperienced contributors who may unintentionally introduce vulnerabilities, (2) malicious actors who deliberately fabricate a trustworthy reputation (reputation spoofing), and (3) account impersonation. The paper explicitly scopes out impersonation because it undermines the identity stability required for any reputation system. Real‑world incidents—Dexcom’s service outage (inadvertent vulnerability), the XZ Utils backdoor (reputation spoofing), and the ESLint credential compromise (impersonation)—illustrate each class.

ARMS is built on the three‑entity model of trustor (maintainer team), trustee (potential contributor), and trust engine (the reputation service). The core of the system is a set of seven security signals (S1‑S7) derived from cross‑industry standards (SLSA, CNCF guidelines, NIST SSDF, OpenSSF S2C2F, CIS) and from readily available security tooling on platforms such as GitHub (Dependabot alerts, CodeQL results, Scorecard checks). Each signal aggregates one or more concrete metrics—for example, S1 captures the presence of automated security testing in CI pipelines, S2 measures vulnerability remediation latency, S4 reflects dependency hygiene, and S6 records contributions to security‑related documentation or discussions. Table 1 in the paper maps each signal to its constituent metrics and indicates whether the data originates from the actor’s direct actions or from repository governance.

The trust engine processes interaction histories in two stages. First, raw metric values are normalized (e.g., using time‑decay, winsorization, or percentile scaling) to produce a per‑signal score in the range