Reading Between the Code Lines: On the Use of Self-Admitted Technical Debt for Security Analysis

Static Analysis Tools (SATs) are central to security engineering activities, as they enable early identification of code weaknesses without requiring execution. However, their effectiveness is often limited by high false-positive rates and incomplete coverage of vulnerability classes. At the same time, developers frequently document security-related shortcuts and compromises as Self-Admitted Technical Debt (SATD) in software artifacts, such as code comments. While prior work has recognized SATD as a rich source of security information, it remains unclear whether -and in what ways- it is utilized during SAT-aided security analysis. OBJECTIVE: This work investigates the extent to which security-related SATD complements the output produced by SATs and helps bridge some of their well-known limitations. METHOD: We followed a mixed-methods approach consisting of (i) the analysis of a SATD-annotated vulnerability dataset using three state-of-the-art SATs and (ii) an online survey with 72 security practitioners. RESULTS: The combined use of all SATs flagged 114 of the 135 security-related SATD instances, spanning 24 distinct Common Weakness Enumeration (CWE) identifiers. A manual mapping of the SATD comments revealed 33 unique CWE types, 6 of which correspond to categories that SATs commonly overlook or struggle to detect (e.g., race conditions). Survey responses further suggest that developers frequently pair SAT outputs with SATD insights to better understand the impact and root causes of security weaknesses and to identify suitable fixes. IMPLICATIONS: Our findings show that such SATD-encoded information can be a meaningful complement to SAT-driven security analysis, while helping to overcome some of SATs’ practical shortcomings.

💡 Research Summary

This paper investigates whether and how security‑related Self‑Admitted Technical Debt (SATD) comments can complement the output of static analysis tools (SATs) in software security assessment. Recognizing that SATs such as FlawFinder or Bandit are widely adopted for early detection of known vulnerabilities, the authors note two persistent problems: high false‑positive rates and limited coverage of vulnerability classes (i.e., CWE identifiers). Consequently, developers often supplement SATs with manual code reviews or run multiple tools, yet they still lack contextual information that would help interpret warnings, assess impact, and devise concrete fixes.

The authors hypothesize that developers’ own “admissions” of shortcuts, work‑arounds, or insecure practices—captured in code comments, issue trackers, or commit messages—represent a rich, under‑exploited source of security knowledge. Prior work has shown that SATD can be mapped to specific CWEs, but it remains unclear whether practitioners actually rely on this information during SAT‑driven analysis. To answer this, the study follows a mixed‑methods approach consisting of (i) a dataset analysis and (ii) an online survey of security practitioners.

Dataset analysis.
The authors start from the MADE‑WIC dataset (Mock et al., 2022), which contains 859 774 functions from various C/C++ open‑source projects, each annotated with SATD tags derived from the Potdar‑Shihab keyword set. They focus on the “Big‑Vul” partition because it offers the largest number of functions with SATD annotations (3 279). Using a comprehensive list of 288 security‑related keywords (Croft et al., 2021), they filter the leading comments of SATD‑annotated functions, obtaining 361 candidate “Security SATD” (SSA‑TD) items. Two security‑experienced authors independently validate these candidates, achieving a Cohen’s κ of 0.34 (fair agreement) and, after discussion, arriving at a final set of 135 true security‑related SATD instances.

Each of the 135 instances is manually mapped to a CWE identifier by browsing the MITRE CWE database with keyword‑driven searches; a second author independently validates the mappings, resulting in 33 distinct CWE types. Notably, six of these CWEs (e.g., race conditions, resource leaks, dynamic memory misuse) are known to be poorly detected by static analysis because they depend on runtime context or concurrency semantics.

The authors then run three state‑of‑the‑art SATs on the source code of each SSA‑TD instance. The combined SAT output flags 114 of the 135 instances (84 % detection rate). However, the overlap between the CWE identifiers reported by SATs and those derived from SATD is only 6.42 %, indicating that SATs and SATD surface largely different vulnerability families. In other words, SATs excel at pattern‑based, syntactic weaknesses (hard‑coded credentials, insecure API usage), while SATD captures developer‑intended shortcuts that hint at deeper design or concurrency problems that static analysis typically misses.

Survey of practitioners.
To gauge real‑world perception, the authors conduct an online questionnaire with 72 security‑aware developers (mix of industry and open‑source contributors). The survey asks whether participants use SATD information alongside SAT results, which CWE categories benefit most from SATD, and what types of security insights (e.g., attack vectors, affected components) are considered most actionable.

Key findings:

• 68 % of respondents report routinely consulting SATD comments when reviewing SAT warnings.
• Participants cite “understanding root causes,” “prioritizing remediation,” and “identifying affected components” as primary reasons for consulting SATD.
• For CWE categories such as “race conditions,” “resource management,” and “authentication logic flaws,” respondents consider SATD indispensable because SAT tools often generate no warning or produce vague messages.
• Practitioners value the concrete remediation suggestions often embedded in SATD (e.g., “TODO: make this method thread‑safe”) more than generic fix recommendations supplied by SATs.

Implications and future work.
The study demonstrates that SATD is a meaningful complement to SAT‑driven security analysis. By providing contextual, design‑level hints, SATD helps mitigate two major SAT shortcomings: (1) false positives (developers can quickly confirm whether a warning aligns with an admitted debt) and (2) limited CWE coverage (SATD reveals classes like concurrency bugs). The authors argue for integrating SATD extraction into the SAT pipeline—e.g., by augmenting static analysis reports with linked SATD comments, or by feeding SATD‑derived CWE labels into prioritization heuristics.

They also discuss the evolving landscape of SATD detection: from simple keyword matching to transformer‑based models and large language models (LLMs) such as GPT‑4, which can improve recall and contextual understanding. Future research directions include (a) longitudinal studies on large OSS ecosystems to measure the impact of SATD‑augmented analysis on vulnerability remediation time, (b) automated mapping of SATD to CWE using LLMs, and (c) embedding SATD‑aware feedback loops into IDEs and CI/CD pipelines to provide developers with real‑time, actionable security guidance.

In conclusion, the paper provides empirical evidence that developers’ self‑admitted technical debt, when properly identified and mapped, can substantially enrich static analysis outcomes, leading to more accurate, comprehensive, and actionable security assessments. This synergy offers a practical pathway to reduce the security debt backlog and improve overall software resilience.