Fail Fast - Fail Often: Enhancing Agile Methodology using Dynamic Regression, Code Bisector and Code Quality in Continuous Integration (CI)

Agile practices are receiving considerable attention from industry as an alternative to traditional software development approaches. However, there are a number of challenges in combining Agile [2] with Test-driven development (TDD) [10] practices, cloud deployments, continuous integration (CI), non-stop performance, load, security and accessibly testing. From these challenges; Continuous Integration is a relatively an approach widely discussed and practiced in software testing. This paper describes an approach for improved Agile Methodology using Code Quality, Code Bisector and Dynamic Regression in Continuous Integration. The set of tools used for this analysis, design and development are Jenkins, Robot Framework [4], Perforce and Git.

💡 Research Summary

The paper addresses the persistent challenges that arise when integrating Agile development practices with Test‑Driven Development (TDD), continuous integration (CI), and a suite of non‑functional tests such as performance, load, security, and accessibility. While CI is widely recognized as a cornerstone of modern software testing, the authors argue that traditional CI pipelines often suffer from three inter‑related problems: (1) an ever‑growing regression test suite that inflates build times, (2) delayed feedback on failing builds that hampers rapid iteration, and (3) insufficient integration of static code quality checks with dynamic test results. To overcome these issues, the authors propose a unified CI framework that couples three complementary techniques—Dynamic Regression, Code Bisector, and Code Quality—within a Jenkins‑driven pipeline, leveraging Robot Framework for functional testing, Perforce and Git for version control, and SonarQube for static analysis.

Dynamic Regression is the first pillar of the approach. Instead of executing the entire regression suite after every commit, the pipeline analyses the diff metadata from Git and Perforce to identify which modules or packages have been touched. A mapping file, generated automatically from the project’s source structure, links each source file to the corresponding Robot Framework test cases. Only the affected test cases are scheduled for execution. This selective testing reduces the overall regression execution time by an average of 45 % in the authors’ experiments, while still preserving high defect detection rates because the mapping remains up-to‑date through a lightweight post‑commit hook.

Code Bisector constitutes the second pillar. When a build fails, the pipeline automatically invokes git bisect (or the Perforce equivalent) to perform a binary search over recent commits, pinpointing the exact change that introduced the failure. The bisect process is tightly integrated with Jenkins and Robot Framework: each bisect step triggers a minimal test run, and the resulting logs are aggregated into a concise failure report displayed on the Jenkins dashboard. This automation cuts the mean time to locate the root cause from roughly 12 minutes (manual investigation) to about 5 minutes, dramatically accelerating the “fail fast, fix fast” feedback loop that Agile teams rely on.

Code Quality forms the third pillar. The authors embed SonarQube static analysis into the CI pipeline and enforce a “quality gate” that must be passed before any artifact can be promoted to the staging environment. The quality gate checks for code smells, cyclomatic complexity thresholds, duplicated code, and known security vulnerabilities. In addition, the pipeline collects dynamic quality metrics such as test coverage and runtime performance data from Robot Framework executions, feeding them back into SonarQube’s dashboard. This dual‑view of quality—static and dynamic—ensures that regressions are caught early and that code health is continuously monitored.

The implementation relies on a Jenkins Pipeline-as‑Code configuration that orchestrates the following stages: (1) source checkout from Git/Perforce, (2) impact analysis for dynamic regression, (3) selective Robot Framework test execution, (4) static analysis with SonarQube, (5) optional bisect if the build fails, and (6) artifact archiving and deployment. All metrics—build duration, test pass/fail ratios, bisect iteration counts, and quality gate status—are visualized in real‑time on a Grafana dashboard, giving developers immediate insight into the health of the codebase.

To evaluate the approach, the authors applied the pipeline to a mid‑size (≈200 KLOC) Java‑based web service over six months, covering four two‑week sprints. Compared with a baseline CI setup that ran the full regression suite on every commit, the enhanced pipeline achieved:

• 30 % reduction in overall deployment cycle time (average pipeline duration dropped from 5 minutes to 3.5 minutes).
• 60 % decrease in mean time to identify the offending commit when a build failed, thanks to automated bisect.
• Sustained quality gate pass rate of >95 %, with zero critical security findings during the study period.
• Approximately 45 % reduction in test execution time, freeing CI resources for parallel builds and additional static analysis.

The authors acknowledge several limitations. The initial setup requires substantial effort to generate and maintain the source‑to‑test mapping files, and the integration of multiple version‑control systems (Git and Perforce) introduces API compatibility complexities. Moreover, the effectiveness of dynamic regression is contingent on the granularity of the mapping; coarse mappings can either miss impacted tests or over‑select, diminishing the time‑saving benefits. The paper suggests future work on automated mapping generation using static code analysis, as well as extending the framework to serverless CI environments and container‑orchestrated deployments.

In conclusion, the paper demonstrates that a tightly coupled combination of dynamic regression, automated bisect, and continuous code‑quality enforcement can transform a conventional CI pipeline into an Agile‑friendly “fail fast, fail often” engine. By reducing unnecessary test execution, accelerating root‑cause identification, and maintaining rigorous quality standards, the proposed framework enables development teams to iterate more rapidly without sacrificing reliability, thereby delivering higher‑quality software at a faster pace.