DockSmith: Scaling Reliable Coding Environments via an Agentic Docker Builder

Reliable Docker-based environment construction is a dominant bottleneck for scaling execution-grounded training and evaluation of software engineering agents. We introduce DockSmith, a specialized agentic Docker builder designed to address this challenge. DockSmith treats environment construction not only as a preprocessing step, but as a core agentic capability that exercises long-horizon tool use, dependency reasoning, and failure recovery, yielding supervision that transfers beyond Docker building itself. DockSmith is trained on large-scale, execution-grounded Docker-building trajectories produced by a SWE-Factory-style pipeline augmented with a loop-detection controller and a cross-task success memory. Training a 30B-A3B model on these trajectories achieves open-source state-of-the-art performance on Multi-Docker-Eval, with 39.72% Fail-to-Pass and 58.28% Commit Rate. Moreover, DockSmith improves out-of-distribution performance on SWE-bench Verified, SWE-bench Multilingual, and Terminal-Bench 2.0, demonstrating broader agentic benefits of environment construction.

💡 Research Summary

**
DockSmith tackles a critical bottleneck in scaling execution‑grounded software engineering agents: the unreliable construction of Docker‑based environments. Rather than treating environment setup as a mere preprocessing step, the authors reframe it as a core agentic capability that demands long‑horizon tool use, dependency reasoning, and systematic failure recovery. Building on the SWE‑Factory pipeline, DockSmith introduces two key extensions: a loop‑detection controller that prevents endless repair cycles by monitoring repeated agent invocations, and a cross‑task success memory that reuses verified Dockerfile‑and‑eval‑script pairs from previously solved repositories as lightweight demonstrations for new tasks.

The system orchestrates four specialized LLM agents: (1) Context Retrieval Agent gathers repository metadata, dependency manifests, CI configurations, and test entry points; (2) Dockerfile Agent synthesizes or patches a Dockerfile based on the retrieved context and execution feedback; (3) Eval Script Agent creates a script to configure the container workspace and invoke tests; (4) Test Analysis Agent runs the build and test pipeline, extracts raw logs, and produces structured failure summaries for subsequent repair iterations. When the same combination of agents fails to improve outcomes for several rounds, the loop‑detection controller forces diversification—activating alternative agents or strategies—to break deadlocks.

Data collection spans over 15 k GitHub repositories across ten major programming languages (Python, JavaScript, Go, C++, Java, Rust, C, Ruby, PHP, and TypeScript). From these, roughly 200 k high‑quality pull‑request (PR) trajectories are curated, each representing a test‑backed code change. PRs are filtered for merge status, substantial code modifications, and test relevance, with additional de‑duplication against benchmark suites to avoid contamination. Language‑model‑assisted refinement expands ambiguous PR descriptions, preserving technical intent while improving clarity.

To transform raw rollouts into a high‑signal training corpus, DockSmith applies a three‑stage curation pipeline: (1) language‑balanced token quotas prevent long‑tail languages from dominating training; (2) redundant or excessively long trajectories—identified by repeated agent calls or unusually high turn counts—are discarded; (3) a complexity‑based curriculum scores each Dockerfile by line count, number of RUN instructions, and number of installed packages, yielding a scalar that stratifies instances into Easy, Medium, and Hard buckets. Sampling follows a 1:2:2 ratio to ensure sufficient exposure to challenging builds.

The model itself is a 30 B‑parameter A3B architecture fine‑tuned on the curated Docker‑building trajectories. To avoid over‑specialization, DockSmith is jointly trained with general coding trajectories using token‑level mixing, keeping the total compute budget constant while varying the proportion of Docker‑building tokens. This joint training injects generic edit‑run‑debug patterns that complement environment construction skills.

Evaluation on the Multi‑Docker‑Eval (MDE) benchmark shows DockSmith achieving 39.72 % Fail‑to‑Pass (F2P) and a 58.28 % Commit Rate, surpassing open‑source baselines such as DeepSeek‑v3.1 (37.72 % F2P) and GPT‑OSS‑20B (26.65 % F2P). Language‑specific analysis reveals consistent gains across Python, JavaScript, Go, and even dependency‑heavy languages like C++ and Rust. Beyond environment setup, DockSmith’s training signal transfers to out‑of‑distribution tasks: on SWE‑bench Verified it improves average scores by ~2.8 points, on SWE‑bench Multilingual by ~3.1 points, and on Terminal‑Bench 2.0 by 3.37 points, demonstrating that robust environment construction benefits broader software‑engineering capabilities.

The paper’s contributions are threefold: (1) redefining environment construction as a core agentic task; (2) introducing a multi‑agent pipeline augmented with loop detection and cross‑task memory, which together raise Docker build success rates dramatically; (3) releasing a large, multilingual Docker‑building trajectory dataset and showing that training on it not only improves environment reliability but also enhances general code‑generation and debugging performance.

Limitations include reliance on a 30 B model—scaling effects with larger models remain untested—and the added complexity of the loop‑detection and memory modules, which may increase deployment cost. Future work should explore lightweight controllers, more efficient memory retrieval, and integration with diverse CI/CD pipelines to further reduce overhead and broaden practical applicability.