Validated Code Translation for Projects with External Libraries

Large Language Models (LLMs) have shown promise for program translation, particularly for migrating systems code to memory-safe languages such as Rust. However, existing approaches struggle when source programs depend on external libraries: LLMs frequently hallucinate non-existent target APIs and fail to generate call-enabling imports; moreover, validating semantic equivalence is challenging when the code manipulates opaque, library-defined types. We present a translation and validation framework for translating Go projects with external dependencies to Rust. Our approach combines (i) a retrieval mechanism that maps Go library APIs to Rust APIs, and (ii) a cross-language validation pipeline that establishes language interoperability in the presence of opaque library types by synthesising adapters exclusively from public library APIs, prior to validating I/O equivalence. We evaluate our system on six real-world Go repositories with non-trivial external dependencies. Our approach significantly increases both the compilation and equivalence success rate (up to 100% in the most dependency-heavy case; approx. 2x on average) by enabling validated translation that manipulate opaque, library-defined types.

💡 Research Summary

The paper tackles two fundamental obstacles that have limited the practical adoption of large‑language‑model (LLM) based source‑to‑source translation: (1) hallucination of external library APIs and missing import statements, and (2) the inability to verify semantic equivalence when the source and target functions manipulate opaque, library‑defined types. The authors focus on translating whole Go repositories to Rust, a scenario that is both industrially relevant and technically challenging because Go has a clear package system while Rust’s crate ecosystem involves trait‑based APIs and re‑exports that make import resolution non‑trivial.

The proposed solution consists of a two‑stage pipeline. In the first stage, a Retrieval‑Augmented Generation (RAG) component builds a compact knowledge base for each candidate Rust crate (K_crate). For every Go package the system first selects a single Rust crate that is likely to provide equivalent functionality. This selection combines LLM‑suggested matches based on package documentation with keyword search on crates.io, followed by a reranking step. Once a crate is chosen, its public API surface—including function signatures, documentation strings, and all valid public import paths—is pre‑extracted into K_crate. When translating a particular Go function, the system queries K_crate with a textual description of each Go API call (e.g., “sha512.New returns a new hash.Hash computing the SHA‑512 checksum”). The top‑ranked Rust candidates (typically three) are then injected into the LLM prompt together with their import paths. This dramatically reduces the chance that the LLM will invent a non‑existent crate or forget the necessary trait import (e.g., sha2::{Sha512, Digest} for Sha512::new).

In the second stage, the system generates the Rust translation using a modular, function‑level prompt that includes (i) an in‑scope type summary, (ii) signatures of direct callees, and (iii) the three candidate Rust APIs with import information. The LLM produces idiomatic Rust code that compiles against the suggested crates.

To validate the translation, the authors introduce a cross‑language testing framework that works even when the involved types are opaque. They synthesize a language‑neutral intermediate data format (IDF) by automatically generating a protobuf schema (T_Proto) from the original Go type definitions. The LLM creates this schema, after which the protobuf compiler emits language‑specific carrier types (T_GoProto and T_RustProto) and (de)serialization code. Using only public constructors and (de)serialization functions from the libraries, adapters are synthesized that convert opaque Go values (e.g., ed25519.PrivateKey, rsa.PrivateKey) into protobuf messages and back. Once both the Go and Rust binaries can exchange data via this IDF, differential testing is performed: the same inputs are fed to the original Go function and its Rust counterpart, the outputs are serialized to protobuf, and the messages are compared for equality. This approach avoids any need for field‑by‑field structural comparison, which is impossible for opaque types.

The evaluation covers six real‑world Go projects that depend on a variety of external libraries (cryptography, networking, database drivers, etc.). Compared with a baseline LLM‑only translator, the proposed system achieves an average compilation success rate of 95.83 % (up to 100 % for the most dependency‑heavy repository) and an average I/O equivalence validation rate of 95.83 %. In the baseline, compilation often failed because the generated Rust code referenced non‑existent crates or omitted required imports, and equivalence testing succeeded on less than half of the functions due to opaque types. The new pipeline therefore roughly doubles both metrics.

Key contributions are: (1) a RAG‑driven library‑mapping mechanism that supplies concrete crate and import information to the LLM, (2) a protobuf‑based adapter synthesis that enables cross‑language validation for opaque library types, and (3) an end‑to‑end modular translation and verification workflow that operates at the repository level. The authors acknowledge limitations: the current implementation assumes a one‑to‑one Go‑package‑to‑Rust‑crate mapping, focuses solely on Go→Rust, and may struggle with highly generic or interface‑heavy Go types. Future work includes extending the approach to other language pairs, handling multiple crates per package, and integrating formal invariant checking via SMT solvers.

Overall, the paper demonstrates that by augmenting LLMs with precise external‑library knowledge and a robust cross‑language validation strategy, it is feasible to automatically migrate large, dependency‑rich codebases to memory‑safe languages while providing strong evidence of functional correctness.