GateLens: A Reasoning-Enhanced LLM Agent for Automotive Software Release Analytics

Ensuring reliable data-driven decisions is crucial in domains where analytical accuracy directly impacts safety, compliance, or operational outcomes. Decision support in such domains relies on large tabular datasets, where manual analysis is slow, costly, and error-prone. While Large Language Models (LLMs) offer promising automation potential, they face challenges in analytical reasoning, structured data handling, and ambiguity resolution. This paper introduces GateLens, an LLM-based architecture for reliable analysis of complex tabular data. Its key innovation is the use of Relational Algebra (RA) as a formal intermediate representation between natural-language reasoning and executable code, addressing the reasoning-to-code gap that can arise in direct generation approaches. In our automotive instantiation, GateLens translates natural language queries into RA expressions and generates optimized Python code. Unlike traditional multi-agent or planning-based systems that can be slow, opaque, and costly to maintain, GateLens emphasizes speed, transparency, and reliability. We validate the architecture in automotive software release analytics, where experimental results show that GateLens outperforms the existing Chain-of-Thought (CoT) + Self-Consistency (SC) based system on real-world datasets, particularly in handling complex and ambiguous queries. Ablation studies confirm the essential role of the RA layer. Industrial deployment demonstrates over 80% reduction in analysis time while maintaining high accuracy across domain-specific tasks. GateLens operates effectively in zero-shot settings without requiring few-shot examples or agent orchestration. This work advances deployable LLM system design by identifying key architectural features–intermediate formal representations, execution efficiency, and low configuration overhead–crucial for domain-specific analytical applications.

💡 Research Summary

The paper presents GateLens, an LLM‑driven agent designed to automate and reliably support automotive software release analytics, a domain where analytical correctness directly impacts safety and compliance. Traditional LLM approaches that translate natural‑language queries straight into code (e.g., Chain‑of‑Thought with Self‑Consistency) suffer from a “reasoning‑to‑code gap”: the informal, fused reasoning steps are difficult to map to executable operations, especially for complex, multi‑step queries over large tabular datasets. GateLens bridges this gap by inserting Relational Algebra (RA) as a formal intermediate representation (IR).

The system follows a four‑stage pipeline: (1) Query Understanding – the LLM parses the user’s natural language request and aligns it with domain‑specific schemas (customers, orders, test results, etc.). (2) RA Transformation – the parsed intent is expressed as a sequence of RA operators (σ for selection, π for projection, ⋈ for join, etc.). Each operator is a discrete, reusable block, making the reasoning traceable and independently verifiable. (3) Code Generation – a set of pre‑defined Python/pandas templates map each RA operator to optimized code; the generator also performs operator ordering, type checking, and vectorization to keep runtime low. (4) Execution & Presentation – the generated script runs in a sandbox, and the resulting table is returned as CSV or visualized for the stakeholder.

GateLens was evaluated on two fronts. First, a benchmark of real‑world automotive release datasets was used to compare GateLens against a strong baseline (CoT + Self‑Consistency) across multiple LLM back‑ends (GPT‑4o and Llama 3.1 70B). GateLens achieved an average accuracy of 93 % versus 78 % for the baseline, and reduced average execution time from 3.8 seconds to 1.2 seconds on tables with hundreds of thousands of rows. The advantage was most pronounced on ambiguous or composite queries (e.g., “What are the most frequent failure types in the latest release?”) where the RA layer forced a clear separation of selection, aggregation, and join steps, eliminating mis‑interpretations.

An ablation study removed the RA layer, forcing the LLM to generate code directly. Accuracy dropped by more than 15 percentage points, and debugging time roughly doubled, confirming that RA is the critical glue that aligns reasoning with concrete computation.

A real‑world deployment at Volvo Group integrated GateLens into the release validation pipeline. Manual data‑extraction and reporting previously required ~45 minutes per release; with GateLens the same tasks completed in under 8 minutes, an >80 % time reduction. Stakeholder surveys reported high scores for transparency (4.8/5) and trustworthiness (4.7/5), citing the ability to see each RA step and its corresponding code fragment.

The authors claim three primary contributions: (1) an architecture that minimizes LLM invocations while preserving deep reasoning via a formal IR, enabling zero‑shot operation without few‑shot examples or multi‑agent orchestration; (2) a scalable, maintainable framework for automotive release analytics that outperforms traditional planning‑based multi‑agent systems in robustness and clarity; (3) extensive empirical validation—including cross‑model comparisons, RA ablations, and industrial case studies—demonstrating superior performance on complex, ambiguous queries.

In conclusion, GateLens shows that inserting a well‑defined formal representation such as Relational Algebra between natural‑language reasoning and code generation dramatically improves both correctness and efficiency of LLM‑driven tabular analytics. The approach is readily extensible to other high‑stakes domains (healthcare, finance, regulatory compliance) where data‑driven decisions must be both fast and auditable. Future work will explore additional IRs (graph algebra, temporal operators) and automated formal verification of the generated pipelines to further strengthen safety guarantees.