PoTable: Towards Systematic Thinking via Plan-then-Execute Stage Reasoning on Tables

In recent years, table reasoning has garnered substantial research interest, particularly regarding its integration with Large Language Models (LLMs), which have revolutionized natural language applications. Existing LLM-based studies typically achieve step-by-step thinking for table reasoning guided by task semantics. While these approaches emphasize autonomous exploration and enhance fine-grained table understanding, they often overlook systematic thinking in the reasoning process. This oversight can lead to omitted steps, disorganized logic and misleading results, especially in complex scenarios. In this paper, we propose PoTable, a novel stage-oriented plan-then-execute approach that incorporates systematic thinking into table reasoning. Specifically, PoTable involves several distinct analytical stages with clear objectives to provide adequate guidance. To accomplish stage-specific goals, PoTable employs a plan-then-execute mechanism: it first plans the operation chain based on the stage objective, and then executes operations sequentially through code generation, real-time running and feedback processing. Consequently, PoTable produces reliable table reasoning results with highly accurate, step-wise commented and completely executable programs. It mirrors the workflow of a professional data analyst, offering advantages in both accuracy and explainability. Finally, we conduct extensive experiments on four datasets from the WikiTQ and TabFact benchmarks, where the results demonstrate the effectiveness, efficiency and explainability of PoTable. Our code is available at: https://github.com/Double680/PoTable.

💡 Research Summary

PoTable introduces a stage‑oriented “plan‑then‑execute” framework for table reasoning that tightly integrates large language models (LLMs) with a real‑time Python interpreter. The authors observe that most recent LLM‑based approaches to table question answering (QA) and fact verification follow a flat, step‑by‑step chain‑of‑thought paradigm: given a question, the model selects an operation, generates code (or SQL), executes it, and proceeds to the next step. While this works for simple queries, it suffers from two major drawbacks when the task is complex. First, the operation chain becomes long, increasing the chance of omitted steps or hallucinated actions. Second, the overall logical flow is often disorganized, making it hard to trace errors or verify intermediate results.

To address these issues, PoTable draws inspiration from professional data analysts who decompose a data‑analysis workflow into well‑defined stages (initialization, row selection, data‑type cleaning, reasoning, and final answering). The framework enforces this decomposition on every table‑reasoning instance. For each stage, PoTable runs a two‑phase loop: (1) Planning – the LLM receives a concise description of the stage’s objective together with the current context (the table, any intermediate results, and the original question). It then outputs a textual “operation chain” that enumerates the specific actions needed to satisfy the stage goal (e.g., “filter rows where Country = ‘USA’, convert column ‘Gold’ to int, compute sum”). (2) Execution – the system translates each operation in the chain into Python code, sends it to a sandboxed interpreter, and runs it immediately. If execution succeeds, the resulting data is passed to the next operation; if an error occurs (syntax error, type mismatch, out‑of‑bounds index, etc.), the interpreter returns the error message to the LLM, which revises the code and retries. This feedback loop continues until the code for the current stage runs without error.

The key technical contributions are:

• Systematic stage design – By limiting each stage to a narrow, well‑specified goal, the length of the operation chain is dramatically reduced compared with a monolithic chain‑of‑thought. Shorter chains lower the probability of missing steps and make the reasoning process more transparent.
• Dynamic code generation with automatic debugging – The plan‑then‑execute loop couples LLM creativity with deterministic execution. Errors are caught early, and the LLM is prompted to fix them, effectively turning the LLM into a self‑debugging programmer.
• Fully executable, commented programs – At the end of the pipeline PoTable outputs a complete Python script where each block is annotated with the stage it belongs to. This script can be inspected, rerun, or modified by a human analyst, providing strong explainability and reproducibility.

Empirical evaluation was conducted on four datasets derived from the WikiTableQuestions (WikiTQ) and TabFact benchmarks: two standard test sets and two “complex” test sets that contain longer queries, multi‑step aggregations, or noisy tables. PoTable was instantiated with GPT‑4 as the LLM backbone. Compared with strong baselines—including chain‑of‑thought prompting, dynamic operation selection (Chain‑of‑Table), and tool‑augmented methods such as ReAct‑Table and Self‑Debugging—the proposed system achieved notable gains. On the standard sets, PoTable improved accuracy by an average of 4.3 percentage points, and on the complex sets it outperformed the runner‑up by 3.68 points. In addition to accuracy, PoTable demonstrated higher efficiency: because each stage executes a compact code block, the overall runtime was roughly 15 % faster than baselines that generate and run a single large script.

Ablation studies confirmed the importance of the two core ideas. Removing the stage decomposition (collapsing the workflow into a single stage) reduced accuracy by up to 5 %, while disabling the error‑feedback loop caused a 2–3 % drop, highlighting that both systematic planning and automatic debugging are essential for the observed performance.

The paper also discusses qualitative aspects. Sample outputs show that PoTable’s generated code closely mirrors what a human analyst would write: clear variable names, explicit type conversions, and step‑wise comments. This alignment not only aids debugging but also satisfies regulatory or audit requirements where traceability of decisions is mandatory.

In conclusion, PoTable successfully injects human‑like systematic thinking into LLM‑driven table reasoning. By structuring the problem into a sequence of goal‑oriented stages and coupling each stage with a plan‑then‑execute loop that includes automatic error correction, the framework delivers higher accuracy, better explainability, and improved efficiency. Future work may explore extending the stage taxonomy (e.g., adding visualization or external API calls), learning optimal stage transitions, or applying the same paradigm to other structured‑data domains such as knowledge graphs or relational databases.