Affordance-Graphed Task Worlds: Self-Evolving Task Generation for Scalable Embodied Learning

Training robotic policies directly in the real world is expensive and unscalable. Although generative simulation enables large-scale data synthesis, current approaches often fail to generate logically coherent long-horizon tasks and struggle with dynamic physical uncertainties due to open-loop execution. To address these challenges, we propose Affordance-Graphed Task Worlds (AGT-World), a unified framework that autonomously constructs interactive simulated environments and corresponding robot task policies based on real-world observations. Unlike methods relying on random proposals or static replication, AGT-World formalizes the task space as a structured graph, enabling the precise, hierarchical decomposition of complex goals into theoretically grounded atomic primitives. Furthermore, we introduce a Self-Evolution mechanism with hybrid feedback to autonomously refine policies, combining Vision-Language Model reasoning and geometric verification. Extensive experiments demonstrate that our method significantly outperforms in success rates and generalization, achieving a self-improving cycle of proposal, execution, and correction for scalable robot learning.

💡 Research Summary

The paper tackles two fundamental bottlenecks in robot learning: the prohibitive cost and safety concerns of collecting real‑world data, and the lack of physical and logical consistency in synthetic data generated by current generative simulators. To bridge this gap, the authors introduce Affordance‑Graphed Task Worlds (AGT‑World), a unified framework that (1) reconstructs interactive, physics‑aware simulation scenes from a single real‑world RGB image, (2) formalizes the entire task space as a structured graph, and (3) equips the system with a closed‑loop self‑evolution mechanism that continuously refines policies using hybrid feedback from a Vision‑Language Model (VLM) and geometric verification.

Scene reconstruction is cast as a Bayesian inference problem: given an input image (X_0), the system samples an initial physical state (S_0) from a posterior distribution (p(S|X_0;\epsilon_0)). The sampled state is instantiated in a high‑fidelity physics engine (OmniGibson powered by Isaac Sim), preserving both semantic affordances (e.g., “cup can be grasped”, “refrigerator door can be opened”) and geometric layout. This approach yields low‑cost digital twins that are both visually diverse and physically manipulable, overcoming the static nature of NeRF‑based reconstructions.

Task representation builds on a 3‑D semantic‑action tensor. The universal directed graph (G = (V, E)) has vertices (V = O \times A \times \mathbb{N}^+), where (O) is the set of all manipulable objects, (A) the set of atomic actions, and (\mathbb{N}^+) an ordered temporal dimension. For a concrete scene (S_0), a subgraph (G_{S_0}) is sampled conditioned on the scene’s affordances. A user‑level instruction (I) (e.g., “place the cup in the refrigerator and close the door”) is transformed into a path‑finding problem on (G_{S_0}). The path consists of two edge types: intra‑task edges that encode an action flow (\pi(T_k)) for each simple task (T_k), and inter‑task edges that encode action transfers (e_k) linking the terminal state of (T_k) to the initial state of (T_{k+1}). Action flows are not deterministic; they are sampled from a language‑model‑driven distribution (p_F(\pi|T;\epsilon_1)). Action transfers are modeled by another conditional distribution (p_T(e_k|T_k,T_{k+1};\epsilon_2)). Consequently, the overall success probability of a long‑horizon task is the product of the success probabilities of each atomic flow and each transfer, explicitly accounting for uncertainties (\epsilon_1) (LLM reasoning) and (\epsilon_2) (task decomposition).

Self‑Evolution is the core novelty. During execution, the system monitors both VLM‑generated textual feedback (e.g., “the cup slipped”) and geometric verification results (collision checks, pose deviations). When a failure is detected, the framework diagnoses whether it originates from (\epsilon_1) (incorrect action flow) or (\epsilon_2) (invalid transfer). It then automatically revises the LLM prompt, adjusts temporal parameters (\Delta \tau), or refines the action primitives, producing a new candidate plan. This closed‑loop refinement replaces the open‑loop reinforcement‑learning exploration used in prior work, dramatically reducing error accumulation in long‑horizon tasks.

Experiments evaluate 102 autonomously generated scene‑task pairs across four complex, multi‑step scenarios (e.g., “pick up cup → open refrigerator → place cup → close door”). AGT‑World achieves an overall success rate of 71.6 %, outperforming random‑placement baselines and expensive digital‑twin pipelines by 15–20 percentage points. In the four‑step refrigerator task, self‑evolution raises success from 58 % to 84 %, demonstrating the efficacy of VLM‑guided correction. Ablation studies confirm that both the graph‑based task decomposition and the hybrid feedback loop are essential for the observed gains.

In summary, AGT‑World contributes a scalable pipeline that (i) builds low‑cost, physics‑consistent digital twins from real images, (ii) encodes the entire task space as a mathematically tractable graph, and (iii) introduces a VLM‑driven self‑evolution loop that continuously improves policies. The authors suggest future extensions such as integrating hierarchical Vision‑Language‑Action (VLA) models, multi‑agent collaboration, and online real‑time learning, positioning AGT‑World as a promising foundation for next‑generation embodied AI.