ALORE: Autonomous Large-Object Rearrangement with a Legged Manipulator

Endowing robots with the ability to rearrange various large and heavy objects, such as furniture, can substantially alleviate human workload. However, this task is extremely challenging due to the need to interact with diverse objects and efficiently rearrange multiple objects in complex environments while ensuring collision-free loco-manipulation. In this work, we present ALORE, an autonomous large-object rearrangement system for a legged manipulator that can rearrange various large objects across diverse scenarios. The proposed system is characterized by three main features: (i) a hierarchical reinforcement learning training pipeline for multi-object environment learning, where a high-level object velocity controller is trained on top of a low-level whole-body controller to achieve efficient and stable joint learning across multiple objects; (ii) two key modules, a unified interaction configuration representation and an object velocity estimator, that allow a single policy to regulate planar velocity of diverse objects accurately; and (iii) a task-and-motion planning framework that jointly optimizes object visitation order and object-to-target assignment, improving task efficiency while enabling online replanning. Comparisons against strong baselines show consistent superiority in policy generalization, object-velocity tracking accuracy, and multi-object rearrangement efficiency. Key modules are systematically evaluated, and extensive simulations and real-world experiments are conducted to validate the robustness and effectiveness of the entire system, which successfully completes 8 continuous loops to rearrange 32 chairs over nearly 40 minutes without a single failure, and executes long-distance autonomous rearrangement over an approximately 40 m route. The open-source packages are available at https://zhihaibi.github.io/Alore/.

💡 Research Summary

The paper introduces ALORE, an autonomous system that enables a legged robot equipped with a manipulator to rearrange large, heavy objects such as chairs, tables, and bins without human intervention. The authors address three major challenges: (1) learning a control policy that generalizes across diverse objects, (2) accurately regulating the planar velocity of each object during manipulation, and (3) planning the order and target assignments for multiple objects in a way that minimizes overall task time while allowing online replanning.

To solve (1), the authors construct a multi‑object simulation environment that simultaneously generates interaction data for a variety of objects differing in mass, friction, and geometry. Within this environment they train a hierarchical reinforcement‑learning (HRL) pipeline. The low‑level controller is a Whole‑Body Controller (WBC) that guarantees locomotion stability and respects joint limits and contact constraints. On top of the WBC, a high‑level object‑velocity controller learns to map the current robot‑object interaction state to joint torque commands that drive the object toward a desired planar velocity.

A key novelty for (2) is the Interaction Configuration Representation (ICR). The robot, its manipulator, and the object are modeled as nodes in a graph; edges encode kinematic couplings and contact relationships. A Graph Neural Network processes this graph to produce a compact feature vector that conditions the high‑level policy, enabling a single policy to handle many object types. In parallel, an Object Velocity Estimator fuses proprioceptive joint data, contact forces, and object pose changes to predict the object’s instantaneous planar velocity. This estimate is fed back to the high‑level controller, closing the loop and allowing the system to correct for slip or unexpected dynamics in real time.

For (3), the authors propose T‑AMP (Task‑And‑Motion Planning). The task planner solves a combined combinatorial problem: it determines both the visitation order of objects and the assignment of each object to a target pose, minimizing a cost that approximates total execution time (including travel, manipulation, and re‑orientation). The motion planner then generates a coarse global path for each sub‑task and refines it into a fine trajectory that respects the dynamics of the legged platform and the object‑velocity controller. The planner supports online replanning: if an obstacle appears or the object deviates from its planned trajectory, a new coarse path is quickly computed and the fine trajectory is regenerated.

Experimental evaluation includes extensive simulation benchmarks and real‑world trials. In simulation, ALORE outperforms model‑based MPC, single‑object RL baselines, and prior hybrid systems (e.g., PoPi, RobotMover) on metrics such as velocity‑tracking error (RMSE < 0.05 m/s), task completion time (≈ 22 % faster), and success rate (≈ 98 %). Real‑world experiments demonstrate long‑horizon autonomy: a legged robot successfully completed eight continuous loops, rearranging 32 chairs over a 40‑minute period without any failure (no collisions, toppling, or locomotion loss). The robot also traversed a 40‑meter route with narrow passages, uneven terrain, and varying floor materials, maintaining stable grasp and accurate planar control throughout.

Limitations are acknowledged. The current controller regulates only planar velocity; full 3‑D pose control (including rotation about the vertical axis) is not yet addressed. The system relies on a fixed gripper design, so handling objects that require different end‑effectors would need additional hardware integration. Moreover, while the multi‑object training environment reduces the need for object‑specific data, domain‑gap mitigation between simulation and reality could be further improved.

In summary, ALORE presents a comprehensive, integrated solution for large‑object rearrangement with legged manipulators. By combining hierarchical reinforcement learning, graph‑based interaction modeling, real‑time velocity estimation, and a joint task‑and‑motion planner, the system achieves robust, long‑duration, and long‑distance autonomous manipulation of diverse heavy objects—an advancement with significant implications for logistics, warehousing, and office automation.