Bench-Push: Benchmarking Pushing-based Navigation and Manipulation Tasks for Mobile Robots

Mobile robots are increasingly deployed in cluttered environments with movable objects, posing challenges for traditional methods that prohibit interaction. In such settings, the mobile robot must go beyond traditional obstacle avoidance, leveraging pushing or nudging strategies to accomplish its goals. While research in pushing-based robotics is growing, evaluations rely on ad hoc setups, limiting reproducibility and cross-comparison. To address this, we present Bench-Push, the first unified benchmark for pushing-based mobile robot navigation and manipulation tasks. Bench-Push includes multiple components: 1) a comprehensive range of simulated environments that capture the fundamental challenges in pushing-based tasks, including navigating a maze with movable obstacles, autonomous ship navigation in ice-covered waters, box delivery, and area clearing, each with varying levels of complexity; 2) novel evaluation metrics to capture efficiency, interaction effort, and partial task completion; and 3) demonstrations using Bench-Push to evaluate example implementations of established baselines across environments. Bench-Push is open-sourced as a Python library with a modular design. The code, documentation, and trained models can be found at https://github.com/IvanIZ/BenchNPIN.

💡 Research Summary

Bench‑Push introduces the first unified benchmark suite for evaluating mobile robots that solve tasks by pushing or nudging objects, a capability increasingly required in cluttered, unstructured environments such as homes, hospitals, disaster sites, and even maritime settings. The authors identify a critical gap in the literature: existing pushing‑based works are tested in disparate, ad‑hoc simulators with inconsistent performance metrics, making reproducibility and cross‑method comparison difficult. To address this, Bench‑Push provides four representative environments, each implemented in both lightweight 2‑D (Pymunk) and high‑fidelity 3‑D (MuJoCo) simulations, and optionally on a physical testbed.

1. Maze – A rectangular indoor maze with static walls and randomly placed movable obstacles. A TurtleBot3 Burger moves at constant forward speed while the policy controls angular velocity. Observations consist of four egocentric channels: static occupancy, movable occupancy, robot footprint, and a goal‑centric distance transform. Rewards encourage progress toward the goal, penalize collisions, and grant a terminal reward upon success.

2. Ship‑Ice – A maritime scenario where an autonomous surface vehicle (ASV) must traverse an ice‑covered channel. Ice floes are irregular polygons with concentrations ranging from 0 % to 50 %. The ship also moves at constant forward speed, with angular velocity as the only control input. Observations mirror the Maze setup but add a heading line channel to aid fine‑grained maneuvering around dense ice clusters. Drag forces approximate fluid dynamics on the xy‑plane. The reward combines collision penalties, a heading‑alignment term, and a terminal reward for crossing the goal line.

3. Box‑Delivery – A manipulation task where the same TurtleBot3 must push a set of boxes into a designated receptacle using only its front bumper. The robot receives a heading command that moves it a fixed distance each step. Observations include static/movable occupancy, robot footprint, an egocentric distance transform, and a goal‑centric distance transform from the receptacle. Rewards are given for moving boxes toward the receptacle, delivering a box, and penalize collisions with static obstacles or moving boxes away from the goal.

4. Area‑Clearing – Similar to Box‑Delivery but the objective is to clear all boxes from a predefined rectangular area. The observation and action spaces are identical to Box‑Delivery; the reward structure rewards each box removed from the area and penalizes unnecessary pushes.

Bench‑Push also defines a novel set of evaluation metrics that go beyond traditional navigation success rates and collision counts. The metrics capture (i) efficiency (path length, time to completion), (ii) interaction effort (cumulative pushing force, total distance objects are displaced), and (iii) partial task completion (e.g., percentage of boxes delivered). These metrics are tailored separately for navigation‑centric and manipulation‑centric tasks, allowing fair comparison of methods that may trade off speed for reduced environmental disturbance.

To demonstrate the benchmark’s utility, the authors implement several baseline policies: generic model‑free reinforcement learning algorithms (Soft Actor‑Critic, Proximal Policy Optimization) and task‑specific approaches (Spatial Action Maps for multi‑object pushing, ASV planners for ice navigation, and a greedy GTSP‑based planner for area clearing). Experiments show that task‑specific baselines achieve higher success rates and lower interaction effort in their target domains, while RL baselines can still learn competent behaviors, especially when trained in the fast 2‑D simulations before fine‑tuning in 3‑D.

A key contribution is the zero‑shot sim‑to‑real transfer study. Policies trained entirely in simulation were deployed on a physical TurtleBot3 testbed and an ASV prototype. Performance metrics on hardware closely matched simulation results, confirming that the benchmark’s physics parameters, observation design, and reward shaping are realistic enough for direct transfer.

Bench‑Push is released as an open‑source Python library built on the Gymnasium API, with modular configuration files that let users vary maze topology, obstacle density, ice concentration, box count, and robot dimensions. Comprehensive documentation, pretrained models, and scripts for training, evaluation, and visualization are provided at the public GitHub repository. By standardizing environments, observations, rewards, and metrics, Bench‑Push aims to accelerate reproducible research, enable fair benchmarking of new pushing‑based algorithms, and ultimately facilitate the deployment of mobile robots that can intelligently reshape their surroundings to achieve their goals.