FeudalNav: A Simple Framework for Visual Navigation

Visual navigation for robotics is inspired by the human ability to navigate environments using visual cues and memory, eliminating the need for detailed maps. In unseen, unmapped, or GPS-denied settings, traditional metric map-based methods fall short, prompting a shift toward learning-based approaches with minimal exploration. In this work, we develop a hierarchical framework that decomposes the navigation decision-making process into multiple levels. Our method learns to select subgoals through a simple, transferable waypoint selection network. A key component of the approach is a latent-space memory module organized solely by visual similarity, as a proxy for distance. This alternative to graph-based topological representations proves sufficient for navigation tasks, providing a compact, light-weight, simple-to-train navigator that can find its way to the goal in novel locations. We show competitive results with a suite of SOTA methods in Habitat AI environments without using any odometry in training or inference. An additional contribution leverages the interpretablility of the framework for interactive navigation. We consider the question: how much direction intervention/interaction is needed to achieve success in all trials? We demonstrate that even minimal human involvement can significantly enhance overall navigation performance.

💡 Research Summary

FeudalNav introduces a clean, hierarchical framework for image‑goal visual navigation that completely dispenses with odometry, explicit topological graphs, and reinforcement learning. The system is organized into three levels. At the top, a high‑level manager maintains a Memory Proxy Map (MPM), a compact 2‑D representation of the environment built in a latent space learned via self‑supervised contrastive learning (SMoG). Images observed during navigation are clustered by visual similarity using SuperGlue key‑point matching; each cluster defines a point in the latent space, which is projected to 2‑D by an isomap‑imitator network. A Gaussian kernel is placed at each projected point, yielding a density map that serves as a graph‑free memory of visited locations and a proxy for relative distance.

The middle level is the Waypoint Network (WayNet), a supervised model trained on human point‑click trajectories from the LA VN dataset. WayNet receives the current RGB observation and a cropped region of the MPM centered on the agent, and predicts a pixel coordinate that acts as a sub‑goal. When the confidence of SuperGlue matches between the current view and the goal image exceeds a threshold, the average matched key‑point location replaces the learned waypoint, ensuring goal‑directed behavior. This design captures the intuitive human strategy of selecting salient points such as hallway ends, doorways, or deeper points in a room, and generalizes zero‑shot to unseen environments.

The low‑level worker is a lightweight MLP classifier that maps the current depth map together with the waypoint from WayNet to one of three primitive actions: turn left 15°, turn right 15°, or move forward 0.25 m. The agent stops when (i) the SuperGlue confidence α_k between the goal image and the current observation is high (≥ 0.7) and either (ii) the depth reading indicates the agent is within 1 m of the goal or (iii) the proportion ψ of matched key‑points to total image area exceeds 0.85.

Evaluation follows the NRNS protocol on previously unseen Gibson scenes within the Habitat AI simulator. Each episode terminates after 500 steps or upon successful stopping within 1 m of the goal. FeudalNav achieves success rates and SPL scores comparable to or surpassing state‑of‑the‑art methods that rely on odometry, graph construction, or RL. Moreover, a minimal human‑in‑the‑loop intervention—simply allowing occasional waypoint corrections—dramatically boosts performance, demonstrating the framework’s interpretability and ease of interactive control.

Key contributions are: (1) a self‑supervised MPM that provides a graph‑free, memory‑efficient proxy for spatial reasoning; (2) WayNet, which learns human‑like sub‑goal selection from point‑click data and transfers zero‑shot; (3) a feudal hierarchy that separates global planning, waypoint generation, and local motion without reinforcement learning; and (4) competitive SOTA results on image‑goal navigation in Habitat’s indoor environments. The work shows that visual similarity alone can replace explicit metric or topological representations, opening new avenues for lightweight, interpretable navigation agents in GPS‑denied or odometry‑unavailable settings.