User-Feedback-Driven Adaptation for Vision-and-Language Navigation

📝 Original Info

• Title: User-Feedback-Driven Adaptation for Vision-and-Language Navigation
• ArXiv ID: 2512.10322
• Date: 2025-12-11
• Authors: Yongqiang Yu, Xuhui Li, Hazza Mahmood, Jinxing Zhou, Haodong Hong, Longtao Jiang, Zhiqiang Xu, Qi Wu, Xiaojun Chang

📝 Abstract

Real-world deployment of Vision-and-Language Navigation (VLN) agents is constrained by the scarcity of reliable supervision after offline training. While recent adaptation methods attempt to mitigate distribution shifts via environmentdriven self-supervision (e.g., entropy minimization), these signals are often noisy and can make the agent amplify its own mistakes during long-horizon sequential decision-making. In this paper, we propose a paradigm shift that positions user feedback, specifically episode-level success confirmations and goal-level corrections, as a primary, general-purpose supervision signal for VLN. Unlike internal confidence scores, user feedback is intent-aligned and insitu consistent, directly correcting the agent's decoupling from user instructions. To effectively leverage this supervision, we introduce a user-feedback-driven learning framework featuring a topology-aware trajectory construction pipeline. This mechanism "lifts" sparse, goal-level corrections into dense path-level supervision. It does so by generating feasible paths on the agent's incrementally built topological graph, enabling sample-efficient imitation learning without requiring step-b...

📄 Full Content

...(본문 내용이 길어 생략되었습니다. 사이트에서 전문을 확인해 주세요.)