StratXplore: Strategic novelty-seeking and instruction-aligned exploration for vision and language navigation

Document Type

Conference Proceeding

Publication Title

IEEE International Conference on Intelligent Robots and Systems

First Page

12093

Last Page

12100

Publisher

IEEE

School

Centre for Artificial Intelligence and Machine Learning (CAIML) / School of Science

RAS ID

70355

Comments

Gopinathan, M., Abu-Khalaf, J., Suter, D., & Masek, M. (2024, October). StratXplore: Strategic novelty-seeking and instruction-aligned exploration for vision and language navigation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 12093-12100). IEEE. https://doi.org/10.1109/IROS58592.2024.10802128

Abstract

Embodied navigation requires robots to understand and interact with the environment based on given tasks. Vision-Language Navigation (VLN) is an embodied navigation task, where a robot navigates within a previously seen and unseen environment, based on linguistic instruction and visual inputs. VLN agents need access to both local and global action spaces; former for immediate decision making and the latter for recovering from navigational mistakes. Prior VLN agents rely only on instruction-viewpoint alignment for local and global decision making and back-track to a previously visited viewpoint, if the instruction and its current viewpoint mismatches. These methods are prone to mistakes, due to the complexity of the instruction and partial observability of the environment. We posit that, back-tracking is sub-optimal and agent that is aware of its mistakes can recover efficiently. For optimal recovery, exploration should be extended to unexplored viewpoints (or frontiers). The optimal frontier is a recently observed but unexplored viewpoint that aligns with the instruction and is novel. We introduce a memory-based and mistake-aware path planning strategy for VLN agents, called StratXplore, that presents global and local action planning to select the optimal frontier for path correction. The proposed method collects all past actions and viewpoint features during navigation and then selects the optimal frontier suitable for recovery. Experimental results show this simple yet effective strategy improves the success rate on two VLN datasets with different task complexities.

DOI

10.1109/IROS58592.2024.10802128

Access Rights

subscription content

Link to Full Text

Share

 
COinS