Rephrase Before You Act: Characterizing and Mitigating Language Sensitivity in Vision-Language-Action Models
Vision-language-action models (VLAs) are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-language models they are built on. A one-word edit can move success by tens of points: $π_{0.5}$ turns on a LIBERO stove 100% of the time for "switch on the stove" and 2% for "switch on the hot plate", and a $π_0$ checkpoint finetuned wi
Reading in Plain English mode — toggle below to switch
Vision-language-action models (VLAs) are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-language models they are built on. A one-word edit can move success by tens of points: $π_{0.5}$ turns on a LIBERO stove 100% of the time for "switch on the stove" and 2% for "switch on the hot plate", and a $π_0$ checkpoint finetuned wi
This paper highlights a critical vulnerability in Vision-Language-Action (VLA) models: their extreme sensitivity to instruction phrasing. For ML engineers, understanding and mitigating this 'language sensitivity' is crucial for deploying robust and reliable AI agents in real-world applications, especially in robotics and automation where a single word change can lead to drastic performance drops.