TANGO: Whole-Body Vision-Language-Action for Humanoid Navigation
TANGO is a novel framework enabling humanoid robots to navigate complex, cluttered indoor environments using natural language instructions and visual input. Unlike traditional 2D path planning, TANGO focuses on whole-body adaptation, coordinating arms, torso, and gait for collision-free movement in 3D spaces. This allows humanoids to perform tasks requiring intricate physical interaction with their surroundings.
Reading in Plain English mode — toggle below to switch
TANGO is a novel framework enabling humanoid robots to navigate complex, cluttered indoor environments using natural language instructions and visual input. Unlike traditional 2D path planning, TANGO focuses on whole-body adaptation, coordinating arms, torso, and gait for collision-free movement in 3D spaces. This allows humanoids to perform tasks requiring intricate physical interaction with their surroundings.
Imagine a highly skilled ballet dancer navigating a room full of furniture and people, not just avoiding collisions but gracefully adjusting their entire body—arms, torso, legs—to move through tight spaces while following verbal cues. TANGO gives humanoid robots this level of coordinated, language-guided, whole-body intelligence for navigation.
How TANGO Works
- 11. Language Instruction & Visual Input: The robot receives a natural-language command (e.g., 'go to the kitchen') and processes real-time egocentric RGB camera observations of its environment.
- 22. Whole-Body Action Prediction: A vision-language-action model directly predicts 29-Degrees-of-Freedom (DoF) joint-space actions, coordinating all parts of the robot's body for movement.
- 33. Simulation-Based Training: TANGO is trained entirely in simulation by synthesizing diverse, collision-free traversal behaviors. This involves global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and reinforcement learning for tracking.
- 44. Zero-Shot Deployment: The trained model is then deployed directly onto a physical humanoid robot (e.g., Unitree G1) without any real-world training data, demonstrating robust language-guided navigation in cluttered real-world scenes.
A = f(V, L)
The robot's whole-body actions ($A$) are a function ($f$) of its visual observations ($V$) and the language instruction ($L$). This signifies a direct mapping from multimodal input to complex motor commands.
A humanoid robot, given the instruction 'walk around the table and pick up the cup,' would use TANGO to not only plan a path but also adjust its torso to squeeze past a chair, extend an arm to maintain balance, and modulate its gait to step over a rug, all while continuously processing visual feedback and the language command.
TANGO shifts humanoid navigation from 2D path planning to continuous, geometry-aware, whole-body adaptation, enabling more natural and effective traversal in complex 3D environments via a vision-language-action model.