Quality-Aware VLA Pretraining: Benchmarking for Better Robotics
This paper investigates how to effectively use demonstration quality during Vision-Language-Action (VLA) pretraining to improve robotic task performance after fine-tuning. It compares different methods like weighting, filtering, and conditioning across various reward definitions and label granularities.
Reading in Plain English mode — toggle below to switch
This paper investigates how to effectively use demonstration quality during Vision-Language-Action (VLA) pretraining to improve robotic task performance after fine-tuning. It compares different methods like weighting, filtering, and conditioning across various reward definitions and label granularities.
Imagine teaching a robot chef by showing it many cooking videos. Some videos show perfect technique, others are a bit messy. This research is like figuring out if the robot learns best by: 1) giving more attention to the perfect videos (weighting), 2) ignoring the messy videos altogether (filtering), or 3) being told how good each video is as it watches (conditioning). The goal is for the robot to cook perfectly after its 'training'.
Improving VLA Pretraining with Quality Signals
- 11. Collect suboptimal human demonstrations: Gather a large dataset of human actions for robotic tasks, acknowledging that these demonstrations might not always be perfect.
- 22. Define quality metrics: Establish ways to quantify the 'goodness' of each demonstration, such as success rates or specific action efficiencies.
- 33. Apply quality-aware strategies: Experiment with different methods during pretraining, like weighting more successful demonstrations, filtering out poor ones, or explicitly conditioning the model on quality tags.
- 44. Fine-tune on target tasks: After pretraining, adapt the VLA model to specific robotic tasks to evaluate how well the quality-aware pretraining improved its performance.
- 55. Benchmark results: Use a controlled benchmark like AXIS-Bench to systematically compare the effectiveness of different quality integration strategies.
P(a|s,o,q)
This represents the probability of taking action $a$ given the current state $s$, observation $o$, and crucially, a quality signal $q$. The paper explores how to best incorporate $q$ to improve learning.
In a real-world experiment, pretraining with simulation data improved robotic success from 9.6% to 31.7%, demonstrating the significant impact of structured pretraining on VLA systems.
Explicitly incorporating demonstration quality signals during VLA pretraining, especially through conditioning, significantly improves downstream task performance in robotics.