LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation
LIBERO-PeRM introduces a new benchmark for Personalized Robotic Manipulation (PeRM), focusing on how robots can learn and satisfy individual user preferences in household environments. It addresses the critical gap in evaluating preference learning independently from task completion in general robotic manipulation.
Reading in Plain English mode — toggle below to switch
LIBERO-PeRM introduces a new benchmark for Personalized Robotic Manipulation (PeRM), focusing on how robots can learn and satisfy individual user preferences in household environments. It addresses the critical gap in evaluating preference learning independently from task completion in general robotic manipulation.
Imagine a tailor who not only makes clothes that fit but also understands your unique style preferences—whether you like a slim cut, specific fabric, or certain embellishments. LIBERO-PeRM is like creating a benchmark for robots to become such 'tailors,' learning and adapting to individual user preferences beyond just completing a task.
How LIBERO-PeRM Works
- 11. **Define Preferences:** The benchmark first establishes a taxonomy of observable, language-expressible preferences (e.g., motion, skill, user-profile choices) that don't change the main task goal.
- 22. **Generate Paired Demonstrations:** A procedural pipeline creates high-quality robot demonstrations, some with specific preferences embedded and some without.
- 33. **Create Task Suites:** Five distinct task suites, comprising 312 tasks, are developed with rich metadata and metrics to evaluate preference satisfaction.
- 44. **Evaluate Policies:** Robotic policies are tested on these tasks to assess their ability to follow preferences in zero-shot settings, learn from demonstrations, generalize preferences, and handle conflicts.
- 55. **Analyze Results:** The performance of various policies (e.g., VLA, SFT, mixed training) is analyzed to understand their strengths and weaknesses in personalized manipulation.
P(success) = P(task_completion) ∩ P(preference_satisfaction)
This conceptual formula represents that overall success in personalized robotic manipulation depends on both successfully completing the primary task and satisfying the user's specific preferences. It highlights the dual objective of PeRM.
A robot is asked to clear a table. The task is to remove all items. A preference might be 'place all glasses on the top shelf' or 'stack plates neatly'. LIBERO-PeRM evaluates if the robot not only clears the table but also adheres to these specific instructions.
Personalized robotic manipulation requires benchmarks that can disentangle task completion from preference satisfaction, revealing that current policies struggle with zero-shot personalization but improve with fine-tuning.