UniMate: Unified Motion Generation for Diverse Skeletons
UniMate is a novel foundation model designed to generate articulated motion for any 3D skeleton from a text prompt. It overcomes limitations of previous topology-constrained methods by integrating skeletal structure directly into its diffusion transformer architecture, enabling zero-shot generalization across diverse body types.
Reading in Plain English mode — toggle below to switch
UniMate is a novel foundation model designed to generate articulated motion for any 3D skeleton from a text prompt. It overcomes limitations of previous topology-constrained methods by integrating skeletal structure directly into its diffusion transformer architecture, enabling zero-shot generalization across diverse body types.
Imagine a choreographer who can teach any dancer, regardless of their body shape or number of limbs, to perform any dance move just by giving them a text description. UniMate acts like this universal choreographer for 3D digital characters, adapting motions to bipeds, quadrupeds, or even serpentine forms without needing specific training for each.
How UniMate Generates Motion
- 1A rigged 3D asset (skeleton) and a text prompt describing the desired motion are provided as input.
- 2UniMate's topology-aware diffusion transformer processes this information, integrating the skeleton's unique structure into its attention mechanisms.
- 3It uses a graph-aware attention bias, spectral rotary position embeddings, and a global topological conditioner to understand the kinematic tree.
- 4Through a diffusion process, the model iteratively refines a noisy motion sequence until it matches the input prompt and skeleton, producing a realistic animation.
L = D - A
L represents the Graph Laplacian, where D is the degree matrix (diagonal matrix with node degrees) and A is the adjacency matrix (representing connections between joints). This matrix captures the connectivity and structure of the skeleton.
A user could input a 3D model of a dragon and the text prompt 'dragon flying gracefully,' and UniMate would generate a realistic flying animation for that specific dragon model, without needing prior training on dragon animations.
UniMate's ability to generalize motion generation across arbitrary skeletal topologies, driven by text prompts, represents a significant step towards truly universal animation tools.