Learning the 4D World through Point Trajectory Forecasting
Reliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene. To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. We extend a pretrai
Reading in Plain English mode — toggle below to switch
Reliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene. To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. We extend a pretrai
This work introduces a novel world model capable of forecasting future videos from arbitrary viewpoints and 3D point trajectories, which is crucial for developing more robust and interpretable AI agents. For ML engineers, understanding such multi-modal, multi-view forecasting systems is increasingly important for applications in robotics, autonomous driving, and augmented reality, where predicting dynamic 3D environments is key.