SGF+: Decoupling Gradient Flows for Autoregressive Video Generation
Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint optimization of visual quality and temporal consistency. We in
Reading in Plain English mode — toggle below to switch
Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint optimization of visual quality and temporal consistency. We in
This research addresses a fundamental challenge in autoregressive video generation by improving both visual quality and temporal consistency. Understanding how to decouple and optimize different gradient flows for complex generative tasks is crucial for developing more robust and high-performing video models, which have broad applications in content creation and simulation.