MFlow: Adaptive Parameterization for Pixel Diffusion Transformers
This paper introduces MFlow, an adaptive parameterization for pixel-space diffusion transformers that learns to dynamically interpolate between predicting the clean image ($\boldsymbol{x_0}$) and predicting the velocity ($\boldsymbol{u}$). This approach aims to improve generative performance by adapting the prediction target based on data characteristics and model capacity, addressing limitations of fixed parameterizations in pixel diffusion models.
Reading in Plain English mode — toggle below to switch
This paper introduces MFlow, an adaptive parameterization for pixel-space diffusion transformers that learns to dynamically interpolate between predicting the clean image ($\boldsymbol{x_0}$) and predicting the velocity ($\boldsymbol{u}$). This approach aims to improve generative performance by adapting the prediction target based on data characteristics and model capacity, addressing limitations of fixed parameterizations in pixel diffusion models.
Imagine you're trying to guide a particle through a complex vector field (the diffusion process). Instead of always pushing it towards a fixed target (clean image) or always along the current flow (velocity), MFlow is like having an intelligent guide that learns when to do which, or a combination of both, depending on the local landscape and how powerful your 'engine' (model capacity) is. This adaptive strategy helps the particle reach its destination (the generated image) more effectively.
How MFlow Adapts Diffusion Predictions
- 1MFlow integrates an adaptive parameterization into a vanilla pixel diffusion transformer, allowing it to learn a continuous interpolation between $\boldsymbol{x_0}$-prediction and $\boldsymbol{u}$-prediction.
- 2This interpolation is learned end-to-end during the diffusion training process, dynamically adjusting the prediction strategy across principal directions of the data.
- 3The model observes that the optimal prediction strategy shifts with model capacity and data representation, with higher capacity models often preferring velocity prediction.
- 4To mitigate persistent high-frequency deficits in generated images, a lightweight convolutional module called ConvRefiner is introduced. It's trained post-hoc with an adversarial loss, keeping the main diffusion backbone frozen, to restore fine details.
vₜ = αₜ x₀ + βₜ ε
$\boldsymbol{v}_t$ is the noisy image at timestep $t$. $\boldsymbol{x}_0$ is the clean image. $\boldsymbol{\epsilon}$ is Gaussian noise. $\alpha_t$ and $\beta_t$ are scalar coefficients that depend on $t$, controlling the noise level.
MFlow demonstrates that adaptively interpolating between $\boldsymbol{x_0}$-prediction and $\boldsymbol{u}$-prediction in pixel diffusion models, based on data and model capacity, significantly improves generative performance and flexibility.