One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
In this work, we show that a single AI architecture block, applied recurrently, can match the accuracy of a full-depth vision encoder at comparable inference FLOPs without intermediate feature distillation. reViT restores depth-specific transformations by representing the FFN at each recurrent depth as a convex combination of a small shared expert bank. A continuous normalized-
Reading in Plain English mode — toggle below to switch
In this work, we show that a single AI architecture block, applied recurrently, can match the accuracy of a full-depth vision encoder at comparable inference FLOPs without intermediate feature distillation. reViT restores depth-specific transformations by representing the FFN at each recurrent depth as a convex combination of a small shared expert bank. A continuous normalized-
This paper introduces a novel approach to recurrent vision transformers, significantly reducing the parameter count and computational cost while maintaining accuracy. For ML engineers, understanding recurrent architectures with depth-programmed experts can lead to more efficient model deployments and innovative ways to manage model complexity.