Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
Weak-to-strong generalization asks whether stronger models can learn from weaker supervisors and surpass them. This question is particularly important for successive model generations and multi-domain consolidation, where repeating frontier-scale post-training from scratch can be prohibitively expensive. Yet conventional distillation treats the weak teacher as an optimization t
Reading in Plain English mode — toggle below to switch
Weak-to-strong generalization asks whether stronger models can learn from weaker supervisors and surpass them. This question is particularly important for successive model generations and multi-domain consolidation, where repeating frontier-scale post-training from scratch can be prohibitively expensive. Yet conventional distillation treats the weak teacher as an optimization t
This research introduces a novel distillation technique that allows stronger models to learn from weaker supervisors without being capped by their performance. This is crucial for efficient model iteration and consolidation, especially as models become increasingly large and expensive to train from scratch.