Stratified Retention for Continual Adaptation in World Models
This paper addresses a critical challenge in continual learning for world models: how to manage knowledge when the environment itself changes. Unlike traditional models where old knowledge is always considered correct, world models must learn to selectively forget outdated information while retaining fundamental truths, a process termed stratified retention.
Reading in Plain English mode — toggle below to switch
This paper addresses a critical challenge in continual learning for world models: how to manage knowledge when the environment itself changes. Unlike traditional models where old knowledge is always considered correct, world models must learn to selectively forget outdated information while retaining fundamental truths, a process termed stratified retention.
Imagine a map that constantly updates. Some features, like the laws of physics (gravity always pulls down), are invariant and should never change on the map. Other features, like the location of a temporary construction site, are instance-level facts that must be revised as soon as they become outdated. A good map-maker knows what to keep and what to update, rather than just throwing out the whole map or keeping everything, even if it's wrong.
How Stratified Retention Works
- 11. Identify Invariant Knowledge: The model first distinguishes between fundamental, unchanging knowledge (e.g., laws of physics, object permanence) and instance-specific facts that can change over time.
- 22. Differential Retention: Instead of uniform forgetting, the model applies different retention policies. Invariant knowledge is protected from revision, ensuring its permanence.
- 33. Revision of Outdated Facts: Instance-level facts are actively revised or discarded when the environment changes, preventing the model from holding onto false information.
- 44. Evaluate with Differential Metrics: New metrics are proposed that can differentiate between correctly revised knowledge and catastrophic forgetting, focusing on invariant regression testing and revision latency.
K = I ∪ E
K represents the total knowledge of the world model. This knowledge is composed of invariant knowledge (I) and ephemeral knowledge (E).
Consider a robot navigating a dynamic environment. It must retain the knowledge that solid objects cannot pass through each other (invariant physics) but must update its knowledge about the location of a moving obstacle (instance-level fact). Stratified retention allows the robot to efficiently adapt without 'catastrophically forgetting' how the world fundamentally works.
World models require a nuanced approach to forgetting, distinguishing between invariant knowledge that must be preserved and ephemeral knowledge that must be revised as the environment changes.