LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially important for AI agents, whose observations, tool outputs, documents, and prior decisions accumulate over long
Reading in Plain English mode — toggle below to switch
A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially important for AI agents, whose observations, tool outputs, documents, and prior decisions accumulate over long
This infrastructure update advances MoE and Serving Infrastructure, directly affecting how engineers build, optimize, and ship AI systems. Teams running production LLM workloads need to understand these techniques to stay competitive in hiring and architecture decisions.