Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
This paper introduces Tev-Gym, a novel dataset and training recipe designed to improve the performance of long-horizon AI agents. It tackles the challenge of sparse rewards in complex tasks by providing dense, verifiable progress metrics, enabling agents to learn more efficiently and achieve autonomous operation over extended periods.
Reading in Plain English mode — toggle below to switch
This paper introduces Tev-Gym, a novel dataset and training recipe designed to improve the performance of long-horizon AI agents. It tackles the challenge of sparse rewards in complex tasks by providing dense, verifiable progress metrics, enabling agents to learn more efficiently and achieve autonomous operation over extended periods.
Imagine an AI agent trying to climb a mountain (a long-horizon task) with only a single reward at the very top (sparse reward). It's hard to learn. Tev is like giving the agent a GPS that constantly tells it how much closer it is to the summit with each step, even if it's just a few feet. This 'progress reward' helps it learn the best path much faster.
How Tev Enables Better Long-Horizon Agents
- 11. Latent World Construction: A complete hidden specification of a task's domain, workflow, and expected state after each step is created. This 'latent world' acts as a ground truth for progress.
- 22. Progressive Environment Rendering: Tasks are rendered from this latent world into an agent-facing environment, allowing for controlled complexity and a natural way to measure progress.
- 33. Progress-Potential Reward Shaping (PPRS): The natural progress metric is converted into a dense, per-step reward. This reward function guides the agent more effectively than sparse, end-of-task rewards.
- 44. Agent Training: Agents are trained using this dense reward signal, leading to improved efficiency and the ability to tackle previously intractable long-horizon tasks.
Rₜ = R_s_pₐᵣ_sₑ + λ (Pₜ_+₁ - Pₜ)
$R_t$ is the total reward at time $t$. $R_{sparse}$ is the original sparse reward. $\lambda$ is a scaling factor. $P_{t+1} - P_t$ represents the change in progress potential between steps $t$ and $t+1$, providing a dense reward signal.
Tev-27B, an agent trained with this recipe, achieved substantial gains on the Agents' Last Exam computing and math sciences domains (ALE-CM), matching GPT-6 Astra and Claude Opus 5 at Pass@5. It also generalized to Terminal-Bench (TB) 3.0 and 4.0, surpassing GLM 5.2 and Gemini 3.7 Flash, and reached Claude Opus 4.8-level performance on middle-horizon tasks such as TB 2.1.
Dense, verifiable progress rewards derived from a latent world model significantly improve the training efficiency and performance of long-horizon AI agents, making complex autonomous tasks tractable.