DolphinBench: Mapping the Pareto Frontier of Agent Memory
Agents today often take real-world actions that depend on long-term memory and context recall over time. However, most current memory benchmarks are built for a conversational question-answer format, where the question itself signals that some fact must be retrieved, and often which one. Moreover, benchmarks rarely require anything beyond accuracy from submissions, allowing mem
Reading in Plain English mode — toggle below to switch
Agents today often take real-world actions that depend on long-term memory and context recall over time. However, most current memory benchmarks are built for a conversational question-answer format, where the question itself signals that some fact must be retrieved, and often which one. Moreover, benchmarks rarely require anything beyond accuracy from submissions, allowing mem
This paper introduces a novel benchmark for evaluating agent memory, moving beyond traditional QA formats to task completion. For ML engineers, understanding such benchmarks is crucial for developing and optimizing agents that can perform complex, long-term tasks in real-world scenarios, directly impacting product development and performance.