Into the Unknown: Benchmarking AI agents on open scientific research problems
Can AI agents carry out research on open scientific problems? We introduce Into the Unknown, a benchmark that provides an agent with a science paper along with a research direction to pursue and tasks it to write a follow-up paper. We work with scientists at US Department of Energy (DOE) national laboratories, who each contribute an unpublished follow-up direction from their ow
Reading in Plain English mode — toggle below to switch
Can AI agents carry out research on open scientific problems? We introduce Into the Unknown, a benchmark that provides an agent with a science paper along with a research direction to pursue and tasks it to write a follow-up paper. We work with scientists at US Department of Energy (DOE) national laboratories, who each contribute an unpublished follow-up direction from their ow
This benchmark addresses a critical challenge in evaluating AI agents' ability to perform complex, open-ended scientific research. For ML engineers, understanding how to design and assess agents for such tasks is crucial as AI moves beyond well-defined problems into more autonomous roles, especially in scientific discovery.