BrickBench: Evaluating Agentic Brick Design
We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and
Reading in Plain English mode — toggle below to switch
We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and
This paper introduces a novel benchmark for evaluating AI agents in a complex, real-world design task, pushing the boundaries of embodied AI and physical reasoning. For ML engineers, it highlights the challenges and opportunities in developing agents that can interact with and understand physical constraints, a critical step towards more capable and reliable AI systems.