Teams of artificial intelligence programs, each a large language model (an LLM: software trained on text that generates responses), can split work and coordinate. Testing whether they truly collaborate is hard: existing tests are either simply difficult, so one program can grind through alone, or check a narrow fixed pattern of interaction.
This benchmark mines reusable structures from existing tasks, small recurring interactions between subtasks and whole workflow shapes, then recombines them into new task graphs. Each task runs in an environment that automatically scores partial progress as well as completion.
Several multi-agent systems and a single-agent baseline, tested on more than one underlying model, showed mixed benefit from teamwork. No configuration resolved more than 6.25% of the tasks.