TURTLE uses three language-model agents.
Evolving existing tasks creates harder, more diverse tool-use benchmarks
TURTLE recombines tasks and tool sequences to generate evaluations calibrated to a model’s current success rate.
Big Tech
Hyunji Alex Nam · Michael Dennis · Natasha Jaques · Minsuk Chang
Google Deepmind · Stanford University · University of Washington
Research Digest··3 min read
Nam and colleagues introduce TURTLE, a multi-agent system that evolves existing benchmark tasks rather than constructing new tools and environments.
§