PeakBench is a new benchmark for evaluating LLM agents that invoke multiple tools. It addresses a gap: existing benchmarks focus on tool selection and argument generation but ignore parallel execution and resource constraints. PeakBench provides executable workflows with dependency annotations and measured resource profiles, plus a two-part evaluation that separates logical planning (which steps can run in parallel) from physical scheduling (how to allocate resources). The key finding is that strong logical planning does not automatically lead to safe or efficient resource usage. Showing resource information helps reduce overflows and improve utilization.
Good tool plans can still fail under resource constraints
PeakBench separates dependency planning from resource-constrained scheduling to evaluate whether agents can execute parallel tool workflows safely and efficiently.
Independent
Zhi-Kai Chen · Xu-Xiang Zhong · Song-Yan Li · De-Chuan Zhan · Han-Jia Ye
Research Digest··2 min read
Thread:Agent Harness Optimization
In one line
PeakBench benchmarks LLM agents on resource-aware tool scheduling, showing logical planning doesn't guarantee safe or efficient parallel execution.
Before you read this
An idea this paper builds onRead it as
Chen and colleagues introduce PeakBench, a benchmark of executable multi-tool workflows annotated with dependencies and measured resource demands.
The authors constructed executable workflows involving multiple tools, with execution-grounded annotations describing which calls depend on others and measured profiles of their resource use.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.
What it released
Code