The authors built SkillHorizon, a benchmark with seven persistent business environments (finance, field maintenance, social campaigns, logistics, research, documentation, and manufacturing).
SkillHorizon benchmark reveals that language agents struggle to complete long business workflows as task scope grows
By composing reusable skills into scalable business tasks, the authors show that mean goal completion falls from 90.4% to 46.8% as workload increases from 64K to 1M tokens
Chinese Tech
Yuxuan Liu · Zhaochen Su · Hangyu Guo · Shijue Huang · Zhixiong Zhang · Kawai Chung · +8 more
Hong Kong University of Science and Technology · Alibaba Group · Shanghai Jiao Tong University
Research Digest··2 min read
Thread:Process Agent Benchmarks
The authors introduce SkillHorizon, a benchmark that constructs long-horizon agent tasks by composing reusable skills across seven business environments.
Why this paper
From Alibaba Group and 2 others · Part of Process Agent Benchmarks, now 27 papers
In one line
Long-horizon agent tasks built from composed skills cause completion rates to drop from 90.4% to 46.8% as workload grows.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§