StepBench is built from 6,885 verified wikiHow procedures spanning 81 categories.
Step ordering is the hardest planning skill for AI agents, benchmark shows
A new benchmark diagnoses missing, incorrect, misordered and next-step errors in open-domain agent plans without requiring execution.
Big Tech
Hossein A. Rahmani · Bin Wu · Chaeeun Lee · Fanghua Ye · Sahan Bulathwela · Sujay Kumar Jauhar · +3 more
University College London · University of Edinburgh · Microsoft · Cohere Labs
Research Digest··3 min read
The authors introduce StepBench, a benchmark of 6,885 wikiHow procedures that separates open-domain planning failures into distinct diagnostic tasks: missing, incorrect, misordered and next-step problems.
§