AgentVQA was built with a human-in-the-loop, model-assisted curation pipeline.
Offline benchmark shows agentic failures stem from grounding, not reasoning
AgentVQA's 1,000 verified questions across five domains rank 22 vision-language models, with the best scoring 58.5%.
Top University
Advait Gupta · Sachin Konan · Tom Yin · Gabriel Sarch · Zhuang Liu
University of Maryland, College Park · Princeton University
Research Digest··2 min read
The authors introduce AgentVQA, a curated offline benchmark of 1,000 multiple-choice questions drawn from 23 datasets across five agentic domains: web agents, robotics, egocentric video, games, and spatial planning.
§