Hu, Bell and Zheng designed GenOlympiad prompts to stress hard constraints rather than subjective style, such as counting many objects, following complex rules, enforcing physical consistency, and performing spatial or 3D reasoning.
Frontier text-to-image models score as low as 23.9 on hardest benchmark
GenOlympiad tests 801 prompts across nine capability axes using an automated, grounding-based evaluation that reveals persistent failures in counting, physical consistency, and spatial reasoning.
AI Startup
Xi Hu · Martin Bell · Liang Zheng
Australian National University · Canva
Research Digest··3 min read
The authors introduce GenOlympiad, a benchmark of 801 deliberately difficult text-to-image prompts spanning nine capability axes, with an automated pipeline that decomposes each prompt into atomic, visually verifiable questions.
§