The authors introduce LongPuzzleBench, a benchmark of 114 levels in six puzzle games played through native GUI actions, designed to test GUI agents' long-horizon visual reasoning. They find that even the strongest agents solve most easy objectives but fail on harder, longer boards; no general-purpose agent completes the hardest level, Bolt Unscrew Hard. Controlled diagnostics reveal that agents fail because they evaluate each move based on immediate visible progress rather than the future options it preserves, a limitation that persists even when rules, state hints, or failure memory are provided.
GUI agents sacrifice future options in long visual puzzles
LongPuzzleBench shows that agents often choose moves for immediate visible progress rather than preserving later solvability.
Chinese Tech
Bingo Zhang · Haochuan Lu · Zongjie Li · Genjian Li · Ari Yu Zhang · Chaozheng Wang
Vera Praxis · Tencent · The Hong Kong University of Science and Technology · Independent Researcher · The Chinese University of Hong Kong
Research Digest··2 min read
In one line
GUI agents judge each move by visible progress, not future options, causing failure on long-horizon visual puzzles.
Before you read this
Three ideas this paper builds onagentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real304 papers →consequence-aware evaluationEvaluation that assesses model performance by considering the real-world impact or risk of errors, not just standard accuracy.33 papers →computer-use agentAn AI agent that interacts with computer interfaces—GUI, terminal, or code—to autonomously perform tasks.50 papers →
Read it as
Zhang et al. evaluate general-purpose GUI agents on 114 levels across six visual puzzle games, using persistent boards where early moves can create unannounced dead ends.
The authors created LongPuzzleBench, comprising 114 levels in six puzzle games, grouped into 16 objectives.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.