CaptchaArena is a new training dataset for that solve interactive CAPTCHAs. It contains 50K puzzles spanning 20 CAPTCHA types and 5 interaction modes, with every solution verified by execution in a real browser. The dataset includes 50K screenshot-action trajectories, 46K with step-by-step reasoning annotations, plus pixel-mask annotations for irregular targets.
Fine-Grained Training Data Improves Agents on Interactive CAPTCHA Tasks
A 50,000-puzzle dataset with executable trajectories trained one 9B policy to handle 20 CAPTCHA types across five interaction modes.
Top University
Zhenhao Zhang · Zhaoyu Fan · Haohan Ying · Jingwen Hu · Hancen Fan · Junhao Zhou · +2 more
Columbia University · Zhejiang University · University of Rochester · University of Illinois at Urbana-Champaign
Research Digest··2 min read
In one line
CaptchaArena supplies 50,000 execution-verified interactive CAPTCHA puzzles across 20 types, and CaptchaAgent trained on it reaches 71.7 Pass@1.
Before you read this
Three ideas this paper builds oncomputer-use agentAn AI agent that interacts with computer interfaces—GUI, terminal, or code—to autonomously perform tasks.50 papers →reinforcement learning with verifiable rewardsReinforcement learning that uses rewards from objectively checkable signals like correct answers or verification feedback.163 papers →verificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.341 papers →
Read it as
Zhang et al. built CaptchaArena, a dataset of interactive CAPTCHA tasks whose reference solutions were verified by execution in a browser.
The authors generated 50,000 puzzles spanning 20 CAPTCHA types and five modes: single-click, multi-click, arrow-cycle, real-time interaction and text entry.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.
What it released
Code