The paper introduces DAYJOB, a of 130 realistic professional tasks in healthcare and finance that take skilled humans 13-17 hours each. Each task provides only a brief request and a workspace of files, and agents must deliver work products that satisfy an expert rubric of dozens of binary criteria. The strongest model, Claude Opus 5.5, passed only about 24% of attempts, while the median model passed under 3%, showing that current AI agents are far from capable of autonomously performing complex professional work.
Top AI agents complete fewer than one-quarter of professional tasks
DAYJOB tests whether agents can interpret underspecified requests, inspect workplace files and produce fully correct healthcare and finance deliverables.
Industry
Stephanie Finley · Liudas Panavas · Thomas Mikkelson · Cam Hinton · Stacey Ganss · Bradley Monton · +9 more
Surge AI
Research Digest··2 min read
In one line
Current AI agents fail most long-horizon professional tasks, with the best passing only about a quarter.
Before you read this
Three ideas this paper builds onbenchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.366 papers →agentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real304 papers →verificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.341 papers →
Read it as
Finley and colleagues introduce DAYJOB, a benchmark of 130 long-horizon assignments designed by professionals in healthcare and finance.
The authors built 50 healthcare and 80 finance tasks around brief, realistic requests accompanied by workspaces containing documents such as PDFs, spreadsheets and Word files.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.