The authors combined four components: a workspace retained across attempts, a process reward agent that audits ongoing work, automatically discovered evaluation rubrics, and task-specific rubric adaptation.
Persistent workspaces and adaptive checks improve AI data science agents
Across five attempts per task, AdaPRA raised an 80-billion-parameter Qwen agent’s mean leaderboard percentile by 24.5 points on easier tasks and 14.8 points on harder ones.
Big Tech
Atharva Naik · Vivek Chari · Pol Llado · Daniel Fried · Carolyn Rosé · Benjamin Van Durme · +2 more
Carnegie Mellon University · Microsoft
Research Digest··2 min read
Thread:Agent Harness Optimization
Naik and colleagues report that data science agents perform better when repeated attempts build on previous work and a supervising agent checks their code, outputs and intermediate decisions.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.