New benchmark reveals agents fail to detect and recover from their own errors in long tasks

CUA-Recovery provides 2,250 verified erroneous states and 8,600 recovery trajectories across desktop applications, showing that even the best agents notice inherited errors only 44% of the time.

Academic
Xu Hu · Haoming Li · Yuxiang Wang · Jinxin Yang · Kangshuo Li · Licheng Wang · +8 more

University of Texas at Dallas · Independent Researcher · University of Manchester · Rice University · University of Texas at Austin

Research Digest··3 min read
Hu et al.

The authors developed ReRail, a framework that synthesizes long-horizon multi-application workflows from seed tasks.

§