Process-based benchmarks establish a method for evaluating agent performance by decomposing tasks into sequentially dependent subgoals, enabling fine-grained error analysis. The recent OSWorld-Pro benchmark from NVIDIA extends this approach to computer-use agents, decomposing over 300 tasks into more than 2,800 subgoals with 67,000 human annotations and a human-aligned LLM judge.

The papers collectively establish that while language-model agents often fail to improve reliably from their own experience (S³Gym, University of Washington and collaborators), recent work introduces structured methods—checkpoint testing (Zhejiang University + Alibaba), trajectory shortcut trees (Fudan University + Meituan), synthetic demonstrations (Fujitsu + ISM), cooperative role training (Waseda + Adelaide), environment evolution (Tencent), and regression replay (Microsoft)—that enable targeted self-improvement without external labels. The line of inquiry has moved decisively from diagnosing failure to building and evaluating specific training loops that leverage an agent’s own data, with the newest benchmarks (EvoPathBench) revealing hidden weaknesses that emerge only when agent memories are tested at sequential checkpoints.

The accumulated work establishes that agent memory must be simultaneously consolidated, compressed, retrieved, and filtered under task-specific and resource constraints, with methods spanning visual workspaces, checkpoint-based testing, hierarchical trees, and adaptive retrieval. Most recently, researchers have introduced bounded visual workspaces to isolate generated artifacts (NLPR&MAIS, Chinese Academy of Sciences), checkpoint-based testing to detect hidden degradation in evolving memories (Zhejiang University, Alibaba), and task-completion benchmarks that move beyond prompted recall (Mem0).