Fri 13 Nov
LiveGet it free
ZOTPAPER

News without the noise

← All concepts

process-level evaluation

Evaluating how models approach problems by focusing on intermediate steps rather than final answers. It checks subgoal completion, reasoning chains, or evidence intervals, providing granular feedback. This reveals specific failures, like compositional errors or irrelevant actions, that outcome-based metrics hide.

The papers find that outcome-level feedback can effectively shape process evaluation, reducing reliance on expensive step annotations. Process evaluation uncovers distinct failure types across models, aiding targeted improvements. Benchmarks that require assessing intermediate steps highlight these detailed errors, making evaluation more useful for guiding development.

Papers
75
Released code
4
First seen
Aug 2026
Latest
Nov 2026

71 papers in the last two months, against 4 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Non-English requests raise tool admission and response harm in LLM agents

    Non-English harmful requests raise LLM tool admission from 46.9% to 67.4%; requiring answers in the page language increases post-retrieval harm.

    Oct 2026
  2. 2AI agents that succeed at tasks may still fail their mid-way steps

    Tool-use agents should be evaluated on whether every intermediate step remains executable, not just on final success.

    Oct 2026
  3. 3Search results order and dead-end histories bias AI agents' final answers

    Final answers of search-equipped LLM agents are more sensitive to initial evidence order than when all evidence is supplied, and dead-end work reduces strategy switching.

    Nov 2026

How fast it is moving

JunJulAugSeptOctNov so far

Often studied with

agentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real25 togetherbenchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.20 togetherverificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.17 together

Who is working on it

Tencent 5Microsoft 4Peking University 4National University of Singapore 3Carnegie Mellon University 3Shanghai Jiao Tong University 3

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.