What they did

The authors trained a search agent to decide when to request feedback and how to act on it. They initialized recovery behavior using trajectories organized around failures made by the base agent, then alternated optimization of the model's agent and critic roles.

Online, CAFE estimates the value of requesting feedback from the success-rate difference between calling and skipping feedback for the same prompt, and adjusts token-level learning signals around the intervention. Offline, it trains the critic through preferences derived from matched successful and unsuccessful rollouts. The system was evaluated on seven agentic search benchmarks, including six out-of-domain tests.

Key findings

• CAFE outperformed the evaluated reinforcement-learning-based search agents on average across seven benchmarks.
• Its improvements persisted across all six out-of-domain benchmarks, suggesting that the learned feedback behavior transferred beyond the training distribution.
• The authors report fewer answer-level hallucinations, although the abstract does not provide the reduction's magnitude.
• Updating only the search agent or only the critic eventually reached a plateau; alternating updates between them continued to improve performance.

Why it matters

Search agents trained only from final outcomes receive little information about where a trajectory went wrong. CAFE shows that learned interventions can provide more localized correction, but that the feedback mechanism must adapt as the search policy changes and generates new kinds of failures.

Caveats

The supplied abstract does not report absolute scores, effect sizes, statistical uncertainty, model scale, or inference costs, making it difficult to judge the practical size and efficiency of the gains. The evidence is also confined to agentic search benchmarks; it does not establish that co-evolving critics are necessary for other agent types or deployment settings.