CAFE is a framework that trains a single model to alternate between acting as a search agent and as a critic that provides in-trajectory feedback. The agent learns when to request feedback, and the critic learns to generate corrective feedback from the agent's own failures. This co-evolution prevents the agent and critic from stagnating when improved in isolation.
On seven agentic search benchmarks, CAFE outperformed existing based search agents. It maintained performance on six out-of-domain benchmarks and reduced answer level hallucinations. Ablations showed that improving only the agent or only the critic plateaued, while alternating updates continued to improve.
This is interesting because it shows that for long-horizon search, feedback must adapt as the agent improves. A static critic quickly becomes misaligned with the evolving agent policy.