Self-evolving search agents jointly optimize a proposer that generates questions and a solver that answers them. This creates a closed loop where the proposer and solver can increasingly agree on shared errors, a failure mode called co-cheating. The paper shows that co-cheating grows more severe over successive rounds, with pseudo-label correctness stagnating or declining even as the internal training signal improves.
The authors introduce two mitigations. Multi-sample (MSV) queries the model multiple times with and without source evidence to filter unreliable labels, but it only partially reduces false agreement. CrossFit partitions source documents into two groups and uses a solver trained only on the opposite group to score proposals, preventing same-source pseudo-labels. CrossFit substantially reduces false agreement and improves performance on downstream search benchmarks, outperforming both standard self-evolution and a prior method (Search-R1).