Language models that search the internet for answers are often trained only on whether the final answer is correct. This means every search gets the same reward, making it hard to learn which searches actually help.
The researchers introduced an extra training signal that measures only how much each search's retrieved documents improve the chance of generating a correct answer, ignoring the model's own reasoning and query wording. This signal is added during training but not used when the model is finally deployed.
On several question-answering tasks, this method improved exact match accuracy by about one percentage point for a small model and about five for a larger one, compared to using only final answer rewards. This shows that giving finer feedback on each search helps these models learn to find and use information more effectively.
