When people search for information, the order of results can bias their final answer. Similarly, pursuing a dead end can make it hard to change direction. These biases also appear in language model systems that search for answers.
The researchers built a controlled system where a language model searches documents to answer questions. They varied result order, records of prior work, or an initial guess, while keeping the documents fixed. They tracked queries, which documents were retrieved and opened, and the final answer.
They found that order affected the model's answers more when it had to search than when given all documents at once. Dead-end work reduced strategy switching beyond what context length alone would cause. Opening more opposing evidence did not always improve accuracy.