Large language models (LLMs) power that repeatedly send their full conversation history to the model. This history gets longer as the agent searches code, edits files, or runs tests. Developers can add useful code to help the agent, or remove old tool outputs to shorten the history. Each choice changes the number of tokens the model sees. But the cost also depends on how the serving system reuses previously processed text (prefix caching) and whether the agent takes extra steps because it misses information.
Researchers tested these effects by running over 20,000 coding-agent tasks on standard benchmarks using open models. They separately counted three kinds of tokens: uncached input (first-time text), cached input (reused text), and output (the model's answers). They compared runs with and without added code context, and runs where old observations were removed.
The main finding: adding retrieved code cut search commands by 18–34% but increased total cost by 33–42% even when cached tokens cost a tenth of fresh ones. Removing old observations could save 19–25% without cache discounts, but cost 38–116% more when cached tokens were cheap. The reason is that removing information made agents take more steps, canceling prompt savings.
