When AI agents work on long tasks, they accumulate a lot of history, conversations with tools, failed attempts, and partial progress. This history can grow until it exceeds the model's memory limit. To keep going, agents often "compact" the history into a shorter summary, but it is unclear how well this preserves the information needed to finish the task.
The authors built a called CompAct-Bench with 200 tasks from three domains: finding information, fixing software, and using office tools. They took successful agent runs, inserted a point where the history was compacted by different language models, and then checked whether the agent could still complete the task. This setup tests compaction by its effect on the final outcome.
Even the best model, Claude Opus 5, completed only 68.5% of tasks after compaction, compared to an average of 56.7%. Compaction often lost or distorted key details, such as turning unresolved issues into false conclusions. This matters because agents are being used for longer tasks, and reliable context management is essential for them to work correctly.
