What they did

The authors developed SelfCompact, a scaffold that equips language model agents with a compaction tool they can call to summarize the context, and a lightweight rubric that guides when to use it. The rubric recommends compaction when a sub-task resolves or the trajectory is converging, and suppresses it mid-derivation or when stuck. They tested SelfCompact on six benchmarks covering competitive math (e.g., GSM8K, MATH) and agentic search (e.g., WebArena, AssistantBench) using seven open-weight models, comparing against no-compaction and fixed-interval summarization baselines.

Key findings

• SelfCompact improved accuracy over no summarization by up to 18.1 points on math benchmarks and 5-9 points on agentic search tasks.
• It matched or exceeded fixed-interval summarization while reducing per-question token cost by 30-70%.
• The rubric was critical: models used the compaction tool effectively with the rubric but almost never at appropriate times without it, revealing a "meta-cognitive gap" in unprompted models.
• The approach required no fine-tuning or external supervision, working across multiple model families and sizes.

Why it matters

This work challenges the assumption that context compaction must be triggered by a fixed token budget. By letting models self-regulate, SelfCompact reduces waste and improves performance, suggesting that scaffolds can supply effective meta-cognitive capabilities without modifying model weights. It opens the door to more efficient long-horizon agentic tasks.

Caveats

SelfCompact was evaluated on math and search tasks; its generalizability to dialogue, code generation, or multi-modal agents is untested. The rubric is manually designed and may need task-specific adjustments. The compaction tool's summarization quality depends on the underlying model, and the approach may not scale to very long contexts where summary errors compound.