The authors analyzed concurrent coding-agent traces in which each model call repeats much of the preceding conversation.
KV cache offloading works when host memory fits agent reuse
EfficientAgent predicts the reusable context of concurrent agents and adapts cache writes to host-memory capacity.
Chinese Tech
Kunming Shao · Jierun Chen · Jiangnan Yu · Xiao-Hui Li · Chaofan Tao · Yanli Wang · +4 more
The Hong Kong University of Science and Technology · Huawei Technologies Ltd. · The University of Hong Kong · Sun Yat-sen University
Research Digest··2 min read
The authors study why moving language-model key-value states from GPU to host memory can either accelerate or slow concurrent agents.
Why this paper
From Huawei Technologies Ltd. and 3 others · Released code
In one line
KV offloading accelerates concurrent LLM agents only when host memory holds the pool’s reuse working set and hardware economics favor reloading over recomputation.
What it released
Code
What we could check
- ✓Code link in the paper (github.com)
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (3 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§